Mecca originally had many, many idols. The lack of the ka'aba will not reduce Mecca's role in the area. The Prophet Mohammed destroyed most of the local idols, but choose to keep the ka'aba in order to reconcile the existing elites of the city (who depended upon the pilgrimage trade of the city) to himself and his new religion.
Without the ka'aba, Mohammed simply picks some other idol to keep to reconcile the city to his new faith. Presumably he can come up with some sort of rationale to justify the new idol. If not and no other idols can be used by him, then he still probably promotes Mecca as a pilgrimage spot and comes up with some other way to soothe relations with the Meccan elite.
I really don't see this as being a real turning point of any kind.