I absolutely see the good sense that the OP is alluding to.
As a good way to achieve this, I propose upholding the value of human (and animal) lives in legal liability issues over the newly-invented "rights" of motorists to drive.
In Germany, the 1909 "Law on Motorised Traffic" paved the way; it is often primarily seen as an early regulation IOTL, but what really changed things back then was § 7, in which the car driver's liability for the damage of goods and animals, the hurting and loss of human lives was excluded in cases in which the accident was caused "by an unforeseeable event, which was not caused by an unsuitability of the car [...] and against which the driver had taken any reasonable precautions".
This was later taken to mean that if you walk around on that part of a street which was now reserved for motorised traffic, it was your fault if you got hit. (Same with your cows, sheep etc. who wander onto a street.)
Scrap this, and have jurisdiction converge on the view that, whenever someone is killed or harmed by a car without unequivocal intention of suicide or self-harm, or something is damaged by a car without it being hurled into the car's pathway with malicious intention, the driver is fully and exclusively liable for negligent homicide / assault / material damage, and motorised traffic is going to be too risky to become anything other than a flamboyant hobby of the wealthy in reasonably populated areas - which at the same time doesn't preclude the widespread use of internal combustion engine-driven vehicles in areas where accidents are much less likely, like tractors or combines on fields.
I think you have the right of it. Self propelled vehicles are useful and perhaps inevitable, but any vehicle that is travelling faster than a cantering horse or a scorching cyclist should, like a train, have its own exclusive right of way so that it does not mix with slower traffic. There is another simple way to restrict the impact of motor traffic on normal roads and streets - ban ths use of googles and windshields.
Exclusive rights of way would of course be prohibitively expensive, especially when high-speed motors are still rare. Perhaps some enterprising cities might experiment with running dual-mode tramcars on and off existing tramways, evolving eventually into something like the BladeRunner concept?
IOTL the motor manufacturers felt entitled to usurp the existing road system because they were the same people who had lobbied for its improvement when they were bicycle manufacturers. See Roads Were Not Built For Cars