You have a point there, as the closure of passenger railroads corresponds precisely to the biggest moves in civil rights. (Of course, we can’t forget trucks, highways and the postal service.) Another issue is that many communities were glad to see curtailment in railroads. I live in such a town. I moved to Quincy, Illinois in 1979 and people born in the thirties spoke of a raucous “little Chicago” reputation of the rail town. There was an attitude of “good riddance” as the hobos and red light districts disappeared. Other towns shared similar feelings. In 1978, movie producers wanted to make a film about Chicago and gangsters set in the thirties. They wanted to film in Rock Island, some 130 miles north. Rock Island said “no,” they worked hard to dispel that reputation. Movie producers managed to find a town in Canada. [That same year, the University of Missouri and City of Columbia said “no” to Hollywood as their choice to film “Animal House,” sending the project to second choice, Eugene, Oregon.]
Racism is an overused explanation. It is the go-to explanation for "Something I don't like". Most problems in the world have little or nothing to do with racism. When someone uses the word "racist" the first thing that comes to mind is "What are they trying to manipulate into believing?". At this stage it so overused that people will disbelieve accounts of real racism assuming it is just a trick for getting them to agree about something.
The biggest reasons had nothing to do with racism. The price of cars was falling relative to wages, the states and the federal government were building more roads, transit companies were bought by car companies who wanted to use busses which they made instead of trams which they did not. Cars were seen as modern while trains were seen as 19th century. All these and probably more are a better explanation than racism.