An accidental collision, let alone one severe enough to take down one of the towers, is highly unlikely due to the use of airspace around lower Manhattan. In good weather with the right winds, the approach up the Hudson and the airspace therein is reserved for flights landing at LaGuardia runway 13. They fly northeast up over the river and make a right turn to land at LGA; large planes bound for JFK or EWR are nowhere near the area. Theoretically, an extended ILS approach to JFK's runway 13L would take planes over lower Manhattan. However, there is no such approach; the ILS is to 13R and there is no extended approach over Manhattan, as such a pattern would conflict with traffic heading into Newark and departing LaGuardia. Instead, planes are vectored over Brooklyn to TELEX intersection, about 8 miles from the runway. Moreover, all planes in that airspace are under radar control and modern aircraft have navigation capabilities that make such a large error highly improbable. To get a plane that far off course, you'd need a combination of both a ground control radar failure and an on-board navigation system failure. You can construct a scenario of cascading failures that gets you there, but it isn't very likely and gets less likely the larger and more complex the aircraft is.
The second part of the problem is that an aircraft that off-course is not under any reasonable scenario going to be flying anywhere near as fast as the 9/11 planes. There's a 250 knot speed limit below 10,000 feet in the US (a byproduct of the 1960 midair collision over Staten Island). As I recall, the 9/11 planes hit the towers at speeds of around 500 knots for UA175 and 400 knots for AA11, far faster than any scenario involving an accident. Since momentum is mass times velocity, the momentum of an accident aircraft is going to be something around half that of the 9/11 planes if one assumes a speed of 200 knots (230 mph). Less momentum at impact = less damage. Now, I suppose you could create a scenario with a just departed intercontinental flight, loaded with even more fuel than the 9/11 planes and an even larger aircraft, say a 747-400 headed for Tokyo, that develops an emergency after takeoff and winds up hitting one of the towers with the same momentum as one of the 9/11 planes, but this is really a stretch. Possible? Perhaps. But it's like a one in a billion scenario of cascading failures and mistakes that is really hard to imagine actually happening. If it did, it would be regarded, rightly, as a fluke accident.