The F4 was foisted onto the USAF in 1962, at that time SAC bomber Generals controlled the USAF. Boyd's fighter mafia didn't gain a voice until the late 60s, after the poor exchange rate in air to air combat in Vietnam. The question is what aircraft will the USAF use in the mid-late 60s and I seriously doubt they'd be using any century series fighter, they'd buy some new design from the early 60s.
I don't agree, the USAF did a fly off between the F4H and the F-106 in 1962 (Operation Highspeed) in which the F4H won hands down. Further evaluations showed the F4H to be superior to the F-105 as an attack aircraft and superior to the RF-101 as a recce platform. So the Air Force moved forward with the acquisition of the F-110 Spectre/F-4 Phantom.
Bottomline - jazz up Century Series fighters all you want, the F-4 was simply a superior combat aircraft across the board. However, without the F-4, the Air Force has nothing to evaluate the Century Series fighters against and so it probably soldiers on through the 1960s with Century Series fighters until a combination of combat results in Vietnam and the rise of the fighter mafia (probably even more pronounced that it was OTL) push the Air Force to develop something better which will ultimately lead to F-15 and F-16 type airframes.