Xachiavelli not to sound too harsh, but your trying to 'retcon an idea' into a timeline that doesn't like it. That shows that you've done little reading to get an idea of the general course of events that lead to the Manhatten Project and the dropping of the Atomic Bombs.
Why is Einstein going to die in the Holocaust? He left Germany in something like 1932, well before much of the Anti-semitism hit the streets. Even then, Einstein is as big a figure in science at the time as Heisenberg. Hitler might have been insane, but his aids were smart enough that if he goes round ordering hits on top interllectuals the 3rd Reich is dead from litterally shooting itself in the head!
Why is Fermis reactor going to cause a great fire in Chicargo? Look, reactors at this time weren't much more than a few kg of not so enriched material used for generating heat and conducting experienments on activity rates and neutron counts and things like that. Absolutely nothing like 3 Mile Island, Chenobyl or Fukashima. The very idea of 'an explosion or fire getting out of control' is frankly a bit ludicris. No offense.
Singular wrong calculations will not affect anything, becauses there is a good process in science to get others to correct them.
----------------------------------------------
The only idea that makes sense in all of this, is for their not be the will from the top of government in America to invest in the scheme. Just like Britian, France (while it still exsited), Germany, Japan, and the Soviet Union (although the Soviets stole a lot of tech, so arguably one might consider their project 'piggy backed' on Allied Research).
Indeed, it is very unlikely, but still possible, that both Britian and Japan could have themselves got to the testing/design stages of a Bomb by the end of the war with the right political will. However, there wasn't that will, so it didn't happen.
That said, you need to concentrate on the political angle, not the technological R&D angle to get the effects you want. The two best points are;
1. No famous letter to Roosevelt
2. No Szilard* Mission/Different effect of it
A 3rd way might be;
3. Trinity is a fizzle (fairly likely) and that retards the program several months.
A 4th way might be;
4. One of the scientists accidentally irradiates a large number of other scientists taking out a chunk of the research team for a few months/kills them off. (No superhero jokes here).
But now we start to wander in the realms of conjecture. Where everything fits, but we are making up fictional events. Therefore while still valid, it would be better to stick to the known events that had the cause to stimulate the Manhatten Project into being.
*Cheers for the spelling correction