You are probably right, but this--"they had a workable conception of the atom (well, they will once they discover radioactivity)"--is where I think things could go wrong. But the OP is asking for the best possible space flight scenario, so, sure, lets say that experimentation with radioactivity gives them an atom model that's workable for developing fission earlier than in OTL.
Space flight may be the least interesting of the effects of this POD.
That's pretty much what I was thinking. IOTL, atomic theory seems to have been developed linearly from exploring around radioactivity. TTL research will suffer from the absence of 60 years of complementary advancement in other field of physics, which I tried to compensate for by immediately dropping the discovery of radioactivity directy into the laps of Gauss and Weber, whose work on electromagnetic field theory (already well underway by the POD, which was near the end of Gauss's career and early-mid-career for Weber) gives them the best understanding of complementary fields of physics necessary to develop atomic theory available at the time. I also tried to compensate by giving the research a political patron who would soon become extremely powerful (Bismark) so research into atomic theory would be actively pursued and fully funded from the beginning, rather than occurring in fits and starts until military applications become apparent and urgent as in OTL.
Admiral Matt brings up a good point about electronics and precision tools necessary to build a nuke. For an early nuke, even if the atomic theory is there, there's still a dillema in the engineering of the bomb. Once atomic theory is mature enough to build an experimental heavy-water reactor, purifying plutonium chemically from spent fuel rods is relatively straightforward, but building an implosion device is difficult with 1940s tech. A basic enriched uranium bomb is very easy to build from an engineering standpoint and could have been done with 1890 tech, but enriching uranium to weapons grade is extremely difficult even with 1940s tech. I don't know enough about the history of the techiques and prerequisite knowledge involved in either option to evaluate how plausible it would be for an 1890s Manhattan Project with accellerated knowledge of atomic theory to develop the techniques necessary on the fly.
Guidance, control, and life support are major concerns for any attempt to accelerate a moon landing, as is the metallurgical and design skills to build a craft that's light enough and strong enough for the job. That's why I focused on an Orion-type spacecraft -- the massive lifting power gives you a bigger margin of error for guidance and control, allows you enough surplus payload to carry more compressed oxygen to compensate for less effective pressure seals and less tech available for scrubbing CO2 (although by 1900, life support tech was already good enough to support submarines and deep ocean diving). It might not be good enough, but it's the best shot I could come up with to get to the moon by only accellerating one facet of technology.