That is what I mean by using a gun, just getting something into space.
Well, it depends on what you mean by "in space." If you just mean "get some inert object above some arbitrary altitude," the easy way is to fire gun or rocket straight up. Say we call "space" 200 kilometers up, and all we ask is that a shell coast up to there, stop, and fall back down. Well, that would only require a muzzle velocity of 2 kilometers/sec, just a quarter of orbital. (Actually muzzle velocity needs to be higher to overcome air drag). However, bear in mind that during WWI, German "height-climber" Zeppelins hoped to bomb Britain with impunity by flying five kilometers or so high; later a big part of US strategic bombing doctrine was to fly very high, again so as to get beyond the effective range of ground based German or later, Soviet, guns. Effective range might not mean "higher than the shells absolutely reach," it might have been mainly a matter of spoiling their aim by forcing a very long time lag between firing the gun and the arrival of the shell. Still I believe even the best German artillery of the second war era would have been very hard pressed to push a shell even 50 kilometers up, let alone 200. So you are talking about making a much better gun in terms of muzzle velocity than the best things Hitler could order, a couple generations after the OP deadline. Also check out the "V-3," Hitler's third-string Vengeance weapon--it was to be a cannon that could strafe Britain. Note, not a cannon that could strafe New York or Moscow, note also what a big and complicated monster it was, with second and third stage booster shots to further accelerate the shell stationed like chevrons along the barrel. Could this contraption have fired a shell straight up to reach 200 km height? Maybe. But it was being designed in the 1940s, not before 1900!
We could start with someone other than the British getting ahold of Tipoo Sultan's rockets in the Anglo-Mysore war of 1799.
Or why not have the British just make the better use of them you think a non-Brit would have?
No matter, a gunpowder rocket will not cut it for reaching orbit. It isn't clear to me how much gunpowder combustion is a matter of reacting with oxygen in air and how much it explodes based on reacting materials it already contains. The more oomph it gets from air, the more clearly its function would be degraded on the edge of space.
Vice versa, rocket performance is degraded when the gases need to rush out against air pressure, and by the late 19th century improved forms of gun explosive that clearly did not need air to work at all were in development. A rocket fuel is different from gun explosive anyway; the latter you want to go to complete reaction as quickly as possible so its complete force is available to propel the projectile in the barrel; you want rocket fuels to be burning over an extended time. Solid fuels can indeed be used to get to orbit; modern ones have performance about 2/3 as good as fair liquid fuels.
The shuttle main engines have ISP in the range of 420 or so as did the hydrogen/oxygen burning upper stages of the Saturn V; using kerosene and oxygen instead the booster stage had one around 300, which is also typical of "storable" liquid propellants such as those used in the Titan missile, which also launched Gemini. Mercury was launched into orbit on oxy-kerosene Atlas rockets. IIRC the modern solid fuel boosters used on the Shuttle had ISP in the ballpark of 200; I might therefore believe that a dedicated 19th century program of solid fuel rocket development might achieve something like 150. The basic rocket equation is exponential; divide total change of velocity required (including virtual velocity as in overcoming gravity and air drag) by the effective exhaust velocity, and take the exponent of that and you have the ratio of total rocket mass before firing to what finally reaches the target speed. ISP such as I have cited is the amount of kilograms of mass a kilogram of propellant per second can thrust at one G; a G is 9.81 meters/sec^2, so roughly speaking multiply them by 10 (less roughly, by 9.81) to get the effective exhaust velocity of various fuel mixes--bear in mind they are lower in the lower atmosphere, and that a nozzle designed for good performance in vacuum is different from one that gives the best performance in air.
"Mission delta-v" is a conceptual way of summing up all the velocity change needed, including the virtual parts, to achieve a given goal; to achieve its 200 km nominal circular orbit the Shuttle had a mission delta-V of about 9 kilometers/sec. With rockets putting out exhaust at about 1500 meters/sec clearly the ratio is exp^4, or about 55. So less than 2 percent of the pad mass is going to get to orbit with these kinds of solid, and that 2 percent includes the mass of the rocket itself as well as payload. Using the idea of staging we can improve things a bit; it solves many problems. (The lowest stage engine is the one that has to worry about thrusting in atmosphere for instance; we can design them for thrust there and the upper ones for thrust in vacuum). OTOH when I try to estimate necessary launch pad masses for systems of a given performance using just the rocket equation I always wind up underestimating it, even though I'm not attempting to factor in the economies involved in staging.
What are we trying to get into orbit? If all we want to do is orbit something that people on the ground can see is orbiting in their telescopes, the rocket will still be big but not nearly as enormous as one that puts up a capsule that contains an astronaut. Since we are talking 1900, clearly the only way to get anything useful done in space is to send up a person.
But now we have to worry about getting that person safely down to Earth again. 19th century science would be only moderately useful in designing a survivable reentry method. Tsiolovsky was I believe then already on the right track in suggesting some kind of ablative dumping of the orbital energy--he suggested carrying water and keeping the outer hull acceptably cool by evaporating it. That would require a lot of water, which greatly adds to the minimum mass that has to be launched along with even a very small person.
So yes, I suppose with rockets, solid fuel rockets, and a POD much earlier than 1885 (this stuff is going to take a lot of time and effort to develop) some sort of steampunk Vostok might be launched by 1900, and its astronaut even returned safely to Earth. The effort and time involved in just getting some silvery sphere into orbit just so you could point at at and say "I did that!" is somewhat less, still more than a 15 years project I think.