Only if it is artificial object. Natural gas giant will collect too many hydrogen and helium from protoplanetary cloud when it will formedcomb
Oxygen is the third most common element, though I grant it is withal far less common than hydrogen.
Still, might we not have a situation where the hydrogen is depleted by some nearby star heating a very massive protoplanetary cloud, leaving a higher fraction of oxygen, which then, when the planet forms, combines with the hydrogen to form water, and thus get not an oxygen giant to be sure, but anyway a water giant laced with some spare hydrogen and helium? And then perhaps the hydrogen can be reduced further by dissipating away, along with much of the helium, leaving mainly water molecules behind?
In fact if the reason the hydrogen were initially unusually low is nearby stars in a cluster forming, and a large bright star forms which the water giant is orbiting closely, and it is high heat input from this new star that bakes out much of the hydrogen, then water molecules will also be split by UV radiation and solar particles, leaving a surplus of oxygen--what starts as a very heavy object with a water core and hydrogen-helium upper atmosphere could thus evolve first into a water giant and then increasingly oxygen-rich mix with water, which might even go to essentially all oxygen with little water left. On Venus this process led to a carbon dioxide atmosphere but that was because the oxygen had a solid surface to interact with and strip it of carbon chemically; if there are no such relatively large carbon reserves, pure oxygen might come to dominate. If there is any other substance whatsoever, oxygen is likely to combine with it, but if these substances are a minority of the mass, they will tend to be buried in the core.
Such an outcome seems likely to be rare, and perhaps short-lived if it is necessary for the star capable of cooking away the free hydrogen and helium to be a high power bright start, F or hotter; such stars do not last long and go supernova eventually, which might simply vaporize the brief candle "oxygen giant." But perhaps a small red dwarf could do the cooking, if it formed very near the oxygen giant, and nearby other supernovae might do little more than heat the big remnant planet briefly. A red dwarf would not provide much UV but sheer heat can still result in selective depletion of hydrogen. And a G type star would put out a fair amount of UV while lasting ten billion years or so.
So it might be as simple as a superjovian sized "ice giant" forming very close to a G type protostar. Rare but not unheard of I'd guess.