Well, Legitimist (and far-right issued from Legitimist, according to René Remond) were...well, living caricatures.
They never managed to be in power in France, except when the country was defeated : 1814, 1815, 1870, 1940... and were almost always in the opposition passed 1830.
As they wouldn't have been to enforce their program, some tried to get in touch with Orléanistes since the 1840's, 1850's (with the support of Thiers and benevolence of Count of Chambord) but both Orléanistes and Legitimistes groups didn't really involved themselves : staunch Legitimist couldn't bear to negotiate with the guy that overthrow ed Charles X and many Orleanists didn't wanted to make concessions with guys ISOTed from 1788.
Now, except in great towns that were majorly republican, the population wasn't opposed, neither supporting monarchy. But it would have needed an earlier and better compromise between two monarchists currents.
The issue is that, as said, without defeat of the imperial regime (that benefited from a popular support, outside great towns and traditionally monarchist strongholds), it would have been hard for them to get back in business.
And this crushing defeat wasn't foreseeable.
Finally Henri de Chambord, keeping the subject of his succession in the most vague way, durably poisoned monarchism.
It's not even that they destroyed their chances of success, it what they were that destroyed it.