I think the Statue was touring either France or the US at the time (in various states of disassembly), in a desperate bid to raise funds for the pedestal. IOTL they managed to scrape up enough through private donations, but I imagine they were cutting it pretty close.
I suspect these pieces would simply remain touring for several more years, until someone finally gives up and leaves them on display in a Paris park. They rest there for a decade or so, until they are moved into a warehouse, perhaps being disassembled even more, and that's where they rest to this day.
Alternatively, someone wealthy industrialist comes along and puts together enough funds to buy a site and build the pedestal, without the federal government's involvement. It ends up being erected in the middle of that man's lush estate as the centerpiece of some elaborate garden of monuments in a wealthy resort area. Or, maybe it's donated as the centerpiece of some Olmsted-designed park in New York, Philadelphia, St. Louis, Chicago, etc. Maybe it gets put up along the owner's railroad lines in Nowhere, Dakota Territory, where it slowly falls apart to corrosion and becomes a rather campy tourist attraction by the late 20th century (The Big French Lady in Pierre).
As for any changes to the diplomatic relations between the U.S. and France, I don't know, but I wouldn't suspect much will happen beyond a change in some city skylines.