Does Alaska count as overseas? Don't forget that not long before Alaska was purchased, there was serious discussion of trying to get British Columbia into the US (presumably they would have changed the name

)
By the time of the Alaska purchase, it was fairly obvious that wasn't going to happen, but there were still some people talking about it.
Also, it should be fairly easy for Hawaii to remain a puppet rather than get annexed - does that count for the purposes of the thread?
Caribbean islands are also pretty trivial; just don't have the US buy them off Denmark. They were purchased during WWI because the US was concerned Germany might try to use them as a base for commerce raiding; it seems pretty easy to get rid of that purchase.
Pacific islands are a little harder because they were mostly taken originally for economic reasons (guano), but maybe have the US abandon guano islands rather than defend them once they're "spent".
Anyway, all of these "overseas" holdings are pretty much negligible. Hawaii brings cheap sugar and tropical fruit, and Alaska gold, oil, and other minerals, but I don't think that the lack would have a super huge effect (though butterflies, of course...)
No pacific islands might lead to a smaller, less active navy, I guess. Though that's a bit of chicken-and-egg