wonder how much of that had to do also with the vast US expansion... with a constant influx of new territory that required a lot of work, labor shortages seem to be a natural result...
Yep, that's a big part of it. Not the whole story, of course, but a big part. It's about alternative choice/opportunity cost. You HAVE to pay your workers a lot if they have the alternative of setting off West and working their own land. (EDIT: That is, you have to pay them more than their subjective evaluation of the benefit of setting off west to work their own land: an open frontier with cheap land raises the 'reservation wage' of prospective employees and employers have to big above that reservation wage to successfully get hires)
The 'problem' also generalizes: A society with a relatively small population but a great deal of capital is going to have pay workers a lot because they have so many other alternatives other than working for one particular employer.
This is just the underlying framework of supply and demand: A high capital stock demands a lot of labor, which raises its price, other things being equal. It also highlights just how quickly the US was growing in the 19th century, when you consider the huge influx of immigrants/dizzying birthrates over the course of that century against a relatively stable/increasing cost of labor.
A US that stayed low net immigration throughout the 19th century is going to have an almost prohibitive cost of labor. The industrial development that does happen will look more like that of the 1820's and 1830's, rather than the burgeoning behemoth of the 1870's and 1880's. It will be extremely capital intensive with an intense focus on energy consumption and mechanized production. That's kind of scary because those are trends that
already existed. There's going to be a tipping point where the US simply can't import capital at the rate that it's being demanded, which is going to cut off development as the cost of capital rises.
The US will go into the 20th century looking more like Scandinavia: Relatively poorer, relatively more rural, but quickly developing as capital scarcity cuts off wage growth, lowering the overall cost of labor vis the rest of the industrialized world.
The problem is that the US, historically, had a lot of problems with its capital markets that Scandinavia didn't share. The US' financial system would have to see serious reform away from the strangled system of OTL in order to enjoy the early 20th century takeoff the Scandinavians did.