[New] Plymouth/Plimouth:
Capital: Plymouth (Plymouth City, Plymouth Town, Plimouth)
Largest City: New Bedford
I think it's safe to say a city on the south coast would become the largest city, although I think the capital would stay in Plymouth for historic reasons and relatively short distances between it and any other part of the state. If the capital did move it would likely to be OTL New Bedford (or maybe Fall River or somewhere near New Bedford).
I assume Nantucket, Martha's Vineyard, and The Elizabeth Islands (Gosnold) would become part of "Plymouth", although they never were in OTL. The islands were part of NY until they became part of Mass, but are closer to "Plymouth" geographically and culturally.
Although Massachusetts and Boston would still dominate New England, it would be slightly less than OTL since there would be another NE state with significant population.
I'm curious what places would be named. Having a state with a capital (and county) that shares it's name would be an interesting quirk. Instead of places like Oklahoma City, the town might simply be called "Plymouth" just like the state. Then again, both could be "New Plymouth" and could also be spelled with an i instead. I would guess everything is called simply "Plymouth" or the capital is called "Plymouth Town".
The perception of Massachusetts in the eyes of other Americans would change. (If we can for a second ignore major butterflies) Plymouth would be known as the home of the Pilgrims and one of the birthplaces of the country. Massachusetts would be more known for the Revolutionary period and less known for the early colonial period. Plymouth would also be thought of as a very nautical state, more so than any OTL state I would think. It would be very important in the early to mid 1800s due to Whaling. Now it would be famous for the beaches of the Cape and the islands. It's general reputation would be most similar to Rhode Island of the OTL states, but not exactly the same.
But who knows, really. Some major changes could happen. Maybe Boston and Providence and bigger Whaling ports than "New Bedford" in this timeline. Maybe a state that was built on religious fundamentalism even more than Massachusetts would be less likely to transform into a center of intellectualism and progressive politics. I see Plymouth staying more socially conservative than Mass or Rhode Island since it doesn't have the big city of MA, or the liberal history of RI. (I do not mean strictly the modern political definitions of liberal and conservative. I just think Plymouth would be a more 'traditional' society, and probably more Protestant than OTL Mass. Although it would attract immigrants, probably not as many as Boston) Then again, maybe they attract as many Portuguese people as they do in OTL and it becomes the only majority-Portuguese state or close to it.
A lot could happen. I agree that people are really underestimating the changes this would cause.