I think the biggest problem would be demographics. Adopting European tech and social organization is very difficult if you don't have cities, for example.
When we're talking about Japan we're talking about a country which already had a dense population settled in cities, with a hierarchical government. Let's look at the places a lot of native american nations would be starting from:
If you want a 19th-century POD we're talking about relatively few nations who still control territory. Maybe the Sioux could be used as an example. According to
http://www.nebraskahistory.org/publish/publicat/history/full-text/1994-Teton_Sioux.pdf they had a population of around 20 000 in 1900. That's not very many people for a nation to draw on to fight Americans numbering in the millions. As far as I know, by that point in time they had already adopted many European innovations such as horses, gunpowder weapons, etc. They lost to the US not because they weren't "modern" but because there simply weren't enough of them left.
Ok, so let's go back a little farther - maybe we want the Haudenosaunee (AKA Iroquois) to modernize. From a little research (
http://en.wikipedia.org/wiki/Iroquois), it seems that even at their peak they only had 12 000 people. Again, even though there were fewer European colonists in the 17th and 18th centuries, there's still no way the Haudenosaunnee could become more powerful.
Really I think your best chances for a nation able to defeat European colonists are the Aztecs or the Incas. They both had cities and the population base to support the production of European-level tech. And that would have to be a fairly early POD because both were conquered pre-1600. I'm thinking the Incas would be the best bet as they're less accessible to conquistadors than the Aztecs. If you can have the Incas spend a couple hundred years in contact with Europeans without being conquered, then perhaps they can learn something from their visitors.
I think your best bet would be to have a nation other than Spain make contact with the Americas. Either have a different nation which is less conquest-minded in Iberia, or have some other nation begin exploring first. What you would need is sustained contact and trade without conquest. Then, it becomes possible for the native american nation in question to begin to modernize.
BTW there were lots of examples of native american nations which did have sustained trade without conquest: the Haudenosaunee are a great example. And, to a large extent they did acquire Euorpean tech (I'm thinking muskets here), and used them to conquer their own neighbours. But, they never had the population base to found cities and develop the industry necessary to produce muskets themselves...