I'm sorry if I sound like I'm short circuiting your OTL, I'm trying to game it out. Yes, the ships were assembled in Tokyo Bay to go too Attu, but the Japanese hesitated till it was too late. If they acted with greater speed, and resolution they could have sent a force to challenge the Invasion, but it would have fewer ships. The Japanese caught the USN by surprise with their rapid reaction to the landing at Guadalcanal and got a big tactical victory at Savo Island. The list of Allied mistakes that led to that disaster are long sad, and unforgivable. About everything went wrong.
The biggest mistake leading to Savo Island was the basic error of underestimating the Japanese. A lot of "Water" was under the bridge since then. An IJN commitment to a strategic counteroffensive in the North Pacific with what was left of their carrier forces, and half their battleships into the open, but foggy waters around Attu would take the Americans by strategic, but not tactical surprise. Both sides had been feeding more, and more of their forces into the South Seas, but the Japanese do hold a more central position so, they can more easily shift the fleet north than the Americans can. The question would be how the Americans would respond.
In the OTL both sides were very cautious about the commitment of battleships. Lossing a battleship or carrier could mean the loss of 2-3,000 men, which is why the USN almost always chose to use aircraft to deal with Axis Battleships wherever possible. Nimitz was a very aviation minded admiral, maybe even more than Yamamoto. He sent no BBs to Midway, and Kinkaid only did it once at the most critical moment of the war in the South Pacific. I'm just trying to think about what options the USN had given the timeframe of a 2-week period in late May 1943, with a head on battleship clash being the last option.
Bottom line is that late May is really too late to affect the outcome of the Battle for Attu. Even some battleships shelling the island won't change the outcome, just inflict some casualties. They really needed to disrupt the invasion in the first few days as they did at Guadalcanal, and make a long-term commitment to supplying, and reinforcing Attu, which given declining resources, and the ongoing war in the South Pacific isn't very realistic.
If a head on collision is the plan, I think it would be dependent on having the North Carolina & Washington being able to get there in time. The USN won't go into a battleship fight knowing they're at a numerical disadvantage, even if they think they have better ships, and technology. Would they really send BBs against IJN carriers, without USN fleet carriers supporting them? At this stage in the war the USN has a decided advantage in radar fire control. The Japanese have surface search radar, but no FC capability. The USN might want a night action, giving them the best advantage in long range gunnery. However, the nights are very short, but visibility is very poor most days, which is to their advantage. The situation just seems a little too Dicy for the Americans to want to play this game, when they have other options.