In terms of political and maybe geographical balance, Laxalt would probably be a good choice for Bush especially since there is a Republican governor of Nevada at the time to appoint his successor. The only issue I see possibly arising would be if Democrats wanted to make the nomination difficult they would probably be able to do so with some of the dirt from Laxalt's time as governor. Another possible option along the same vein (conservative with regional diversity) would be Phil Crane of Illinois, but again Democrats could make his drinking/alcoholism an issue if they wanted to do so.
Like
@James G said, I don't see Bush going for Rumsfeld. Dole would be out for many of the same reasons, including personal animosity between him and Bush. You mentioned Cheney also, but I don't think he has the stature to make a VP in 1981 when he's only an ex-C.O.S and two-term member of the House of Representatives.
If Bush is looking to reassure conservatives, he could always go with former Senator James Buckley, Senator Orrin Hatch (who was considered a hard-line conservative back when he was first elected), Representative Jack Kemp, Governor Pete DuPont, or Secretary of Energy James Edwards.
On the other hand, if Bush isn't all that worried about conservatives he may go with someone like Senator John Warner, Senator John Heinz, Senator Dick Lugar, or Senator Howard Baker.
If he wanted to make an historic appointment there are two female candidates who spring immediately to mind: Senators Paula Hawkins of Florida and Nancy Kassebaum of Kansas, but I would consider those to be truly long-shot choices.