You can't exclude ANYONE from a general vote, and that's actually the only reason you can't exclude a LGBT (there is no protection for discrimination based on LGBT, you can legally in most states discriminate, it's sad but true); I hate when people only mention the minorities or women as if it is only minorities, civil rights go both ways, despite conservatives thinking whites are so not covered. And no you can't limit your electors in such a manner as that violates the 14th, 17th, and 24th amendments and the precedents of the SCOTUS rulings based on those amendments and their "penumbra".
There are explicit textual provisions forbidding disenfranchisement on the basis of race, color, previous condition of servitude (15th amendment), sex (19th amendment), failure to pay poll tax or other taxes (24th amendment), or age if age is >= 18 (26th amendment). There are other textual provisions (section 2 of the 14th amendment) that reduce congressional (and thus electoral college apportionment) in proportion to the number of adult male citizens over 21 who are disenfranchised for reasons other than criminal conviction or participation in rebellion.
The equal protection clause of the 14th amendment has also been read as putting limits on what kinds of restrictions can be put on voting rights, but in general, equal protection cases rely on either facial or as-applied discrimination against a "suspect classification" (race, national origin, religion) or "quasi-suspect classification" (gender, legitimacy of birth). If the provision discriminates on a non-suspect class, the court generally applies the very deferential "rational basis" test, which merely requires that the justices be able to imagine a vaguely plausible justification related to a legitimate state interest.
Disenfranchising everyone but left-handed dogcatchers probably wouldn't fly, since it's hard to imagine a rational basis for the disenfranchisement, and because as-applied it is likely to be found to discriminate indirectly against suspect classifications (many interpretations of Islamic Law consider dogs to be unclean (religion), and many Middle-Eastern cultures strongly discourage the use of the left hand for hygiene reasons (national origin)). And if it did get upheld, the state would lose most of its electoral votes under section 2 of the 14th amendment.
On the other hand, an electoral college law intended to work around most or all of the state being under foreign occupation is much more likely to fly. It's narrowly tailored to serve a legitimate public purpose (holding elections as best as possible in the face of a foreign invasion), so there's definitely a rational basis, and depending on the details, it's likely to pass muster under intermediate or strict scrutiny as well. The apportionment clause of the 14th amendment might still cause problems: it could be interpreted that the citizens of the state in occupied territory are eligible to vote but are being physically prevented from doing so by factors outside the state's control; or it could be interpreted that appointment of electors by the in-exile state legislature indirectly enfranchises those who elected the legislature in the first place; or the state could be reduced to the minimum of three electoral votes because almost all of its citizens are unable to vote.