Cook
Banned
The great difficulty is that blacks were a majority in some states, and a large majority in many localities. Full enfranchisement of blacks meant black control of local government, including law enforcement.
It was my understanding that 'people of colour' did not constitute a majority of any state in the Union at the time and represented only a quarter of the population of the Confederacy during the Civil War. While no doubt there were counties where they outnumbered whites, I'd be very surprised if there was any state where whites were the minority.
I don't see how this would be a problem anyway, if anything it should have made reconstruction easier and more effective.