Personally I'd suggest giving the Balearics and Africa to this Middle Empire, but excluding attempts at reunification it does mean that each Emperor is better placed to hold their territory, and would develop distinct characters.
If we put aside "reunification" because that didn't happen IOTL...