The Upper Nile especially seems like it could quite plausibly be conquered by a powerful east African state, not least since IIRC at thus time it was still majority Coptic. Going downstream on the one of the biggest and tamest rivers in the world is a massive logistics boost, add in the Crusades, Fatimid weakness and the existing Coptic populations and I think the most difficult part would be getting a powerful state around Nubia in time for the Crusades. Accomplish that and I think all of Egypt falling is quite plausible indeed, it never happened OTL which is the only reason Egypt never really had to worry about their southern border.