They had some expeditions to Somalia.
https://en.m.wikipedia.org/wiki/Land_of_Punt
And ptolemaic Egypt traded with India - so why not securing the way to India by conquering the coast of Arabia?
Sadly, after a brief read that doesn't make it clear whether these expeditions were peaceful, or trade-wars, or intimidations. But I wasn't aware it was considered God's Land - perhaps an interesting cultural justification for conquest and colonisation.
The bolded part just isn't true. The Egyptians built excellent boats and sailed all around the Red Sea and the Eastern Mediterranean. It's true that they didn't establish (very many) colonies (outside of the Western Desert, parts of Canaan, and Nubia); but that was mostly because they didn't want to.
You need a reason for Egypt to become culturally expansionist, not just politically. Egyptian of its culture as universal, but it was also bound up with notions of insularity - think "Anyone can be Egyptian, provided they live in Egypt". You need to do away with that second part.
Could Security be a justification? Military colonies in the same manner as the Romans. Build a new city, filled with Egyptians, to control the "provinces". The idea of an Egyptian Sicily (whether united with Egypt or not) is a cool idea. Resources seems the most obvious, especially if it enables access to wood. I suppose something like that might have happened on Cyprus, which would be a perfect target for a Mediterranean colony.
It could always be changed to "Anyone can be Egyptian, provided they've stood in
Kemet" or "swam in the Nile". The latter seems to be a bit of a copy of bathing in the Ganges. It does seem a bit like the Hajj too. Perhaps another alternative is a baptism with
Kemet (Put a thumbprint of Kemet on your forehead?). More practical - ships filled with soil. Considering the role of Pharaoh, I would expect they could pull it off. -> Suddenly soil capture becomes an industry!
Yeah, the Egyptians are known to have sent trading expeditions as far north as Lebanon and Syria and possibly as far south as Somalia, but they never stayed anywhere outside of Egypt for very long because they held Egypt to be sacred. The Egyptian state religion outright stated that any Egyptian who died outside the borders of Egypt would not resurrect and go to the next world. If you want Egypt to be more culturally expansionist, somehow you need to change that. Maybe an ATL Amarna period?
On the ATL Amarna Period - If you haven't, read NikeZnates TL, it's fantastic and bathes in that. As I mentioned, above, perhaps that act of bathing, or having the soil of Kemet on your forehead is another way to assure that they would resurrect and pass to the next world. Whether once in life, or perhaps repeated before death.
Assuming these cultural ideas are plausible, what new resources besides (I hope) coffee could an Egypt with colonies around the Red Sea acquire? Could they manage to maintain colonies around the entire Red Sea?