The romans tried to invade Nubia during Augustus reign, and they failed. Like they failed in Arabia, at the same time.
If they ever would have been succesful, I doubt, the romans would conquer more than Erytrea. They have been only interested in controlling the Red Sea and the traderoute to India. Everything they could get from East Africa was available via trade anyways.
Obviously, they made it, finally. Latest in the 2nd century they controlled the entire Red Sea with a base on an island near the shores of Yemen. It seems the romans had best relationships to the Kingdoms of the Hyamar in Yemen, Meroe in Nubia and Aksum in Erytrea. What else should they goal for? Well an expedition led by 2 praetorian officers tried to find the sources of the Nile. But that was probably rather by scientific interest.
Nevertheless, things became worse after the fall of Meroe. The following chaos led to barbarian raids into southern Egypt in the 3rd century. And the fact, that the romans never conquered Arabia led to the Fall of Rome as an empire in 636 after the Battle of Jarmuk.