I'm not seeing what Japan gets out of this alliance.
The 1920s Imperial Japanese Navy's battleships and battlecruisers, at this time, are mixed coal and oil firing. Where are the friendly ports for them to refuel, or are they going to develop long-range colliers and fleet oilers earlier?
The Anglo-Japanese Alliance is no longer in effect by 1922, and the Washington Naval Treaty is in effect. Japan has fewer ships now, and the Royal Navy would not appreciate them being near the Gulf of Aden.
Japan gains access to... a poor country with an uncertain amount and type of resources to be utilized, which it can't feasibly supply, loses a lot of the value in resources due to the expense of shipping them literally halfway across the world, spread the IJN and IJA even thinner.
All this, and China is on their doorstep. It just doesn't add up.
For very small assistance, Japan may get small yet valuable returns in Ethiopia (especially in now Eritrea) mining concessions:
Here resources in order of decreased availability (simply distance from coast)
1) Alid Volcano - mischmetal (useful for lighters and incendiaries) and zircons (refractory crucibles)
2) Danakil depression - barium (useful for vacuum tube getters and cathodes - IOTL, Japan imported all barium untill 1943) and platinum (again, refractory industrial goods)
3) Gimbi- iridium, rhodium, ruthenium, platinum (refractories)
4) Shakiso - poly-metallic, including nickel (alloy steel), molybdenum (light bulbs and marine-grade alloy steel - IOTL Japan started domestic mining of molybdenum from 1931), tantalum (refractories), bismuth (fuses, medicines) etc. Actually molybdenum has huge synergistic effect on industry - more productivity due better lighting, more pressure to develop power networks, less tungsten diverted from tool steel applications. Also, reduced maintenance and better longevity of existing assets due rust reduction in salt spray zone (very important for Japan).
5) Kenticha mine - tantalum (refractories) and niobium (rockets, capacitors)
Some of application were not recognized by Japan in time IOTL - but with extra foresight and consistent policy the resources above can be obtained for small transport expenditures and virtually garbage - obsolete firearms (especially Murata rifles) plus ammunition stocks for them. The Murata rifles stock would be pretty average compared to OTL mix of rifles the Ehiopia had in 1935, may be increasing Ethiopian infantry power by 30-50%.
Navy: obsolete destroyers up to Harusame-class whose value for Ethiopia would be mostly deterrent rather than battle force.
Some automobiles (Japan produced Ford model A since 1928 too), plus radios for better liaison.