As to the corruption: Harry Daugherty, Edwin Denby, and Albert Fall would never have become members of the cabinet, especially not Daugherty. The whole so-called Ohio Gang would have remained a bunch of borderline low-level political grifters on the outside looking in. Long story short: forget Teapot Dome and the Veterans' Bureau scandals. No idea about Lowden's views on prohibition: could be he could readily see the waste and inefficiencies early on and declare it good in theory / unworkable in practice and come out for repeal to save tax dollars spent on enforcement, replacing them with tax revenue from excise taxes. Not that they'd need it, but a repeal plank in the '24 or '28 GOP platform guarantees the GOP at least four years more.
Side note I: expect Hughes and Hoover to become part of a Lowden cabinet; they were too competent as administrators to be overlooked. I'm unconvinced Andrew Mellon would have been tapped for Treasury, though. Even though he turned down the offer of the post IOTL, I wonder if Lowden himself and/or members of his coterie could have persuaded Charles Dawes to take Treasury. That could have significant implications on the Wall Street bubble we knew in our version of the 1920s.
Side note II: likely there would be a role for Calvin Coolidge, given his popularity in the wake of the 1919 Boston police strike. Perhaps he'd head Interior; can't see him running any other department if you assume Hoover gets Commerce one way or the other (although I suppose you could transpose Hoover and Coolidge).
Side note III: Warren Harding remains an amiable nonentity "representing" Ohio in the Senate at least on the short term. I'd bet a scandal about his running around would surface sooner or later--like Nan Britton and all the noise about Harding fathering an illegitimate child. While there were more restraints on the press in general then, senators were somewhat more available targets than the presidency, so it's not unlikely this sort of thing might make headlines for a short while. It would probably mean divorce for Harding and possibly the end of his career in national politics, but he wasn't all that ambitious IOTL anyhow.