In a world where Washington has only one term, the United States is likely to experience earlier and more severe political polarization. There is no obvious universally popular candidate to cool tensions between pro and anti-Administration officials. If Adams comes to power, which is probable, he'll be as much a polarizing figure as he was in the original timeline.
A related consequence is that Hamilton would be much less influential. Although supported by Washington, both Adams and Jefferson personally loathed Alexander Hamilton.