The biggest stumbling block is the fact that huge numbers of left wing platforms would never win referenda, and a great many right wing causes celebres would win them.
I don't think we'd be looking at a big fed left here. Direct democracy would limit the kind of issues you can vote on by the level at which you're voting for them. A county or municipality might be able to vote in things like prohibition, and maybe even most counties or municipalities will vote for something like that, but it wouldn't be binding on a state-wide or federal level. I think even things like welfare would be left on a state by state basis, not federal.
Politicians would be concerned with drafting legislation in response to electorate demand, determined during the election process. One would vote for a representative in order to get a policy on the ballot, not in the expectation that they will always vote in your best interests.
The concept of
delegative democracy would be closer to how this would work in practice. One vote that you can either use directly on any issue on the ballot, or give to a representative to use on your behalf.