From the 2000s onwards, there's two ways the Libertarian Party could go in terms of becoming more influential: the Ron Paul way or the Gary Johnson way. Both of these involve alienating the die-hard base while gaining to new ones. The Ron Paul way is to appeal to the right: focus on significantly shrinking the government, take an anti-immigration stance, and have a states' rights approach to social policies. This will turn off a lot of cultural liberals in the Libertarian Party, but attract a large number of Tea Party conservatives. The Gary Johnson way is the appeal to the center (i.e. socially liberal, fiscally conservative) while still having a small-government, libertine approach to issues. This would definitely turn off hard-core libertarians, but attract a lot of moderates turned off by both major parties.
The best way for the Ron Paul way to work is if the Republicans are perceived as going to far too the center, allowing the Libertarians to attack them from the right. I could see them possibly merging with the Constitution Party in this scenario, since Paul is basically a paleoconservative. In this case, the Libertarians would have a solid, but very limited base and would be seen as a fringe party that just takes votes from Republicans.
The Gary Johnson way would work best if Republicans moved to the right while Democrats moved to the left (imagine Ted Cruz vs Bernie Sanders in 2016). Johnson and Weld-style moderate Republicans join the Libertarian Party and try to appeal to the center. This would be difficult to pull off, since libertarianism is an inherently radical ideology, but it's not impossible. In this case, the Libertarians wouldn't have much of a solid base, since their support would mainly come from moderate swing voters, but if the two parties became unpopular enough, they might be able to make some gains.