They say prog rock died with punk in the mid 70s, but Kansas had several hits in that time period (you can hear "Carry On Wayward Son" and "Dust in the Wind" to this day, although Kansas's best prog moments came on their earlier albums), and of course Rush shifted from a Led Zeppelin-inspired hard rock band to doing their own thing with albums like
2112. Outside of the Anglosphere you can find plenty of notable prog music in the mid-late 70s like Krautrock in West Germany (or bands like Eloy who had a more Pink Floyd-inspired sound) or French bands like Ange. And in addition to Kansas, the US had bands like Starcastle (obvious Yes ripoff, but solid enough).
The trick is getting the attention to the right songs, since there's some very catchy songs from bands I've mentioned here which could easily attract mainstream attention.
Another thing I've noticed is how post-80s neo-prog seems to have had a lot of crossover with progressive metal (Dream Theater's biggest inspiration was Rush, to the point where the singer on their first album was a Geddy Lee clone). Arena, probably the most notable neo-prog band of the 90s, did very prog metal-inspired albums by the early 00s (incidentally, keyboardist Clive Nolan--also with Pendragon, another notable neo-prog band who did heavier works in the 00s, has done work with Dragonforce). 80s neo-prog seems simple to make popular--not just Marillion, but other bands too. Dream Theater gained mainstream attention with "Pull Me Under", and there's plenty of material on the same album and the subsequent
Awake and
Falling Into Infinity. Most Dream Theater albums have songs which could have mainsteam appeal, and rock radio would play all sorts of nu metal/post-grunge. This is where you blend into other prog metal like Fates Warning, who released rather pop-sounding albums like
Parallels and
Inside Out in the 90s (although there's songs from their 80s albums like
Awaken the Guardian which could have been rock hits but they're just straight up metal and are a different topic) which have obvious mainstream appeal. Or songs like Dream Theater's "Constant Motion", which seems like a Metallica knockoff yet was popular at the time (metalcore band Trivium did plenty of Metallica knockoffs). Or at the other end of the spectrum, Dragonforce's "Through the Fire and Flames", since Dream Theater also inspired endless progressive power metal bands (i.e. Pagan's Mind, Circus Maximus, etc.)--that's another topic.
Prog rock will always be fringe, but it can also always stay on the fringe of popularity (i.e rock radio). I mentioned Spock's Beard--prog fans tend to like their debut
The Light, but for mainstream appeal (
The Light has elements you can shorten to singles, but overall it isn't something non-prog fans might like), the subsequent albums
Beware of Darkness or
The Kindness of Strangers or
Day for Night could capture the same appeal classic 70s albums had. There's a diversity of influence on those albums (notably the Beatles), so it's as "updated" of prog as Marillion was in their day. Spock's Beard could have been much more popular IMO.
Then we get to 00s prog again, where again, I'll cite Coheed and Cambria (their progressive moments and not their alt-rock/pop-punk moments, most notably the
Afterman albums in recent years), Porcupine Tree (who also fused with prog metal in the 00s as many neo-prog bands did, although a lot of their 90s work could have been rock radio hits), or the atmospheric rock sound exemplified by Katatonia or Anathema. In the mid/late 00s, bands like Disturbed, Linkin Park, Breaking Benjamin, etc. were popular on rock radio. While I don't like to compare something like
Viva Emptiness or
We're Here Because We're Here to mainstream rock radio hits, there is obvious crossover appeal. As for the 10s, where mainstream rock seems to be dead, I think Katatonia, Anathema, and Steven Wilson of Porcupine Tree's solo work could power rock radio that decade, in addition to Coheed and Cambria, and other related bands.
An issue that needs to dealt with (and it's one that is occasionally referred to as being a driving factor in the demise of prog rock), is song length. By the mid 70s, many prog bands were going overboard with songs that in some cases took up the entire side of an album, or even longer (I'm talking about you Yes - "Tales of a Topographic Ocean", with 4 songs for a double album, come on!). I'm a musician (I played symphonic music through my sophomore year of college, and I still play guitar in bands [though mainly in a church band nowadays]), and I even got bored with that. The average listener got worse than bored. Even Jethro Tull's "Thick As a Brick" album was basically one whole song for the entire album. The only thing that saved that album, was that there were song movements, that broke up the monotony, and made it possible for the "Thick As a Brick" movement to be released in and of itself as a song of sorts, that garnered a fair amount of radio airplay.
Thick as a Brick was a parody of prog rock, yet a strangely effective one since it's probably the best Tull album.
Tales from Topographic Oceans is excess even for me as prog fan, but there's a lot of redeeming factors there--it's an album before it's time IMO. It's the prototype for something like The Flower Kings, although ironically guitarist Roine Stolt of that group (and Kaipa, Swedish prog masters), recorded an album with Jon Anderson a few years ago (
Invention of Knowledge) which sounded like a lost Yes album (as opposed to everything Yes has done since the 80s barring
Fly From Here). Roine Stolt tends toward insane excess (hence why Flower Kings will never be mainstream, even if they do have a few catchy songs 90s rock radio might like), but he's a fantastic guitarist, and I did convince my father--a Yes fan in their classic era before I was born--that he was listening to a lost Yes album with Anderson/Stolt's
Invention of Knowledge.
This was wasn't the case with most long prog songs, and shortening the songs by editing to make them more radio friendly didn't work. When it was done (Yes' "Roundabout" for instance), it basically ruined the song listening-wise. So, cut back on the quantity of jam band length songs, and maybe prog has more of a chance of maintaining its visibility. Ditto for the "Lord Of The Rings"/fantasy fiction vibe so many prog albums had in the 70s (it got old listening to songs alluding to "days of the long lost magical golden age"). Scale back on these two elements, and maybe prog has a better chance of avoiding becoming just another underground music genre - one that punk, and metal musicians still occasionally deride as being bloated, overly self-indulgent, and pretentious sounding.
I disagree. Take Yes's
Going for the One from 1977. The title track, "Parallels", and ballad "Wonderous Stories" are all easily able to be singles with minimal editing, while appealing to both prog fans and general rock fans. 1980's
Drama has Buggles members replacing Jon Anderson and Rick Wakeman, yet has songs which could easily be hits despite being obvious prog rock songs.
Hmmm, maybe prog buyers watched Spinal Tap and became aware of the pretentiousness...
Agree with what's been said, one thing that's not been covered is the drugs element. To fully appreciate the musicianship you needed to spliff up or drop some acid. Maybe 'Operation Julie' cut off the supply of LSD necessary to drench yourself into a 20 minute track.
Nah, I was hypnotised the first time I heard "Close to the Edge" and "2112" in high school. Before then I liked Dream Theater's prog stuff (I didn't even know "Octavarium" was quoting Pink Floyd's "Shine On You Crazy Diamond" back then). No drugs needed.
I dont think the fantasy lyrics were the issue actually. Plenty of metal bands from the 70s on up used (and still do use) lots of "mystical" imagery (ie, Black Sabbath). It was more the length of the songs. If they could somehow write shorter songs that still were able to put what they were trying to say musically and lyrically into the song, it may have somewhat lasted longer.
Very true. I could see any NWOBHM band from Iron Maiden to a more obscure group like Cloven Hoof doing a cover of Genesis - "The Knife" (from
Trespass, it's got some straight up proto-metal riffs, blazing organs inspired by Keith Emerson which wouldn't sound out of place in Deep Purple), but in a concise form (power metal band Dark Empire did a cover of this song, speeding it up, and omitting the flute/key solo, and I can imagine a version like this becoming a hit early 80s metal single), and that being a defining version of the song.
Progressive rock and early metal drew much influence from the same 60s psychedelic rock style, and Deep Purple and first three Rainbow albums is the best example of crossover (aside from perhaps
Sabbath Bloody Sabbath which Rick Wakeman played keys on). The last Ozzy Osbourne Sabbath album,
Never Say Die, had a lot of that Deep Purple/proggy style (it's utterly inferior to Dio Sabbath, yes, but that's another matter).