The Medici have had several family members who became popes: Leo X (1513-1521), Clement VII (1523-1534), Pius IV (1559-1565) and Leo XI (April 1-April 27, 1605). There are also a few other major italian families that have had at least one family member who became Pope: I remember seeing a Visconti and a Farnese on the list.
Regarding the possibility of a Cardinal related to a regning house becoming Pope in the 17th or 18th century, I must say it would probably only work if said Cardinal is Italian by birth: before the election of John Paul II in 1978, the last non-italian pope was Adrian VI (born as Adriaan Floriszoon Boeyens), who was pope from January 9, 1522 to September 14, 1523. And even then, Adrian VI is only one of the three popes that weren't Italian after 1378 (when Gregory XI, the last French pope and the one who returned to Rome, died): the other two were the two Borgia popes (the Borgia being Spanish), Calixtus III (1455-1458) and Alexander VI (1492-1503). And even before the Great Western Schism, a grand majority of Popes were Italian.
That's all I can say though as I'm not that knowledgeable on the Papacy and on Papal elections and politics.
Only Leone X and Clemente VII were male line Florentine Medici, the other two were Milanese Medici de Marignano (or somesuch (Pius), and Leone XI was Medici of Ottoaiano (female-line grand-nephew of Leone X). Paul III was the founder of the house of Farnese, while Ascanio Sforza represented the Sforza interests at the Vatican.
As to foreigners - they seemed to come rather close in the runnings for pope (however, Italian was preferred it would seem). Two foreign candidates in the Renaissance were Georges d'Amboise and Tamás Bakócz, Archbishop of Esztergom (Gran), a Hungarian [one author has even gone so far as to call him the "Hungarian Wolsey"]. And of course, Thomas Wolsey aspired to the papal tiara.
Another one that was pushed forward by the French was an Italian himself - Ippolito d'Este. But after the counter-reformation IDK of any foreign candidates who stood a chance of (possibly) winning, let alone quasi-royal candidates. Although, the duc de Vendome (father of the great general) renounced his titles for a cardinal's hat, and Louis XIV's (middle) bastard son by La Montespan - the comte de Vexin - was created abbé of St-Germain probably en route to a career in the clergy. Likewise Philippe II d'Orléans pushed his one bastard son into the clergy, and another into the knights of Malta .
Other royals who became cardinals included George of Hesse-Darmstadt, Joseph of Hesse-Darmstadt, both Rinaldo d'Estes, there was a Farnese cardinal in every generation, as well as for the Medici, Archduke Albrecht (who ruled with Isabella in the Netherlands) was archbishop of Toledo and cardinal, another royal archbishop of Toledo was the Infante Luis (youngest son of D. Felipe V); and a cascade of cardinals came from the Wittelsbach family in Bavaria, usually getting the Archbishopric-Electorate of Cologne, the Prince-Bishopric of Liège, too. Younger sons from the Hapsburgs were named cardinal - Archduke Leopold Wilhelm, Sigmund Franz and Maximilian Franz, as well as a Hapsburg-d'Este, who became archbishop of Gran. All three of King Pedro II of Portugal's children of Pahlava were given ecclesiastical positions (ICR if cardinal was amongst them). And of course, Henry Benedict Stuart was carrying the torch for the Jacobites - while another Stuart - a Fitzjames - also served as a French cardinal.
Then there's all the Rohans (who liked to believe they were kings/royalty) who amassed various ecclesiastical benefices and became court cardinals (I think). And at one point, I seem to remember four/five of the electoral votes in the empire belonging to Wittelsbachs, Franz Ludwig of Neuburg was Elector of Trier, Josef Clemens of Bavaria was Elector of Cologne, the obvious Electors of Bavaria and the Palatinate and I seem to imagine there was another Neuburg/Wittelsbach at Mainz too. Although - AFAIK some pope had a problem appointing Clemens August/Johann Theodor of Bavaria to some post (IDK if it was cardinal or not) because of his scandalous behavior.
So, IMHO, the potential for a royal pope exists - it is simply unlikely that the other powers will take too well to it.