I agree that the younger infantes could've taken a larger role (in theory) with the death of the prince of the Asturias, but by the time of his death, both their first cousins archduchesses are already married, one to Munich and one to Warsaw. Also, Olivares played up the rivalry between Felipe IV and his brothers and made the king believe they were after the throne. Ergo, the king basically left Carlos as being able to do nothing, and Fernando himself was hamstrung by the Conseil d'Etat in the Netherlands, that were under secret orders to refer everything to Madrid.
But as to defrocking Fernando, Habsburg cardinals had been defrocked before (Archduke Albert is an example), and if he's the only other male in the succession besides Felipe himself, the king may have no choice but to consider that (if Queen Isabel (who, much like her double-sister-in-law in France, Anne of Austria, a dedicated opponent of the prime minister, but powerless to do anything about him) lives) or to legitimize Don Juan (which Queen Isabel was against in the first place).