The problem from Hitler's perspective is whether or not invading Spain first to get at Portugal is acceptable. That's going to tie down 20+ German divisions indefinitely, hand valuable island bases to the British and inflict a real blow to relations with neutrals and even other fascist regimes(such as Italy).
It may be that a Portuguese brigade or so and British bases in the Azores may have to be swallowed.
Then again, Hitler is not always known for his calm, detatched, rational decision-making process. It also bears mentioning that Portugal and Spain would both have useful naval bases for the Battle of the Atlantic, and if Gibraltar can be taken Britain's position in the Mediterranean is going to be more troubled than OTL.