Armies were generally recruited for one campaigning season at a time, since everyone had responsibilities back home. The economy just wasn't strong enough to support a full-time professional army. Even the nobility and gentry, who largely functioned as a military caste, also served as the civilian officials, local governments, and police. And commoners needed to go home in time for the harvest or their families would starve.
Fixed fortifications had several advantages in this environment. They bought you time to call up and assemble your army if an invasion caught you by surprise. They often even enabled you (the defender) to win without a relief force, if the attacker couldn't storm your castle or starve you out before they need to go home.
And as an attacker, having fortifications back home meant you could get away with calling up more of your theoretical forces and keeping more of your army active between campaigning seasons, since it meant fewer nobles and knights were needed back home to maintain order, and building or occupying fortifications in conquered territories meant you didn't need as much of a permenant force to occupy the conquered territories.
That said, Charlemagne did order most of the castles in his empire destroyed, so rebellious subordinants couldn't use them to protect themselves from him. This worked as far as it went, but with the unfortunant consequence that when the Viking raids started, there was no good way to defend against them. Charlemagne's successor states got right to work building the castles again, and of course, the Vikings who stuck around to carve out their own territories immediatelys started building castles of their own, to hold down revolts and defend against French/German/Etc counterattacks.