On the other hand, it does make a certain amount of strategic sense. If I were about to be attacked, I would generally prefer a thousand men at my back than a thousand men scattered in ten or twenty locations; or worse, just a hundred men at my command. Agricultural civilizations can generally support more troops, and quantity has a quality all its own.
And we have to remember that this was not an immediate transition. what we are talking about is one group making a more or less gradual transition to an agrarian lifestyle, and growing as a whole as a result. Then they wind up flexing their muscles by expanding and most likely defeating their hunter-gatherer neighbors, who may as a result take up their opponent's techniques.
Finally, consider the actual likelyhood of a city getting sacked. Yes, smaller settlements would have almost always been exposed to risk of attack. But that also applies to isolated hunter-gatherer groups as well. And the more a city grows, the less vulnerable it is to tribal forces. Yes, rome was sacked in 410, and before that as well. But in between the first and second sacks is an 800 year gap, during which Rome grew immensly. But it was never sacked for centuries, because no nomadic groups threatened them. By 410, the city was in decline, and was sacked by the goths while it was a weakened shadow of its former self. As cities develop, the gap between them and their tribal neighbors grows, as the cities can raise larger, more sophisticated armies with better equipment. And finally, it is interesting to note that often after sort of sack that you describe, the nomads who did so set themselves up on top of the social pyramid in the conquered lands, and the people were never forced from their agrarian lifestyle.