A country losing territory or moving a capital doesn't turn it into another country. Similarly, simply owning Rome doesn't mean much to actually being the Roman state, especially since Rome the city had long been in decline and wasn't even the center of power in the West by the end. No one claims modern Italy is the Roman state just because it controls the city.
As for religion, language, culture, and customs, the Rome of 500 BC and the Rome of 100 AD was drastically different in nearly all those areas and more, but there's no doubt that they are both still the same government over different time periods. Every single civilization and state changes over time, especially Rome since it is the longest lasting state in human history. This is probably starting to veer away from the point of the thread, however, and would be better fit for the pre-1900 forum.