In a scenario that for some reasons (and with any PoD after the schism) the byzantine empire conquers the papal states, what they would do to the pope and to the vatican in general? Would them convert it to a orthodox church?
Why would they?
Anti-Roman Crusade as revenge for 1204?
How about Manuel Komnenos succeeding in Italy and taking the PS a decade or three later if they can convince the HRE to remain at peace?
They would simply install their own Patriarch of Rome and either exile or kill the old Pope (in case of capture) or leave him in exile or hunt him down and kill him (in case he managed to run away).
Any POD for this should occur before 1040 or so, since after that the Byzantine presence in Italy was minimal and any attempt to take Rome would only occur after the presence in southern Italy solidified, which was impossible when they had bigger fish to fry all over the place.
Could the pope create a papal state somewhere else in europe? Maybe moving to Köln
They would probably either force the Pope to submit to Orthodox doctrine and the status of the other patriarchs, or they will find someone else who would. In the latter case, the Pope is forced to retire, with very pointy emphasis.