It would likely take a pretty biug POD, unless the primacy is supposed to be mostly ceremonial. The command structure of the church emerghed more or less in keeping with the Empire's administrative system. Jerusalem really didn't play a major role in that structure for the good reason that the city was comparatively small, had a bad history and a contexted present. Yes, the bishopric of Jerusalem would always become an important one on the strength of its apostolic foundation, its control of major pilgrimage centers and its length of tradition, but in an era when decisions on dogma and church policy were made in officialdom and around the emperor, it is hard to see how it can keep up with the likes of Antioch or Alexandria, let alone Constantinople.
That said, itz is entirely plausible for the See of Jerusalem (The Throne of James?) to claim that supremacy. Rome, too, developed the habit of doing so, on much less convincing grounds, and Rome had the historical good luck of being able to make it stick regionally. Perhaps if you hae Jerusalem raised to a Patriarchate along with Antioch and Alexandria at Nicaea rather than later? I can't quite see Jerusalem pulling its own papacy, but a more important role for the Patriarch is certainly possible.