If James II dies a few days after the birth of Jimmy, his mother will see him when the Regency Council decides. He will be raised Protestant and his mother and her known allies will not be allowed unsupervised visitation or private conversations with the child. If Jimmy is taken away by mum and her buds, he will be declared "dead" and any child Mary of Modena claims is her son will be deemed a pretender. Enter William and Mary. Without James II (hell, even with him), Mary of Modena is not a power-wielder. (Although I understand she was more charming than her husband.)
V asked if Anne had witnessed the birth, would it stop her from spreading rumors of a changeling (and the warming pan wasn't the only way to smuggle a changeling in). I don't think so. And: what it boils down to is the fear of a Catholic monarch; that would be the impetus for CoE lady witnesses to support Anne's story of a stillbirth and a switch. And if James II dies on top of this, Dowager Queen Mary is shit out of luck. If Anne and her cronies win the day, the child will not be allowed to leave England with Mary of Modena (she's not his mother, you see). If James II dies before the rumor gets around, I imagine the child will be safer in England, raised as a good CoE as above by the Regency Council (although Anne might be his physical guardian since she's in England) - with the above restrictions.