Possibly. I'm not sure of the exact wording of the rules of secession. In the case of Kennedy's assassination, all the dithering over Johnson taking the oath is moot. Because it was stated that the vice president becomes president upon the death of the former president, oath or no. So in that case the oath is simply a formality. I'm not sure how it applies to the president elect or what the wording is, or what it was back then.
The Constitution requires him to take the oath "Before he enter on the execution of his Office". ie before he performs any of his duties as President.
Afaik, Taylor did not perform any of his Presidential duties on that Sunday, so there was no need for him to take the oath that day.