IMHO even if Buchanan gets aggressive he's going to try to stall while as much war material as possible is evacuated from the Slave States or destroyed. This probably results in more flash points which could spur secession efforts much like Fort Sumter did in OTL.
That being said, depending on how things go down history could be decisively changed. For example, in OTL on April 17, 1861, former governor Henry Wise threatened to use his authority as former governor of Virginia to order the state's militia to seize Federal assets at Harpers Ferry and Norfolk's Gosport Naval yard
(link). Would he just stand idly by while Buchanan evacuates these assets in TTL? Maybe, maybe not. He and other secessionists could feel threatened, panic, and force a confrontation anyway. If such a confrontation occurs prior to Virginia voting to secede, it could turn the vote the other way with Virginians voting to fight against reckless lawless secessionists alongside the rest of the country. If Virginia sides with the Union early on, the other border states likely follow its lead. Something like the Corwin amendment is ratified and the nascent Confederacy is rolled up by a massive army headed up by Robert E. Lee.
If Buchanan's withdrawals fail to cause such a spark, when the Civil War does start the South will be in much worse position. Though no
CSS Virginia stands out as a possibility, the real impact IMO will be the thousands of rifles and hundreds (?) of cannon that will be denied to the Confederates. Virginia likely gets the lion's share of whatever does manage to be captured but this leaves the Western Theater even worse off than OTL.