Oh, I wouldn't go that far. A few hunting rifles with good shooters behind them would probably take down the guards. A suicide bomber with an explosive vest won't get THROUGH the gate, but he could probably get TO it and detonate.
From my limited knowledge, a squad-sized unit with small arms could almost certainly breach the fence (killing the gate guards) and run wild until the local PD showed up. Actually making the reactor core do Bad Things will be much harder, of course, but if they're willing to die... hard to stop people like that before they draw blood.
The problem for the terrorists of course, is getting a squad sized group of appropriately armed men into a location where they could carry out such an attack.
I certainly agree that if they got in they could do some damage.