That is the thing though. For much of the war they weren't veteran soldiers. They were volunteers fresh off the farm often with volunteer officers they elected themselves. These guys are the kind of fellows shock and awe could work on. And while it may seem strange elephant hide is sufficient to limit the damage from mid-19th century infantry weapons. I'll try to find the book (I think by Osprey) that mentions it.
Fresh from the farm or not they know it is only an animal and enough .50 caliber bullets will bring it down. The elephant would be shot to pieces.