I know that modern orthodox jews have something similar to excommunication, I've seen it applied when a husband wouldn't give his wife her get, in which case after talking with the elders and refusing he is marked as outcast, no one would buy from him, talk with him, pray with him, allow him...