We want him gone, but we don't want to go to court...?

You want him gone but you do not want to do anything that will result in him leaving. Then just keep wishing. Or realize that you do not control the world. Tell the person who runs the show about why you are uncomfortable.