A man is no longer welcome in Wilby Road, Lakenham following "seven years of hell", according to those who lived next to him.