The producers released an official statement explaining how the real-life homeowners are struggling with fan visits.