Experts say leaving the high-cost buildings unused for years is a huge waste of public money