"Follow the rules – you can come. You don't follow the rules, you can't," said Australia's prime minister.