Tennis-Waiving Djokovic ban would be 'slap in the face' for Australia - ex-minister