As some Tories join calls for Boris Johnson to quit over partygate, his chairman says it would bring uncertainty.