A jury has been dismissed after "misconduct" during deliberations in a trial that has shocked Australia.