Since ex-PM was released from custody he has only left his residence once, citing fear of being re-arrested