Punitive actions will be taken against people who have hoarded US dollars illegally, the Bangladesh Bank said today.