The incident — which sparked an evacuation of nearby streets — comes six weeks after the Christchurch terror attacks by a suspected white supremacist, which killed 50 people in shootings at two mosques.
Police located the package on Tuesday evening at an address in a central suburb, Canterbury District Commander Superintendent John Price said in a statement. An army bomb disposal team made the package safe.
The 33-year-old man was arrested and is talking to police, Price said.
According to an earlier statement, a no fly zone was in place over the area, and streets were evacuated as a precaution.
New Zealand is currently enacting a ban on military-style semi-automatic weapons following the Christchurch terrorist attacks. The new laws were voted in almost unanimously by the New Zealand parliament in April.
Gun owners have six months to hand in their weapons, from the time the buyback scheme comes into effect.