His comments came after a national security briefing with President Donald Trump about North Korea's latest nuclear test. North Korea claimed it had successfully conducted its sixth and most powerful nuclear test, detonating a hydrogen bomb that could be loaded onto a long-range missile.
But speaking to reporters outside the White House, Defence Secretary Mattis said the US had the ability to defend itself and its allies South Korea and Japan.
"Any threat to the United States or its territories - including Guam - or our allies will be met with a massive military response, a response both effective and overwhelming."
However, he said the hope was for denuclearisation, "because we are not looking to the total annihilation of a country, namely North Korea".
North Korea has defied UN sanctions and international pressure by developing nuclear weapons and test missiles that could potentially reach the US. On Sunday, seismologists' equipment started picking up readings of an earth tremor in the area where North Korea has conducted nuclear tests before. The US Geological Survey put the tremor at 6.3 magnitude. Japanese Foreign Minister Taro Kono said there was no doubt this was North Korea's sixth nuclear test, calling it "unforgivable".
Then North Korean state media confirmed it was indeed no earthquake and claimed the country has conducted another nuclear test in spite of warnings not to. North Korean leader Kim Jong-un was pictured with what state media said was a new type of hydrogen bomb.
The latest move by North Korea has drawn international condemnation. The UN Security Council is to hold an emergency meeting on Monday to discuss an international response, according to the US mission.
Meanwhile, President Trump has warned that America may stop trading with any country that does business with the North. Denouncing the test as "hostile" and "dangerous", President Trump described the North as a "rogue nation" which had become a "great threat and embarrassment" to China - Pyongyang's main ally.
He also said South Korea's "talk of appeasement" was not working and that the secretive communist state "only understands one thing".
"The United States is considering, in addition to other options, stopping all trade with any country doing business with North Korea," Trump later said in a tweet. North Korea relies on China for about 90% of its foreign trade.
Leaders of various nations condemned the action of North Korea.
South Korean President Moon Jae-in called for the "strongest possible" response, including new UN Security Council sanctions to "completely isolate" the country.
China, meanwhile, also expressed "strong condemnation" and said the state "had ignored the international community's widespread opposition".
Russia urged all sides involved to hold talks, saying this was the only way to resolve the Korean peninsula's problems.
UK Prime Minister Theresa May said the "reckless" new test represented an "unacceptable further threat to the international community". She called on world leaders to come together to stop North Korea's "destabilising actions".