The 42-year-old man was returning to Vietnam from Africa, according to the office.
The horns, weighing nearly 2kg in total, were cut into small pieces and hidden in cans of milk powder, it said.
The customs office and related agencies are investigating further.
The trade in rhino horn is illegal under Vietnam's penal code.
Around 1,300 rhino were killed illegally in Africa last year, driven by demand from China and Vietnam where rhino horn is believed by to help reduce toxins in the body, reduce body heat and treat fever, and to improve one’s general health or prevent disease.
It is also rumored to help treat cancer or reduce the side effects of cancer treatment, according to Education for Nature Vietnam, an NGO for wildlife protection.
Use of rhino horn has also become somewhat of a status symbol, whereby members of the emerging wealthy class flaunt their success by using expensive "medicines," the NGO said.