The unexpected but historic meeting carries risks for both presidents, political experts said.
It comes at a sensitive time in Taiwan, with presidential elections being held on January 16 amid rising anti-China sentiment, particularly among younger Taiwanese who worry about Beijing's influence over the island and don't believe Taiwan benefits from closer economic ties with its giant neighbor.
Ma's office said in a statement the purpose of his trip was to "consolidate cross-strait peace and maintain the status quo". Ma would not sign any agreements, nor issue any joint statements with China, it added.
Zhang Zhijun, head of China's Taiwan Affairs Office, said the two leaders will "exchange views on promoting the peaceful development of cross-Taiwan Straits relations", according to a statement carried by the official Xinhua news agency.
He called it a milestone in relations that will help manage conflict and disputes and would gain "wide support from all walks of life across the Strait and the international community".
"This is a pragmatic arrangement made in accordance with the one-China principle under the situation where the political dispute across the Taiwan Strait has yet to be resolved," Zhang added.
Ma, who is set to step down next year due to term limits, has made improving economic links with China a key policy since he took office in 2008.
He has signed a series of landmark business and tourism deals, though there has been no progress in resolving their political differences.