The Ministry of Finance has said in its recent proposal that the government needs to stop guaranteeing such loans for new projects starting next year and increase its overseeing of loans assigned to existing ones.
The goal is to reduce these government-backed loans to 15.6% of public debt by the end of 2020, it said.
Figures showed SOEs borrowed nearly US$21 billion with the government's guarantee as of the end of last year, or 17.6 percent of public debt.
In 2011-15, the government guaranteed US$15.6 billion in loans for 35 projects of SOEs, a three-fold increase from 2007-10, according to the finance ministry.
Electricity of Vietnam, oil giant PetroVietnam and national carrier Vietnam Airlines were among top borrowers backed by the government, it said.
In another move, the ministry also sought to reduce government-guaranteed bonds issued by the Vietnam Development Bank and the Vietnam Bank for Social Policies over the next four years.
The growth of the banks' outstanding bonds, estimated at around VND161.47 trillion at the end of last year, must be slowed down to 4-6% a year from 10% at the moment, according to the ministry.
It is "a big risk" when the banks issue more bonds with a maturity of less than five years and then provide loans to projects with terms of seven to 10 years, it said.
Latest figures released by the government in May showed Vietnam's public debt was equivalent to 62.2% of GDP.
It will rise to 63.8% at the end of this year, and then 64.7% in 2018, or slightly lower than the threshold of 65%, according to the World Bank's projections.