The clashes in Tuz Khurmato, 175 km (110 miles) north of Baghdad, marked the latest violence in the town since Islamic State militants were driven back in 2014 by Kurdish peshmerga and Shi'ite militia, nominal allies against the Sunni militants.
Mayor Shalal Abdul said that under the deal, local police would take control of Tuz Khurmato - home to Kurds, Shi'ite Turkmen and Sunni Arabs.
A Kurdish official in the town, Kareem Shkur, said the peshmerga and Shi'ite militias would pull out once the police forces achieved a balance between the town's various ethnic and sectarian groups, estimating that would take around one month.
In the meantime, Tuz Khurmato will be secured by a unit from each force coordinated through a joint operations room.
Previous agreements have broken down and residents of Tuz Khurmato were skeptical the deal would be implemented.
Fighting began several days ago after members of a Shi'ite militia threw a grenade into the house of a Kurdish leader. A ceasefire was declared on April 24, but sporadic mortar and gunfire continued until April 27.
Tensions in towns like Tuz Khurmatu risk further fragmenting Iraq, a major OPEC oil exporter, as it struggles to contain Islamic State, the gravest security threat since a US-led invasion toppled Saddam Hussein in 2003.
Efforts to push back the Sunni insurgents have been complicated by sectarian and ethnic rivalries, including a contest for territory which the Shi'ite-led government in Baghdad claims but the Kurds want as part of their autonomous region in the north of the country.