Ceasefire talks between Cambodia and Thailand were underway in Malaysia on Monday after days of deadly clashes on their disputed border sent civilians scrambling for safety and relations between the two neighbors cratering.

Thai and Cambodian leaders are meeting in Malaysia in an urgent effort to resolve deadly border clashes that entered a fifth day despite mounting international calls for peace.

"Cambodian forces used small arms and grenade launchers, prompting Thailand to respond in self-defence," Thai army spokesman Major-General Winthai Suvaree told reporters.