Warning that terrorists wanted to sabotage the Indo-Pak peace process, Pakistan on Sunday said the two countries should not allow extremists to dictate terms to them.