This State Mosque was built in 1888 by a Yemeni Arab merchant named Habib Hasan. He has donated land for the construction of this mosque. Originally, the roof of the mosque was made of boards. This mosque has been made the State Mosque of Pahang because Kuala Lipis was the State Capital of Pahang in 1898, before it was moved to Kuantan in 1955. 

According to the stories of the elders, one of the specialties and miracles of this mosque is that it has never been submerged in water even though the city of Kuala Lipis has been hit by large floods several times. The position of the mosque on the banks of Sungai Jelai was also not affected even though floods hit in 1926 and 1971. The first imam of this mosque was called Imam Awang who came from Indonesia. While the first local child to become an imam is Imam Kawi, who is the second imam of this mosque. At that time, the salaries of the imams were paid using money collected from zakat. The first imam to receive a salary from the government was Imam Salih Bin Hussin. 

Among the dignitaries who have prayed here are Dato' Abu Samah bin Haji Ali (Pahang Government Secretary), Tun Abdul Razak bin Hussin (Pahang Chief Minister) and HM Sultan Abu Bakar Riayatuddin Al-Muadzam Shah. In 1980, the management of this mosque was taken over by the Indian Muslim community until now, at this point the mosque is known as the Indian Mosque.