Built on a promontory, defended on three sides by formidable deep ravines, the crumbling ruins of the castle cover an area of 16 000m². Held by the Cathar heretic Ramon (Raymond) de Termes, the castle only fell to Simon de Montfort after a siege lasting four months, from August to November 1210, the hardest siege of the first period of the Albigensian Crusade. Following an exceptionally dry summer and autumn, the empty water tanks led Raymond to offer surrender. However, as the crusaders advanced to possess the castle, they were met with a hail of arrows. A heavy storm overnight had replenished the cisterns and the defenders were able to hold out a little longer. Later, weakened from dysentery, and exposed to the fire of numbers of siege weapons, the garrison attempted unsuccessfully to creep out at night. The alarm was raised, the fugitives caught and killed, and Raymond surrendered the castle. After de Montfort's death, Raymond regained possession of the castle but was soon forced to give it up again, this time to the King of France.
Rebuilt in the 13th century as a royal garrison, the castle was one of the "sons of Carcassonne" (five castles defending the border with Aragon and later Spain). When the border moved further south in the 17th century, the castle lost its function. It was taken over by a band of brigands who used it as a base from which to terrorise and pillage the surrounding country. To stop this, it was demolished by royal decree - a master mason from Limoux spent 1653 and 1654 blowing up the walls with gunpowder and reducing them to piles of rubble. The bill for this work exists today: 14,922 livres and 10 sous.
Steps to preserve the site were taken in the 20th century. It has been classified as a monument historique since 1942. Since 1989, it has been the property of the commune of Termes. It is a 15 to 20 minutes walk from the village and open to visitors.