The castle was built in the 11th and 12th centuries, when a Benedictine community decided to surround the small settlement of the area with walls. The real defensive structure itself, however, arose thanks to the German Baron Sancho, who came to Italy following the Emperor Conrad II (990-1039). In July 1155 Frederick Barbarossa was in the Castle and in 1213 it belonged to the Duke Diopoldo, who donated the property to Spoleto in exchange for aid against neighboring Trevi, so it became a fief of the noble Spoleto Sansi.

Later, for a short period, the castle passed to the noble Trinci family of Foligno, and after a few months Pope Martin V assigned it to the governor of Perugia. In the middle of the fifteenth century it had about 240 inhabitants.  

In later centuries it became the property of several important historical personalities, and each of them contributed to the strengthening of the defensive castle. In 1571 the headquarters of the papal post were established in Pissignano. Unfortunately, in June of 1799 the castle was ravaged by French troops, who sacked the entire community, taking away everything was inside the church of San Sebastiano.

Since 1860 it has been part of the municipality of Campello sul Clitunno, and during the Fascist period was site of a concentration camp. Recently it has been restored and turned into a residential center, hosting international conferences.