The side chapels, erected on the orders of the castle's most important families in the 16th and 17th centuries, feature a wealth of frescoes and paintings on canvas.  Among these, the Santissimo Sacramento, known as the 'Great Chapel', deserves particular attention; the panel depicting The Glory of Christ (1528-30) by Rosso Fiorentino was formerly hung here, and is now kept in the nearby Cathedral Museum.  Among the many works preserved inside the church are the paintings by Bernardino Gagliardi, the frescoes by Tommaso Conca, Saint Paul's fall from a Horse, painted by Niccolò Circignani, known as Il Pomarancio, as well as a carved and inlaid wooden choir stall from 1540.
Under the Cathedral is the Lower church, subdivided into two naves by thick pillars that support cross vaults. The relics of Saints Florido and Amanzio are kept here. According to tradition, the cathedral was founded in the 6th century by Saint Florido, although the original structure dates back to the 11th century.

Over the centuries the church has undergone many important restorations:   it was enlarged in 1356; between 1466 and 1529 it was completely renovated following plans by Elia di Bartolomeo, his son Tommaso and Piero di Lombardia, who adapted the church to the new architectural forms of the Renaissance.  Lastly, in 1632, following designs by Francesco Lazzari, the facade was rebuilt, although the upper section remained incomplete.