The fortress commander (or governor) and the main officers usually had comfortable residences in accordance with  their status.

What is known today as the governor’s house is a building that seems to take advantage of the fusion of some houses from the medieval town and was used by the fortress commander as accommodation during the second half of the 17^th  century. Subsequently, the building was abandoned, and the head of the fortress moved his residence to another building, located next to the Sea Gate and of which nothing is currently preserved.

It is a quadrangular building with the main door to the east. Some of the exterior walls, especially the western one, present a slightly inclined profile (scarp) as protection against projectiles. To the west and north, other constructions were eventually added, used respectively as a stable and a warehouse for storing straw. Internally, it had a central courtyard that provided light and where the staircase was located to access the first floor. A curious quadrangular platform, which functioned as a terrace, leans against the south side of the building.