The building was erected in 1776 with Christoffel Brand as its first occupant. Admiral Lord Nelson is presumed to be one of his first visitors after his arrival in Simons Town on 21 May 1776. Towards the end of the 18th century it was converted for use as a naval hospital. After 1814, when Simons Town became a separate district, the building served simultaneously as a court room and magistrates' residence. Today it is used as a museum.