Saffron Walden Castle is situated on a promontory at the junction of two streams, the Madgate Slade and the King’s Slade, in a position which would have commanded the valley westwards to the River Cam.
Only the flint core of the basement remains of the once 3-storey keep. Inside are traces of a circular staircase, a well shaft and a fireplace. Unlike some castles, this one didn’t have a motte (mound) but was a tower keep, built on the ground where the solid chalk bedrock could take the weight of the masonry. An earth mound was raised around the basement level of the keep, but it wore down over time once the castle went of use and the walls were robbed.
The layout of the surrounding streets reflects the original line of the inner bailey.Castle Street, Museum Street and Church Street on the west, whilst on the east side it would have followed the old road, a little to the east of the present Common Hill.
There is no direct evidence about who built the castle at Saffron Walden, but it is likely to have been between 1125 and 1141. Geoffrey had recently been created Earl of Essex and it is highly likely that he built the castle around that time. The first reference to it is contained in Empress Maud’s first charter in 1141 when Geoffrey de Mandeville II was given permission to move the market from the neighbouring village of Newport to his castle at Walden. Geoffrey de Mandeville changed his allegiance more than once during the civil war and in 1143 he was forced to surrender the newly built castle to King Stephen. It was restored to Geoffrey de Mandeville III in 1156. Around 1158, after the civil war, the castle was partially destroyed by order of Henry II. The castle later passed to Maud, the wife of Henry de Bohun, Earl of Essex and Hereford. On her death in 1236 it passed to her son, Humphrey, who became the 7th Earl of Essex.
In 1346, Humphrey VII de Bohun, Earl of Essex was given a licence to crenellate the castle, adding battlements to it. The de Bohuns opposed Edward III and in 1362 the castle was confiscated and endowed to the Duchy of Lancaster. It later passed into the hands of Henry IV and remained a royal manor until the reign of Henry VIII. The manor was given to Thomas Audley, the Lord Chancellor in 1538. It then passed by marriage to the Howard family.
Little is known of its later occupation, but much of the stone was removed before or during the 18th century. The turret on top of the keep was added in 1796 by Lord Howard de Walden. In 1797, it passed to Richard Aldworth Neville and remained in his family until 1979 when the ownership of the castle passed to Uttlesford District Council.
Excavations carried out in 1911-1913 confirmed the location of the castle ditch surrounding the bailey. More recent excavations, in 1973 and 1975, located the northern extent of the bailey, along Castle Street and the extent of the bailey eastwards to Castle Hill House. There have been more recent evaluations and watching briefs by archaeologists as dictated by essential works in the grounds and the recent conservation scheme for the keep, and a geophysical survey in 2012.