Initially it was proposed that the Museum would be set up within the ruins of Walden Castle but it was later decided that a new building would be built on Bury Hill, between St Mary’s Church and the Castle. This land belonged to Richard Griffin, 3rd Lord Braybrooke of Audley End, who leased it to the trustees and the new museum building was erected in 1833-34. The ‘Agricultural Hall’ eastward extension was added to accommodate meetings of the Saffron Walden Agricultural Society, which Lord Braybrooke had founded; this is now the ‘Great Hall’ gallery. The Museum initially occupied the first floor of the west end, with rooms for lectures and other societies below.
In the early days the Museum was open one day a week and known for its display of large animal skeletons and taxidermy specimens. The local newspaper at the time described the Museum’s displays as “Stuffed birds and animals – shells, bird’s eggs, nests, skeletons and several bones of the mammoth – a beautiful rhinoceros, stuffed, which stands in the centre of the room, Indian curiosities, insects, casts of heads, medals, minerals, petrifactions, etc., etc., head of an elephant, of a hippopotamus, horse, cow, etc.”