A more conventional answer than that of Olga would note that visitors should respect signs which state whether a college is open or closed (or ask if it is not clear)!
Off the beaten track, one might have a coffee at Indigo's hanging out with the students, admire the Mathematical bridge from Silver Street (or walk across if Queens' is open), visit Charles Darwin's room at Christs', Pepys Library at Magdalene and Clare gardens if the weather is good