I don't think that this will be of much interest to you guys but the 1871 entry for Four Ashes Inn, 36 Stafford Street has a family headed by William and Frances Matthew and sister in law Martha Oaks.
I'll post the whole think if any one wants it RG10-2937/31
I tried looking for the other streets mentioned at the start of that census section (Charles Street, Carp Street, Chicken Lane) and could not find them so I wonder if it has been redeveloped? Still, if you can get a map ypu should be able to find no 36. The White Hart was at 41 if that helps and Brittanica at 51.