Well I think I've got it. Browsing around the 1911 census scans and what free searches I can do for the 1921 on findmypast, I've found these address written variously as e.g. "4 Chesterfield Cottages, North Row, Cromer" and "No 12 North Row, Chesterfield, Cromer" and "15 North Row, West Street, Cromer"
So I bought the 1921 record for 19, North Row, Cromer. Which I believe is Chesterfield Cottages. Living there are widow Elizabeth Smith and son Frederick and lodger George Neale. No connection to Edward Slipper Gaff or the Gaff family that I know of, but maybe one can be found.
I'm 98% sure that the man pictured in my OP is Edward Slipper Gaff but if I can find a link to this address I'll know for sure.