Hi Juliette,
Have found William for you in Gravesend on the 1851 census:
Ref HO107 1608a folio 39 page 12
39 Bath Street, Gravesend (which is very close to Nelson Place)
William Fitness head wid age 43 greengrocer born Wrotham, Kent
Susannah dau un age 22 housekeeper born Wrotham Kent
Charles son un age 15 greengrocer born Wrotham Kent
Emily dau un age 13 born Wrotham Kent
Elizabeth dau un age 9 born Peckham, Surrey
(no sign of Frederic at this address)
xx Carol.