Found this family, but don't know if they are the right ones as you quoted Ipswich.
1851 census index (names only)
HO 107/1805 - 242 - Lowestoft
JEFFREY
John G. aged 10months
Arthur E. aged 2yrs
Jane G. aged 4yrs
Charles J. aged 6yrs
Edward G. aged 9yrs
Edward, aged 32yrs
Jane G. aged 36yrs
all born Lowestoft, except Edward junior in Yarmouth, Norfolk Edward senior in Martham Norfolk and Jane senior in Collon??, probably means Corton nr, Lowestoft.
from
Sue