This looks the best match for John on the 1851 census:
HO107; Piece 2044; Folio 33; Page 4 - Village, Hinton on the Green, Gloucestershire
George GREGORY Head 31 Groom b. Bicester, Oxfordshir
Ann GREGORY Wife 35 Laundress b. (?)Bowdon, Northamptonshire
Harriet GREGORY Dau 10 Scholar b. Banbury, Oxfordshire
Mary Ann GREGORY Dau 8 Scholar b. Banbury, Oxfordshire
Emmanuel GREGORY Son 5 Scholar b. Hinton, Gloucestershire
John GREGORY Son 2 Scholar b. Hinton, Gloucestershire
Sarah GREGORY Dau 11 months b. Hinton, Gloucestershire