Kingston, Cambs is a mile from Gt Eversden, Cambs, and 5 miles from Gt Gransden, Hunts. If you check the IGI you'll see a Caroline Ellis birth and baptism recorded at the Independent Meeting House, Gt Eversden in 1820. Also at the same church was recorded the birth and baptism of Edward Ellis in 1824, plus five other children. **
The problem is that the parents in each case were John and Sarah Ellis, not Elizabeth. Their first child baptised was in 1816, implying a marriage 1815/6.
How sure are you about Elizabeth being the wife of John? (Edit - she was, but his second wife and not the mother of his children)
In any event you need to check Gt Gransden parish register for the marriage of Caroline, to see if a parish of residence other than Gt Gransden is given, and also to see if the witnesses provide a clue.
In 1841
HO107/ 453/8 folio 11A
South St, Gt Gransden, Hunts
John Ellis 45 farmer
Sarah Ellis 45
Edward Ellis 17 ag lab
Hannah Ellis 13
All except John born in Hunts
What makes you think John was from Caxton? (Edit - just seen the 1881 census)
David
** The baptism index on the Cambs FHS site that I mentioned in my previous post has the added detail for five of the baptisms that the abode of John and Sarah was Gt Gransden. It's beginning to look as though your information about Elizabeth as the wife of John is wrong.
John Ellis married Sarah Rich in Gt Gransden in 1814 (Hunts Marriage Index)
Just found John & Sarah in Gt Gransden in 1851. Forgot that Ellis might have been enumerated as Elliss
HO107/1758 folio 99A
Gt Gransden, Hunts
John Elliss head marr 55 farmer 10 acres employing 2 men born Cambridge
Sarah Elliss wife 58 b Kingston, Cambs
Hannah Elliss dau unmarr 23 b Hunts Gransden
Mary Ann Flinders grand dau 14 scholar b Gransden Hunts
Alice Barley visitor unm 26 milliner b Yaxley Hunts
William Peck visitor unm 22 tailor b Wrestlingworth Beds
Stephen ?insby servant 12 Farm lab b Lt Gransden, Cambs
James Williamson visitor 18 unm Farm lab b Kingston Cambs
The presence of Mary Flinders, grand daughter, who must have been the Mary aged 5 with William and Caroline in 1841, I think closes the loop