Jennalyn, I think Williams marriage to Caroline may have been his second as here are the census details from 1841 and 1851.
1841
HO107/790/3 - Norwich St. Michael at Thorn
Folio 47, Page 16
Thorn Lane, Middle Square
FREEMAN
William - 25 - cabinet maker
Susan - 25
William - 4
John - 2
James - 5 months
=================
1851
FREEMAN
William - 30 - cabinet maker - born Norwich St. Edmunds
Caroline - 25 - born Norwich St. Lawrence
William - 13
John - 11
James - 10
Frederic - 5
Caroline - 2
Walter - 1 month
all the children born in Norwich=Thorn
HO107/1814 - Norwich St. Michael at Thorn
Folio 224, Page 32
Topers Square
==============
In 1841, Caroline is working as a servant in Norwich, so not with any of her family.