I can't find the younger William on the 1861 census - but mother & father, are still in Bardwell:
Brother Edward, too. RG09/1139/61/8
CAWLEY, William Head M 54 Bricklayer b Hepworth
CAWLEY, Dinah Wife F 60 b Bardwell
CAWLEY, Edward Grandson M 4 Scholar b Bardwell
CAWLEY, Aron Grandson M 1 b Bardwell
Brother Edward, too. RG09/1139/52/15
CAWLEY, Edward Head M 31 Bricklayer b Bardwell
CAWLEY, Harriet Wife F 29 b Bardwell
CAWLEY, James Son M 4 b Bardwell
CAWLEY, William Son N 3 b Bardwell
and sister Celia/Cecilia. RG09/1139/64/13
SMITH, Sarah Widow F 54 Independent House Proprietor b Midhurst, Sussex
CAWLEY, Ceilia Servant F 21 Servant b Bardwell