Hi - I hope you didn't get too excited when you saw you had a reply - I havn't solved it I'm afraid, but I looked some things up this afternoon and thought I would run by you what I found. I found William in the 1861, 71 and 81 census as you said and what looks like his death registration in 1886.
Then I looked for Ann Breary/Brearly, I found an Ann Breary in the right sort of area in every census up to 1881 and her death registration in 1887 she was born abt 1820 in Spofforth, Yorkshire but seems to have moved down with her family to the St Neots area quite early, according to most of the census she never married - it is a very uncommon name and I could not see any others in the right area at that time.
I found Lucy Franklin with William in the census's and although she is calling herself Lucy Villiers in 1881 I can find no evidence that she & William ever married, the children's births seem to be registerd as Franklin and I found what looks like her death registered as Franklin in 1887 - even though the children are calling themselves Villiers in later census.
I can find no evidence of anyone born in Burton-on-Trent 1810 - 1830 with the surname Villiers in any of the census, which makes me wonder if it was not William's original name.
I did find in 1851 in Brixworth Northants a William Wallers aged 31 born Burton-on-Trent - profession Sadler with his wife Mary (aged 26 born Buxton Derbyshire) lodgeing with George Gammage, and I can find no evidence of this couple in later years - but this is guesswork (they could have emigrated or something).
Sorry I havn't found him for you but I thought I would tell you what I had done so that you would know what has been checked

Debbie