Hi,
Although the visitor in 1851 is transcribed as Thomas PAMLER age:38, birthplace: Hankerton, looking at the image I think he could be Thomas PAINTER.
In 1861 I found this family,
1861 RG9/402 f.108 p.28 Greenwich St Alphege, Kent
11 George St
Thomas PAYNTER Head M 48 Labourer Sewer Ankerton Wiltshire
Ann Wife M 40 Gloucester Minty
William Son 14 Carter Wilts Ankerton
Elizabeth Daur 9 Scholar " " " "
f.109 p.29
Margaret Ann Daur 6 Scholar Kent Greenwich
Thomas Son 4 " " " " Lewisham
In 1851 Thomas isn't with his wife and family, but this is a bit of an odd family

1851 HO107/1835 f.83 p.19 Hankerton, Wiltshire
Wm HOPKINS Head Widr 71 At Home Wootton Bassett Wilts
Ann PAINTER Daur-in-law U 29 Ag Lab Minety Wilts
Frederick YOUNG Grandson 12 " " Hankerton Wilts
William PAINTER " " 4 At home " " " "
Jane " " " daur 2 " " " " " "
Going back to a birth for Joseph BROWN, I see in 1851 he is Joseph Thomas BROWN, so I wondered what you'd think of this one,
Thomas Painter BROWN Mar Q 1851 Cricklade v.8 p.344
maybe she changed his name to Joseph after Thomas PAINTER left

Hope this helps.
Daisy