OK I'm confused now.
Hi, where did you find the baptism record? I looked on the Wolverhampton Archives website and it shows Joseph as illegitimate son of Mary, no mention of the father.

Willenhall St Giles's Baptisms 1642-1812 A-Y
SMITH Joseph son of Mary - illegitimate 14 Jun 1812
http://www.wolverhamptonhistory.org.uk/resources/indexesAdded: FreeREG also shows the same information - illegitimate son of Mary
https://www.freereg.org.uk/but the alleged reference to his death:
Joseph buried 31.1.1854 Union Street Wesleyan Methodist Chapel Willenhall aged 52 of Lane Head Engineer puts his age at 52, which means he would have been born around 1802 - some ten years before his christening mentioned above and out of sync with ages given in the the census.