I personally haven't found the 1861 census record for the family - where were you looking, CaroleW? I haven't spotted him in 1841-51 either.