Hi John,
I tried the Dorset OPC site:
http://www.dorset-opc.com/no 1851 census, but I found these Hartleys for 1861:
http://www.dorset-opc.com/1861DorchesterHTrinity-1.htm (just in case you haven't got them already)
56; 51 High West St. ; John; Hartley; Head; Married; 33; Tea Merchant employing 11 men & one boy; Scotland ; RG9/1353/F44p13
56; 51 High West St. ; Jane; Hartley; Wife; Married; 31; ; Poole; RG9/1353/F44p13
56; 51 High West St. ; William J; Hartley; Son; Unmarried; 4; Scholar; Dorchester; RG9/1353/F44p13
56; 51 High West St. ; Eaustace; Hartley; Son; ; 1; ; Dorchester; RG9/1353/F44p13
My guess is that Dad died, and son John, now married, has taken over Dad's business. The other two son's seem to have moved out of dorchester. might be worth trawling Dorset OPC for them ...
Hope this helps.