Well, after a lot of digging around, I think I've finally found the elusive William Hogg senior

He died on 16 Dec 1861 in West Linton - which these days is in the Scottish Borders, but was formerly in Peebleshire - it's practically on the border of Midlothian, only about 15 miles from Edinburgh city centre. William was 79 years old, listed as a joiner on the death registration, and widower of Catherine Bryson

He was living with his sister, Ann Ramage (nee Hogg) and her husband Alexander Ramage in West Linton - they all appear together in the 1861 census in West Linton.
William Hogg was born on 20 Sep 1782 in Penicuik, Midlothian (which is about 8 miles north of West Linton) to John Hogg - a miller, according to William's death record - and Anne Porteous. I've found two other children for John and Anne so far, both born in West Linton - John born around 1789 and Ann, already mentioned, around 1800. Ann Hogg married Alexander Ramage in West Linton in 1822 - I've found two children for them so far.
So... this means that the 1851 census entry that I found for William Hogg and his daughter Jane/Jean (and the off-spring of dubious parentage...) is correct, as that William Hogg's birth place was noted as Penicuik (albeit spelled very phonetically

) - they were in Uphall, Midlothian - still need to work out who the two grand-children actually were - not documentary trace of them as yet...
It also means that all those trees on Ancestry who have Catherine Bryson down as dying in 1884 are probably wrong - William was a widower by the time of the 1851 census.
Phew...
Added: Oh, and as a final clue that this probably is the correct William Hogg - William Hogg junior (he who married Margaret Minto Thompson) named one of his children James Porteous Hogg...