Hugh and Mary, and their daughter Phyllis, are in the 1851, 1861 and 1871 censuses, and all of them say that Hugh was born in Troqueer.
There's no listing of Hugh in the LDS 1881 census transcription at Scotland's People.
There are four deaths between 1871 and 1881 of Hugh Smith, born 1815 plus or minus 5 years: see screenshot.
There are no baptisms of Smiths, mother's surname McDougall, Kelly or Glen in Troqueer, but there are three baptisms of children in Troqueer to William Smith and Agnes Gilchrist; John 1815, Helen 1818 and William 1820.
Would it be worth a look to see if Hugh is a brother whose baptism record has not survived?
Also note two more death of Smiths with mother's surname Gilchrist in Loudoun, the same parish where Hugh's death is registered. Might they be sisters?
And there is John Smith, died 1892 in Keir aged 76, mother Gilchrist. In 1891 John Smith, 75, born Troqueer, is in Keir with wife and family.