I've managed to track it back via my books, newspapers and censuses to 1813/1814 when it was advertised  for sale in the  Caledonian Mercury.   I believe it was built on the land that Sir William Douglas acquired fron the Gordons in 1790-91 - see histories of Castle Douglas.  In 1861, it was occupied by Capt James Stoddart, RN. I think he bought it  soon after 1856 when the previous owner,  
Hannay Inglis died (1856), and 
? InglisHannay before then. The Valuation Rolls of 1865 have Stoddart as Proprietor/occupier. 
I'll see if I can find any more but I think Lesley's suggestion  is probably best. There is/was also a very good local studies section in the library at Castle Douglas - I was able to find a great deal there.
Gadget 
Have names and dates on my other machine so will add them/amend if I've got this wrong  
