Just found one more snippet on a Google book that is talking about his son James Hamilton Browne and references Birnie Brown
He was, I believe, the eldest son of Mr. Birnie Brown of Morpeth (a reputed son of one of the Dukes of Hamilton — born at Dalserf, Lanarkshire, and died at Morpeth, 10 July, 1825, aged sixty) by his marriage with Catherine Grace.
This is from the book Notes & Queries Oxford Journals 1907.
Sort of confirms Dalserf as well but @1765