Scotlandspeople.
I went with Ladybird's assessment as McGhie looked quite good and I did birth searches for children between 1790 and 1854 with the surname
ferg*s*n and a parent called
m*cg*e.
There are, however, 2 sets of couples

a Robert Ferguson (various spellings) and an Ann McGhie in Dumfriesshire but there is also a Robert with a Mary McGhie same areas at the same time.
They aren't all in the same parish but within a reasonable distance - I have the same patterns of movement.
Robert & Mary
1796 John Ferguson in Terregles KKT
1798 Sarah Ferguson in Closeburn
1799 Elesabeth Fergeson in Keir
1802 John Ferguson in Tinwald
1808 John Ann Grier Fergusson (female) in Mouswald (same date as Mary)
1808 Mary Fergusson in Mouswald (same date as John Ann)
The Robert and Ann couple are in Kirkmahoe similar time period 1799-1814.
No James but given they moved so much he may have been baptised in a parish where the records are no more?
Do you have James in 1851?