This looks a possibility for James's mother Elizabeth in the 1841 Census, everyone showing as born out of the county:
Elizabeth Baxter 47
Thomas Baxter 20, miner
Peter Baxter 18, engineer
Isabella Baxter 13
Margret Baxter 8
James Bailie 20, lab. b. England
Address: Village Of Newarthill, Bothwell Holytown