Hi robytherob,
No signs of a death for James in Scotland, but I think your family did move to England. Rose still shows as married in 1881.
Address: 126 Green St, Liverpool, EnglandRose Boyland, f, head, mar., 48, dressmaker, b. Ireland
Patrick ", m, son, mar., 29, window blind maker, b. Ireland
Mary ", f, dau., mar., 28, stay maker, b. Ireland
Francis H. ", m, grandson, 8, scholar, b. Scotland
James Joseph ", m, grandson, 5, -, b. Scotland
Patrick Ed. ", m, grandson, 2, -, b. Scotland
In 1891, Rose shows as widowed, so death between 1881 and 1891 for James. Only one between those dates for Scotland and it's not the right person.

hume24