This makes no sense at the moment, but since James was in Great Baddow in 1871 & 1881 as a servant to John Crabb it seems too much of a coincidence not to be connected.
There is an article in the Chelmsford Chronicle 31 Dec 1880 about the suicide at Great Baddow of David Spackman, coachman to Mrs Crabb (who I think is the wife of Richard H Crabb). A witness at the inquest was "Harriet Harris, wife of James Harris of Great Baddow"
The only James Harris I can see in Great Baddow in 1881 is 'your' James, and the only Harriet Harris is on the same census page, just a few properties away:
Harriet Harris 54 b Wyke, Dorset
Harriet Hansford 22 Niece b Great Power(?) Street, City, Middlesex
Beatrice Young 8 Visitor b Woolwich, Kent
By 1891 James & Harriet are back in London, at 73 Pulford Street, St George Hanover Square
James Harris 63 born Brentford, Middx, Coffee Tavern Keeper
Harriet 63 wife b Wyke, Dorset
Harriet 28 Niece b City of London
also three boarders and a servant
(Edited to correct spelling!)