Hi
Did you see James Richardson lodging with Elizabeth and sons in the 1881 census?
It might be worth getting one of Elizabeth's sons' birth certificates to see who was named as their father.
John Thomas BREAKINGBURY March Qtr 1875 Wimborne Vol 5a Page 281 (census says Devonshire!)
James BREAKINGBURY March Qtr 1879 Andover Vol 2c Page 231
I can't see one for Fred

Kind regards
Gaie