Elaine,
I think you may be referring to John Burgess (1848-1937), your grand-mothers' grand-father, who lived most of the second half of his life in Kitchener Road, Walthamstow with a second partner who he never married. He was a General Dealer and had 7 children by his second partner, after having 11 by his wife, Susan Martha Harvey. He lost an eye and lived a rather scruffy existence, but he wasn't Irish, or a Gypsy. He was also estranged from some of the children from his first marriage, including my great-grandfather, who only knew that he had left the family home and didn't realise he had set up a new one a few miles away.
The reason that the Burgess' are on here is that some of us are descended from them and have been studying the family history. There are also some who have tried to help answer some of the peculiarities and confusion in the line.
I am happy to share some of what I know, including a photo of John Burgess and his father Richard Burgess if you want to send me a Personal message with your e-mail address.
Neil