I have been tracing back the Grays (James, son of Thomas and Mary was my great-great-grandfather). As far as I have been able to find out, his older brother Thomas was still unmarried in his 60's. Jame's son Thomas married Rebecca; in the 1901 census they are shown as having the following children - Charles, Emily, Ernest, Annie and James.
Hope this is some help.