Neale,
Do you have James Taylor (b. 1838) marriage cert, or the birth certs for any of his children?
I can't find any records of Thomas and Martha marriage or baptisms of their children, or other census records. But there is a James and Martha and family who fit the bill, so I'm wondering if the 1851 census for Thomas is a mistake (or perhaps he was James Thomas).
A tree on Ancestry has James Taylor (b.1838) marrying Mary Vowell, Newcastle St.Nicholas, 23 April 1860, then with your Samuel and lots other children. But his father at the marriage is James, not Thomas (also on Familysearch marriage record), witnesses Henry and Margaret Taylor.
I can't find Thomas/James or Martha in 1861, but this looks to be James (1838) brother Henry and sister Isabella.
21 Pitt St., Westgate,
Henry Taylor, 26, Morocco Leather Dresser, b. Newcastle.
Margaret Taylor, wife, 25, b. Newcastile
Henry Taylor, son, 6m, b. Newcastle,
Isabella Taylor, sister, 14, b. Newcastle.
RG9/3810/77/61
A marriage on Familysearch : 7 Sep 1856, Newcastle All Saints.
Henry Taylor, 21, and Margaret Fairs, 20. Grooms father James Taylor, Brides father Robert Fairs.
There is a marriage of a James Taylor and Martha Graham, Newcastle St. John, 3 Nov 1834.
There is a baptism of an Isabella Taylor, Newcastle St. John, parents James and Martha, 12 April 1847.
1841 census, Pilgrim St., Newcastle,
James Taylor, Ostler, 35, Y
Martha Taylor, 30, Y
James Taylor, 4, Y,
Samuel Taylor, 1, Y
A burial at Newcastle St Andrew, Samuel Taylor age 2 1/2, 13 Nov 1842.
It seems to fit, but you might have to buy a cert or two to confirm.
Alan.