On the 1851 census at familysearch.org there is this family living at Pentyrch, Glamorganshire:
John Phillips, 65, married, quarryman, head, b 1786
Deborah Phillips, wife, 57, b Ystrad
Howell Phillips, son, 37, b 1814 Ystrad, unmarried ag lab
William Phillips, son, 20, b 1831 Ystrad, unmarried coal miner
Phillip Phillips, son, 17 b 1834 Lantwit Vardre, Glamorganshire, unmarried coal miner
Deborah Phillips, daughter, 14, b 1837 Lantwit Vardre
Does that sound like your man?