I've had a hunt around for John Jones, it's not an uncommon name, to say the least, and I can't identify one for sure as the son of James & Martha. If that is the correct family, you'd expect there to be more children in between, but no obvious baptisms. Parish records for the area have coverage on FreeReg & DustyDocs.
I did find a possible burial for Martha
Martha Jones 80 buried in the parish church of Llanfihangel Pontymoel 5 April 1863, abode Albion Terrace, Pontypool.
There's also a possible older sister for Elizabeth, Sarah bp 25 Jan 1807 in Panteg. There had been another daughter to the couple also called Sarah bp 9 Jan 1806, but no obvious burial.