I don't have William's birth certificate to be absolutely sure, but in 1881 he was staying with Rebecca's sister Hannah where he was described as "nephew", so it looks pretty likely. Rebecca and her sister must have both been rather naughty girls, each having an illegitimate son! I haven't found any more children for either of them after their marriages.
I have Edward Holberry as dying in 1904; I haven't been able to find Rebecca in the 1911 census, and I haven't tracked down a death or remarriage for her, either. So what happened to her I really don't know!