Did you get my PM with the different details? There was a birth for Rebecca 1833. I couldn't find a birth for William. I may have missed it, but the IGI has nothing I could see either. Have you found anything interesting? There are plenty of William Penn's on the IGI as you must have noticed but I definitely think these (on 1851) look very good.
Also, I couldn't resist having a look on familysearch:
WILLIAM PENN
Christening: 14 NOV 1784 Bayton, Worcester, England
Father: WILLIAM PENN
Mother: ELIZABETH
Batch No.: C039821 1710 - 1812
Matches perfectly the 1851 Census info !
I'd definitely go for this rather than the Walsall birth. What document do you base William's age on?
Tanja
PS.The record on familysearch is a member-submitted record. If the person says 'about', it means he hasn't found any christening so it's just his guess, obviously.
Also, they had a daughter Sarah born 1827.