Hi Bryant,
There's a Daniel & Martha Penn in the 1851 census in Neithrop, Banbury but I don't think it is the same Daniel?. Census Ref HO107; Piece: 1734; Folio: 513; Page: 37; GSU roll: 193644.
Daniel Penn head age 40 plush weaver born Banbury
Martha Penn wife age 38 born Adderbury
Thomas Penn son age 10 born Neithrop
Henry Penn son age 8 born Neithrop
Mary Penn daughter age 7 born Neithrop
Emma Penn daughter age 3 born Neithrop
Frederick Penn son age 1 born Neithrop.
Your Edward age 16, can be found still in St Aldates in census ref HO107; Piece: 1728; Folio: 148; Page: 51; GSU roll: 193637.
Edward (Edwin?) Penn servant age 16 servant born St Aldates.
Sarah, Joseph, Eliza can be found Church Street, Oxford St Ebbes at census ref HO107; Piece: 1728; Folio: 344; Page: 46; GSU roll: 193637.
Henry Penn head age 25 cook confectioner born Hinksey, Berks
Sarah Penn sister age 19 born Oxford
Joseph Penn brother age 12 born Oxford
Eliza (Elizabeth?) Penn sister age 11 born Oxford
Esther Penn sister age 10 born Oxford
Mary Ann Penn sister age 7 born Oxford
This one supports your statement that Daniel & Maria have died and it seems the children are living with their older brother.
Hope this helps
Carol
