Emily is living with her parents in 1881 and her father in 1891.
There is a possible candidate for Joseph on the 1891 living in Castleford, whom I can't see on the 1901, but I agree with Valda, Emily and Josephs marriage details would provide the necessary info.
I think the Emily you found Silas is a very likely candidate, as you say Wetherby, is close to Boston Spa and of course the family were living there in 1891.
Regards, Kath