Have you asked people to look at photos of you alongside potential "suspects"
Though my dad, brother and I were/am 6'2", they, although both 'skeletal', did not share the same facial features that my sisters can recall. I am bigger in build, looked nothing like my brother, and my sisters can neither confirm or dismiss that resemble my father.
My brother and sisters had all left home by the time I was 5yrs old and none of us are/were close.