Thanks for the info folks, it's great that you're sharing the headache with me.
Well there's no address given on marriage certificate but from parish records Edward's parish is given as Netherton, Dudley which is the same as for Hagar.
Don't know about the Edmund Long in Birmingham (I'll check that one out) but in the 1901 census Edward was a bit more specific as he said he was born in Queens Cross, Dudley. Although I've checked the 1851/1861 census returns (albeit on Find my Past!) and cannot find any Long family living in Queens Cross.
I think you're right and he is illegitimate but I would just like to find at least who his mother was and where he was in the 1861/1871 censuses.