To MargP many thanks, I somehow missed it when I was writing details up.
I was contacted an hour ago by a family elder who has heard of my research efforts and carried out some many years ago. This information might help with the Joseph Allard search.
Sometime possibly in the 1870s Joseph left Lydia and lived with Emma Collins, they had a son called Thomas Joseph Allard Collins. They are on the 1871 census and my relation believes Joseph used the name Hallard, he also gives his place of birth as Knowle.
Joseph's son Henry (1842) lived with Mary Anne Orme (possibly not married) their son was named Harry Allard(1873). They had another child Annie who died young.
My relation also believes that Lydia lived in Vincent Place and that the address might have come under Kings Norton. I dont understand the address 2 Handley Place, Kings Norton which she is also shown as living at.