Goodness, I do believe I've found Ellen and Joseph in 1861:
RG9/173 Folio 114 page 54
St Giles North, Finsbury, Middlesex
14 Sunker or Sember(?) Court
Joseph Clark, hd, mar, 38, green grocer, b. Hampshire
Ellen Clark, wife, mar, 32, b. London, Middlesex
Ellen's age gives same approx birth date as the 1881 one.
_______________________xxxx_______________________
Now I am totally and utterly confused. This is what I've found in 1871. It's the same person. His wife is the same age but different name, different place of birth and an 11 year old daughter

St Giles North, Finsbury
RG10/352 Folio 11 page 16
26 St Earl Street
Joseph Clark, hd,m, 48, Green gorcer Foreman, Hampshire, Christchurch
Margaret, wife, 42, b. Cornwall, Helston
Sophia, d, 11,scholar, b. Middlesex, Upper Holloway
_______________________xxxx_______________________
I don't see Joseph or Margaret Clark in 1881
I don't see Margaret Clark in 1861 - maybe she married Joseph afterwards 
Gadget