Re Riding House Lane.
It is still in existence, and now called Riding House Street (is in Westminster)
In the 1851 census in Marylebone, number 29 is on piece 1486 folio 418 page 45
There are 7 households, but there are no Edwards of course.
On the previous page are the unmarried Sarah Wood at 27, and Thomas Lawrence at 28 (as per directory entries)
Unfortunately we can't see the rate books featuring Edward Edwards, and no number is given in the transcript. Also they don't seem to have the preceding and following years to 1850 and 1851.
I haven't come across any more exams for Jane in nearby places. I was surprised that she was in the St Pancras rough exam book for 1861-2, since she went into the workhouse there several times in 1860, firstly in August (when Edward disappeared) and had been there since October. The rough exam book covering 1860 seems to be lost.
However, since it appears to be a desertion case, Jane might be in the Pancras sworn exams for 1860, as they would probably have wanted a magistrate to issue a warrant for Edward. They are arranged chronologically, I don't think they are indexed (and I don't have access to them today anyway!)