Hi Neal, in your post you implied that you found James and Elizabeth in 1851 and 1861 in Montgomeryshire. There is a family that may fit the bill but the birth places don't fit. Here they are, firstly in 1851.......
HO107/2498/f190/p4 Churchstoke Parish, Carden Marsh, all surnames Ellis except where shown.
James, h, m, 39, lead miner, SAL Pontesbury
Elizabeth, w, m, 36, MGY Hyssington
Ann, d, 11, SAL Chirbury
Elizabeth, d, 10, MGY Churchstoke
Isaac, s, 7, ditto
Edwin, s, 6, ditto
Hannah, 1, ditto
Martha Lloyd, lodger, w, 25, pauper, MGY Hyssington
Ann, lodger, 2, SAL Pontesbury
1861 RG9 4253/f99/p29 Churchstoke all surnames Ellis
James, h, 48, lead miner, SAL Chirbury
Elizabeth, w, 46, MGY Hyssington
Ann, d, u, 21, dairymaid, SALChirbury
Isaak, s, u, 17, MGY Churchstoke
Hanna, d, ditto
Jacob, s, 9, ditto
Martha, d, 4, ditto
Probably not your family but worth a try, Dave