Hi,
The address on the 1841 for Ellen Luger/Sagar is actually Lowerhouse Fold, Habergham Eaves. Which places her nearly in Padiham

However, she is still a possible.
Robert Towler was living at Brunshaw, Burnley (civil parish of Whalley, but not the town Whalley) on the 1841. He and Ellen were also living there on the 1851 census next door to his parents.
As they married at St. John`s Cliviger I would have expected her to be living around the same area.
The 1841 images for part of Cliviger are quite faint so it`s possible she may have been missed or really badly transcribed!!

Regards,
Mo