Levi did marry the Ann you refer to, as his first wife died in child birth.
Who was his first wife 
Eliza Sharpe - they married 1858, had a daughter Hannah Maria, but both mother and daughter died in 1860, after child birth.

Interestingly, Ann was listed in the 1861 census as working in the same place as Levi...