The 1901 census for Eliza and Mozart shows them living at the Royal Naval Hospital East Stonehouse and surname is shown as Vining
Caroline Eliza Vining 54 b Bursledon, Hampshire,
Mozart 56 b Gloucs Bristol paymaster & clerk of the hospital
Marriages December qtr 1874
Caroline Eliza Blundell Southampton 2c 26
Mozart Vinning Southampton 2c 26
They don't appear to have had any children by 1891 - here's the 1881 and I checked the 1891
https://familysearch.org/pal:/MM9.1.1/XQ6T-52NI doubt there is a connection to your Finnings
The birthyear of 1877 for Emma Vinning is incorrect - it should be 1827. The 1901 shows her as Vining - single and b Bristol. She died in 1911 as Vining