Hi Caitlin
I'm afraid I couldn't resist the temptation, and I just had to search a little further, but without purchasing certificates or transcripts (cheaper), it may be a little diffiuclt to work out.
You indicate your gg grandfather married twice, his first wife dying shortly after the birth of a still born child.
I found an 1871 marriage of Henry Craggs SWEETING to Margaret Annie CARNS (Ref. 1479/1871), but also with same reference and place, a marriage to Mary Josephine SWEETING
Anyway, I found a 1872 birth and death of Henry Richard SWEETING to Henry and Margaret SWEETING in 1872, and then the 1873 death of Margaret SWEETING, which ties in with part of your story.
His second marriage was in 1878 to Maria M CLAYTON, however, all of his 9 children had the mother of Emma M. Perhaps there was an error with the name Maria/Emma?
Les