I thought this was excellent too. At first I thought it was simply going to follow what has been a strong theme in this series, finding an ancestor with something in common with the celebrity. Fortunately we moved on from the baker to another strand in Mary's family.
I thought she did feel a bit dubious about the baker, especially when it was revealed what an extra- ordinarily large amount of money he made from supplying bread to the workhouse etc. I suppose she gave him the benefit of the doubt, as we all might. I wanted to know more about how he became a builder as well, it seems a strange combination but he had to put all that money to work somehow! Perhaps he was what we would call a developer now.
As for the four illegitimate children, that does suggest to me that the mother had an ongoing relationship of some kind.