I wonder if Henry George did die?
Name: Henry George Morgan
Event Type: Death
Registration Quarter: Apr-May-Jun
Registration Year: 1888
Registration District: Edmonton
County: Middlesex
Event Place: Edmonton, Middlesex, England
This is the only birth record that fits for Henry Charles:
Name: Henry Charles Morgan
Event Type: Birth Registration
Registration Quarter: Jan-Feb-Mar
Registration Year: 1889
Registration District: Edmonton
County: Middlesex
Event Place: Edmonton, Middlesex, England
So maybe Henry John and Caroline moved from Walthamstow after Henry was born, to Edmonton, where wee Henry George died and then Henry Charles was born? From memory, back in the day when I lived in London, I think those 2 places are quite near each other, so it's not too much of a stretch to imagine they moved from one spot to another.
I think this will end up being one of those things that I just never fully sort out and can only make guesses at.