Include them...absolutely!! I have two or three branches that show babies that died young all with the same name. One of my lines had 3 Roberts, before one survived. I think apart from anything else, it shows the determination of some of our Ancestors for family names to be carried on.
Sooziecats, my Nan too had a stillborn son. He only survived 2 days & they didn't name him. Again, like yours, he's in an unmarked grave & I did the same as you & experienced my Nan smiling down on me.
Debbie