It seems Dudley died in June 1779. There is a burial at St Mary, Hampton for Dudley NORTH, infant on 24 June 1779. (There are some letters after the name but I can't read them on the register.) There is this item in newspapers, though perhaps they have the number of sons incorrect:
"... The reason the sense of the House appeared against having the Honourable member's motion put, was not from any dislike to it, or that it was not thought a matter truly deserving of their most serious consideration, but merely from the absense of Lord North (occasioned by the death of his third son) ..."
June 18, 1779 - June 21, 1779
Publication: Lloyd's Evening Post
Modified to add: a couple of death notices for Dudley -
"Yesterday morning early died the youngest son of Lord North, a child a little above the age of two years."
Saturday, June 19, 1779
Publication: Morning Chronicle
"Friday died, the youngest son of Lord North, at his Lordship's House in Downing -street."
Monday, June 21, 1779
Publication: Morning Post