Hello Donnalynn,
Has taken me a while to search as there are so many Irvings in Annan (75 in the 1801 census and 61 in the 1801 census.)
Looked at the 1801 for a Robert IRVING with a Richard. Found one but doesn't say whether he is a son or not:
1801 Annan Census BURGH SOUTH SIDE
Robert Irving, Jean Irving, Richard Irving, Elizabeth Bell, Agnes Gass (ag 2)
1821 Annan Census HALLMEADOW
Richard, Mrs Irving, William, John, David , Robert Lawson, Maid, (ag 2)
If that is the family in the 1821 Census - could Richard's father be a William ?
Looked up both the Annan (Town Hall) Old Burial Ground Memorial Inscriptions and the Annan Old Parish Church graveyard Memorial Inscriptions - but there aren't any Richard Irving burials.
Sorry I can't be of more help.
cupcake