Intriguing indeed.
G F Black's The Surnames of Scotland says that it is of local origin from old lands of the name in Angus, and the eariest reference to it dates from 1656 in Brechin.
Using the 'quick Search' on Scotland's People for D*ld*rg produces 63 baptisms, 21 banns and 7 burials, all of which are in Angus except two of the banns, one of which is in Marykirk and the other in Colinton. There are no post-1855 records and no census records.
Five of the baptisms are one family, in Forfar between 1827 and 1835, but there are no others later than 1800. The last marriage is in 1852 - one of the Forfar family - and there are three other banns and one burial after 1800.
The 19C Forfar family is in the 1841 census as Dargie (two transcriptions agree) and the widowed wife and one daughter are still there in 1851. The daughter's banns are the very last ever record of the surname.
So at least one lot of Dildargs can be accounted for by a change of name to Dargie, but it doesn't explain why the name seems to have become all but extinct by 1800.