Same here. My father's maternal grandfather was an only child, who married an only child(!). Doesn't make for many matches, especially when we suspect that his father may also have been an only child! We've been looking for his (my 2g grandfather's) birth and parents for 50 years.
I've had 4 cousins contact me. One took a DNA test, and found she was adopted. The other three vanished into the woodwork when I mentioned "DNA test". So no help there.
I have a few matches who have the fairly unusual surname in their trees, nearly all with roots in one village in Norfolk. And one friend, who looked startlingly like my father, turned up in my matches last year. He matches two others, and all three are descended from one illegitimate lady in Hackney. They match none of my other matches at all. But that is all I have after several years waiting.
I have dozens of matches in my mother's tree. One couple have 35 matches descended from them! But my father's tree is much poorer. What with only children, and illegitimate children, it's been much more challenging.