Don't close it yet RBC!
The weight of evidence that you have (from what you have said) could mean you have identified your gt-grandfather. But Mr X is also correct in that it COULD be the brothers.
That's where other circumstantial evidence (i.e. paper trail) might help. For example, if any of the brothers were out of town 8-10 months before your grandfather was born that would rule them out.
For some kits I manage here are the amounts of shared DNA with various cousins. If you and this man have the same gt-grandfather you would be half-second cousins. If you have the same gtgt-grandparents you are 3rd cousins.
Person S:
MM (2nd cousin) - total 165cM, longest 40cM
BW (3C) - 120 and 40
RM (3C) - 116 and 19
MD (3C) - 86 and 23
JP (2C1R) - 88 and 41
RS (2C) - 61 and 25
HT (3C) - 55 and 12
JB (3C) - 39 and 10
FA (3C1R) - 36 and 13
CG (1/2 4C) - 36 and 14
CJ (1/2 4C1R) - 32 and 8
Person P (sibling of S):
MM (2C) - 157 and 52
RM (3C) - 149 and 26
BW (3C) - 108 and 21
RS (2C) - 82 and 35
MD (3C) - 79 and 23
JP (2C1R) - 79 and 21
HT (3C) - 68 and 16
JB (3C) - 59 and 18
FA (3C1R) - 38 and 13
CG (1/2 4C) - 32 and 12
CJ (1/2 4C1R daughter of above - must have some segments that match from her father) - 51 and 13
So, as you can see, sometime a more distant relative shares more than a closer one.