Has it ever been mentioned that James Brady might not be his birth name?
Even if that were so, though, it wouldn’t account for the lack of a marriage record.
Have you contacted the church in Dunmore? I see there is an email address.
If the children were baptised there, there might be information in the baptismal records.
Added
Have you considered a DNA test?