New York State did not require vital registration then, so I don't think they would have to obtain a birth certificate unless the locality required it. If they had the child baptized, there may be a church record, but it may not be easy to find if you don't know specific details.
If they married and had a child there, it doesn't sound like they were "just visiting". More likely, they moved, one thing or another didn't work out, and they returned home.
