Although the census says that Robert wasn't born in Nottinghamshire, it might be worth looking at Robert Hough who was born just over the border from Lincolnshire in Coddington, Nottinghamshire, in1801.
He had a brother, Thomas, who married and had children in North Clifton, although he had moved to Whiston, Yorkshire by 1841. He was still there in 1851, only a few miles from Catcliffe, where Ann was living.
Regards,
David