This is the extract from the 1871 census. There is no way you could misread this as anything other than Sussex, Newhaven.
Indeed i've seen that record, and spent many long hours trying to verify it and drawn a complete blank. Which is what led me to wonder if he was only ever told he was born in Newhaven as a child, and as an adult simply assumed it was the one in Sussex rather than the one in Edinburgh. Easy assumption to make.
Back then people often didn't know where they were born, especially if their parents died when they were young. As it happens his 2 sons both thought they were born in Cornsay (as that was where they were living when their mother died) when in fact they were born 20 miles away in Sunderland. Hence why i'm looking at other options, as part of my detective work :-)