I may have this wrong - Is the only evidence re Charles at the moment the 1880 census and the headstone?
1880 has his birth, New York but in other censuses for his children, they have Virginia as their father's birthplace.
I do realise re errors, lies and ignorance though.
Additionally, Charles, son of P &B in New York is also in 1880 census - hasta's post.
Heywood