The 1870 US census has him entered as - B (Black) in the 'Colour' column and age 3 so born c 1867.
There is a Major Stewart age 13 in the 1880 US census Race M, adopted child of Perry Milliam(?) 41 and his wife ? Missario(?) 48, no first name. Major is working on his adoptive father's farm. Major's father's birth place North Carolina, mother's birth place Virginia
James 11 and Eliza Scott 18, are also in the household as grandsons to Perry's wife and in the 1870 census I looked on the previous page to see whose household the 3 year old Magor may have been living with, and there is a Susane Scot 40 and a Delia Scot 15, Race Black, both farm hands