"Since would have been born around 1820, and on a ship in 1840s with his daughter born in Western Australia about 1850, and because of slavery i’m guessing there probably weren’t records for him"
Records are poor for everyone during this time period because, before 1850, the censuses only listed heads of households by name and the sea-faring lifestyle meant that some sailors were not counted at all. Slavery is obviously an additional complication to tracing this man's origins. Presumably his parents or grandparents were slaves but he himself was a free man. There were free Black people and they did get censused. Some of them were mariners - as I recall, the cook on the Brig Pilgrim in 1834 [Richard Henry Dana, Two Years Before the Mast] was Black and had a family in Boston.
You could start by the tedious process of trawling through the 1850 census of the whaling towns of New England looking for appropriately named Black families engaged in the maritime trades. You can search the 1850 census by race.