So, as you have eliminated the Johnny Taylor chap, have you considered any other possibles for Agnes and Mary's DAD ....
For example, have you looked at ordering any official transcriptions (around $20 each) for anyone with the apparent family name of ABORIGINAL as per the NSW BDM online index... Here's a small sample from that NSW BDM online index under their heading “Births” but actually its from their Early Church Records of baptisms:
John Chidd ABORIGINAL, son of Robert, and unknown mother. #3026/1848
John L Smith ABORIGINAL, son of Philip, and unknown mother #4230/1846
John Potter ABORIGINAL, son of John, and unknown mother #1219/1849
John T ABORIGINAL, son of Thomas and Nanncy #1864/1851
NONE with Firth as a given name.
Cheers, JM