Hi Jenn,
I was looking on Trove using Nellie Chapman as the search because she was an artist Olly had performed with and I came across a wedding notice printed 17 November 1898 in The Advertiser in Adelaide:
Gilbert-Chapman: on 18 October as the Baptist Manse Flinders St, by the Rev Mr Raws, Olly Mee Gilbert of Angas St, to Ellen Maud Nellie Chapman the youngest daughter of Mr George Chapman, Pirie St Adelaide.
And another notice of 20 May 1905 also in the Advertiser after an adultery case had hit the news:
I am not the Nellie Chapman concerned in the "Moore" court case of May 18 - Nellie Gilbert nee Chapman, 171 Pirie St.
So it looks to me that Olly stayed in Adelaide using the surname Gilbert and there is an address for which you can search voting, land or rates records.
Good luck, Colin