I see that his father James Samuel Drury died relatively young - 11 Nov 1865, aged 43, at 13 Radnor Place, Paddington. His estate at probate was under £300 so the family was apparently not all that affluent, unless funds had been hived off into trusts.