I'm coming into this thread very late for which I apologise but there were a number of large houses in South Stoneham, particularly around the Portswood area. I carried out a study on the origins of the name of a local pub in the Highfield area and it caused me to discover a number of moderate to large sized houses - most of which are now gone.
The area included Highfield House, Portwood Lodge and Portswood House, Westwood House as well as South Stoneham itself. The subject of my study was a Lieutenant-Colonel William Charles Drummond who was resident at Myrtle House, sometimes called Myrtle Cottage, which is a deceptive name for the grounds include all the land lying bounded by Bowden Lane, Brickfield Road, Portswood Road and a line west of Somerset Road. The 'cottage' was on the grounds of what is now Portswood Primary School. It is unlikely that Colonel Drumond or his neighbours in the burgeoning villas at the top of Highfield would have done their own gardening.
The ideas about market gardeners are perfectly valid and are likely to be the solution to your quest, but it remains that there will have been a reasonable number of gardeners for reasonable sized houses in the area.
regards
John