Hi Sandra
Well, you won't believe this but I did finally make it to the library

Not sure how popular I am there since one of the librarians thanked me for enabling her to expand her knowledge of Stroud area information (I'd asked her a question she couldn't answer....... she was really nice......)
Anyway - I checked Rodborough Parish church records between 1829 and 1838 and no sign whatever of your Walters family. The only baptisms on Oct 9th 1836 were for Daniel and Caroline, children of Daniel and Eliza English.
I was really pleased I went there because I didn't realise they also had a set of non-conformist records copied from the Kew archives, so checked the Rodborough Tabernacle records as well. They are, or were, 'Calvinistic Methodist' and also had no Walters baptisms over the whole period I looked (about 20 years).
I looked at various other Stroud churches with no luck - you wouldn't believe how many there were in the mid-1800s

I'm not completely despairing about it - it seems the whole area around Stroud was known as Stroudwater at the time, so I could well have missed one. I'm pretty sure I've got all the non-conformists now. No idea about the Catholics (could they have been Catholic?) but I could look at some of the other Stroud C of E churches when I go back (next few days, hopefully)
Sorry not to have been more help, but I'm fairly sure your contact who said they were baptised at Rodborough was wrong. Did they mention a particular church? (Because I haven't found any Catholic baptisms and don't know where the nearest church was at the time.)
In all the churches I looked at I only found one mention of a Walters, so thought I'd mention it in case it rang any bells - at the Stroud Weslyan church on 27 June 1837, as Martha Walters, daughter of Thomas and Martha Walters was baptised. She was born on 2nd April 1811 (near your Joseph?) and Thomas was described as a farmer at Rodborough.
Best I can do at the moment. but I'm happy to have a look at other parishes in the Stroud are if you like!
Regards
Angela