I have just looked at all the parish records we have from the relevant period and can find no possible connection between your James an any of our Harveys.
As you know I am trying to create a 'family tree' of Tharston and, when I've completed the current family under research, the Bensleys, I'll have a look at the Harveys and let you know if I find anything interesting.
Just a quick thought - there are several villages around here with very similar names, are you sure it is Tharston as there are Starston, Thuston, Stuston to name but three, and I have often found people have confused these both in transcriptions and on original documents.
I hope you have a good trip when you visit us and, should the Tharston contact be fruitful, we would be pleased to meet you in our village.
Regards. Nigel