Hi there
James was noted as a tailor and draper also in articles in the Ipswich Journal in 1801 and 1816, so think can rule him out. I do have a number of millers in my Kerridge database, either K's themselves or spouses. Happy to share if likely to be of interest
BTW James married again later the same year as Joanna died, to a Sarah Kerridge, not sure if she was a relative or not
Andy Kerridge