I'm fairly sure Peter and Mary Ann stayed in England: although he is very vague about his age, Peter's occupation is "blacksmith" on all censuses and their birthplaces remain the same:
1871
RG10/3867/6/6 - 1 Worsley Brow Court, Sutton, St Helens, Lancs
Peter RIGBY - Head - 65 - Blacksmith - Bold, Lancs
Mary A RIGBY - Wife - 63 - Warrington, Lancs
Sarah RIGBY - Sister-in-law - Wid - 70 - Annuitant - Warrington, Lancs
Peter RIGBY - Grandson - 6 - Scholar - St Helens, Lancs
1881
RG11/3739/116/8 - 90 Sutton Rd, Sutton, St Helens, Lancs
Peter RIGBY - Head - 62 - Blacksmith - Bold, Lancs
Mary Ann RIGBY - Wife - 73 - Warrington, Lancs
Peter RIGBY - Grandson - 16 - Apprentice Boiler Maker - Sutton, Lancs
Rambler