This could possibly be James on the 1841 -
Thomas Boothman 45
Betty Boothman 40
William Boothman 17
Ellen Boothman 18
Priscilla Boothman 20
Thomas Boothman 13
Elizabeth Boothman 11
James Boothman 9
HO107 Piece 540 Book 11 Folio 45 Page 6
There is a marriage of a Thomas Boothman to a Betty Nutter (wonder if this should be Nuttall) in 1816. Perhaps James took his mother's maiden name as his middle name later in his life.
Marriage at St. Bartholomew, Colne.
Baptisms at St. Bart's:
1820 Priscilla; 1822 Ellen. Abode Hey Head, Little Marsden, Chapelry of Colne. Occupation weaver.
James put Marsden for birthplace on 1871 census.
Someone has put a post on Ramsbottom Heritage Society FB page about James Nuttall Boothman. It concerns a photograph, taken in 1889 of him + his former colleagues from Nuttall Mill. Contemporary newspaper article has been reproduced. Poster asked if a copy of the photo exists.