These were the only Wilson Summers in Fraserburgh appearing in the 1881 census. Hope it helps!
Dwelling: 8 Castle Street
Census Place: Fraserburgh, Aberdeen, Scotland
Source: FHL Film 0203456 GRO Ref Volume 196 EnumDist 3 Page 9
Marr Age Sex Birthplace
Wilson SUMMERS M 51 M Rathen, Aberdeen, Scotlan
Rel: Head
Occ: Grocer
Elizabeth SUMMERS M 49 F Rathen, Aberdeen, Scotland
Rel: Wife
James SUMMERS U 19 M Fraserburgh, Aberdeen, Scotland
Rel: Son
Occ: Grocer
William SUMMERS U 17 M Fraserburgh, Aberdeen, Scotland
Rel: Son
Occ: Cabinet Maker
Andrew SUMMERS U 15 M Fraserburgh, Aberdeen, Scotland
Rel: Son
Occ: Flesher App.
Martha SUMMERS U 12 F Fraserburgh, Aberdeen, Scotland
Rel: Daur
Occ: Scholar
John SUMMERS 10 M Fraserburgh, Aberdeen, Scotland
Rel: Son
Occ: Scholar
Joseph SUMMERS 7 M Fraserburgh, Aberdeen, Scotland
Rel: Son
Occ: Scholar
David SUMMERS 5 M Fraserburgh, Aberdeen, Scotland
Rel: Son
Occ: Scholar
Alexr. SUMMERS 3 M Fraserburgh, Aberdeen, Scotland
Rel: Son
Occ: Scholar
………………………………............................................
Dwelling: 9 North Street
Census Place: Fraserburgh, Aberdeen, Scotland
Source: FHL Film 0203456 GRO Ref Volume 196 EnumDist 3 Page 22
Marr Age Sex Birthplace
Wilson SUMMERS M 26 M Fraserburgh, Aberdeen, Scotland
Rel: Head
Occ: Cabinetmaker
Jane SUMMERS M 24 F Crimond, Aberdeen, Scotland
Rel: Wife
Wilson SUMMERS 1 M Fraserburgh, Aberdeen, Scotland
Rel: Son
Mary A. SUMMERS > 1 F Fraserburgh, Aberdeen, Scotland
Rel: Daur