In 1891 the entries preceding Arthur Parker in Alley Yard are the families of Matilda Wilding and William Howe, then the preceding addresses are Pump Square (in brackets for some reason). In 1881 those two families are neighbours but their address is Pump Court, then the preceding entry is again Pump Square.
Since address search at FindMyPast only finds Alley Square in 1891 it looks like perhaps it was the same as Pump Court, or adjacent to it. It's also suspicious that I can't find a single newspaper reference to Alley Yard.
Pump Court is still there today.
Added:
26 October 1955: Suffolk and Essex Free Press
A report of the compulsory purchase of dwellings at Pump Court, Lavenham.
At the inquiry, the Medical Oficer, Dr. Robert Clayton, described the properties in Pump Court as more than 150 years old with neither historic nor architectural value
Looking at Streetview, Pump Court today seems to be the backs of various houses, although it was last surveyed 15 years ago.