Just looked at details of 1870 newspaper article with mention of Pembrokeshire Hotel. The descriptions put the building in the Morriston area. Several other names are mentioned such as New Cross, Wharf Wall, Penwern, Morgan's place. I don't have access to the original census pages, but in my opinion 4 Llangyfelach Road and Pembrokeshire Hotel are in two different locations.
The photo Talacharn posted is next to a current public house, but in the vicinity of 2 other former public houses. The Joiners Arms nearby, and the other around the corner in Pengors Road. The one in Pengors Road used to be kept by the Gambold family. This house is still standing, but I don't think this was the Pembrokeshire Hotel! I will continue to make enquiries next week.