Hi Keith
I've determined that Rose Hill Cottage is a sizable property, now flats, in London Road near to a solicitors. This link has it grade II listed from 1970 when it was already converted to its current use:
http://britishlistedbuildings.co.uk/en-233778-rose-hill-flats-1-2-and-3-10-wellingboroThere is nothing on the building to show that it is / was called Rose Hill but the 1899 OS map has it clearly marked as that.
You can probably get as much as you need from Google Maps / Street View but I've put a few of their images together which might help to give some context in addition to the ones I've already attached so if it's any help PM me I'll mail them to you.
Unfortunately there is a possible alternative as I notice that the 1851 Census also shows a William Dulley (born Kimbolton, Hunts [or Kimbolton, Hungary if you believe Ancestry's transcription

]), brewer and employer, in Sheep Street - next door to an Innkeeper / farmer. This has to be the Golden Lion which was originally built as a farmhouse and converted to the pub that it still is in the C19th. This puts William in "Rose Cottage" (also a listed building and currently home to a firm of solicitors). Assuming that your aged David Dulley had retired from the brewing business and that William had taken over: the original Castle Lane entrance to Dulley's Brewery / Baths (which is now the unused back entrance to the Museum - see picture) neatly bisects the 150 or so yards between "Rose Hill" and "Rose Cottage".
The London Road address looks pretty certain though as the alley down the side of that building terminates in a gated wall which has a "Rose Hill" sign attached together with a no parking sign pointing to the flat entrance to the rear (ahem.. I took a photo of that as well but I haven't figured out my new camera and the shot out of the dark alley whited the sign out so it looked completely blank - I'll try again if it's of any interest).
Finally, I tried to figure out where in the High Street David may have been living, but the disappearance of all the yards referred to in the census together with the enumerator's refusal to provide any useful information like pub names (of which there were plenty) means I can't even be sure which side of the road I should be looking at.
Hope this helps
Ian