We'll never really be able to disentangle memory from photos.
I was able to, to a certain extent, in some cases, because all our family photos of that time were in black and white, and I apparently even then had an excellent colour memory, and described accurately the colours involved even when I had no later evidence available - for example, the colour of the benches in a garden that I only ever visited before I was 2. (The woodwork of the window frames etc, was different, and the bench colour must've made such an impression on me because of that).