There is but one badge on his hat, and most deffo a RA one...The one that Kyt has posted is somewhat squashed to a "normal" one...
As for being mounted within the RA there is nothing wrong with that at all!

The British Army in WW1 relied most heavily on horses, and ammunition, food fodder, in fact everything!
It wasn't just the RHA that had horses, and somebody had to drive them!!!