There is a long death notice in the Folkestone, Hythe, Sandgate & Cheriton Herald - Saturday 20 January 1906, for a Henry Herbert who died on 11th January 1906.
It's all about his life as a mason, the masons who went to his funeral and other details of his life. It's too long for me to copy out the whole thing, but here are some of the main details.
"He was a Past Grand Standard bearer of the Province of Kent, a Past W.M. of the military jubilee lodge and at the time of his death, he held the office of treasurer of the Lodge. He was also a Steward of the Boys' and Girls' Masonic institution."
There is a brief list of condolence notices, one of which reads "From Lieut-Colonel Hamilton, his old commander in 'The Buffs'."
So I would say, if it is Herbert, his regalia could be connected to one of those offices.
EDIT: I see upthread someone had asked which lodge he was a member of. The lodge 'Military Jubilee Lodge, Royal Arch Chapter No 2,195' is specifically listed in the obit.