There appear to be two specific sets of documents at TNA which might have details of your man, depending on how long he remained at the War Office (which was then located at Horses Guards):
WO 25/3957 - Headquarters Staff, and correspondence on the re-organization of Branches. 1803-1860
WO 25/3959 - Lists of Employees and Annual Expense Accounts. 1809-1828
If either or both of these prove fruitful there are later staff records in WO 25/3960 - Nominal Rolls by Branches, Military and Civilian. 1811-1862
The records have not been digitized so a visit to TNA, or employment of a researcher, would be required.
It is possible that he retained his commission and was on half pay while employed at Horse Guards.
There's a brief history of the War Office here:
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/49053/history_of_mod.pdf