There are two separate offences under military law: desertion and being absent without leave (AWOL). The former occurs when a soldier absconds with the intention of never returning to his unit. Absence is a short term thing, for instance due to some family or domestic situation or drunkenness etc, with the soldier often returning to his Regiment voluntarily.
Desertion was viewed in two ways: ordinary desertion would usually be punished by imprisonment, loss of pay and seniority, and reduction in rank if appropriate. But desertion in the face of the enemy (ie during active operations) was much more serious and had overtones of cowardice and letting down one's mates, and frequently lead to capital punishment by firing squad. This was particularly so during WW1 when it was deemed necessary to maintain discipline in an army largely made up of civilians. Obviously for James to be missing for a period of years, that would have been classed as desertion.
During the period 1801 - 1805 the 25th Regiment of Foot were in briefly in Gibraltar then England, so this wasn't desertion in the face of the enemy.
FindMyPast only has one reference to a James Whitaker (or Whittaker) serving in the 25th at that time, and it is taken from
WO25/949 at TNA. This series of records has not been digitized, but the entry has been transcribed as James Whitaker having enlisted in the 1st Battalion 25th Regiment of Foot on 14 Apr 1801. If he was in the band, the chances are that he enlisted as a boy soldier and as such he would have treated much more leniently if he deserted before reaching the age pf 18.
Where did you find the references to his service?