Hi Ru, thanks for the reply. There's been a bit more progress with this one. I asked NRS to check the original death certificate and they believe it's the 25th (King's Own Scottish Borderers). Another Record Office looking just at the scan suggested the 26th (Cameronians). With the latter in particular, I've been advised they were based more at home and in the colonies so more possible for children to have been conceived at that time. It's also been suggested that, during this time period, men volunteered for the Crimea but never actually went to war, then were stood down when it was over - which also would fit with children being conceived and the 'civvie' occupation being recorded on an 1859 death cert.
I've had another look at the on-line records for these two regiments specifically (FindMyPast, TNA) but no luck. Also asked Scottish Museums for some background information, even if they don't actually hold any records.
I'll see what comes of these two regiments first, given they're both Scottish which would fit with the family's location at this time. I've had a quick look at the 24th and this was the South Wales Borderers who seem to have been out of the country a fair bit in the 1850s (fleeing from cannibals on the Andaman Islands in May 1857, apparently) - but another one to add to the list.
Thanks for the link to the Fleming page - this is a different family but will make a note for other research