I don't think you are clutching at straws; the marriage looks quite likely.
Even assuming the ages are correct, which is not always the case, Catherine could be as young as 70 and Elizabeth as old as 24, so Catherine need only be 46 when Elizabeth was born. The register entries are pretty conclusive but William and Catherine would be 33-37 when they married in 1804, so I suspect the ages have been exaggerated on the census.
Presumably you looked for Catherine and William's birth on the CD? Showing that there is a Catherine White bc 1770 would strengthen your case, particularly if her parents were called the same as some of her children.
In an ideal world there would be a White will mentioning an Aplin, or a White family in the 1841 census with an Aplin staying.