Could be children from previous relationships - or one or both might not be. Off the top of my head, I'd suggest in the first place that you find baptisms for them, if possible, and check bastardy bonds if they're available for that period.
Other people may have different suggestions, but I think if the above doesn't obviously decide the matter (sometimes a baptism of a child out of wedlock can include the father's name, for example) then I'm not sure you'll really ever be able to satisfactorily decide the paternity issue.
In my own case, I've an obituary, marriage and birth certificate that says one thing, and a will, surname and baptism that says/suggests another, and I'm unable to decide paternity categorically. (Fortunately, I literally know where the bodies are buried and am tempted to explore exhumation and DNA tests should I win the lottery!!)