Have you seen the recent post "Ancestry Oddity", currently showing directly below yours? You don't say what length your matches are, or over how many segments. Autosomal tests are not phased, consequently mismatches can occur on segments. With larger match lengths, mismatches, if they occur across a relatively small segment of your total match length, are relatively inconsequential. But the chances of a match being false increase below match lengths around 18 to 20 cM, and can be false in approximately half of declared matches of 6 to 8 cM across a single segment.
Ancestry also aren't telling you that the match is a 3rd to 5th cousin. They are informing you that for a genuine match of the length declared, the highest probability of the relationship between you and the match could be at that level. But there could be other relationships between you, at a lower probability. Not everyone can be in the highest probability, otherwise there would be no need to express the possible relationships in degrees of probability.
And of course, if the match is false, there is no relationship between you at all.
I would treat any matches within that range as unproven unless you have corroborating evidence, such as having identified or verified the exact relationship between you by more traditional genealogical research, othe proven shared matches, etc