No, the Border Reivers were not clans, historically speaking.
The clan system was a social feature of the Gaelic-speaking Highlands and Islands.
The modern clan system has its roots in the early 19th century. It is only since then that Lowland and Border families have reinvented themselves as clans and adopted all the tartanalia.
In the 16th and 17th centuries most Lowlanders would have regarded the Highland clans as a bunch of dangerous savages, and would have been horrified at the notion that they had any connection with them.
Things began to change in the 18th century, with the movement of people away from the Highlands to the towns, and to England and overseas, and especially with the clearances in both Highlands and Lowlands from the middle of the 18th century onwards. There was also a deliberate policy by the government to destroy the clan system, which was seen as being at the root of the various Jacobite Risings against the Hanoverian king and government.
Unfortunately the Brigadoon industry has managed, by romanticising the Highlands and the clans, to peddle successfully the idea that every Scot belongs to a clan, and that all Scots are passionately attached to their clan. This is untrue, but it tends to distort how people from other countries, and especially the descendants of those who were forced to leave Scotland in the 18th and 19th centuries, imagine that their displaced ancestors lived.