You have to work with the DNA results. If the 2 Gt. GF's shared even one parent from your x 2 Great Great Grandparents then you and your brother would show a % of shared DNA with the other person.
So you have to work back, have you found the Great GF's living in the same household as children etc. on census.
Remember that there was no formal adoption in the UK until 1926, children changed hands if a parent died , which mothers did frequently in childbirth, social conditions could be very poor for many.
Children became known by a surname and possibly didn't know their original family surname. They didn't need or carry the amount of ID that we do today. So look at any other close DNA matches to you and your brother , try contacting them or looking at trees and areas where they were born, any census with them showing. A lot of detective work, you have to take things slowly. And I'm sure someone else would give you similar advice