Ages are little bit inconsistent for both James and Edward between the two census returns .. but it looks as though James was born before the start of civil records (1864) but younger brother Edwards ages of 30 and 47, puts his year of birth between 1864 and 1871, so he may have a civil birth record.
There is one Edward Rossiter listed on the BMD Index in Co. Wexford during this time :
Name: Edward Rossiter
Registration district: Wexford
Event type: Birth
Year: 1864
Volume: 9 / Page : 1005
and also an extracted civil record for this birth :
Name: Edward Rossiter
Birth date: 24 May 1864
Birthplace (District): Crossabeg, Wexford
Reference : 1005
Parents : Charles Rossiter & Bess Dempsy
So if the father listed on the marriage cert for Edward and Elizabeth is Charles then you could order this birth cert for Edward to establish exactly where your Rossiter family lived at the time... and possibly where James was baptized.
Shane