The 1851 census disk from the Latter Day Saints covers the counties of Devon, Norfolk and Warwickshire only. It has one Anthony 'Gillipsy' born circa 1808 in Ireland on census night 1851 in Birmingham.
Another index that has full coverage for England in 1851 also gives you
Anthony Gillaspie born circa 1829 in Carlisle in Cumberland in 1851 and
Antony 'Guslaspy' born circa 1843 in Derby and in Derby in 1851.
There are 8 Anthonys in Birmingham in 1851 born Birmingham or Ireland, born within two years of a circa 1843 birthdate - none outside Birmingham with a Birmingham birth. None of them have the surname Gillespsie but in future years for various reasons (but the most likeliest would always be illegitimacy) anyone of them may have chosen to use the surname Gillespsie which might account for the difficulty in finding Anthony, or simply that though he gave Birmingham as his birthplace he arrived there after the 1851 census (or is very difficult to spot on the census, or was not enumerated) and was a merchant seaman by 1861. If he wasn't actually born in Birmingham the most likely place for his birth may very well be Ireland.
There is no Anthony Gillespsie on The National Archives electronic database for Royal Navy service 1873 - 1923.
Since the censuses do not help in the search, the main piece of evidence which may move the search on, remains Anthony's marriage certificate. Without it anything else remains mere conjecture.
Regards
Valda