Hello, Sanz,
Here are the following census entries for Georg Job Elvey for the years 1871, 1881 & 1891.  I have already given you the 1851.  I could not trace him in the 1861.
1871
RG10/1300  folio 87  page 87
12, Clarence Crescent, Cleaver,  Windsor
George J, Elvey  head  age 53,, Knight & Music Doctor  b  Canterbury,  kent
Hellen N G  wife age 52    b   London
Ellen M.  dau  age 10     b  Windsor,  Berks
Chas.G.  son  age 7       b   ditto
Elizabeth  Jarvis  visitor  Unm.  age 34         b   London
Caroline  Jarvis   visitor  Unm  age 31          b     ditto
Cd. Norris  unm  age 25  Footman  Domestic   b Sudbury Derby
Eliza Gerrard  unm  age 25  Lady's maid  b Burnham, Bucks
Ann Newell  unm  age 26  Cook   b  Cuxham  Oxon
Louise Woodwood  unm age 25 Housemaid, b  Middsx Northwood
1881
RG11/1323  folio  7  page 7
12, Horseshoe Cloister Windsor Castle
Sir George Job Elvey  wid.. age 65  Music Doc.Oxon  b  Canterbury  kent
Ellen  Maud  dau  unm age 20     b   Windsor, Berks
Augusta Hogg  Unm  age 34   Cook  b   Southampton  Hants
Louisa Woodward  unm  age 34  Housemaid  b Pinner Middsx
Albertina Cloghurst unm age 19     ditto    b  Ascot,  Berks
Frederick W.Eaton  unm  age 24  Coachman   b  Chester
1891
RG12/554  folio 100 page 15
The Tower, London  Windlesham, Surrey
George J. Elvey  age 75    Dc. of Music,   b Canterbury  kent
Mary  Wife  age 44          b   Upper Clapton
George F H   son  ge 7     b   Ascot.  Berks
Wilhemina Pauly  S.  age 26  Governess, School   b Zurben, Switzerland
Agnes Surtees  S.  age 27   Nurse  b  Renhold, Berks
Karolina Sclossen  S  age 21  Cook   b  Kaisers Langern Germany
Olive Jeal  S  age 24  Housemaid   b  Oxted,  Surrey
Fanny  Jeal  S  age 24  Parlour maid     b   ditto
Ada Wheeler  S  age 16  under housemaid  b  Cowes, Isle of  Wight
John Jerrow  S  age 29  Coachman  b  White Parish  Wilts.
Marriages
George J. Elvey  =  Elizabeth Shelton
June  ¼  St George Hanouver Square   1   49
                          =  Eleanor Grace Jarvis
June ¼  1865  Windsor   2c   696
                          =  Mary Savoury
June  ¼  1882     Windsor  2c  747
Death
George J Elvey
Dec ¼  1893  age 73   Windsor  2c  259
Parents  Joh  Elvey  & Abigail  Hardiman
married  30th  Nov  1799  St Mary, Northgate, Canterbury
(John   bapt  1780  Kent
          died   Mar ¼  1844  Blean (Kent)  5  24)
baptised the following children at
Union Chapel-Watling St-Lady Huntingdons, Canterbury
James Job  Sept. 1801
John   22nd May  1803
Stephen  born  27th June 1805
               bapt  4th Aug   1805
               died   Dec ¼  1860  Oxford  3a  353
(Stephen was also a Musciian & Organist)
Abigail  31 st Dec 1809
The followng baptisms took place at
Guildhall St-Independent & Presbyterian Canterbery
Elizabeth 5th Dec 1813
George Job  14th Apr. 1816
Sarah   11th Apt.  1819
John Samuel  5th Jan  1823   
If you wish to try to trace the descendents of any of the Elvey children shown in the censuses as a way of being able trace any living relatives you will first have to obtain the marriage certifiates for them, then follow that up with their children & so on to the present day.  This can be a time consuming effort, trawling thro' the Birth, Marriage & Death indexes of the General Register Office.  There is a free site for this 
http://freebmd.rootsweb.com/ but it is not complete, you can view he complete BMDs at either 
www.ancestry.co.uk or the pay to view site 
http://www.findmypast.com/HomeServlet.
Certificates can be bought online at
http://www.gro.gov.uk/gro/content/certificates/at a cost of £7 pe certificate
I hope this has all been a help to you & good luck with your search,
Meliora