I'm certainly no expert in this area. I do remember reading (and just found) this from Genuki:
"From 1851, the Registrar-General grouped together the census returns from those parts of Middlesex, Kent, and Surrey which formed the "metropolitan" area commonly called London - i.e. the built-up areas - as a separate "census county". This area was used as the basis of the area governed by the Metropolitan Board of Works and later formed with few alterations the new County of London when it was created in 1889. The areas covered include Paddington, Kensington, Fulham, Chelsea, St George Hanover Square, Westminster, Marylebone, Hampstead, Pancras, Islington, Hackney, St Giles, Strand, Holborn, London City, Shoreditch, Bethnal Green, Whitechapel, St George in the East, Stepney, Mile End Old Town, Poplar, St Saviour, Southwark, Lambeth, Wandsworth, Camberwell, Greenwich, Lewisham and Woolwich.
The rest of the county remained under the name of Middlesex."
https://www.genuki.org.uk/big/eng/MDX/censusDoesn't this imply in 1841 it was under Middlesex (or Kent or Surrey for other areas)?