Not being allowed to have clarinet lessons at grammar school. My primary school music teacher had suggested I take it up when I started. There was a form that your parents were supposed to send back before your first term started, but my mum didn't realise that and when I handed the form in at the start of term, it was too late. The music teacher said I'd have to go on the waiting list for the next year. So the next year I asked about it and was told that because other teachers had told her I was so rubbish in several other subjects - mainly maths and history (history?), then they couldn't allow me to take time out of regular lessons for extra music. My parents later paid for me to have organ lessons outside school and I actually played the school organ a couple of times (I was the only pupil who could - the music teacher was gobsmacked!).
Nevertheless, because I didn't actually have my lessons through school, I didn't meet the pre-requisite for taking O'level music, so I missed out on that too.
I've made up for it, though. Been playing clarinet and saxophone for around twenty years now, lead alto sax with my concert band, have a Diploma in Music from the Open University and recently got my grade 5 singing - and yes, I did eventually take that O'level (well, GCSE) along the way, played my clarinet for the practical part and got m'self an 'A' - missing out on A* by just two marks.

Oh, and I've also picked up a GCSE in Maths plus an honours degree in History. Failure, eh? Well yah boo sucks to you, teach!

I still wonder, though, whether I could have been anything more if I could have started earlier.