I spent several years working for a local authority, three of which were working as part of a team focused on developing the management plan for a local AONB. I've never worked for an AONB directly but did develop a bit of knowledge which might help to answer some of your questions.
Area of Outstanding Natural Beauty - what on earth does that mean?
It is a legal designation recognising the defined area as having "outstanding natural beauty". Does what it says on the tin! With this legal designation comes protection against development which might threaten said natural beauty, but note that this does not prevent development happening at all (which is also the case with National Parks).
National Parks make sense to me,
AONBs are essentially the same as National Parks, but they do not have a separate national park authority which covers planning, economic development and so on. Rather it is up to each local authority in the AONB area to provide these functions. Sometimes there's a separate trust set up to manage the AONB, but they have no statutory powers and compared to National Park Authorities have very limited capacity.
however AONB to me seems like a vanity project for all homeowners in that particular area.
That seems a bit unfair. AONBs are not designated by homeowners in the local area at all and any influence homeowners are likely to have on the designation is likely to be extremely minimal. Homeowners may be consulted on a designation, the views of other stakeholders is likely to take precedence: e.g. local businesses, farmers, environmental experts, etc. Homeowners certainly do not have the final say, and an AONB designation would certainly not be given just to please local homeowners.
What does AONB mean and how does an area get such an accolade?
Already covered what it means above, but in terms of how such an accolade is given (well the short version anyway). A government body known as Natural England (loosely attached to Defra) will make the proposal that an area should receive the designation. This proposal will usually be based on sound evidence and consultation with a range of relevant stakeholders. The final decision is then taken by the Secretary of State of Defra.
What makes the Chilterns a more beautiful area than say the valley of the Great Ouse? or the Severn Valley?
Nothing as far as I can tell. I would caution against thinking National Parks and AONBs are some sort of 'ranking' system. They're legal designations which convey various protections. They're not a statement that an area is more "beautiful" than another (although the name is rather misleading!).
Why are there gaps between the "naturally beautiful" areas? Are the bits in between just terrible?
Not at all. There could be a range of reasons why the are not designated. Often AONBs receive their designation not only because they are naturally beautiful, but because the landscape or environment might be unique so deemed worthy of a higher level of protection to ensure the uniqueness is not lost. Perhaps some areas are not at threat from development (which by the way does not just mean the building of houses, but could also mean over-farming, misuse of natural resources, etc), so it's not necessary to go through the long (and often expensive) legal process of designation when it's not required. Perhaps an area is already well protected by the local authority looking after it. You could always write to Natural England and suggest an area they've overlooked
Look at this map -
https://www.google.co.uk/maps/@53.2965252,-2.1652172,7z
This is a genuine question, seeing as beauty is in the eye of the holder, how on earth has this been decided? Whether you're in London commuter belt?
Yes, it is subjective. But the need to consult on a designation and its boundaries does ensure as many views as possible are taken into consideration. Indeed, the boundaries are regularly reviewed and sometimes amended to reflect changing circumstances (as they are with National Parks too). As mentioned above the decision is taken by the government so I guess Whitehall falls into the London commuter belt category! But in reality the areas are not just plucked out of thin area and designated because some minister likes the look of it.
North-west to south-west London looks as if peoples houses are protected from urban sprawl, yet when a railway needs to be built through it, they seem to think that they live in an area where no construction can take place as a result of the areas designation.
As I mentioned above there is no guarantee against development if an area is designated as an AONB or a National Park. Indeed there is plenty of development going (much more than in many non-AONB areas up and down the country). The designation just means the criteria is stricter (the criteria themselves usually based on a locally agreed development and management plan), and it is up to the local planning authorities to determine whether or not proposals for development meet said criteria.