Cardiff Airport actually fares quite well in comparison to rail links from South Wales to other UK Airports.
Cardiff Airport from West - Train to Bridgend, change to Rhoose, 10 min shuttle bus (1h22 from Swansea)
Cardiff Airport from East - Train to Cardiff Central, change at Rhoose for 10 min shuttle bus. (40 mins from Cardiff Central)
Bristol - Train to Temple meads, which requires a change at Cardiff Central if coming from West, bus link to BRS (Approx 25-30 min drive) (2hr44 from Swansea)
Birmingham - Train to Cardiff Central, change to XC to New St, Change at New St for international, monorail to terminal. (3hr30 from Swansea)
London Heathrow - Train to Reading, change to Elizabeth line and change at Hayes. Or Train to Paddington and change to Heathrow Express. (Approx 3hr30 from Swansea)
London Gatwick - Train to reading, change to Gatwick train. (Approx 4hr10 from Swansea)
Manchester - Train to Wilmslow, change to Airport. Or Train to Piccadilly, change to Airport. (Approx 4hr50 from Swansea)
Most of the above Airports don't have direct train links from mainline stations. Also look at Luton that has just had the light rail link built from it's main station, and passenger numbers far exceed anything Cardiff Airport can only dream of. There's absolutely no reason for a rail spur for Cardiff Airport.
What's needed is a 2tph service serving Rhoose. The benefits of that would also service Rhoose and Llantwit Major, as well as the Airport.
It would be good if the service improvements can be made to better serve West Wales with a direct link.
What I would like to see is a Cheltenham - Swansea/Carmarthen service. It would provide a direct link from Chepstow area/Newport as well as West Wales to Rhoose. The issue would be squeezing them in between Cardiff Central and Barry.