On the contrary, your position is completely reasonable and understandable. All of us have had to make some degree of risk assessment over the whole thing, including over taking the vaccine (I'm still in the "hold off and wait for a bit" camp, as are pretty much all the under-40s I know - but on the contrary personally I would encourage anyone over 40-50 to take it as the balance of risk probably tips the other way then).
The key is making a reasoned judgement - I wish the media would simply present the figures, not put a slant on it like "terrifying doubling of cases in a week", or whatever.
Part of the issue is that it's very apparent that many people have consistently over-estimated the level of risk posed to themselves by Covid, for example grossly under-estimating the median age of death, or the odds of ending up in hospital. For me this is a distraction, as the more people get sick of having to go along with measures which are simply there to be seen to be doing something, the less it is taken seriously for those cases where there really is a tangible risk.
This isn't to do it down, however on the contrary I know people who have ended up in hospital (and in one case sadly passing away) due to having allowed their shielding to become compromised. So for me the focus is making sound individual judgements, rather than the puerile government nonsense like "let's all do our bit", "travel with confidence", or whatever. In any event, the government don't give a toss whether you or I catch SARS-CoV-2, they're only interested in whether hospitals fill up.
If ones immune system is compromised, other illnesses such as measles (UK has not been measles-free for some years now), mumps, chickenpox, tuberculosis (surprisingly prevalent in some areas and often drug-resistant these days) and things like the "common cold" and flu or even an outbreak of usually-benign bacteria or yeast which live on our skin, in our gut etc can be seriously damaging. Tetanus can be caught from the soil too.
Measles is particularly nasty- it has an R number of 12-18 and has the additional effect of transient immune system supression. Vaccination rates were below the level required for herd immunity even before COVID so I'd guess they are lower still now with the NHS having been the NCS (National Covid Service) for over a year.
Perspective is important and over-estimating the COVID risk can blind one to other higher risks.