I do partially feel sorry for them, in that a great deal of people are probably watching TV and simply don't want to pay. And they'll obviously refuse entry too.
It's hard to tell the ones telling the truth and those that aren't, given the fact that there are no magic detector vans. Their only tool is trying to scare people into paying, or getting someone to slip up and give them the evidence they need for a warrant.
I happen to believe in the fee, so I pay... although I still had one of these idiots on my doorstep when I moved house (but some time after I'd transferred the license and got it at the new address).
He was obviously going door to door on the assumption that a lot of people having moved in might not have a licence - but not having any paperwork to know who did or didn't. [Just as the energy companies did, all claiming/lying to own the meters and to get me to 'just sign below' that I'd confirmed they'd been - as an incredible con trick to get me to switch supplier. Yes, npower and British Gas, I'm looking at you)
I was quite rude to the guy and told him to 'go away', and he threatened me with what could happen and coming back with a warrant etc, so I said fine.
I actually thought it quite funny that he went away thinking that he'd found someone without a licence (and my faked attitude implying I needed one) but obviously sod all happened.
The fact they'll even go after people WITH a licence makes it hard to have any sympathy for them - but at the same time, I do think anyone watching a TV live does need to pay, and there needs to be some way to make them do so.
At the moment, there is no way to do that. Those without the need for a licence feel harassed, while those who need one just ignore the letters and slam the door in the face of doorstep 'agents'.
Maybe general taxation is the answer, I don't know.