I think we need to look at voting in person and postal votes separately.
I don't think we know if there is much voting in person fraud - presumably it will only show up if the genuine person turns up and finds that someone else has already used their name to vote. There could be widespread fraud: a sharper fraudster could identify people who may have moved away, are apathetic, may be ill but not registered for postal votes etc., so the chances of being caught are remote, and we therefore do not know how widespread it is. Thus we do not know how much voting in person fraud there is, and I have always found it strange that some id is not required, even if it is not photo id. For example, rates bills, utility bills, bank statements, etc.
Your logic is kinda correct, but there are some good pointers to levels of in-person voting fraud being very low. Consider that it's a crime for which there is virtually no personal gain (other than to the candidate). The scenario you describe requires the fraudster to do some planning and risk (I'm guessing, a jail term, I don't know what the exact penalties are) in order to make a difference of - presumably, one or two votes at most in an election where you typically need thousands of votes to make a difference in most parliamentary constituencies, or hundreds of votes in most council areas. You'd have to be politically *extremely* motivated to want to do that.
Also, I've spent many years in the Labour Party in various constituencies. In that time I've never once encountered any hint of a culture of anyone thinking it would be OK to try to vote fraudulently, nor can I recall encountering anyone who voiced any thoughts of doing so. And of course, if we ever got any hint of members of a rival party seeking to commit fraud, the agents and legal guys within the party would be onto it and kicking up a
huge fuss straight away, yet I can't recall that ever happening. You'd think if voter fraud was widespread nationally, then a reasonable number of people in various political parties would be encountering hints of a culture of others being prepared to commit fraud - yet I'm not aware of any reports of that. That doesn't prove, but strongly suggests, that in-person voting fraud is at insignificant levels nationally.
(Obviously, events in Tower Hamlets show that there may well be some localised groups more willing to commit fraud within their own areas. But wasn't the fraud in Tower Hamlets more around postal voting? I can't recall the details).