To clarify, I don't think the Tories are ACTUALLY neo-Nazis (well, maybe one or two could be, just like maybe one or two of Labour could be described as "Communists" or whatever)
So why compare the Tories with neo-Nazis? Can you not see how inflammatory that kind of language is? Consider for example, that, in another part of Europe, a certain autocratic leader has used the exact same 'Nazi' smear as an excuse to invade a neighbouring sovereign country, an invasion which includes raping, torturing and abusing civilians, which from all reports, some of the invaders seem to feel is OK because their victims supposedly are or have supported 'Nazis'.
Using inflammatory language has consequences: Consider for example, Angela Raynor's describing the Tories as 'Scum' last year. Was it entirely a coincidence that about a week or so later, Iain Duncan Smith was actually assaulted by a group of people yelling 'Tory Scum'? (To be fair, Angela Raynor did subsequently apologise for her remarks, but only a few months later and after a Tory MP was murdered).
- it was the way in which dissent was stifled and led to the departure of the centrist wing as Boris was pulled to the right by having to appease the Brexit party so they wouldn't stand against him in elections and split the right-wing vote.
The 'dissent' being 'stifled' was that some Conservative MPs voted against the Government on a 3-line whip, and therefore had the whip withdrawn. Personally I think it was harsh not to restore the whip to allow them to stand as Conservatives at the next election, but that is how the whip system works. Indeed, Keir Starmer has done much the same thing: He's withdrawn the whip from Jeremy Corbyn, so that - as things stand - Corbyn won't be able to stand as a Labour candidate at the next election. Starmer also recently threatened to do the same thing to some other Labour MPs who signed a letter part-blaming NATO for the Ukraine invasion - until those MPs backed down and withdrew their signatures (And this was over a matter where - unlike the Tory 'dissenters', those MPs hadn't even voted against their party). But I'm going to hazard a guess that you're not intending to compare Keir Starmer to a 'neo-Nazi' for doing much the same as Boris Johnson did in 2019?
The reality is of course that there is absolutely no comparison between on the one hand, withdrawing the whip from an MP who votes against you, and on the other hand, promoting the kind of race-hate-fuelled genocide and militarism that characterizes actual Nazis. It's an utterly ludicrous hyperbole to bring the 'Nazi' word into that kind of comparison. I think that, particularly given that you say you're an elected representative, you ought to consider withdrawing that comparison.