Language changes with the times, it is what it is.
The history of the word is that "man" meant human beings of any gender. As in "makind" or landing on the moon and saying "one small step for man". The specific gendered term for a female was "wif". So a female human was a "wif-man" when became "woman". See also "wife" and "midwife".
The specific gendered term for a male was "wer". With the initial 'w' pronounced as a 'vee' sound (like in German) it lasts today as "virile". With the initial 'w' pronounced as in modern English like 'w as in water' it has the word 'werewolf' (a male human that can turn into a wolf).
Gendered job-titles (themselves now still in use but increasingly sounding archaic and old fashioned) don't use the word "man" to mark them, but have gendered suffixes such as Actor / Actress, Seamster / Seamstress, that come from "master" or "mister" and "mistress".
These distinctions would have made perfect sense and led to no ambiguity whatsoever to vernacular speakers of English in the 18th Century / Georgian times, and would have been obvious and understood in the 19th Century / Victorian times. Contemporaneously, as we don't tend to use the underlying words in day to day speech, we don't really understand why the words are as they are, other than that's what we were taught when younger. When words enter this phase (of just being used but with nobody knowing the reason) they tend to get dropped, or change, or subject to what is called "hyper-correction" - where they sound kind of wrong to prestige speakers so people use what they think is "right" instead.
So we say the Arctic Circle (with two 'k' sounds) because it sounded "right" to pronounce it that way in the 15th Century, even though before then it was pronounced "artic" because speakers hyper-corrected it, it came from the Latin "articus" (which itself came from Greek Arktikos, which has two 'k's) and originally meant "something to do with bears" (from the star constellation of the Great Bear which is in the north). The word "bear" in Welsh is "arth" even today.
The current prestige speakers (politicians generally - even though most of us dislike them) are moving away from gendered language. Subconsciously we hear people in a leadership or academic role using these terms, so if you don't want to sound like you're one of the people at the coal face you will also use the same types of words. At that point using the older words marks you out as unsophisticated or a dinosaur and you either don't use them at all or "code switch" and say 'signalman' down the pub, but 'signaller' at a work meeting. Of course putting it like that sounds a bit jarring and no doubt many will say "I'm not speaking in a particular way because I heard someone else do it! I am speaking how I choose to speak!", except we all do it whether we like it or not, because language it about communicating with others and if we all made our own words up nobody would understand us.
