Another early railway which doesn't get much coverage, the Surrey Iron Railway, was a 'plateway' with a 4ft 2in gauge. It was a public railway, horse drawn, my understanding is anyone could use if they paid the tolls and had the required wagons. I assume the gauge would have been suitable for wagons already in use in the area.
Like a lot of things something which started as a local undertaking spread key parts of its technology to the rest of the country, I suspect that the 4-5ft range for gauge was probably because this resulted in wagons which had sufficent capacity to be useful, but were not too heavy to be horse drawn.
In the early days 4ft 8in wasnt the standard, the Eastern Counties Railway (precursor to the Great Eastern) started with 5ft gauge until 1844. There were also a number similar but not identical gauges in Scotland, and of course the Glasgow Subway is still 4ft gauge.
Brunels broad gauge was conceived as a steam operated railway from the outset, and you could argue that the 7ft gauge was the right choice from an engineering viewpoint, but was flying in the face of what was rapidly becoming a standard. A good modern example would be the Betamax v VHS format wars. Betamax was the better system but VHS won out because of wider adoption.