"Even last week, the clear divisions within the party could be seen here in Kent, as Mr Bolton and Chris Wells, the leader of Ukip's only council in the country in Thanet, publicly clashed over the future of Manston airport and the general running of the council".
Waters finished second in the leadership contest.
He promised to pressure the government into a hard Brexit.
Bolton, 54, is a former police officer and served in the army.
"The other thing is, I do not believe for a moment that we should be pushing this out and allowing the negotiations to be endless before we leave".
"Brexit is our core task", Bolton announced.
"We want to be a confident and optimistic nation and we want to be a nation that is proud to be called British". There is an issue to be discussed.
He said: 'I do not see myself now as simply being your leader, I see myself as serving this party.
Tatiana Smurova-Bolton gave birth on platform 13 of the St Pancras, a UKIP spokesperson confirmed, and the couple subsequently named their daughter Victoria.
Farage told Channel 4's Michael Crick that he voted for Bolton, describing him as the "most competent" candidate in the running.
Farage hailed Bolton's election saying on Twitter that he was "a man of real substance".
Bolton's victory was a shock to nearly all in attendance at the announcement in Torquay.
Mr Etheridge was one of a number of UKIP MEPs who had threatened to quit the party if bookies favourite Ms Waters had won.
The little-known candidate beat the two front-runners in the contest - controversial Sharia Watch director Anne Marie Waters, who is originally from Dublin, and London Assembly member Peter Whittle - to win with 3,874 votes.
Reacting to the news, UKIP MEP David Coburn tweeted: "Congrats to Henry Bolton thank God the bad @AMDWaters or ghastly @prwhittle didn't win".
There were a total of 12,915 votes cast in the leadership election, compared with the more than 300,000 secured by Jeremy Corbyn when he was re-elected as Labour leader previous year.
New leader Paul Nuttall left in June after failing to win a parliamentary seat in a general election.
It also used anti-Turkey slogans in its campaigns.