How many bots are on the internet? Way more then you may have ever expected. Everything from search engines crawling websites to bot armies on social networks.
It all started about a month ago. I was working on a PHP script to give a live view of users that are on a website. Because it was PHP based it would work even if javascript was blocked. But as I was testing it on a website I started to notice some oddities. I was testing on a domain that should have very little traffic and I was seeing way more traffic from my script then I anticipated. Even large spikes at times.
So I went to work expanding my script to give me more details about the users instead of just the count of who was there. That’s when I noticed a good bit of the user agents were reporting they were bots. I have to admit I did not think so many bots would be crawling the website but here they were. This got me curious so I started to do some research and what I found was surprising.
Website Traffic from Bots
I started with research on website traffic and found a report from 2016 that said on average 51% of website hits come from bots. Of course, this can vary depending on the type and size of your website but that is way more than I ever expected. So I wanted to see what the bot traffic looked like on one of my low traffic domains. I changed my script around so I could record human hits vs bot hits and let it run for about a week. I found that 57% of the hits were bots that I could detect.
Of course, I would rather have real people looking at my sites so 57% is a lot higher then I like. Some of these bots are search engines. I seen lots of agent strings like AlphaBot, Baiduspider, DotBot, Googlebot, GrapeshotCrawler, SeznamBot, Qwant, YandexBot, and many more. But others do not give you much information about what they are for. Such as all the user agents I have seen that just say something like “Python-urllib”
I learned that a lot of the bots that do not announce they are bots could be hacking tools or other malicious software probing sites for exploits. Even doing things like comment spamming all the way to faking view counts on youtube and other sites.
Bots on Twitter
Looking at website bots got me thinking about other types of internet bots. So I decided to look at Twitter. Research says as much as 15 percent of the active Twitter accounts are bots. I have also seen numbers that say between 20 million and 48 million accounts are bots. Based on my own experience I would say that number is much higher. The reason I say that is because I know many people with real accounts that have some kind of automatic system hooked to them to do and post various things while they are away. A simple google search for “Twitter Auto Tools” shows a whole list of options to do this. And as far as I am concerned that counts as a bot too.
Some bots are innocent and are just trying to be helpful. But there are many more posing as real people and it can be hard to tell if one is a bot or not. Then these real looking bots are used to show fake support or outrage on any number of topics. Bot followers are also sold to people to make them look more popular or important. Another google search for “buying bot followers on Twitter” shows lots of options but I rather not link to them here.
But maybe the worse thing of all is when a bot steals a real person’s identity and uses that identity for nefarious purposes.
Bots on Facebook and other social media
I stopped using Facebook years ago and I do not use anything else other than Twitter. But while researching twitter bots I saw it mentioned a few times that Facebook has even more bots then Twitter. So I did some searching and it did not take long to see a claim that Facebook has more than 270 million fake or clone accounts. Facebook is not the only one with problems. All social networks seem to have similar bot issues.
One thing to watch out for is what information you put on social media. Internet privacy can be an issue because bots are there to scoop up any data they can find on you in order to target ads or do whatever else their masters can think of to make a profit off your data.
Game Bots
Another type of bot I have seen is a game bot. A program designed to automatically play a game in order to level up your character or to gather items. They are almost always for online games because it is easier to just hack memory than to write a bot for an offline game.
Most online games do not welcome bots. So it is hard to get a count of the number of game bots that are out there. Not too long ago, a ban wave hit bot users in the pokemon go game. There are also countless examples of FPS and MMO accounts being banned for using bots. But people keep making and using them anyway.
In closing
I am sure there are lots of other types of bots out there I did not mention but one thing is for sure. Bot usage is only growing and I do not see any signs of it stopping. With further developments in AI technology maybe we will see a day were bots outnumber real people on the internet. At that point, we may not even know we are interacting with bots anymore. But there are also companies like Kasada that are fighting back. Who will win? I guess time will tell.
