TikTok parent company ByteDance has a tool that's scraping the web 25 times faster than OpenAI
TikTok parent company ByteDance is amassing huge volumes of web data way faster than the other major web crawlersByteDance may be planning to release its own LLM, and is aggressively using its web crawler, "Bytespider," to scrape up data to train its models, Fortune reported. Bytespider showed up on the scene in April, and since then, its rate of consumption puts web scrapers from OpenAI, Google, Meta, and Anthropic to shame.
SEE ALSO:
Texas adds to TikTok's legal woes over children's safety concerns
Sam Crowther, CEO of Kasada, a company that specializes in bot management, told the outlet that Bytespider's scraping rate is 25 times more than OpenAI's GPTbot and 3,000 times the rate of ClaudeBot, which is Anthropic's web crawler for its Claude LLM. Crowther also said that Kasada's data has seen "huge spikes in scraping activity" from Bytespider in the last six weeks. As Bytespider voraciously consumes the web, the U.S. government is trying to inhibit potential access of American user data to the Chinese government. In April, President Biden signed a bill forcing the ban of TikTok unless it was sold by ByteDance within the year. Given ByteDance's ticking clock for selling TikTok, the sense of urgency fits the massive rate of its web crawling activity — whether for an LLM, a better algorithm, or something else, we don't know.What ByteDance plans to do with all of its newly-mined data remains to be seen. But TikTok has launched several AI-powered features for the platform. In May, it announced a suite of tools for advertisers to create AI-generated ads, and AI-generated avatars for brands and creators. TikTok is also rumored to be working on an internal search engine, with results powered by AI — possibly using ChatGPT.
Welcome to Billionaire Club Co LLC, your gateway to a brand-new social media experience! Sign up today and dive into over 10,000 fresh daily articles and videos curated just for your enjoyment. Enjoy the ad free experience, unlimited content interactions, and get that coveted blue check verification—all for just $1 a month!
Account Frozen
Your account is frozen. You can still view content but cannot interact with it.
Please go to your settings to update your account status.
Open Profile Settings