🔙 Throwback Thursday for 2025-06-12
🤖 + 🕸️ One story caught my eye this week was the growth in AI crawlers that are gathering instant results for users of various alternatives to web search. Some coverage uses the “bot” name as an inside baseball style reference to robots.txt.
https://www.techmeme.com/250612/p13#a250612p13
As for “bots”, if you look back at coverage of early web crawlers, there are some familiar patterns. 20 years ago, coverage of Google Base was mixed but provided a roadmap for where the increasingly machine readable web was heading.
https://googleblog.blogspot.com/2005/11/first-base.html
