|
|
The AI web now has a bouncer. |
Cloudflare is giving site owners sharper controls over which bots can search, act like agents, or train on their pages. That sounds technical until you remember the modern internet bargain: publishers need traffic, AI companies need content, and nobody agrees on who owes whom dinner. |
Search bots can still be invited in. Training bots may get stopped at the door. Agent bots will need a better reason than “my user sent me.” Somewhere, a robots.txt file just became a velvet rope. |
Here’s what happened in AI today: |
😼 Cloudflare drew new lines for AI crawlers 📰 Amazon designed AI chips for future devices 📰 Zoom bought Common Room for sales agents 🍪 Z.ai launched a free coding tool 😹 AI job posts entered time-travel mode
|
… and a whole lot more that you can read about here. |
Hey: Want to reach 700,000+ AI-hungry readers? Advertise with us! |
P.S: We just launched a robotics newsletter! Sign up for it here. |
|
|
😼 Cloudflare Drew A New Line For AI Crawlers |
The old web bargain was easy to understand: search engines crawled your site, then sent readers back. |
AI broke that bargain. A bot can now read a page, summarize it somewhere else, train a model on it, or act on behalf of a user without sending much traffic back. Cloudflare’s new answer is to split those behaviors apart. |
Here's what happened: |
Cloudflare introduced new AI traffic controls for all customers, including Free users. The controls separate bots into three big uses: Search, Agent, and Training. Starting September 15, 2026, new domains joining Cloudflare will block Agent and Training bots by default on ad-supported pages, while Search stays allowed. Multi-purpose crawlers that combine Search and Training can be blocked under the stricter rule if a site owner blocks Training.
|
How to try it: If you run a site on Cloudflare, go to the Security settings and review AI bot traffic controls before September 15. Decide whether you want search indexing, agent visits, model training, or some mix. The important part is separating discoverability from data extraction. |
Why this matters: This turns “AI crawling” from one blurry category into a permissions system. Search helps people find you. Agent bots may help a person complete a task. Training bots absorb your content into a model. Those are different deals. |
That matters for anyone who publishes online, from media companies to solo creators to SaaS businesses with public docs. If your content drives attention, leads, or ad revenue, AI access now has to be managed like a business decision. |
Our take: Cloudflare is trying to make the web’s new negotiation visible. The counterargument is obvious: stricter defaults could make smaller sites harder for AI tools to reference. But the bigger shift is overdue. AI companies have treated the open web like a pantry. Site owners are starting to label the shelves. |
Hampton took $440K in planned hires off the calendar |
|
|
Hampton co-founder Joe Speiser had three roles budgeted: a data engineer, an ops manager, a PM. $440K. He installed Viktor on April 12. Forty-four days later, none are on the calendar, and 18 of his team work with Viktor daily. His VP: we are editors now, not creators. |
Get Started for Free |
|
|
🎓 AI Skill of the Day: Audit Your Site's AI Crawler Rules |
If you publish anything online, your AI crawler policy is now part of your content strategy. |
Cloudflare's new categories make the audit simple: Search bots help people find your site, Agent bots visit on behalf of a user, and Training bots use your pages to improve models. Treat those as separate decisions. |
Use this checklist for your website, docs, blog, or knowledge base. The goal is not “block everything.” The goal is to decide what kind of machine traffic actually helps your business. |
Audit my AI crawler policy for this site:
[URL]
Business model:
[ads / subscriptions / lead gen / ecommerce / documentation / personal brand]
Create a crawler access plan with:
1. What Search bots should be allowed to do.
2. What Agent bots should be allowed to do.
3. What Training bots should be allowed to do.
4. Pages that should have stricter rules.
5. Pages where discoverability matters more than protection.
6. The risks of blocking too much.
7. The risks of allowing too much.
|
Want more tips like this? Check out our AI Skill of the Day Digest for July. |
Have a specific skill you want to learn? Request it here. |
|
|
|
|
*Pave by QuickBase turns spreadsheet workflows into usable business apps. We tested it by building The Neuron CRM. Watch here. NOX puts your messages from iMessage, WhatsApp, Slack, email, and more into one Mac inbox, then drafts replies in your voice, free to try, Pro $30/mo. ZCode gives you a free GLM-5.2-powered coding tool positioned against Cursor, Claude Code, and GitHub Copilot, with no pricing details listed. Eve gives developers a filesystem-first framework for building durable agents from folders for instructions, tools, skills, channels, and schedules, free on GitHub. Kimi Code gives developers a coding agent and CLI powered by Kimi K2.7 Code, including autonomous goal execution through a /goal workflow, with no pricing details listed. claude-real-video lets Claude or another model process video through scene-aware frames plus transcript extraction from URLs or local files, free on GitHub. deptrust checks package versions for known vulnerabilities across npm, PyPI, crates.io, Go modules, RubyGems, NuGet, Maven, and GitHub Actions, free on GitHub.
|
 |
Click above to watch |
|
New episodes air every week on Wednesdays: Spotify | Apple Podcasts | YouTube |
|
|
📰 Around the Horn |
Amazon said it is designing custom AI chips for Echo, Fire TV, and future devices. Zoom agreed to acquire Common Room, adding buyer-signal intelligence and RoomieAI agents to Zoom Revenue Accelerator. Microsoft reportedly began merging consumer and enterprise Copilot chatbots into one app as it tries to make Copilot a stronger rival to ChatGPT and Claude. Anthropic emails showed how its Pentagon relationship frayed over guardrails, access, and military-use boundaries. Kling AI reportedly raised an initial $2B as Kuaishou spins off its video AI unit. SoftBank established SB Neo to operate its U.S. neocloud business. Symbotic acquired ARMS Innovations to push warehouse robotics toward AI-powered orchestration. Tripo AI raised another $150M for 3D foundation models and world-model tools.
|
Everything is coming into focus. |
|
|
Join beehiiv live on July 16th at 1PM ET for a first look at the future of audience-led business. |
This isn’t just another feature launch (though there will be plenty of those). It’s a look at a more connected future for creators and brands that are tired of juggling disconnected tools, platforms, and data. |
If you care about building an audience online, this is worth your time. |
Reserve your spot. |
|
😹 Monday Meme: Time-Travel Experience Required |
|
|
A Reddit thread spotted a Yahoo job ad asking for 10+ years of Claude Code experience, which is impressive because Claude Code is still new enough that most people's first memory of it includes asking, “Wait, can it actually edit the repo?” |
The job post may be fake, sloppy, or copied from some recruiter template that escaped containment. Still, it captures the vibe perfectly. AI tool requirements are now moving so fast that the only qualified applicant is a time traveler with a LinkedIn Premium trial. |
But hey, my 100+ Claude Code sub-agents have probably put in about 10+ years of work between them in the hyperbolic time chamber, so does that count? |
|
|
 |
this is literally our style guide!! |
|
|
|
|
|
That’s all for now. |
|
What'd you think of today's email? |
|
|
|
|
P.S: Before you go… have you subscribed to our YouTube Channel? If not, can you? |
 |
Going for an anime aesthetic this month! |
|
P.P.S: Love the newsletter, but only want to get it once per week? Don’t unsubscribe—update your preferences here. |
No comments:
Post a Comment
Keep a civil tongue.