Episode 186

A Spotify model for AI

Bots are taking over the internet, previewing a new world where agents interact and humans consume bot-created content. Cloudflare, a leading content delivery network, saw this coming.

Cloudflare Chief Strategy Officer Stephanie Cohen told me that publishers reported less traffic to their sites and less human traffic. “Those numbers were 10 times worse than a decade ago.”

This has broken the open web trade of the crawling right for traffic promise. Cohen said, “For some bots, the crawl-to-traffic ratio is tens of thousands of times worse than it used to be.”

Five takeaways from our conversation:

  1. Scarcity is required. Cloudflare’s pay-per-crawl model aims to restore balance by creating enforceable scarcity. “If you don’t have scarcity, there’s no market. Changing the defaults was the only way for a market to develop.”
  2. AI engines need publishers. Models rely on fresh, high-quality publisher content to stay accurate. “If you got asked a question and didn’t know what happened over the last two weeks, your answer is going to be stupider.”
  3. Pay-per-crawl is a first step, not the finish line. Cloudflare’s system lets publishers decide whether to block, allow, or charge AI crawlers. It’s a way to test how a content economy might form. As Cohen put it, “It’s the simplest thing you can do because you can count a crawl.”
  4. The goal is the Spotify model. Cohen pointed to Spotify’s emergence after Napster as a precedent for compromise. “There’s lots of money going to creators in that model… it took a while for that market to develop.”
  5. The problem is Google. Publishers face a lose-lose choice: block Google entirely or allow AI Overviews to rewrite their work. “If they want search, they’re also in AI overviews… They can’t both crawl and pay nothing.”
Transcript
Brian:

Welcome to the Rebooting Show.

Brian:

I am Brian Morrissey.

Brian:

This week's conversation is with CloudFlare Chief Strategy Officer Stephanie Cohen.

Brian:

An era when most of Big Tech has turned its back on publishers.

Brian:

I'm not gonna name anyone, but you know who we're talking about.

Brian:

You know, one company has quietly become something of a trusted ally, and that's CloudFlare.

Brian:

CloudFlare serves a vital role.

Brian:

In the open web, it is a content delivery network and that basically means that

Brian:

it is infrastructure for the open web.

Brian:

its tools underpin everything from security to performance to how modern

Brian:

media companies deliver content at scale, without relying on, the big tech platforms.

Brian:

And on this week's episode, I speak to Stephanie, who leads.

Brian:

Cloudflare's media and entertainment business about how the company's network has become the

Brian:

connective tissue of the modern internet and why it's found traction with publishers when

Brian:

others in tech have not, and what it means to be a neutral partner at a time when the neutrality

Brian:

of the information ecosystem is in short supply.

Brian:

Now, CloudFlare is making aggressive moves to, stand on the side of publishers when

Brian:

it comes to AI slurping up their content.

Brian:

this is.

Brian:

I don't think it's exaggerated to say it's an existential threat to many publishers.

Brian:

And so, CloudFlare as an ally, is noteworthy.

Brian:

But before we get to that conversation, I wanna tell you about something that I think you'll

Brian:

find, useful from The Rebooting partner Beehiiv.

Brian:

When you're running a media business, email is usually the backbone of it, and getting it right

Brian:

is unfortunately usually harder than it should be.

Brian:

And that's where Beehiiv comes in.

Brian:

This is a platform that was built by the people who scaled Morning Brew to millions of

Brian:

readers, and they created the platform they wish they had when they were in your shoes.

Brian:

so it is no surprise that companies like Time and the Texas Tribute are already

Brian:

running their newsletters on Beehiiv.

Brian:

It is designed for serious media operators, not marketers, bolting

Brian:

on, a newsletter as an afterthought.

Brian:

if you are running a publication with more than a hundred thousand

Brian:

subscribers, Beehiiv is built for you.

Brian:

Their team has scaled newsletters at places like the information puck and Morning Brew.

Brian:

They understand the challenges big publishers face from seamless migration to enterprise

Brian:

grade deliverability and monetization.

Brian:

Beehiiv makes it easy to grow, retain, and engage audience.

Brian:

At scale.

Brian:

So if you want to see what your next stage of growth could look like, go to Beehiiv.com/trb.

Brian:

That is spelled BEEHIIV.com/TRB.

Brian:

You can meet with their team of growth and newsletters.

Brian:

Experts today.

Brian:

Thank you so much to Beehiiv.

Brian:

be sure to check out actually last week's conversation I had with Tyler Dank Beehiiv's, CEO.

Brian:

for more on how Beehiiv is helping publishers, large and small, put, the

Brian:

audience at the center of their strategies.

Brian:

Now onto my conversation with Stephanie.

Brian:

All right, Stephanie, thank you so much for joining me.

Brian:

We're like, across the World Trade Center Plaza from each other.

Brian:

You're in World Trade One.

Brian:

I'm a World Trade three.

Brian:

so I can't see you across, but you're across the plaza.

Stephanie:

across the internet and across the yard basically.

Brian:

Yeah.

Brian:

And probably this data is, is flowing, through, CloudFlare.

Brian:

I wanna get into, how CloudFlare is, is doing paper per crawl with every single publisher,

Brian:

obviously is very focused on the changes to distribution patterns and therefore their,

Brian:

their monetization with, AI in particular.

Brian:

There's tons of AI crawling going out there, but I just wanna sort of.

Brian:

Level set with the role CloudFlare plays, which is unique.

Brian:

I think a lot of times we think about the, the sort of giants of, of

Brian:

technology on the distribution side 'cause they control the distribution.

Brian:

but CloudFlare operates on the infrastructure layer, so give us a CloudFlare 1 0 1.

Stephanie:

So infrastructure is, is really cool by the, by the way, um, I did a lot of

Brian:

I like roads, I like plumbing.

Stephanie:

yeah.

Stephanie:

So I.

Brian:

I love it.

Stephanie:

I did a bunch of things before I came to CloudFlare, all at Goldman

Stephanie:

Sachs, and one of 'em was that I ran the general industrial business, which was,

Stephanie:

you know, physical infrastructure, and I always found it incredibly interesting.

Stephanie:

So CloudFlare is just the internet version of infrastructure.

Stephanie:

So today CloudFlare works with millions of customers to help them build and run their

Stephanie:

applications so that they're fast, and secure.

Stephanie:

Our mission is that we help build a better internet.

Stephanie:

Two most important words there, our help and internet, which I'm sure we'll get

Stephanie:

into as part of this conversation, but, but your question is, we do that by having

Stephanie:

this very large, interconnected network.

Stephanie:

So the CloudFlare network today sits in 325 locations in more than 125 countries

Stephanie:

where it's our server running our software.

Stephanie:

And like the magic of the CloudFlare network is that we're capable of running

Stephanie:

every piece of software in every single.

Stephanie:

Server.

Stephanie:

And so that makes us incredibly reliable, fast and secure.

Stephanie:

And so what do we do on top of that like layer of infrastructure is that we help your

Stephanie:

application so that traffic gets to you fast and that the traffic you don't want gets deflected.

Stephanie:

We also protect your data, your employees and your network when they're

Stephanie:

on the internet and then we help you.

Stephanie:

You can build applications on that, so, so we are this incredible network that help

Stephanie:

other people then build their businesses

Brian:

So you're, you're a CDN right?

Brian:

Content Delivery Network and, and a lot of that is.

Brian:

Basically, I mean, there's a lot of data moving back and forth.

Brian:

I'm gonna sort of like, but there's, there's a lot of data moving back,

Brian:

but that's what the internet is.

Brian:

It's data moving back and forth.

Brian:

Right.

Brian:

And, and you, you guys sit on the side to make sure that whether it's through caching,

Brian:

that the data is very fast when you go on a website and like the, it renders very quickly.

Brian:

That is thanks to, to CDNs.

Brian:

Correct.

Stephanie:

Absolutely.

Stephanie:

And you know, one of the amazing things about the internet is that there's this great

Stephanie:

diversity of what's developed on the internet, but you have to be able to be on the internet.

Stephanie:

In order to do that, one of the problems is if you put a website on

Stephanie:

the internet, it's gonna get attacked.

Stephanie:

Like that's just the world we live in.

Stephanie:

CloudFlare showed up and said every one of those websites, legal websites has a right

Stephanie:

to be on the internet and free from attack.

Stephanie:

So it means that all of these millions and millions of individual

Brian:

it's security.

Brian:

'cause there's all kinds of malicious characters out there, whether they're part

Brian:

of a foreign government or just someone who's trying to grab a credit card information.

Brian:

And so that's a critical role.

Brian:

That you play in defending like the sort of internet, but also I think what, and the reason

Brian:

I want to get into this is because there's like a quasi regulatory role, almost like a

Brian:

policy role because you have to make decisions.

Brian:

And some of them are, they're, they're.

Brian:

So, you know, it fishers, you know, phishing.

Brian:

It's pretty easy.

Brian:

It's not very controversial decision to make though, I don't think, well, I

Brian:

guess the fishers are very, pro phishing, but most people would, would agree.

Brian:

You're not gonna get a lot of people protesting that, but you know, because you

Brian:

have this, I think it's like 20% of the internet end up going through CloudFlare,

Stephanie:

More than 20% of the internet sits behind club.

Stephanie:

The way to think about that is the surface area of the internet, so the breadth of the internet.

Brian:

So, I mean, you, you make decisions that affect, you know, it has like a. don't know.

Brian:

How would you put it?

Brian:

Like a policy?

Brian:

It's kind of like, to me it's like, it's like when California puts emission standards

Brian:

out, like they become, it's more than just California because like, you can't serve all of

Brian:

the country and, and without like, California.

Brian:

So the, the emission standards come to de facto.

Brian:

So like when you guys make a decision, it has a greater impact even than just quote unquote

Brian:

just 20 odd percent of the internet traffic.

Stephanie:

So I think it's important to now dig into.

Stephanie:

Some of the complexity around this, which will make not complicated, when we talk about it,

Stephanie:

which is that like every piece of traffic that comes through the CloudFlare network, we're

Stephanie:

we're scoring to figure out like, is this a real, a really bad guy, gal? Is this a bot?

Stephanie:

Is this a human?

Stephanie:

Like we're trying to figure out what's coming at the CloudFlare network and the real like

Stephanie:

benefit, amazing part of the CloudFlare network is because we have millions of

Stephanie:

free customers in all of those locations.

Stephanie:

We have really, really great information about what's going on.

Stephanie:

On the internet.

Stephanie:

And so that allows us to be able to tell what's going on.

Stephanie:

And the reality is, I, I would say the vast majority of the decisions that are being

Stephanie:

made, one, they're being made by machines, but two, they're, they're pretty obvious.

Stephanie:

Like DDoS attack is deflected, right?

Stephanie:

Like bad actors deflected everything else that is, quote unquote, a judgment call is really made.

Stephanie:

At the individual domain level.

Stephanie:

So the website owners are making decisions about what gets through and what doesn't get

Stephanie:

through, and so they're making that decision.

Stephanie:

And so some of them will say, I want nothing but a human on my site.

Stephanie:

You can imagine the types of sites that say nothing but a human.

Stephanie:

There's others that really want that type of traffic, and so they're

Stephanie:

letting more of that through.

Stephanie:

And so we're giving them the tools and we're giving them the ability to

Stephanie:

do that, but they make those choices.

Brian:

So the ultimate decision is made, at, at, at the end point.

Brian:

and, and we'll get into the, the, the devil being in the defaults and, and whatnot.

Brian:

Um, so let, let's talk about, Pay Per Crawl.

Brian:

I was across the plaza.

Brian:

You guys had a, had a very late party for Monday night.

Brian:

I'm still shaken by, but it was, it was the Herald, a new, um, a new internet.

Brian:

And obviously look, AI is, it's inescapable in any conversation, particularly with

Brian:

publishers and usually it can be a little bit of a depressing conversation because there's a

Brian:

lot of fear of what First of all, it's bigger than ai, but the, the changes in distribution

Brian:

patterns of the internet are just shrinking.

Brian:

a lot of, of publishers, you know, the traffic to them and they built business

Brian:

models around that, that traffic.

Brian:

And so they're trying to rapidly.

Brian:

Diversify their models.

Brian:

They always wanna talk to me about their events and, and various things that have

Brian:

nothing to do with putting ads on webpages.

Brian:

They still have a, a, a large business in putting ads on webpages, and they

Brian:

need to get people to those webpages.

Brian:

And, you know, AI is, it's an existential threat.

Brian:

I mean, you put in like Armageddon and media, you get like many results.

Brian:

It's very disturbing.

Brian:

Like, I think we, we used to stop using that.

Brian:

it's not very good, vibe.

Brian:

So the question is like, how is there going to be a new economic bargain for the internet?

Brian:

Google, to me, was always the linchpin of the internet.

Brian:

Before we started, I was talking about how, you know, covering them since before

Brian:

they were public and you know, they've played this arbiter role on the internet.

Brian:

Also a defacto regulator.

Brian:

really because, because they just had such an outsized role in distribution

Brian:

and broadly, I always felt like they almost got too much grief because like

Brian:

they were balancing a bunch of different constituencies and they did it imperfectly

Brian:

and obviously in a self-interested manner.

Brian:

but you know.

Brian:

It's a tough job.

Brian:

It's a good job, but it's a very lucrative job, by the way.

Brian:

but it, it's a kind of tough to, to please everyone all the time, and at

Brian:

least their interests were broadly aligned with, a, a healthy open internet.

Brian:

They're, they're, they, they were making all that money off of the open internet,

Brian:

you know, that that's been changing.

Brian:

And AI to me is just an accelerant on a lot of that stuff.

Brian:

So what did, what were you guys seeing on, and it's beyond obviously Google and

Brian:

open ai, but what were you, you seeing.

Brian:

On the CloudFlare side, 'cause you see the data, right?

Brian:

that was a little bit worrying and, and made you want to, you know, devote

Brian:

resources to even get involved in this.

Stephanie:

So you spent a lot of time with the publishers, you know, they're our

Stephanie:

customers, and so we started hearing from.

Stephanie:

Them.

Stephanie:

It's a while ago now.

Stephanie:

Actually.

Stephanie:

The first thing we did around, this happened about a year ago.

Stephanie:

We started to hear exactly what you were hearing, which was that traffic.

Stephanie:

There was less traffic coming to their sites

Brian:

Human traffic.

Stephanie:

Less human traffic coming to their sites, so less eyeballs on, on their pages.

Stephanie:

And we heard lots and lots of complaints.

Stephanie:

By the way, I do at some point wanna get into this.

Stephanie:

We're now hearing that from lots of other industries.

Stephanie:

It's not just the, the publisher space, but, but it was, but hit them first, which

Stephanie:

is unsurprising, given the business model.

Stephanie:

And so then we went back and we looked at the data.

Stephanie:

So we actually, on the CloudFlare site, had this great site called Radar, which

Stephanie:

has tons and tons of information on it, which will allow you to see this real time.

Stephanie:

But we looked at the data, and the data was, was more than we would've expected and was

Stephanie:

changing faster than we would've expected.

Stephanie:

And so we can see how often bots are crawling a site and we can see

Stephanie:

how often they're referring traffic.

Stephanie:

And those numbers.

Stephanie:

So the ra, you want more traffic, human traffic to your site, like per crawl.

Stephanie:

And those numbers were at least 10 times worse than they were a decade ago.

Stephanie:

And for some bots they had, you know, versus where Google had been, they were tens of

Stephanie:

thousands times worse, which is again, kind of unsurprising when you take a step back, which

Stephanie:

is that they're crawling a lot and there's very little traffic going back to the site.

Stephanie:

And we said, you know what?

Stephanie:

This is like a real problem.

Stephanie:

And so we need to

Brian:

and this is beyond, I'm sorry, but this is beyond just quote unquote training data.

Stephanie:

Absolutely.

Stephanie:

This is so interestingly despite, you know what I, I think, you know, some people

Stephanie:

report on our network, we still see the vast majority of crawling comes from training.

Stephanie:

It doesn't mean there isn't other types of crawling, but the vast majority of the crawls that

Stephanie:

are happening are still training related crawls.

Stephanie:

But doesn't mean that that's not gonna change over time.

Stephanie:

And this doesn't mean we're not seeing things for rag and operators and you know, agents, but.

Stephanie:

The vast majority of the crawling is still training related

Brian:

Wait, can you, because you brought up rag, can you explain Rag

Brian:

to the people we're already getting

Stephanie:

Yeah, yeah.

Stephanie:

No, no.

Stephanie:

Yeah.

Stephanie:

the, the idea really is that if you are asking a question to an AI agent, they're, or a chat

Stephanie:

bot, they're gonna use the model to come up with the answer, but they're also gonna check to

Stephanie:

make, see what the most recent information is.

Stephanie:

And so to make sure that they're getting the most recent information

Stephanie:

in answering your question, which

Brian:

Okay, so that's when like when they say like checking whatever, like

Brian:

when you're on chat GBT, is that like, then it's going out with like rag or No.

Stephanie:

when they're, they're processing everything through their, their, I mean, by the

Stephanie:

way, all the models don't run exactly the same way, but they're, they're processing the data

Stephanie:

based on the actual model and what's been trained.

Stephanie:

And then what we have seen, and we've seen this since the announcement, that they are

Stephanie:

heavily dependent on recent information.

Stephanie:

It obviously depends on the.

Stephanie:

What question you're asking, but you can imagine that if you got asked a question and you didn't

Stephanie:

know what's happened over the last two weeks, like you don't have access to recent information,

Stephanie:

your answer is gonna be stupider than if you have access to the most recent information.

Stephanie:

And so it's pulling information from the open internet every time it's answering

Brian:

so the, the, the, the, the bargain used to be.

Brian:

You can crawl our site, particularly with like Google, you know, search was like the real

Brian:

crawler, but like, you can crawl our site.

Brian:

You're gonna basically scrape, I mean, I remember back, you know, covering,

Brian:

it's like they're scraping the data.

Brian:

It's like, okay, you're gonna scrape some data, they're gonna claim it's not scraping or whatever.

Brian:

You're, but you know, the, the trade traffic was the trade.

Brian:

Right.

Brian:

Like at the end of the day, it was like Google had this imperfect system that was like, look, we're

Brian:

gonna, we're gonna basically scrape your sites.

Brian:

We're gonna take little snippets on there.

Brian:

and if you sue, we're, we have way more and expensive and better lawyers.

Brian:

It doesn't really matter.

Brian:

but in return, we're gonna send you traffic, you're gonna monetize that traffic.

Brian:

Oh, by the way, we, we have the ad tech stack too, so here we're gonna get a piece of that.

Brian:

But like, it's better to have customers.

Brian:

To be in business together because your interests are more likely aligned.

Brian:

A little, little bit of a different bargain, I feel like now.

Brian:

because the, the, the amount of crawling that's being, that's taking place just

Brian:

overall does not match up with the amount of traffic that's coming back.

Brian:

That currency has sort of been devalued.

Brian:

Yeah.

Stephanie:

I think the underlying thing you're talking about is that there's

Stephanie:

a view that the internet was free.

Stephanie:

So we've kind of grown up with the idea that the internet was free, but the

Stephanie:

reality is the internet was not free.

Stephanie:

It was being paid for by, by advertisers.

Stephanie:

And Google, to your point, like facilitated that model.

Stephanie:

There are others, but Google is facilitating that model.

Stephanie:

And so when that model breaks down, then that economic model breaks down.

Stephanie:

And so yes, there's a bunch of.

Stephanie:

Crawling that was going on has gone on, continues to go on.

Stephanie:

Previously that would, the amount of crawling would be better linked to the amount of

Stephanie:

traffic that was being sent back to your site.

Stephanie:

But now when people are reading the derivative.

Stephanie:

Rather than the original, they can get the answer.

Stephanie:

So they get the answer from the crawling, but they never go back to the original

Stephanie:

site, which means you cannot monetize subscriptions, you cannot monetize ads.

Stephanie:

And by the way, we think you should not underestimate ego in all of this,

Stephanie:

which is just the idea that you, someone is reading what you wrote.

Brian:

Yeah, for sure.

Brian:

and so explain the, obviously this is, this is an issue.

Brian:

It, it's an existential issue for many publishers.

Brian:

CloudFlare is uniquely positioned to address this, I guess.

Brian:

so why explain, explain the system of like Pay Per Crawl and how it

Stephanie:

So I think what I'd like to do is talk about what we, all the changes we

Stephanie:

made, which are linked to Pay Per Crawl, but Pay Per Crawl in isolation doesn't really

Stephanie:

make a ton of sense without everything else.

Stephanie:

Okay.

Stephanie:

One.

Stephanie:

What we announced on July 1st and made you show up at at midnight was because

Stephanie:

it was content Independence Day.

Stephanie:

So instead of ringing in the new year, we rang in Content Independence Day

Stephanie:

and pressed the big button about we changed the defaults on our network.

Stephanie:

Okay, so what do we do?

Stephanie:

One, we change the defaults for new customers, so new customers, new domains on

Stephanie:

the CloudFlare network, blocking training.

Stephanie:

By default, we blocked that either at the network level, if that works with the specific crawler.

Stephanie:

Or we instruct the crawler to not train using a robots txt message, which we can come back to.

Stephanie:

That's the first thing we did.

Stephanie:

the second thing we did was that we made it significantly easier to see who was

Stephanie:

scraping your site for all different types of reasons, and we gave you the ability

Stephanie:

to turn each one of those on or off.

Stephanie:

And I say all that because it matters.

Stephanie:

Creating a market without having the conditions for a market doesn't work.

Stephanie:

If you don't have scarcity, there's no market, and so the changing of the defaults, but really this

Stephanie:

idea that you would have more transparency and more control over who was on your site was really

Stephanie:

the most important thing that we did because that's the only way for a market to then develop.

Stephanie:

We then launched a Pay Per Crawl beta with the simple idea being that

Stephanie:

you can block, allow, or charge.

Stephanie:

So you can say, I don't want you on my site block, fine, come to my site because I already have a

Stephanie:

deal with you, or because I want you on my site, or I feel like I'm getting traffic, or, neither

Stephanie:

of those really are great solutions for me.

Stephanie:

So what I really like to do is have you pay me to be on my site.

Stephanie:

The simplest way to do that was Pay Per Crawl.

Stephanie:

I think we can all agree that like we're at the very beginning of this and so we'll

Stephanie:

figure out how the market needs to develop and what the right business model is and is

Stephanie:

it one business model or many business models.

Stephanie:

But that is the, the most simplistic thing that you can do because it's, you can tell when

Stephanie:

it's happened, like you can calculate AC crawl.

Stephanie:

And so we launched Pay Per Crawl and, and I'm happy to talk about kind of where

Brian:

So I guess the, I mean, the first thing is like I, because I get this, 'cause like I,

Brian:

I, I I did a podcast with, with Neil Vogel, the, the CEO of, now People Inc. back in, in.

Brian:

And, and we talked about this issue and he was like, I was talking, I was, you know, skipping

Brian:

ahead to the money, to the marketplaces and he was like, we, we gotta develop scarcity first.

Brian:

We have to, like, you have to understand like what is going on and whether you, you can block them.

Brian:

So I mean, Do you see evidence that you can actually block them?

Brian:

Because I mean, you mentioned like robots txt and tell me I'm wrong.

Brian:

There's a lot of nuances here, but like I always thought that the whole point of robots

Brian:

txt was that publishers could say, don't crawl.

Brian:

Don't crawl these pages, they're transaction pages.

Brian:

We don't want that indexed or anything.

Brian:

And then it seemed like, obviously there are no like government governing

Brian:

bodies, seemingly on the internet.

Brian:

So like, then it was just like, oh, some people are just like, yeah, we're we, we, we don't,

Brian:

we're not, we're ignoring we robots at TXT.

Brian:

This is not like some.

Brian:

Is Interpol gonna arrest us?

Brian:

it's, it's not some law or something.

Brian:

but explain like, first of all, like, does, is this effective?

Brian:

Can you, can, can you actually block, this crawling and, and because I just

Brian:

assumed that they would just go around it

Stephanie:

So, yes, yes.

Stephanie:

We can block.

Stephanie:

It's actually the core business, the thing we started out at the beginning of what

Stephanie:

CloudFlare does, we've been doing for 15 years.

Stephanie:

So yes, we can block is one, two.

Stephanie:

The vast majority of actors on the internet are good actors.

Stephanie:

We actually launched a site that's good ai bots.com, which actually tells you which

Stephanie:

bots obey instructions, and the vast majority of bots obey instructions on the internet.

Stephanie:

What if you are using a provider, someone like CloudFlare, but there are others if

Stephanie:

you are using, if you have bot protection.

Stephanie:

You want to block a bot that then disobeys your robots message you, you have ways to

Stephanie:

to then go after them and to stalk them.

Stephanie:

And so the our, we feel quite strongly that yes, you can block, I would tell you that we

Stephanie:

have lots of evidence that blocks are effective.

Stephanie:

Like what is the evidence?

Stephanie:

One large publishers that are.

Stephanie:

Blocking that are like, you'll have deals with some and blocking.

Stephanie:

Everyone else would tell you that the complexion of their conversation has changed.

Stephanie:

Two, we, we have evidence, we've written blog posts about it, of actors that are blocked and

Stephanie:

then try to evade the blocks that we can stop them, and that when they don't get access to the

Stephanie:

information, they start making up what the answer.

Stephanie:

So we've made them.

Stephanie:

Less intelligent in terms of what they're answering.

Stephanie:

And three, we anecdotally hear that there are customer complaints with large models

Stephanie:

because they no longer have access to the same level of up-to-date information.

Stephanie:

And so many of these companies are running consumer businesses that require that they provide

Stephanie:

high quality product to their end customers.

Stephanie:

And so as they have access to less information, as you create scarcity.

Stephanie:

If there will and should be demand for, for that information, for

Stephanie:

content that is easier to make scarce.

Stephanie:

We already see the benefit for content that's harder to make.

Stephanie:

Scarce meaning it's out there.

Stephanie:

it's more difficult.

Stephanie:

We do believe that the world will move towards, you know, high quality original content.

Stephanie:

because that's in, in terms of, that's where most of the value will be, which feels like a, a good

Stephanie:

thing for us to all have an incentive to create.

Brian:

I hope so.

Brian:

I hope so.

Stephanie:

That was my utopian

Brian:

Yeah.

Brian:

No, I love it.

Brian:

I love it.

Brian:

I, it's Friday, it's the weekends coming.

Brian:

I, I like it.

Brian:

I think the question ends up being for a lot of publishers, they're downstream of technology

Brian:

giants fighting, intergalactic battles.

Brian:

Like sometimes literally they're in space.

Brian:

and.

Brian:

The publishing industry while those in it, you know, it's, it's, it's, and

Brian:

it is like societally very valuable.

Brian:

It's a very small, it's a very small industry.

Brian:

there's not a lot of money in it.

Brian:

it's very easily squashed.

Brian:

publishers typically are pretty easy to push around.

Brian:

and I, there's a long history, I don't need to go through that.

Brian:

I think, publishers against technology companies, they're like, it's like the Washington

Brian:

Generals against the Globe Trotters, like, as far as the record goes at this point.

Brian:

and so, you know, ultimately the question is like, do they get some measure of leverage?

Brian:

And I know Matthew Prince has said, I think he said it at, at that event, that like, that

Brian:

these, that these companies want to do deals.

Brian:

That they want to, they, they want someone else to go first.

Brian:

And so I think there, there has to be, there's no like silver bullet

Brian:

like solution to any of these issues.

Brian:

CloudFlare is very influential, obviously, like we talked about, it's still 20 odd

Brian:

percent, like, and it seems like these companies don't want to cut the check.

Brian:

Except for in a few small cases.

Brian:

Right.

Brian:

And like, what do you see?

Brian:

Because I was asking like John Roberts actually one of on, on Neil's team at that event, I was

Brian:

like, are am I gonna see their products get worse?

Brian:

Like, am I like, is that 'cause the only thing that I would see, I don't believe in

Brian:

benevolence when it comes to these companies.

Brian:

I don't think we should, like, the only thing would be is if their products get

Brian:

worse, then it's like, okay, well we will pay to make the products better.

Stephanie:

So we see evidence, and obviously we have good information about who's blocking and

Stephanie:

therefore, which, which models, and therefore where is there gonna be an information gap, right?

Stephanie:

And so the rest of the, it'll take the rest of the world longer to see where there's,

Stephanie:

where the models start to have issues.

Stephanie:

But we see evidence of issues both.

Stephanie:

We can see it in the models we can.

Stephanie:

Tell it from the conversations and we see how the interaction works between large publishers

Stephanie:

interacting with large model companies.

Stephanie:

Interestingly, like, and I know you know this, that's not the only place where the market is.

Stephanie:

There's a long tail in both sides.

Stephanie:

There's a long tail of content and there's a long tail of models, and so that is like.

Stephanie:

In many ways, that's where a lot of the diversity and innovation is gonna come, right?

Stephanie:

If this only, if the only thing we solve is we solve this for really big publishers

Stephanie:

and really big model companies that will be better than the current situation.

Stephanie:

But that's actually doesn't get you to high quality diverse content on the internet.

Stephanie:

But

Brian:

No,

Brian:

the opposite, actually,

Stephanie:

the first place you'll see it is with large publishers, large models, but

Stephanie:

there is real benefit across the ecosystem.

Stephanie:

And interestingly.

Stephanie:

There's the value of the content, which is valuable, but there's also like, how do I

Stephanie:

make sure I get the information quickly?

Stephanie:

How do I make sure that I get it efficiently?

Stephanie:

Meaning do I have to crawl the same site over and over and over again, or

Stephanie:

can I tell that the content is fresh?

Stephanie:

Like there's, in many ways the current situation of a bunch of machines crawling an internet that

Stephanie:

was meant for humans doesn't make a lot of sense.

Stephanie:

There's a much more efficient way to do this.

Stephanie:

One of the benefits of us being a technology company is there's a lot of technical solutions

Stephanie:

to, to making all the crawling more efficient, which has tons of value for these big tech

Stephanie:

companies that spend lots and lots of money on CPUs and GPUs and capital expenditures and

Stephanie:

everyth.

Brian:

It's another line item.

Brian:

Let's go raise

Stephanie:

it.

Brian:

4 trillion.

Brian:

yes, but it's a business decision.

Brian:

Ultimately, what, what is the, what is the holdup on the business decision?

Brian:

Because this is all technological.

Brian:

It, it seems like we do amazing things with technology.

Brian:

This doesn't seem like a technology problem per se or not.

Brian:

It's a solvable problem.

Brian:

Right.

Brian:

It's a, it's a business issue.

Stephanie:

yeah.

Stephanie:

So some of it is scarcity, meaning in the plate, like have you talked about GPUs?

Stephanie:

We can.

Stephanie:

Talk about CPUs.

Stephanie:

We could talk about talent in those places.

Stephanie:

There is scarcity today, right?

Stephanie:

Like some of that, like the history tells you that the cost of compute will continue to go down.

Stephanie:

The cost of talent.

Stephanie:

Now that everyone realizes that this is a place to make a lot of money,

Stephanie:

there'll be more talent in those areas.

Stephanie:

And so there're spending lots and lots of money on those two things.

Stephanie:

But you would argue that the cost for that will go down.

Stephanie:

And so there will be room for.

Stephanie:

For content, but on top of that, you have to have scarcity in order for them to just

Stephanie:

like, they have to have a reason to do it.

Stephanie:

You know, I think we've been quoted many times as saying that no one wants to be the sucker.

Stephanie:

No one wants to be the, the only person that's paying.

Stephanie:

And so we're right now, two-ish months in from kinda the original, the big

Stephanie:

announcement around what we're doing.

Stephanie:

That's time to demonstrate that it's working.

Stephanie:

You know, one of the, the questions of like, is there a demand?

Stephanie:

I know, you know, there's demand to be paid, but you want a stat that tells you there is, there's

Stephanie:

been 1,000,000,004 oh two, there are 1,000,000,004 oh two requests a day on the CloudFlare network.

Stephanie:

And for those wondering what a 4 0 2 request is, that is a site saying, please pay me.

Stephanie:

So instead of saying, so, instead of saying just block it's, it's saying, instead of

Stephanie:

just saying a block, it says, actually what I really want you to do is to pay me.

Stephanie:

And we

Brian:

how many okays came back?

Stephanie:

I mean the, you need a market in order for that, and

Brian:

Okay, so let's talk about the market, because it's like, okay, you

Brian:

gotta, you know, Neil's two step, it's like you gotta establish the scarcity.

Brian:

You have to block and get some kind of leverage.

Brian:

Let's assume that these models.

Brian:

There are signs that they're, they're sort of reaching the limits, right.

Brian:

Of, you know, the, the GPT five was, was broadly seen as a disappointment.

Brian:

I can see having unique access to fresh, high quality data sources when

Brian:

you're making a consumer product that.

Brian:

I would think that would be good, at least in marketing and, and

Brian:

probably on the product level, right?

Brian:

But then you have to figure out what is the bid, what is the ask.

Brian:

and my experience with publishers is they always view, you know, they're like.

Brian:

Journalism or, or you know, they don't like it being called content.

Brian:

And then it's even worse when it's called data.

Brian:

'cause then once you're, once it becomes like ones and zeros, like you're completely commoditized.

Brian:

and so I'm always like wondering what, what is the marketplace gonna be like for this?

Brian:

and whether it makes up for what is lost.

Stephanie:

Yeah.

Stephanie:

I, I, okay.

Stephanie:

So I think those are two different questions, but let's start with the, like, what could

Stephanie:

the, what could the market look like?

Stephanie:

Like how could this actually work?

Stephanie:

And I think, you know, world tends to think in analogies, let's talk about Spotify for a second.

Stephanie:

Right?

Stephanie:

And so I think there are.

Stephanie:

Everyone could have their issues with the, with the Spotify business model, but

Stephanie:

there is lots and lots of money going to, creators in the Spotify business model.

Stephanie:

It took a little while for that market to develop.

Stephanie:

but to us that's a reasonably good, it's not exactly a perfect analogy, but it's

Stephanie:

a reasonably good analogy for, for what happens, which is there's a whole host of

Stephanie:

content in this case instead of multiple subscribers paying for it, individual models.

Stephanie:

Are paying for it.

Stephanie:

And then you figure out how you're gonna distribute that, that amount of money to

Stephanie:

the creators on inside a certain ecosystem.

Stephanie:

So that to us, doesn't feel like a crazy model and feels like a way for the large content creators.

Stephanie:

They can do their own version of that negotiation and they should, but for kinda the long tail.

Stephanie:

Any one of them individually is not going to have the ability to have that level of negotiation.

Stephanie:

And so this is a rational way to do it.

Stephanie:

And there are a bunch of data.

Stephanie:

Data.

Stephanie:

There's a lot of data that shows that's encouraging high quality original content.

Stephanie:

The the second thing is that if you look at Netflix and YouTube, to your point, those

Stephanie:

models, they are competing for the best content.

Stephanie:

They, you want the, the best, newest thing so that people sign up for your system.

Stephanie:

And I do think that that is an interesting world for, particularly for the AI

Stephanie:

companies that are more consumer oriented.

Stephanie:

And so interestingly, let's, like local press in, we've had lots and lots of issues with

Stephanie:

local press over the last couple of decades.

Stephanie:

It's a, it's a business model that's had lots of problems, but in many

Stephanie:

ways they do have very unique access.

Stephanie:

To information that a cohort of people in that geography really care about.

Stephanie:

And so this, this idea that you can create content that's specific and high quality that people

Stephanie:

want and that an AI company will ultimately have a pretty good view on their need for it and

Stephanie:

willing to pay for it, doesn't feel crazy to us.

Stephanie:

And then how that negotiation happens.

Stephanie:

There's a Spotify version where there's kind of one negotiation and then

Stephanie:

it's programmatically distributed.

Stephanie:

There's an auction version of how this all develops, which maybe feels

Stephanie:

more like the ad networks, and then there's one-to-one negotiations.

Stephanie:

I think we have no idea how this will develop, for sure.

Stephanie:

For now, it feels like we're gonna test out as a ecosystem all of those

Stephanie:

models and see which ones work.

Brian:

Okay, so it's not like, so that is still like up in the air, right?

Brian:

Because I think one of the questions ends up being is like, how do you value, it's

Brian:

very qualitative, like, I mean, how do you value a particular piece of content?

Brian:

Like just saying like, okay, you, you know, when you break it down into

Brian:

ones and zeros, everything's the same.

Brian:

And it's like access to, there's a lot of different types of publishers.

Brian:

and the, and you know, words are not all equal, right.

Brian:

And some are more valuable than others.

Brian:

And the question is, how do you arrive at what the value is?

Stephanie:

you can imagine a world where the AI companies themselves have a view on that.

Stephanie:

Meaning they have a view for their own business and their own model.

Stephanie:

What that, what that information is worth, by the way, is it worth more

Stephanie:

immediately than it is worth five minutes?

Stephanie:

Like there's certain content that has real temporal value, right?

Stephanie:

So if you have access to it.

Stephanie:

Immediately versus five to 10 minutes later, that fundamentally changes the value of that content.

Stephanie:

And so you can imagine a world where you don't have to have a central

Stephanie:

place that makes that determination.

Stephanie:

You have every model has their own view and that gets inputted into how they think

Stephanie:

about what they did and what they pay.

Stephanie:

which just feels like a, a rational system

Brian:

Yeah, I mean, because like they have, you know, it's like if you look at the

Brian:

programmatic world, you know, it's always been this, you know, the, the buy side has

Brian:

insights into how valuable a particular.

Brian:

A person or cookie is to them, and so therefore they're gonna bid.

Brian:

You know, specifically to that, I mean, the AI companies, I would think, like ha,

Brian:

have unique insights into the value to them for particular types of content when

Brian:

recency, how fresh there are, et cetera.

Brian:

it might be different than than what the, the, the sort of publisher, I mean, I'm sure the

Brian:

publisher will say, no, it's more valuable, but,

Stephanie:

most markets, having spent a lot of my career in markets, I in financial

Stephanie:

markets, you know, most markets like, there's always that, there'll always be

Stephanie:

that tension between a buyer and seller.

Stephanie:

Today, the tension is greater because there isn't, in most markets, there isn't scarcity.

Stephanie:

So in other markets where there's tension between buyer and seller, it's an opinion, but the

Stephanie:

reality is the option is you just don't buy it.

Stephanie:

Unfortunately, today in a lot of places, the answer is you'll just

Stephanie:

get it from somewhere else, right?

Stephanie:

That's like the problem we have today is that you, there is an alternative

Stephanie:

to paying the price that in this case, the seller wants to sell it to you at.

Brian:

Okay, so the pay per crawl, this, this initial implementation is

Brian:

not necessarily the only implementation.

Brian:

You know, the only way to create like a marketplace,

Stephanie:

Absolutely not.

Stephanie:

The, the idea with Pay Per Crawl is that one, we, we wanted to learn meaning as a ecosystem

Stephanie:

to learn what the right market development was.

Stephanie:

What we set up was very simply, if you are, if you have content, you can, instead of saying block.

Stephanie:

You can say, I would like to charge you.

Stephanie:

You do that through a 4 0 2 request.

Stephanie:

It's today a private request, which means that the world doesn't see what you're asking for.

Stephanie:

Just that one crawler will see what you're asking for.

Stephanie:

They can say yes or no.

Stephanie:

If they say yes, they get to crawl.

Stephanie:

If they say no, they're blocked.

Stephanie:

So today.

Stephanie:

Simple and not particularly programmatic, meaning like, it's like it's a

Stephanie:

human to human one-to-one thing,

Brian:

And it's blunt.

Brian:

It's like

Stephanie:

it's a totally, but, and the idea behind that was if you add a lot of

Stephanie:

complexity into a market that has not yet been developed, you can just have lots and

Stephanie:

lots of fun with the complexity, but then you haven't really learned what the market demands.

Stephanie:

and.

Brian:

started with a fairly basic formula.

Brian:

Like it's like click through rate, you know, times like what was bi, I mean, it's like

Brian:

became much more sophisticated over time.

Brian:

Right.

Stephanie:

For sure.

Stephanie:

And, and then I think you can decide how much complexity do you want on marketplace versus

Stephanie:

how much complexity happens off marketplace and just turns into a reasonably simple back

Stephanie:

and forth between, you know, two, two parties.

Stephanie:

And so.

Stephanie:

We're, we would say exceptionally early in the development of this.

Brian:

Okay.

Brian:

So if you, if you think of the, the demand side in this as being the ai, companies, and

Brian:

then the supply side are the, the publishers, you, you start, I mean, it's in beta, right?

Brian:

Like it's in testing, so it's like, it's clo it's private, right?

Brian:

Like, it's not like it, anyone can, can go into this.

Brian:

I, it's, you know, I, I saw a lot of publishers.

Brian:

I know there, they're, they, they're big businesses.

Stephanie:

Yeah, the, we launched in private beta because that's the right way

Stephanie:

to launch a new product, which means you can request to be a part of it, whether you're

Stephanie:

on the supply side or the demand side.

Stephanie:

And so, you know, we do a bunch of things to, to make sure it makes

Stephanie:

sense and then we, we put you on.

Stephanie:

System with the expectation that you will, that it's not a set it and forget it, meaning

Stephanie:

you're gonna have to actively manage it today because it's not overly programmatic today.

Stephanie:

And the,

Brian:

publishers now are, are.

Stephanie:

the, the, the answer is, I don't know the answer to your question.

Stephanie:

I don't, I know how many, I don't know the answer to your question.

Stephanie:

Um, we have been very careful about.

Stephanie:

It doesn't make sense to send bad deals through our marketplace, if that makes sense.

Stephanie:

And so we're most of it today is still, even if it's gonna run through

Stephanie:

the marketplace, is the bilateral

Brian:

Yeah, this is a different, I mean, how are you gonna make money off this?

Brian:

Don't tell me you're not gonna make money.

Brian:

That would, that would worry me.

Stephanie:

the, a couple of things.

Stephanie:

One.

Stephanie:

CloudFlare is like in the business of having people on our network and

Stephanie:

us, you know, doing content delivery network and us doing bot management.

Stephanie:

And so all of those are

Brian:

Websites pay you app, like they pay you to provide these services to them.

Brian:

This is a different model.

Stephanie:

Yes.

Stephanie:

But to be clear, that is part like meaning in order to be on this system,

Stephanie:

you're running your, you're running your traffic through, through, through

Stephanie:

CloudFlare, which by the way, we both sides.

Stephanie:

Our customers kind of unrelated to, to this product, but, but also creates a better

Stephanie:

experience for people on our network.

Stephanie:

So that's one, but it's like a not irrelevant part of, of CloudFlare.

Stephanie:

The second thing is, you know, we will work through what the right economics are over time.

Stephanie:

I should think about it as a marketplace in terms of us like charging a fee for the

Stephanie:

facilitation of transactions on marketplace.

Stephanie:

We don't.

Stephanie:

To be clear, we will have plenty of publishers and content creators who ask us to do the blocking,

Stephanie:

so to do the enforcement, but we'll, we'll do deals like separate, large, privately negotiated

Stephanie:

deals that won't run through the marketplace.

Brian:

Okay.

Brian:

I mean, would it be like Stripe, how Stripe acts?

Stephanie:

The, I think the right way to think about it is more Spotify like

Stephanie:

those types of Spotify's a better.

Stephanie:

Analogy in terms of how to think about a marketplace, or any other place where you are

Stephanie:

getting paid a marketplace fee for creating the marketplace, doing the facilitation.

Stephanie:

The strike part is only the payments.

Stephanie:

And so yes, we will handle the, the payment side of that, but

Brian:

Yeah, I think I meant with the stripe in that like the, you know, the,

Brian:

the merchant controls the relationship with, you know, the person buying,

Stephanie:

Yeah, the merchant of Rec.

Stephanie:

Yes.

Brian:

Like, I just wanted to make sure, because like a lot of, you know, like Ronald

Brian:

Reagan had a thing, like, you know, when, when, he said the seven scariest words are

Brian:

like from the government, I'm here to help.

Brian:

Like I feel like publishers might have that, but it was just with technology and like, 'cause

Brian:

it's like uhoh, they're gonna get between us and, and maybe these are the next like customers.

Brian:

And I'm just wondering how that like would work, whether it's like you, the, the sort

Brian:

of AI companies are doing business with you and then you are like passing along.

Brian:

Or if it's like the directly, the publisher's working directly, or, or

Brian:

it's just a, a, a variety of models.

Stephanie:

the answer is yes, meaning all of it.

Stephanie:

Meaning the answer is all of it.

Stephanie:

And so we, we, we expect, see, believe that it makes sense for large publishers with

Stephanie:

large model companies to do deals directly.

Stephanie:

And those don't, none of that passes through a marketplace, but many of those publishers

Stephanie:

will use us to enforce those deals, meaning.

Stephanie:

They will make sure that the people they have deals with get through, and that they're

Stephanie:

using their data, meaning they're scraping, they're using the right crawlers in the

Stephanie:

right way, that's one, and that they'll block everyone else they don't have a deal with,

Stephanie:

which is a very important part of making this all work, but doesn't mean that payments

Stephanie:

run through in any way, shape or form.

Stephanie:

CloudFlare, that is CloudFlare really behaving in the traditional CloudFlare way in terms

Stephanie:

of what we do every single day, which is letting good traffic through blocking traffic.

Stephanie:

They don't want.

Brian:

Yeah, so I, No, I was gonna say, 'cause I wanted to get to Google for a minute.

Brian:

'cause they're, they're very unique, in this, in that, like, it seems like there's an asterisk

Brian:

with Google because like, there's one crawl.

Brian:

The way I understand it, like, you can't turn off Google.

Brian:

It's like the mo the person who's who's, you know, they could like be banging

Brian:

on the drum of like, we must stand up.

Brian:

It's like, well, Google's a little different because, you

Stephanie:

We were very careful with what we did on July 1st to treat.

Stephanie:

Everyone exactly the same.

Stephanie:

So, so Google has one bop but they have Google extended, Google extended, does both

Stephanie:

training for Gemini and grounding for Gemini.

Stephanie:

And so we, and anyone

Brian:

ground?

Brian:

I'm sorry.

Brian:

What's grounding?

Stephanie:

grounding basically is indexing.

Stephanie:

It's so the organization of the, the information, think about it in some

Stephanie:

ways as like search for, for Gemini.

Stephanie:

so what we did was.

Stephanie:

We block anyone who has a training only crawler in our defaults.

Stephanie:

We block them at the network level for Google.

Stephanie:

We inject a robots txt message, which instructs Google extended not to train.

Stephanie:

We have.

Stephanie:

Every evidence on the CloudFlare network is that they obey that instruction.

Stephanie:

We have no evidence that they do anything other than if you block, if you tell them to not

Stephanie:

train, therefore basically do the equivalent of blocking Google extended they, they do

Stephanie:

what you have asked them to do, which means in our default block, we treat them exactly

Stephanie:

the same way as we treat everyone else.

Stephanie:

Like interestingly, you know, perplexity, which there's been a bunch of back and

Stephanie:

forth, we don't, in our default block.

Stephanie:

Perplexed doesn't train, so they're not part of the default block.

Stephanie:

Okay.

Stephanie:

Then when an individual site owner is trying to figure out what to do with each bot,

Stephanie:

you're absolutely right that if they want to be in Google search, they, they also

Stephanie:

are in AI overviews, so they don't have the ability to say no search, but yes, yes.

Stephanie:

Search, no AI overviews, and that's where.

Stephanie:

The complexity is in terms of like figuring out how to deal with Google.

Stephanie:

We've been very vocal around our request for all bots that they tell us what IP addresses they're

Stephanie:

announcing from, so they verify who they are, they cryptographically verify so that we can confirm

Stephanie:

they are who they say they are when they are crawling, and that they're very clear on their

Stephanie:

intent and they listen to their instructions or the instructions of kinda the network.

Stephanie:

On being very clear on their intent means that separating things out between search, between

Stephanie:

things like rag ai overviews training, and so that's the thing that hasn't been changed

Stephanie:

yet, meaning some bots already do that.

Stephanie:

Meaning if you look at how open OpenAI runs, like things are happening with a different crawler,

Stephanie:

so you can easily pick one versus the other.

Stephanie:

Google you're stuck with.

Stephanie:

Other than Gemini training, you're stuck with everything else.

Stephanie:

If you want search.

Brian:

What?

Brian:

Gimme the optimistic take for, for what?

Brian:

What happens next?

Brian:

Because I mean, I, I think when we talk about.

Brian:

I almost think that there needs to be, like, they talked about that, like Mar-a-Lago

Brian:

accords, like with current, like there needs to be some kind of new economic bargain.

Brian:

And to me it starts with Google because that they're the, the center of of, of the internet.

Brian:

Maybe, maybe I'm wrong, but, what, what is it, what, what happens like

Brian:

that, that that changes this dynamic?

Brian:

is it like Google moving and saying.

Brian:

Look, we underst like we, we are going to cut a new bargain because we see they,

Brian:

they see what is happening and I know they, they gin up some studies that, that it's

Brian:

like telling people that it's not raining.

Brian:

It's like it's raining.

Brian:

Like every single publisher is dealing with this.

Brian:

You're not going to convince them otherwise.

Brian:

like what, what actually, like, I don't know.

Brian:

What's a tangible thing that will, that will happen that will show

Brian:

that there is progress being made.

Stephanie:

The single most important tangible progress that will be made is that they split out.

Stephanie:

Crawlers.

Stephanie:

And that doesn't mean they actually have to have separate crawlers, but they will allow

Stephanie:

you to be in search and indexing, but they will allow you to opt out of things like

Stephanie:

AI overviews and that one that creates a level playing field across the various AI

Stephanie:

companies and gives publishers the control, the control that they need to be able to say.

Stephanie:

Search was the grand bargain of the internet.

Stephanie:

Like that works for me.

Stephanie:

You get to scrape my site and you send me back traffic.

Stephanie:

But for things where you're producing a derivative, where the likelihood of someone

Stephanie:

coming back to my site is meaningfully less, that's a different bargain.

Stephanie:

And so I can choose to block you.

Stephanie:

Also, by the way, we can negotiate a deal, whether that's directly or programmatically,

Stephanie:

where you just pay me for that.

Stephanie:

But once you get to that world being different, the a business model can develop.

Brian:

So in like five years in this optimistic take, because I, I talked with Neil about this.

Brian:

I'm like, is, is, is the end goal that you're, Your revenue line for

Brian:

licensing is far bigger in five years.

Brian:

Maybe your advertising line is smaller, but that maybe you're more of a wholesaler

Brian:

than a retailer in some ways, and I'm meaning like you're wholesaling content

Brian:

for training to, you know, and you're not.

Brian:

You're not having that direct, you're, you're not as, as reliant on, say, programmatic

Brian:

advertising on all the affiliate and all, you know, yes, you're gonna be doing that,

Brian:

but that your revenue mix is gonna change.

Brian:

Like, is, is the, is the fundamental economics of the internet shift away from advertising then?

Brian:

In some ways, because if attention is finite and if, if people are increasingly going for

Brian:

this information and they're getting it from this, these centralized sources and all of the

Brian:

AI apps that are gonna be built off this, and I don't know if publishers are gonna do that,

Brian:

then it seems like the, the, the fundamental nature of these businesses will change.

Stephanie:

Yeah.

Stephanie:

Uh, projecting five years in the future is petrifying, but the um,

Brian:

I can't get any publisher.

Brian:

I asked him, I'm like, what does your business look like in three years?

Brian:

Nobody has taken me up on the offer to, to even like hazard a guess.

Stephanie:

I do think that, you know, subscriptions, one, you've seen a bunch of

Stephanie:

content creators with subscription models have a reasonably attractive business model.

Stephanie:

So you talked about ads, but you didn't say subscriptions, which we think is relevant.

Stephanie:

Two.

Stephanie:

Today the AI companies actually themselves have reasonable subscription models.

Stephanie:

So if you imagine a world where you're paying AI companies X, and so they earn X

Stephanie:

per monthly active user, and there's some amount of that that's going out to the content

Stephanie:

creators based on monthly active users.

Stephanie:

There is a little bit of this world of you win.

Stephanie:

I win.

Stephanie:

Like you're growing math.

Stephanie:

Monthly active users, and that's going to the AI company.

Stephanie:

So if you wanna call that, I mean, that's, you can call that licensing if you, if you would like.

Stephanie:

And so it feels like that is a, that is a world that that could exist.

Stephanie:

Whether people will figure out advertising, you know, we will, we will see, in terms

Stephanie:

of what, what that, what that looks like.

Stephanie:

The idea that we could get to a world that has less click bait and you

Stephanie:

know, is something that's, you know.

Stephanie:

Better, higher quality from a content perspective maybe means that you have

Stephanie:

some advertising, but not as much.

Stephanie:

But I think it's hard to predict exactly what it looks like.

Stephanie:

I think it's clear that the current, the status quo is unsustainable.

Stephanie:

Every content creator like ceasing to exist does not work for the world.

Stephanie:

A few companies deciding what content gets created sounds scary, and so figuring out a model

Stephanie:

that's more about like, what is the populous.

Stephanie:

Asking for, feels like where we're trying ahead.

Stephanie:

And that would mean, you know, publishers have a business model that that grows with

Stephanie:

the number of people who wanna read or wanna learn from the things that you've created,

Brian:

Yeah, I mean that is the goal, I guess.

Brian:

Like, I mean, because ultimately, I mean, I guess the sort of like the biggest like

Brian:

optimistic take is, you know, I feel like publishers a lot of times and, and you know.

Brian:

The incentives have been set to do lots of things that are outside of the core

Brian:

mission that will sell subscriptions.

Brian:

Like everyone has to gin up all kinds of different, you know, it's like, I used to

Brian:

make fun of all the Game of Thrones recaps.

Brian:

I mean, you had to do everything for these algorithms and it was distasteful to some

Brian:

degree, depending on your taste level, I guess.

Brian:

But it was like, eh, you gotta do what you gotta do.

Brian:

and you know, I think the, the ideal would be.

Brian:

That those kind of like hacks, you know, become less integral to a lot of these models.

Brian:

and yeah, there's more of a strong, subscription element to, to these businesses.

Brian:

I think the incentives of the, the ad supported internet a lot of times, you know,

Brian:

got, got publishers, going in, in directions that were not great, to be honest with you.

Brian:

cool.

Brian:

Well thank you so much for, uh, taking the time and, and, and walking through.

Brian:

This, I don't know.

Brian:

Any final thoughts like for publishers as, as they contemplate, their future, in an AI world?

Stephanie:

One, thank you for having me as fun to, to connect.

Stephanie:

We can do it in person next time across, um, the walkway.

Stephanie:

no, just that, we know that the things are changing rapidly, that lots is going

Stephanie:

on, but there, but one, like you can have transparency, you can have agency,

Stephanie:

you can have control, and it will take a while for the business model to develop.

Stephanie:

But, but it starts with making sure you know what's going on on your site and you,

Stephanie:

you take control over who's, who's on it.

Stephanie:

We're

Brian:

Okay, cool.

Brian:

Stephanie, thank you so much.

Brian:

Really appreciate you, uh, taking the time.

Stephanie:

Awesome.

Stephanie:

Good to see you.

All Episodes Previous Episode

About the Podcast

Show artwork for The Rebooting Show
The Rebooting Show

Listen for free