Related resources
Transcript
0:00
Cliff Crocker
Well, I'm going to go ahead and sort of uh kick us off a little bit. Um, we're really really excited. Tammy and I have been looking forward to this um conversation for uh several weeks now.
0:11
Cliff Crocker
Uh because it's not often that we get to kind of sit and talk with folks like this outside of a Perf Now conference or
0:18
Cliff Crocker
in the hallways of a of a web performance track or front-end performance track. Um but we get three amazing people and amazing humans in one
0:27
Cliff Crocker
room to uh basically um answer questions and talk to us for the next hour. So our focus is going to be on the future of
0:36
Cliff Crocker
front-end user focused observability. Um we've got some great panelists that I'm going to ask to start introducing themselves in a second. Um, my name is
0:44
Cliff Crocker
Cliff Crocker and I am working on products for uh now for PaloAlto Networks where I'm focused on digital
0:52
Cliff Crocker
experience management uh specifically ramen synthetics um building out those product lines. Previous to that was at
0:59
Cliff Crocker
Embrace. It was recently acquired by Paulo and prior to that was Speed Curve which was recently acquired by Embrace.
1:06
Cliff Crocker
So, a little bit of a Russian doll happening but um we're along for the ride and having a great time. So, thanks everybody for joining and I'll go ahead
1:14
Cliff Crocker
and maybe could get to Tammy to introduce herself first before we let our distinguished panelists kick it off.
1:20
Tammy Everts
Yeah, I'll go real quick. Yeah. So, like Cliff said, so excited. I was so thrilled when everyone you see on your screen um agreed to join this panel.
1:30
Tammy Everts
Like what a what a gift. Um great way to start the week. Um I'm Tammy Evers. Uh I also was part of the speed curve to
1:39
Tammy Everts
Embrace to Palo Alto Networks uh migration. Uh and uh if you if you ever want to know what it's like to be
1:47
Tammy Everts
acquired two times in a year, come and uh come and find me. I'll let you know.
1:51
Tammy Everts
I have three laptops. Super exciting. Um uh but yeah, I'm part of the digital experience strategy team at PaloAlto. So
1:59
Tammy Everts
yeah, just really excited about where that's going to go. And I am going to kick it over to Annie to go next.
2:06
Annie Sullivan
Hi everybody. I'm Annie. I'm an engineer on the Chrome web ecosystem team and uh I lead the core web vitals project. Uh
2:14
Annie Sullivan
as for what I've been working on lately, uh we've been obviously really excited to continue working on integrating soft navigations in the core web vitals. Uh
2:22
Annie Sullivan
we've launched uh soft navigations in the performance timeline in Chrome 151 and now we're working on uh integrating into the Chrome user experience report.
2:30
Annie Sullivan
uh and uh as far as the rest of what I do, I've been actually spending a lot of time taking a step back and trying to understand uh the health of the web at
2:39
Annie Sullivan
large and what's going on as we see this big AI shift on the web. Uh how are you know what websites are succeeding uh
2:47
Annie Sullivan
what is changing in the way websites are developed things like that. How about over to you Alex?
2:57
Alex Russell
Hi good morning. I'm Alex Russell. I'm a what do they call me this week? A product architect on the edge team at
3:04
Alex Russell
Microsoft. Um madeup title. Uh mostly I go around the company and try to help folks with the performance of the web applications which if you've used
3:11
Alex Russell
Microsoft products um is a target-rich environment. Let's leave it there. uh and on the other sort of side of my uh
3:19
Alex Russell
my work I focus on uh browser competition particularly on mobile u because you know most of the computers that uh exist on the client side at
3:27
Alex Russell
least are um mid to low-end Android devices that are not um this year's model. So um that's a that's a place
3:35
Alex Russell
where you know mobile being most of computing still kind of drives my thinking.
3:42
Alex Russell
Paul.
3:44
Paul Calvano
Hey everybody. I'm Paul Calvano. I'm a performance architect at Etsy. Uh so I make sure Etsy's uh marketplace is fast and stays that way. And that's primarily
3:53
Paul Calvano
what I what I do there. Um I spent most of my career working on web performance.
3:57
Paul Calvano
In fact, I am like almost embarrassed to say how long ago I worked with Cliff. Um because it will just make me sound old,
4:05
Paul Calvano
so I won't. Um but we go back. Um so I I also do a lot of analysis with the HTTP archive. um part of the admin team for that as well. Uh my current focus at
4:14
Paul Calvano
work is I've been uh trying to get better visibility into the user experience for our iOS and Android
4:21
Paul Calvano
mobile apps. Um like I'm looking at various uh performance metrics, trying to work with the teams to define new ones and um really just kind of get a
4:30
Paul Calvano
get a better understanding of what we can do to make that um that platform faster.
4:37
Cliff Crocker
And for anybody really wanting to know, I think it was 2006 that Paul brought me in and taught me how to break websites for a living. So systems, it's great.
4:46
Cliff Crocker
Great stuff.
4:50
Cliff Crocker
Um, thanks everybody. So, uh, if folks have questions as we go, uh, post them
4:56
Cliff Crocker
in the chat and we will, I don't know, we'll try to answer them organically. Are we kind of keeping one eye on them?
5:04
Cliff Crocker
Um so let's kind of dive into it like you know the elephant in any room when we talk about observability is uh core
5:11
Cliff Crocker
web vitals so it's a good place to start. Um, so it's been 6 years, 6 and a half years since uh, Coral Vitals were
5:18
Cliff Crocker
released and I would just really love to have everyone's take on um, uh, you know, kind of what we've learned. Um,
5:26
Cliff Crocker
because there's been a lot obviously and we could do a whole hour on that and, uh, importantly kind of where vitals are headed and what everyone's kind of
5:35
Cliff Crocker
biggest maybe biggest learning moments or or or biggest surprises have been. So Andy, would you mind uh, taking this one first?
5:42
Annie Sullivan
Uh, sure. uh as far as like biggest learning moments or biggest surprises.
5:46
Annie Sullivan
Uh, as far I just want to talk about like I also go back way back with Quick Cliff and I remember in 2019 him giving
5:53
Annie Sullivan
me some pretty direct feedback that we needed to uh to meet websites where they're at and to really listen uh deeply to developers and I think you
6:01
Annie Sullivan
know we really took that to heart and we've been really trying to you know like really understand uh what
6:09
Annie Sullivan
businesses and developers are trying to do on the web and really relate to that and and be empathetic. And I think that's been like the biggest thing
6:16
Annie Sullivan
that's kind of helped us, you know, uh, keep going, uh, and keep everybody going along with us. And I also think like as
6:24
Annie Sullivan
part of that, really focusing on consistency and simplicity has been our biggest learning. We actually thought like seven or eight years ago that we
6:33
Annie Sullivan
were going to introduce like three new metrics every year and have this like sliding matrix of things that were going to happen. And and I think that's been like I I think it's so obvious in
6:42
Annie Sullivan
retrospect, but it was probably the biggest surprise or learning for me is that we have to keep it very very simple, straightforward, and very consistent. Uh as far as what's next uh
6:51
Annie Sullivan
from what we're thinking about and we we do really love feedback and we love to hear from the community. So please come talk to us. But what we're thinking about is obviously integrating soft
6:59
Annie Sullivan
navigation. So if you have like a single page app and you do a transition in JavaScript, being able to uh show that user experience the exact same way as a
7:07
Annie Sullivan
regular navigation, uh that's what we're like immediately working on next. And we have other really interesting things that we're looking at in the performance
7:15
Annie Sullivan
timeline that that kind of unlocks around container timing and interaction to contentful paint. But we're also looking at uh we're doing a ton of user
7:23
Annie Sullivan
experience research about agentic browsing and co- browsing and trying to understand what are the user experiences that the user is seeing and then also
7:33
Annie Sullivan
trying to understand at the same time like what did developers and businesses actually want right when they're building these experiences or when they're you know actively not building
7:41
Annie Sullivan
these experiences uh and just trying to get a big picture there. So those are the kind of big things that we're
7:48
Annie Sullivan
thinking about. I guess one other thing we're thinking about is like more places we can kind of meet business and developers where they're at. Like where could we be showing these metrics are
7:57
Annie Sullivan
are helping especially like there's already a lot of work on modern web guidance and uh dev tools for agents uh
8:05
Annie Sullivan
in the AI space but also other places where we can kind of highlight to businesses how they're doing.
8:10
Cliff Crocker
Do you see any changes coming for the the thresholds themselves, Annie?
8:16
Annie Sullivan
I I still think that you know if you look what we really wanted again going back to this simplicity anchor is to
8:24
Annie Sullivan
make sure that everybody can meet the threshold and while you know websites are doing better and better and better and more than half of sites meet all the
8:32
Annie Sullivan
thresholds we still see a lot of places especially like Africa uh or low-end devices and it does look like next year
8:39
Annie Sullivan
devices are going to have less RAM you know due to rising costs and things like that where it really isn't quite possible to meet the you know to go lower on the thresholds.
8:49
Cliff Crocker
So is the so sorry I just I so interested in this. So it basically is the the thinking that when
8:58
Cliff Crocker
most like all of us probably in possibility sites are hitting the thresholds that's basically the trigger point for examining that because the thresholds themselves are not goals.
9:06
Cliff Crocker
They're they're kind of a bar.
9:08
Annie Sullivan
Yeah. So it's over half but it's nowhere near all sites that are meeting them. uh it's doable to meet the thresholds right now as they are uh but we also need to
9:17
Annie Sullivan
look at the other side is if we lowered the thresholds would it be possible for all sites and there's a lot of conditions in which it's not possible for that site and so I think what what
9:26
Annie Sullivan
I'm hearing from a lot of the community is they're just you know products like speed curve people like Alex are saying hey we can on our site go lower let's
9:33
Annie Sullivan
look at a higher percentile let's look at a lower threshold um so I I really definitely support that when it's
9:40
Annie Sullivan
possible but it's really hard to to balance that with needing some to very clearly state like just something very crisp that we can just say to everyone.
9:49
Cliff Crocker
That makes sense.
9:51
Cliff Crocker
Kind of like going there a little bit because you you you touched on it about more of like some of the the constraints that are out there that are, you know, sometimes beyond a developer's control.
10:01
Cliff Crocker
Not sometimes, a lot of the times beyond a developer's control. This is obviously an area where Alex has has really been carrying the torch for a while around
10:09
Cliff Crocker
performance and equality. And I know I'm sort of jumping right into that, but it seems so related. Alex, do you feel like
10:16
Cliff Crocker
um things are getting better? Do you feel like core web vitals are something that um folks that are maybe in an underrepresented part represented part
10:24
Cliff Crocker
of the world are actually able to make um headway and traction or are there other barriers that we need to be thinking about?
10:31
Alex Russell
Um I should just say first and foremost, thank you to Annie, right? Like that's just let's just all say thank you.
10:37
Alex Russell
[laughter]
10:39
Alex Russell
um we wouldn't be in nearly as healthy a place as we are today uh without a series of insights some of which aren't
10:46
Alex Russell
public u that Annie delivered both um you know to us all and to the team at Google um and um we're all kind of you
10:54
Alex Russell
know standing in the shadow of that kind of towering research. So thank you. Um and and my perspective has been uh sort
11:01
Alex Russell
of more narrative oriented I suppose and a little bit less kind of unfailingly data driven right like call it a personal
11:10
Alex Russell
failure but like I think that's kind of where I come at this from and so when we started working with teams that were attempting to build PWAs in 2014 2015
11:17
Alex Russell
2016 what we were seeing was a set of folks bringing their desktop oriented
11:26
Alex Russell
assumptions with them to mobile in a way that was fatal to their products. And that was not an intent on their behalf,
11:33
Alex Russell
but it was a nar it was an overhang of stories that worked once upon a time, right? That CPUs always got faster, that
11:41
Alex Russell
networks were going to improve to the rate that they had in the past and that we were all going to live in this kind of post scarcity future. And to the best
11:49
Alex Russell
I can tell, that has never panned out in the last, you know, 10 to 15 years. um about the time that phones started to
11:56
Alex Russell
get really fast for folks like us um they started to you know replace uh feature phones with smartphones for
12:04
Alex Russell
everyone else and the the replacement devices didn't get faster they just got cheaper and and so you asked sort of like what is what does inequality mean
12:12
Alex Russell
one of the interesting things about the RAM crisis now is that um wealthy folks are actually getting hit a little bit
12:20
Alex Russell
right like it's been possible in a sort of financialized device and carrier world to not really think about the fact that you're paying a h 100red bucks a month and 30 of that
12:28
Alex Russell
is a you know a highinterest um loan that you are paying back to a carrier for your latest gen iPhone and so what
12:37
Alex Russell
has happened has been that as the performance of the high-end has delaminated from the loan like the high end keeps doing this and the low end keeps doing this right [laughter]
12:44
Alex Russell
and um one of the depressing things for me has been that every year for the last five or six years the the improvement of
12:51
Alex Russell
the high-end has been larger than the potential gains at the low end. And so in absolute terms, every year that a
12:58
Alex Russell
person replaces their phone in one of these cohorts, the gap grows, right? The difference in feeling between them grows. The good news is that the
13:06
Alex Russell
networks have improved. Um, and that's going to continue, right? That seems to be going a pace. Um, and maybe there is going to be some renewed focus at the
13:14
Alex Russell
high end on certainly memory. And I'm seeing that of course in my work. Um, which is which is great. It's very salutary, right? Like you know we sort
13:21
Alex Russell
of always tell people for a long time moving to HTML first moving to multi-page applications moving away from
13:28
Alex Russell
heavyweight JavaScript you know getting rid of your npm uh uh addiction is going to improve everything including memory
13:36
Alex Russell
and people would be like everything I don't care about memory uh suddenly it's a problem. [laughter] So um I guess that's a nice feature of the current
13:44
Alex Russell
crisis uh if you had to find one. But um is it improving? Yes. Is it improving fast enough or rather is it improving
13:52
Alex Russell
quickly enough to soak up the amount of um emissions that the JavaScript ecosystem continues to produce at pace?
13:59
Alex Russell
Absolutely not. Um so one of the things that we're seeing is that you know Annie said that you know half of websites are above you know are getting getting passing core vitals um but the pace of
14:07
Alex Russell
progress is slowing. So you know there is a there's a um a bit of the kind of residual that is not improving. And so I
14:16
Alex Russell
would suggest that browsers have uh more to do here to increase the browsers, search engines, you know, the ecosystem as a whole have more to do here to
14:24
Alex Russell
increase the strength of the interventions and to um make the failure uh more front and center for users,
14:31
Alex Russell
right? So that they blame the site rather than the browser.
14:37
Cliff Crocker
I think that that's one of the things that's always amazed me is like yes, like your point of like, hey, hardware acceleration for, you know, those of us
14:44
Cliff Crocker
that are privileged enough to have a high-end iPhone or whatever. Yeah, things get faster doesn't really for everyone else. But then when I think about things like the HTTP archive and
14:52
Cliff Crocker
looking at Paul a little bit here, one of the things that always just struck me as just mind-boggling was that we
14:59
Cliff Crocker
weren't getting better. Like our best practices were like getting worse. like all of a sudden like like more and more and more requests like Steve says like
15:06
Cliff Crocker
whatever it was 15 20 years ago like hey make fewer HTP requests and everybody goes the opposite way like we say hey
15:14
Cliff Crocker
use less JavaScript be more efficient with JavaScript like you know um watch the trap of some of these frameworks and it's like like like all over the place.
15:22
Cliff Crocker
So I I don't know Paul if that still continues to hold any of the work that you're doing as you kind of look through the HTTP archive and and I mean what a
15:30
Cliff Crocker
great source of data as well and thanks again to you for stewarding a lot of that and and and uh keeping it fresh and relevant. Are you seeing the same
15:38
Cliff Crocker
pattern or do you am I wrong and are we starting to see things turn around?
15:43
Paul Calvano
Um well so the HTTP archive is is not um is not measuring performance like Krux is. Krux is measuring the performance over time of of the web. the HTTP
15:52
Paul Calvano
archive is measuring how it's built and like there's one of the reports on the HTTP archive uh um website is page
16:00
Paul Calvano
weight and I know Tammy you've written about that a lot is in the chat yes he's written about it a lot as well the page
16:08
Paul Calvano
weight is not going down it has never gone down it is is just continuously rising it's almost like this hockey puck stick and I actually was looking at it
16:16
Paul Calvano
this morning and the first um the first time the HTW archive captured mobile
16:22
Paul Calvano
data. The median desktop web page was 524 kilobytes.
16:30
Paul Calvano
Uh it was 202 kilobytes for mobile, right? And we are over 20 like like 13
16:36
Paul Calvano
times that now for um uh and it just it just continues to increase. And and I I kind of like you know chalk it up to the
16:45
Paul Calvano
the Jevans paradox like as as capacity expands business priorities fill the gap. So like whatever capacity we're getting from all the new hardware that
16:53
Paul Calvano
gets absorbed into feature work and as we've seen from Alex's work the it's almost assumed that everyone has a new
17:01
Paul Calvano
device and then people on the lower end really really suffer to um and and to go back to like like
17:09
Paul Calvano
Alex you had mentioned before the uh um you just the general appreciation we all have for Annie and her and her team's work on on core vitals like I it's for
17:19
Paul Calvano
me that is such a um like the the the the biggest achievement that I kind of feel that core web vitals made was that
17:26
Paul Calvano
it put that business impact of performance spotlight out there. And like when core web vitals first came
17:33
Paul Calvano
out, I was I was doing consulting. I was working for ACMI and um I remember like right after that became like a topic
17:42
Paul Calvano
that like the performance and the SEO industry were starting to talk about.
17:46
Paul Calvano
Um, I would walk into meetings and CEOs and CTO's would ask me about largest Contentful Paint like unprovoked and it
17:53
Paul Calvano
was like that that type of um, you know, beyond just being cool as a performance nerd. Um, when executives understand the value of performance, they invest in it
18:01
Paul Calvano
and that's that's where the change happens.
18:05
Cliff Crocker
Yeah, I I I saw that as well, you know, just suddenly we were talking to different folks. We were talking to um like CEOs, talking to product
18:14
Cliff Crocker
managers, VPs of product, you know, SEO teams, you name it. So, it was just like, you know, well, you you guys know
18:21
Cliff Crocker
because you're more familiar with my work, but for anyone else, like I'm all for expanding the umbrella and I think everybody who touches a page should
18:28
Cliff Crocker
understand the impact of what they do on that page. And so, Corwood Vitals is like just a massive step forward for that. So, yeah, thanks Annie. I'm just
18:36
Cliff Crocker
going to like add my thank you onto the pile. Um I'm curious to know sort of you sort of touched on this Paul but like um
18:44
Cliff Crocker
like what's a Cliff Cliff kind of mentioned this as well like you know Steve recommends all recommended all these Steve Saers for anyone who's not
18:52
Cliff Crocker
familiar with his work recommended um all these performance improvements so like like literally textbook
18:59
Cliff Crocker
improvements. So, what are some textbook improvements that um that
19:07
Cliff Crocker
have like have they did they survive contact with real sites today I guess is what I'm asking.
19:14
Paul Calvano
Um, a lot of them have. Some of them some of them have been implemented um, unsuccessfully, but a lot of them have.
19:22
Paul Calvano
And I think that a lot of the um, the optimizations that are that are out there when you think about web performance best practices, you can often find that they're, you know, adopted by a large portion of the web.
19:32
Paul Calvano
Like look at look at lazy loading. That was so easy to add. as soon as um as soon as Chrome you know added support
19:39
Paul Calvano
for that um Safari followed a little bit later but like I think it's something like 50% of the web is using lazy
19:47
Paul Calvano
loading right now and that's um uh you know incredible when you think about the fact that it used to be like uh like a third party like plugin that you would
19:56
Paul Calvano
need or um some JavaScript that you would need to write now it's just native in the browser um I feel like operational complexity just kind of goes
20:05
Paul Calvano
hand inhand with performance performance optimization. So like if you you know if a performance win can be obtained easily there's little risk to experimenting
20:12
Paul Calvano
it's much faster to experiment with and adopt and that's why solutions like that that are kind of baked into the browser I feel like those wind up getting uh
20:20
Paul Calvano
getting a lot more traction and and if it takes a lot of engineering effort um it's it's harder. Um the uh so at Etsy I
20:30
Paul Calvano
I recently implemented a performance optimization. If anybody's talked to me at conferences this year, they probably heard me talking about it. I was so
20:37
Paul Calvano
excited about it. Um I was able to significantly reduce our JavaScript and CSS size. Um I decided last month not to ship it. It is not live in production.
20:48
Paul Calvano
And this is like the the reason was um so at Etsy we use we use real user monitoring and we also run an experimentation platform. So I won't
20:57
Paul Calvano
make a major platform change without making sure that you know it's the users are telling me the same thing the metrics are telling me. And what I found
21:05
Paul Calvano
was that you know the user behavior didn't match the performance expectations. Um and so I'm planning on revisiting it.
21:13
Paul Calvano
I have some theories and hypothesis as to what I can do to make this have the impact that I was expecting. But for now, I wasn't willing to ship something
21:21
Cliff Crocker
that adds a ton of operational complexity when there was literally no business value to do it.
21:30
Cliff Crocker
It's a great point and it's a hard one for us to stomach sometimes, right?
21:33
Cliff Crocker
Because we always want faster to be the thing and faster is always better. But the truth is like the bigger message is listen to your users and you know is
21:42
Cliff Crocker
[clears throat] it impacting their behavior to to make a lot of those decisions. Um interesting. I would definitely want to hear more about it.
21:48
Cliff Crocker
But you but you've just parked it for now, right Paul? Like is you're still the work's done. We can flip the experiment on like very soon. But right
21:56
Paul Calvano
now, you know, as you know, an e-commerce site, we're prioritizing getting ready for holiday and I'll probably look into it some more afterwards. I think that's the thing
22:04
Paul Calvano
that it's I've always loved about Etsy like from the early days like 15 16 years ago when I was getting started in performance like the fact that you have
22:13
Paul Calvano
an environment and a culture that is about experimentation and you just you you're like pitbulls you just don't you
22:20
Cliff Crocker
don't let stuff go and you'll come back to it and kind of keep poking at it in a lot of different ways which I think is really commendable and it's and is I
22:29
Cliff Crocker
mean it's it's why you're so fast so it's a really fun culture and and I I like I I actually first learned what
22:36
Cliff Crocker
Etsy was at the Velocity conference like that's how much of a nerd I am that like like because it was probably like maybe the first or second year that Etsy even
22:45
Cliff Crocker
existed. But I was like I I became so like like impressed by their by by their engineering team and what they would
22:53
Cliff Crocker
come to conferences and share that eventually I yeah eventually I became um a consultant for them and then I just joined the team.
23:02
Paul Calvano
[laughter] That's Did you just like show up and just be like, "Okay, I'm here now." Like, "Yep, kind of." [laughter]
23:13
Cliff Crocker
Um, so this is something that, um, I will freely admit might be a little bit off topic from what we usually talk about in
23:22
Cliff Crocker
these rooms. Um, and that's the topic of mobile. And when I say mobile, I had to kind of learn this that when people say
23:29
Cliff Crocker
mobile, sometimes they're not talking about mobile web, they're talking about native applications. So, we've, you know, through one of our recent
23:36
Cliff Crocker
acquisitions started working on um really monitoring for native applications. And to me, it kind of opened up a whole new world. And I'll
23:44
Cliff Crocker
admit, I was a little sensitive because I'm a big fan of the open web and and always have been at the same time. I've got more apps than I know what to do
23:53
Cliff Crocker
with on my phone. So reality is as a user I use apps all the time and so do a lot of people and so do a lot of our customers that are doing both. Are we
24:02
Cliff Crocker
seeing any best practices or thoughts that we have around web starting to be applied to mobile? I mean it's years
24:10
Cliff Crocker
later but what are we doing in that uh aspect of the world if we are doing anything at all? And maybe Paul, I know that you've touched mobile a little bit
24:18
Cliff Crocker
more before I have diff more more difficult conversations trying to pry this out of Annie and Alex a bit.
24:25
Paul Calvano
So, um, if like mobile, like specifically native mobile, like I found that there's so much overlap like in the time that I've been spending working with the mobile teams and and digging
24:34
Paul Calvano
into performance, there's so much overlap between what, you know, we're doing with web browsers and the web and what like the way that we analyze mobile
24:43
Paul Calvano
um native mobile. Um, however, the tooling just isn't there, right? The metrics are the the metrics are also, you know, substantially different.
24:51
Paul Calvano
There's no concept of a page. You've got screens and then interactions with the screens. Um, the but the fundamental
24:59
Paul Calvano
principles of performance I find are the same. So, like I might find an optimization for, you know, compressing something better or speeding up an API call or caching something or hey, what's
25:08
Paul Calvano
this third party doing? Right? Like all of that I I see in the in the same vein.
25:12
Paul Calvano
But like I kind of feel like the um the ecosystem for understanding
25:19
Paul Calvano
performance on native mobile is um is kind of like almost 10 years behind or
25:26
Paul Calvano
more behind web. Um back uh I I remember back before Chrome DevTools and before
25:33
Paul Calvano
webpage test um I used to analyze websites by running them through a proxy, right? and you'd look at a proxy
25:40
Paul Calvano
recorder in order to really see like what the HTTP request and response headers looked like. Um, I'm doing that with native mobiles. I actually have a
25:48
Paul Calvano
proxy that's running on my phone that I'm able to like intercept the traffic and and see what's going on. Um, but like that um but like beyond that like
25:58
Paul Calvano
I'm taking that data and I'm throwing it into tools like waterfall tools or just dropping it into Chrome DevTools. I'm able to use like modern tooling to
26:06
Paul Calvano
analyze native mobile. some like I'm I'm kind of bridging that gap where I'm like looking at how you know or am I finding
26:14
Paul Calvano
duplicate JSON? Am I finding like things that should be um should be compressed differently? Are there third parties that are blocking um which isn't really
26:22
Paul Calvano
a concept on mobile native mobile? Um, but it all depends on how you've coded it. And like a lot of these a lot of
26:29
Paul Calvano
these aspects of um of native mobile analysis really are kind of inspired from what we've done on the web for so many years.
26:39
Cliff Crocker
Yeah. And I think there's still some of those things that'll translate over, but I I don't really know enough yet to to be dangerous even. But there is still
26:47
Cliff Crocker
the concept of like, you know, blocking on the on a main thread and and things that we have to sort of think about the contention there. Um but a little bit
26:55
Cliff Crocker
yeah obviously layered different and and much more complex. Annie, does any of this start to translate as we talk about
27:02
Cliff Crocker
things like core web vital and then Android vitals and other things that initiatives that might be happening within Google? Are are your worlds
27:11
Annie Sullivan
colliding at all? Do you all collaborate in this area or I think there's a lot of opportunities to really build more bridges. Uh we
27:19
Annie Sullivan
haven't really deeply collaborated in the past. Uh I think a lot of the time like just specific goals about user experiences can be really different. Um
27:27
Annie Sullivan
the platform always has to focus on things like power, right? Uh where we want to be thinking about page load and they don't like Paul said have a concept
27:35
Annie Sullivan
of page load, right? So I I think it's a really interesting time to be thinking about that about um user experiences as
27:41
Annie Sullivan
you know uh there's a lot of change just happening in the way people are interacting with their devices in general.
27:48
Cliff Crocker
Alex, you dropped a good comment in the in the chat. Did you want to elaborate on that? Because I am interested in learning more about like what's performance culture amongst mobile
27:57
Cliff Crocker
engineers. It sounds like they're just by necessity required to care about performance more in a different way.
28:04
Alex Russell
Yeah, I would just say that um when mobile was born, right, like you know out of out of the bones of Nokia series 60 and series 90, right, you got the
28:13
Alex Russell
iPhone and then you got Androids and the people who gravitated to that universe of development took the lesson early on that to succeed you had to do a good job
28:20
Alex Russell
on a very lowspec machine, right? And so they, you know, if you sit down with a mobile engineer, what you will see are folks who've got phones littered across their desks, lots of them, right?
28:29
Alex Russell
Because that's that's just that's just how you got to roll, right? [laughter] like because because that's that's what that's the target, right? That's that's
28:37
Alex Russell
you know those are the users and um it helped a lot that the tool chains that the mobile oses presented were
28:46
Alex Russell
incompatible there were there was a functional incompatibility with the legacy desktop ecosystem right so um Apple could have taken universal
28:54
Alex Russell
binaries right if you remember those right like the things that let you go from uh you know what was it uh power
29:02
Alex Russell
architecture to Intel and they could have added ARM for mobile, right? And they you could have, you know, used Coco
29:10
Alex Russell
and and all the stuff from your desktop world and brought it all over. It wouldn't have felt good. So, what they did is they they cut hard over, built a new set of thinner things and forced
29:18
Alex Russell
everyone into an incompatible moment where they had to rebuild effectively from scratch. So, they couldn't bring the legacy with them. Whereas on the desktop, our culture was it's all fine.
29:26
Alex Russell
I guess this is just additional TAM, whatever. We're just going to keep doing what we were doing. And what we're doing was catastrophic, right? But I just want to be very clear like the the what Paul
29:34
Alex Russell
said earlier about you know the initial web page you know weighing 200 kilobytes and that would have been in 201 10 2009
29:43
Alex Russell
something like that um you know that that that was a catastrophic payload for a 2G network right like and and the good news at that time was the people who
29:52
Alex Russell
were getting smartphones were relatively wealthy we hadn't yet got entered into the Cambrian explosion to the low end but that was obviously on the horizon um as Android was starting to gain share um
30:00
Alex Russell
and so what happened the the interplay was that um web developers expected that the future would continue to expand before them as CPUs got faster and
30:09
Alex Russell
networks got faster and starting from a high cost baseline and mobile developers said well we can't do any of what we did before so we're going to rebuild and
30:16
Alex Russell
focus heavily on this and culture has carried mobile a very long way to the point that like to your experience Paul
30:23
Alex Russell
of the metrics being 10 years behind that that's partially out of necessity in both in both cases right we haven't had to build those metrics in maybe to
30:31
Alex Russell
maybe the same rigger that Annie and and friends have built them in Chrome. Um because the culture of mobile has been
30:38
Alex Russell
so much healthier in terms of satisfying what users need when they need it and building to the constraints of the device. Now that's obviously not
30:46
Alex Russell
universal and of course mobile is heavily segmented. Um high-end users are all in the top 20% of sales. They're all on iPhones. Those are all you know a
30:54
Alex Russell
class ahead in every way from the mass market which are lower. But because of that, again, because of that segmentation, no one who's building an iOS app thinks that it's going to run
31:02
Alex Russell
well on an Android, right? Like whereas web developers are confused. They all think that building on top of their their MacBook Air or Mac, you know,
31:09
Alex Russell
MacBook Pro, uh, is going to be fine on the Chromebook that costs, you know, two to three hundred bucks new. Um, and of course, uh, no matter how good Chrome OS
31:17
Alex Russell
is, you you can only do so much to make up for the silicon.
31:24
Cliff Crocker
You kind of blew my mind a little bit, Alex, because I've been [laughter] I've been thinking, you know, oh, like mobile is behind because they don't have
31:31
Cliff Crocker
[clears throat] these sophisticated metrics and things like that. And you've kind of made me think like web is behind because [laughter]
31:38
Cliff Crocker
right, we're not we're not thinking about users or you know, like I mean we're trying to, but like not everybody's thinking about users as being as kind of
31:47
Alex Russell
intersectional and nuanced as they actually are. So yeah, I don't want to oversell this. I I don't want to say that mobile is like a you know pang glossian future. I just I I
31:56
Alex Russell
only think that there is a a distinct difference in culture that has carried these communities of practice into different places. And if you go to an
32:05
Alex Russell
average web developer or an average mobile developer and you say X is a problem, you will get a different response in my experience. Um and that
32:13
Alex Russell
is a um that is a a negative aspect of the web's development culture today. Yeah.
32:20
Cliff Crocker
That truly is great perspective and honestly I hadn't even framed it in my own mind that way but we do tend to say like yeah mobile's behind mobile's not
32:29
Cliff Crocker
you know keeping up they don't have the metrics we have look at us but at the same time those experiences are probably profoundly better than most of the
32:37
Cliff Crocker
experiences that we're serving uh on the web because of those constraints that's actually a really interesting framing well I I will say just as a as as a
32:46
Tammy Everts
little caveat though as somebody who lives semi rurally using my apps as soon as I leave my town is
32:55
Tammy Everts
it's like one bar on 5G. What does one bar on 5G even mean? I don't know. I just know it doesn't work. So, um, yeah,
33:03
Tammy Everts
it's it's it's it's interesting to see, you know, how apps come up against like
33:10
Tammy Everts
real world constraints like that because I love the I love the idea of a mobile engineer with their desk covered in
33:19
Tammy Everts
devices, but I want them to come to my house and then like drive 20 minutes out of town with me.
33:24
Tammy Everts
[laughter]
33:25
Cliff Crocker
They need a mobile lab that's like, you know, I'll just hang out with them and we'll go on a road tour and I'll show them how things work in Canada.
33:32
Cliff Crocker
[laughter]
33:35
Cliff Crocker
Um, some really good comments in the chat and I'm trying to keep up and I think we're probably going to have to do a little recap on some of these to yeah,
33:43
Cliff Crocker
to get them answered. But one that was um kind of perked my ears up a little bit because this is an area that Tammy and I are really kind of digging into
33:51
Cliff Crocker
right now was hey it'd be great to have web vitals kind of standards for mobile iOS Android screens. Um, and that is something that we've been looking at and
34:00
Cliff Crocker
really kind of digging into. Um, and trying to take a lot of the same principles that Annie and team and others from outside of Google that
34:08
Cliff Crocker
really kind of participated in Core Vital had, but bringing that to mobile to say like what is that thing that we can start to optimize for um, you know,
34:16
Cliff Crocker
responsiveness and uh, screen load time versus like an actual page load, those types of things. So I like the thinking there and I think that's definitely
34:25
Cliff Crocker
something that I personally am investing a lot of time in just to try to understand without forcing a paradigm on
34:32
Cliff Crocker
a whole group of developers that as we just heard are probably a little bit ahead of where we thought they were. Um so I don't know that's interesting and I don't know if you want to hit on that
34:40
Cliff Crocker
one at all Tammy or anyone else has thoughts about the concept of of building core mobile vitals out in the open like we did with web vitals.
34:49
Tammy Everts
Well, I'm just gonna give a little shout out to the fact that Cliff's going to be talking about this at Perf now. So,
34:55
Tammy Everts
[laughter]
34:56
Tammy Everts
um, so there's going to be a really interesting talk about, um, uh, kind of just the journey to explore what core
35:03
Tammy Everts
mobile vitals can mean. So, we can definitely talk more about it today, but just I was gonna say follow Cliff and, uh, you know, watch this space.
35:12
Cliff Crocker
Yeah. And don't throw tomatoes because I'm talking about mobile performance at what has typically been a web performance conference. But
35:19
Cliff Crocker
well, it's a performance conference. So, you know, true. [laughter] Oh, Dan's uh Dan's got some good
35:28
Cliff Crocker
comments in the Oh, cool. Just about pushing P75 thresholds. So, that's cool.
35:35
Annie Sullivan
Yeah, I can talk about the percentile. I think that that's a feedback. So, uh core vitals as as people are probably aware, right? like your real user traffic follows a distribution and we
35:43
Annie Sullivan
look at the 75th percentile which is you know like 75% of your users are at this value or better and many many performance experts would love to push
35:52
Annie Sullivan
that percentile higher it should be 90% or 95% of your users and I think for individual sites and properties like absolutely right like all the Google
35:59
Annie Sullivan
properties I know are way higher percentiles where they're looking at trying to keep that that good threshold good uh but I think that the one thing
36:08
Annie Sullivan
that really stops us from doing this across the whole program is that we have like kind of a very uh we try to be as inclusive as possible and take like the
36:16
Annie Sullivan
smallest like uh amount of aggregated data that we can to publish and for those smaller sites of which there are
36:22
Annie Sullivan
many millions. Uh unfortunately like if we push past the P75 we end up in kind of a noise territory.
36:30
Annie Sullivan
So that that's the only reason we don't do it. But as an individual developer if you have enough traffic I would highly recommend. Yeah.
36:37
Cliff Crocker
Yeah. I've been talking to more um speed curve slash embrace users who they're fast and they're um they're setting
36:46
Cliff Crocker
their performance budgets at the 95th percentile because that's kind of the next target which I think is super awesome.
36:54
Annie Sullivan
That's uh that's some really good insight. I I hadn't thought about um changing the percentile, but I do know that we focus um on like we we kind of
37:02
Annie Sullivan
look at like the core web vital thresholds as an industry floor rather than as our floor. So like we always look at our like our rum data to try to
37:11
Annie Sullivan
understand what what those thresholds are for our site and for our pages and and that varies by page uh varies by
37:19
Annie Sullivan
device. Uh, and so it's um it's always good to kind of look at your own data and and build those those thresholds for yourselves.
37:27
Cliff Crocker
I like the methodology that Dan's proposing here like at an individual site level as well like um again dating
37:34
Cliff Crocker
myself but back 2010 and at Walmart Labs when we started figured out that front-end performance actually mattered
37:41
Cliff Crocker
for conversion rates and things like that and and sort of shifted the way we're focused at least for a few years.
37:47
Cliff Crocker
um we we had to start at the median because it was like we can't even start thinking about 75th percentile and like you know 35 second product details pages
37:55
Cliff Crocker
and and things like that like we we had to ratchet back and start and then we progressively started to the point where we did start getting into like the you
38:03
Cliff Crocker
know P90 and and looking at some of those things for optimizing edge cases but identifying outliers with so many different things that were impacting the
38:11
Cliff Crocker
site and different third parties and different permutations of experiments and everything else um it became almost a necess necessity as we became, you
38:18
Cliff Crocker
know, a little bit more mature in this space. But certainly for looking at a broader um, you know, ecosystem in
38:26
Cliff Crocker
different sites, I think P75 has been fantastic for that. And I would Yeah.
38:31
Cliff Crocker
And I would again credit the the team and Annie's team for coming up with that because initially I was I was I was skeptical of like is P75 really the
38:39
Cliff Crocker
right number? Like what message are we sending? And but spot on for sure.
38:44
Cliff Crocker
Um there's a question in the chat from Jenna saying um as someone who's worked on long lived applications with a lot of
38:51
Cliff Crocker
interactions between page loads, how do we evolve or adapt Corva vitals for more interactive experiences or is this something that's best left to
39:00
Cliff Crocker
performance engineers for those apps or is core vital just not for them?
39:06
Annie Sullivan
Uh yeah, so I I have a lot to say that we're working on. So the way that we kind of evolve core vitals most of the time is we'll add features to the
39:12
Annie Sullivan
performance timeline so that experts can use them and then we'll slowly like evolve that and when they end up being like really useful to the broader
39:21
Annie Sullivan
community then we'll integrate into core web vital. So the work that we've been doing in this area specifically is uh first off the soft navigations work
39:30
Annie Sullivan
where basically like you know you have a JavaScript application there's a user interaction and you you draw a new view and we're trying to make that work just like LCP uh separate CLS and imp for
39:39
Annie Sullivan
those pages just like regular core web vitals because we think the user experience is really the same. And then I think maybe the the next thing that Jenna might say is that uh well yes
39:48
Annie Sullivan
sometimes we don't change the entire view right sometimes there's a click and then there's a change uh and we can actually measure those things right uh
39:56
Annie Sullivan
but they don't get yet reported in core vitals so what we're looking at is taking all those APIs that would be interaction to content for paint instead
40:04
Annie Sullivan
of interaction to next paint uh looking at the APIs that we made uh that allow you to do that in the performance timeline aggregating it and see if
40:12
Annie Sullivan
there's like a broader use case for that. But those those APIs are available as of Chrome 151.
40:18
Annie Sullivan
You can track the paints of an interaction to see like what happens. So that's one thing that we've been working on. And we've been also working with the broader community on the container
40:27
Annie Sullivan
timing API which allows you to take like a container and see like how it's filling in. Uh Bloomberg's really been doing a lot of fantastic work on this as
40:34
Annie Sullivan
well in Agelia. Um that that was actually just implemented in Firefox as well. So, uh, we love to get like the
40:42
Annie Sullivan
more advanced teams using those metrics and understanding and seeing what they do and then trying to evolve figure out what we evolve out of there that can apply to more sites.
40:55
Cliff Crocker
I was curious about um, ICP, if we're calling it that, interaction content. Is that something that will be rolled out
41:03
Cliff Crocker
as as a more standalone metric or as a LCP for softnabs or is this going to be in crux like how do we start to think
41:11
Annie Sullivan
about that as like a unique metric that's measuring um yeah I'll tell you actually how we think about it and how we approach it which is
41:19
Annie Sullivan
a little bit like not a great like uh what's happening next story. So how we think about it right is we make the API we we see the use case there's enough
41:28
Annie Sullivan
use cases for the API so we talk to partners that are are asking for the use case we make sure the API meets that and then we look at how they're using it and
41:36
Annie Sullivan
what they're doing with it um like Gnome Healthman from uh the Outlook team speaks to the performance uh working group a lot about how he uses some of
41:44
Annie Sullivan
these APIs for example we have other other uses and then what we try to do is like internally in Chrome kind of make an aggregate metric what if we just did
41:52
Annie Sullivan
the very simple thing of interaction to contentful paint, right? Which is like interaction to next paint except it's the last one instead of the first one.
42:01
Annie Sullivan
Uh we can make a metric and we can see on all websites what that would look like. What would the value be? How many people are having ICPS that are even
42:08
Annie Sullivan
different than NP, right? Um, when we look at slow ICP, is it like, well, they
42:17
Annie Sullivan
filled out this really complicated form and then they got these continual status updates and it's actually like the best possible user experience or are there
42:25
Annie Sullivan
some really bad experiences that need improving? Um, can we separate out like the bad long ones from the good long
42:32
Annie Sullivan
ones? uh try to basically just do a lot of analysis over what what the different ways to implement such a metric are and
42:39
Annie Sullivan
how they actually look as a user experience and then decide from there if there's something that we really want to show like all websites or we're also
42:47
Annie Sullivan
thinking about are there maybe some like uh some surfaces where we could show a smaller set of sub websites a metric.
42:55
Cliff Crocker
Gotcha. Gotcha. Makes sense. Makes sense.
42:59
Cliff Crocker
Um, I don't want to do a hard shift away from what we're talking about. Um, but I feel like we got 15 minutes left and we haven't
43:07
Cliff Crocker
talked about AI and I feel like that's that's the other elephant in the room. Right.
43:11
Alex Russell
Before we go there, could I just quickly say that I'm I'm a Jenna, your question is very good. Um, I just wanted to add that I don't think
43:18
Alex Russell
that most of the developers maybe maybe your data is great. Maybe you understand um what your you know percentiles are for session depth. Um but what I can
43:26
Alex Russell
tell you is that most of the teams that I work with live in um they live in beliefs rather than evidence about these
43:34
Alex Russell
questions. And so what that has meant is that folks will be building to a a justified belief about the longest
43:41
Alex Russell
session that they know of for a product rather than the mean or the average.
43:45
Alex Russell
Right? A lot of these things are have have different you know humps in the distribution. Right? There's like a lot of biodality. Um, and so it can be the
43:52
Alex Russell
case that something could be built to do well in a long session, but the prep preponderance of your sessions could be very short. And what I would encourage
44:00
Alex Russell
all teams to do is to go look and find out, right? Just go look at your evidence. Like you're not going to make a good experience without looking at, you know, what people are trying to do.
44:08
Alex Russell
And so um to the extent that people have adopted tools that are incompatible with doing a good job for users in short sessions on the belief that they're
44:17
Alex Russell
going to have long sessions, I think we have as an industry have adopted a whole series of mistakes at scale. And um and I would encourage us to regground our
44:25
Cliff Crocker
our beliefs in evidence about how long sessions actually are. That's great.
44:32
Cliff Crocker
Now, also really quick, just to add on to that, when you are looking at your data and finding things out and learning, you know, about your assumptions and what
44:40
Cliff Crocker
you're able to see both with APIs and metrics, please publish it or give a talk about it, like there's there's just not enough data out there. So,
44:49
Cliff Crocker
we love to have this bigger corpus of information.
44:53
Annie Sullivan
Definitely. plus one that yeah I it's it's kind of why I'm still in this community is because people are so great
45:00
Annie Sullivan
about sharing their findings and we we all learn together which is awesome.
45:05
Cliff Crocker
In the spirit of talking about sharing our data and helping Tammy with the pivot, I did want to say one of the topics that we are we are really digging
45:14
Cliff Crocker
into in the W3C rum community group is this u discussion around agentic traffic
45:22
Cliff Crocker
and agentic browsing and how do we think about agents and are we supposed to be measuring them like they're real users?
45:28
Cliff Crocker
Is this userdriven traffic? Should we be thinking about the same way? It's opened up so many different like discussion
45:35
Cliff Crocker
points and challenges, a lot of challenges when we start thinking about measuring from a traditional sense. So definitely would love to pivot in there
45:43
Cliff Crocker
a little bit. And Annie, I know that you said you would been thinking about this and focusing on this a bit too. So would love your thoughts.
45:50
Annie Sullivan
Uh yeah, so we're kind of just at the beginning of everything right now. It's a really exciting time, but also a time when we don't really know all that much, right? uh we're trying to just
45:59
Annie Sullivan
understand like what are basic user expectations uh what does existing UXR say about what they might be and then where do we need more UXR and then I
46:08
Annie Sullivan
think there's a a whole slew of complexity about like the use cases and um like all the different developer use cases and and whether like it makes
46:17
Annie Sullivan
sense to identify traffic is coming from an agent or not that it's like I I feel like there's it's not I can't personally
46:25
Annie Sullivan
have like an opinion or answer on it because there just so many different things happening right now. I'm sure lots of people do though and I'd love to
46:32
Annie Sullivan
hear them. I guess is I I'd love to hear like from anybody who's thinking about this. I'll be at Perf now or just come talk to me or give web vitals feedback.
46:40
Annie Sullivan
Uh so we're really really just trying to learn as much as we can right now is is what I would say. One line of discussion was like, hey, do vitals even matter
46:48
Annie Sullivan
anymore for an agent? Like, or should we actually be now, I don't want to say regressing, but thinking more about time to first bite or DOM content loaded, you
46:56
Annie Sullivan
know, when these agents aren't necessarily executing JavaScript or doing all the things that that we are measuring in traditional web performance. Are those some of the like
47:05
Annie Sullivan
primitives or that we need to go back to? So I think this is a really interesting question and I see like what the question is kind of a roar text like
47:13
Annie Sullivan
everybody has a different answer to it right and I that's where we want to really do UXR and really understand this right like like one question that comes
47:20
Annie Sullivan
up is like should agents be using MCP right and that's like a totally different path or should they be using hardcoded HTML like
47:29
Annie Sullivan
we don't even kind of know the basics of like best practices for agents or what agents are going to do as far as like I don't know them uh so I'd be really
47:38
Annie Sullivan
interested to, you know, I'm just excited to to see where things are headed. Yeah.
47:42
Annie Sullivan
I have a question for you, Alex, and sorry, I don't mean to put you on the spot. I I feel like you and I sort of share the same brain on some topics. So,
47:50
Annie Sullivan
when I So, I will tell you what I think and then I'm curious to know what you think. So, when I have been hearing people say things like, "Oh, we it's
47:59
Annie Sullivan
time to start optimizing for agents, blah, blah, blah." I'm like, "Wait a second. Did we finish optimizing for humans?" Like [laughter] I feel like I
48:07
Annie Sullivan
feel like we're the humans are going to still be in the loop and we have some work to do. So it makes it gives me
48:15
Annie Sullivan
pause. And I'm not it's not like I'm against optimizing for agents. I just feel like that's a whole other add-on.
48:21
Annie Sullivan
Not a it shouldn't that shouldn't take away from this the other work.
48:26
Alex Russell
I agree. Um, and I would say that my test for whether or not a story makes sense here is whether or not it is a challenge or a permission structure.
48:35
Alex Russell
Right? So, one of the things that happened with mobile was that a bunch of people in the JavaScript ecosystem specifically sold developers the belief
48:42
Alex Russell
that if they just JavaScript hard enough that all the performance problems would be solved, right? And of course, that was It was just a pile of
48:51
Alex Russell
towering nonsense. and it didn't work because the idea that the next compiler was going to save you was a permission structure for you not to actually change
48:59
Alex Russell
what you were doing or look very hard at how it was working out in the real world. Um, and so if we think about whether or not the narrative is
49:06
Alex Russell
challenging us to build to the most uh constrained environment that we're going to need to succeed in, right? Like think about a market, right? Like you you sell
49:15
Alex Russell
a product into the market and the price of that product is is made at the margin, right? It's not what most people would buy it for, it's what the marginal user will buy it for. And that is true
49:23
Alex Russell
of all of the products that we make. So if you're talking about web performance, there is probably some case, some edge case in which the the needs of agents daminate from the needs of actual users.
49:35
Alex Russell
But I would encourage everyone to think about, okay, is it in the case in the main that the things that I would do to make life better for real users will make things better for agents? And if
49:43
Alex Russell
so, I would circle that big pile of work and then add a plus on it, right? I would just sort of, you know, add add a couple of extra check marks and say like we need to do a really good job there.
49:52
Alex Russell
And then we can think about agents as a separate uh population of users. And I think that's true all the way through the metric set and all the way through
50:00
Alex Russell
the the user journey set. Do agents have long sessions? Probably not. Do agents
50:07
Alex Russell
um have uh overlapping interests in in lightweight or semantic markup?
50:12
Alex Russell
Absolutely. Right? they're trying to con, you know, construct a view of the page at as low cost uh as possible in the same way that users at the margins
50:20
Alex Russell
are. So, um I would encourage us to think about what we have in common between these use cases and what our users have in common with these sort of
50:28
Alex Russell
new presentations. And to the extent that someone is trying to create a permission structure for not thinking hard about what's common, um I would call immediately, right? And
50:37
Alex Russell
then say, "Tell me what your evidence is. Show me." Right? Because because the antidote to is evidence and and a good culture, the kind you were
50:45
Alex Russell
talking about at Etsy, uh, Paul and Tammy, is one in which the, you know, everyone expects that the way you make your mark is you present the better evidence and everyone goes, "Oh, yeah.
50:55
Alex Russell
Okay, obviously, yeah, sure, let's do that." Um, which sets the expectation that we're going to go, um, be engineers about engineering, uh, rather than
51:04
Alex Russell
thought leaders, right? So, so if you want to be a thought leader, get a blog.
51:07
Alex Russell
If you want to do a if you want to be an engineer, do an engineering, right?
51:14
Cliff Crocker
Great point. And Ann Andy made a subtle comment in the chat that I think is pretty profound and throwing accessibility into that pile of work
51:21
Cliff Crocker
that we're already still not good enough at that is clearly going to be benefiting agents if they're, you know, looking at the accessibility tree and
51:28
Cliff Crocker
things like that. This to me is a great thing if we could say, "Hey, let's let's optimize this because it's gonna help agents, too, because maybe we'll finally freaking move the needle a little bit
51:36
Cliff Crocker
more on accessibility, right?" Um, so I do I do like that framing of looking at the whole big circle of work.
51:43
Cliff Crocker
Well, and Erin made the point in the chat that it it holds true for SEO as well. Doing the right thing for SEO does the right thing for your users. So it's
51:51
Cliff Crocker
like if we can draw one like make basically make a master checklist of everything that helps performance, helps
51:58
Cliff Crocker
SEO, helps accessibility and helps agents do those things first. That's my very grossly I as a person who doesn't
52:05
Cliff Crocker
have to do the things. It's really easy for me to say make a list and do the things but um it sort of seems like that's that's that's the direction we should be going in.
52:15
Cliff Crocker
So we've got where where's the Sorry.
52:18
Paul Calvano
Sorry. Go ahead, Paul. Please. Oh, I I recently went down a rabbit hole around AI and it was kind of on the opposite spectrum and I was I was looking at like
52:25
Paul Calvano
I was I was trying to understand like you know what the um what the browser stack essentially looks like for an AI
52:33
Paul Calvano
agent. So an AI like you you ask a question to to an LLM and it looks up something from the same site in seven different places like how is it actually
52:41
Paul Calvano
fetching that? What is it fetching? Is it using HTTP 1.1 or two? Is it using compression? Is it use is it decoding
52:48
Paul Calvano
images? Is it able to interpret um different image formats? Um is it even requesting them? What does it do when it
52:56
Paul Calvano
gets something it doesn't expect? And so I've actually been like um this is not not for Etsy, but just for like my own like um uh probably going to be a blog
53:04
Paul Calvano
post soon um just trying to understand like where that ecosystem kind of fits.
53:09
Paul Calvano
And it's it's been really interesting just to see like um you know like like one of the uh one one of the popular AI
53:17
Paul Calvano
solutions does not use persistent connections at all. Um so every single request is its own TCP connection TLS connection. Um some of them one of them
53:26
Paul Calvano
like one of them aborts requests after 5 seconds. Um but if you um trickle data
53:33
Paul Calvano
to it um it will never abort the request. it will always, you know, just continue to like just hold off until it
53:41
Paul Calvano
receives the data. But yeah, there's there's lots of like little little differences in terms of the browsing behavior. And and one of the things that
53:48
Paul Calvano
like when I when I think about taking this back to web and mobile is that we've a lot of these like basic um ways
53:57
Paul Calvano
that web servers and web browsers communicate with each others. It's kind of this almost assumed contract that we have that may or may not exist um with
54:06
Paul Calvano
this client that we don't fully understand yet.
54:12
Cliff Crocker
So, we're coming up on four minutes left. And one of the more more juicy fun things that we wanted to try, you know,
54:20
Cliff Crocker
close with was for each of you to kind of take a look at each other's worlds and maybe pose a question like what are the what are the different asks
54:28
Cliff Crocker
that you might make to a Paul if you're Annie or Annie, you know, if you're Paul or Alex? Like what are some of the questions that you would, you know, if
54:36
Cliff Crocker
you could force the the other two's worlds to change something? What would that be? I don't know if I framed that right, but hopefully you guys are really
54:44
Cliff Crocker
smart, too. [laughter] Anyway, um, who wants to go first?
54:52
Annie Sullivan
I I guess I'll go first. Um, so, um, I mentioned before how like the, you know,
55:00
Annie Sullivan
I felt like the the mobile, um, native mobile industry in terms of like performance tooling was way behind. And Alex, you had a lot of perspective for
55:08
Annie Sullivan
me. So, thank you. I hadn't considered um how the um just kind of engineering practices that went into developing for
55:15
Annie Sullivan
you know on a small scale like actually led further to that more lightweight experience. Um one of the things I'd love to see is like a Chrome user
55:24
Annie Sullivan
experience report type style data set for native mobile apps, right? And um and obviously we need to get to metrics
55:32
Annie Sullivan
even compare them first because we don't have those. Um but like um I think that that would uh serve two main main purposes and one it would help teams
55:41
Annie Sullivan
understand how their apps compare to let's say their industry but by unlocking the ability to share
55:49
Annie Sullivan
competitive insights it also could spur investment.
55:58
Cliff Crocker
Is there something [laughter] Oh no you're just you're just uh coming up with names. Okay. Yeah. Yeah. Yeah.
56:06
Cliff Crocker
I was like I was like wait did someone is there something because that may have just like you know rerouted my entire day.
56:11
Cliff Crocker
So but I'm just going to say I don't like saying annex. That makes me feel weird.
56:17
Cliff Crocker
[laughter]
56:19
Paul Calvano
We've got used to saying weird things before and it's normal. So, but yeah, I think Android vitals does give you some of those things like, you know, ANR
56:27
Paul Calvano
counts and and things like that, which I think are great. And I think a lot of times in mobile, maybe they're a little too overfocused on like what's my crash
56:35
Paul Calvano
um you know, uh my crash rate and you know, what's my you know, what percent of my sessions have ANRS? Um although
56:43
Paul Calvano
those do really have an impact on the user experience, like what goes beyond that? like where do you get pissed off and start raget tapping like you do on web like those types of things I think
56:52
Paul Calvano
would be really interesting to to try to see if we could surface but Annie do you have any any ask
57:00
Annie Sullivan
mine was also very similar I was hoping that Edge could publish data sets similar to Krux about like edge uses and you know like different things that that
57:08
Alex Russell
Alex is really passionate about if we have data sets around there those two I would love that yeah that's a that's a good point well taken and um my colleague Andy dealers
57:16
Alex Russell
uh spent some time in the last couple years rehabilitating our data pipelines around core vitals. So we we do have some of that data. Um I think the
57:24
Alex Russell
missing part is a way to publish and anonymize in in a kind of compatible
57:31
Alex Russell
way. So the way Annie Annie tell me if I get any of this wrong. Um my understanding of the of the crux flow
57:38
Alex Russell
today is you get the yeah you you get the histograms they are bucketed by origin to the extent that users allow us
57:46
Alex Russell
to do that with telemetry they're anonymized using rapport um and rapport is a a differential privacy system that
57:54
Alex Russell
we can link to um the the public implementations of rapport have maybe bit rotted a little bit so I don't know if if you're on the same one that that
58:01
Alex Russell
you were forever ago but um but that is an essential component And then the data that comes out is relative to the total
58:08
Alex Russell
traffic for um that you see and is uh scaled. So uh all of those things
58:17
Alex Russell
present challenges for doing publication in a kind of additive way. They might we might be able to do publication in a comparative way. Um but I'll I'll take it back and sort of you know ask folks
58:26
Alex Russell
if there's appetite for it. Um Microsoft has not historically published that sort of thing, but you know there's no reason that we shouldn't I think is the way I'd put it. Yeah.
58:33
Annie Sullivan
Yeah. Yeah. And I don't think we do publish the way we do in automization, but we can talk. Yeah.
58:40
Cliff Crocker
And Alex, anything any asks of your fellow panelists?
58:46
Alex Russell
Uh, keep up the good work. I would say for um we need you know I would say you know Annie and I have had this long
58:53
Alex Russell
running conversation you know in various forums over the years about the role of uh browser UI effectively to um prod
59:03
Alex Russell
users and developers. You know, my experience of getting the web encrypted was that we were successful because we were able to use the padlock icon,
59:10
Alex Russell
right? We were able to connect the interest of the decision makers, the people who buy and sell um technology
59:18
Alex Russell
products, uh with the interests of users by making it visible to decision makers in technology organizations when their sites were doing a bad job for some people, not all people, but some people.
59:28
Alex Russell
And I would say that browsers have kind of fallen down on the job today. And I I blame myself here. Um, Edge hasn't shipped anything, right? So, let let's
59:36
Alex Russell
be very clear that I'm I'm not succeeding at this myself, so I don't want to like put it on anyone else. Um, but I would love for us to join hands and as a sort of an alliance of browsers
59:45
Alex Russell
agree that um a good experience particularly for for users on mobile uh and at the margins is enough of a motivation or should be a similar
59:53
Alex Russell
motivation to use UI to inform folks inside the privilege bubble of the experience of most of folks. Um, and I
1:00:01
Alex Russell
don't think that we've made that solidarity connection uh the way we did with TLS. And I I think that is um you know maybe one way out of the problem
1:00:09
Alex Russell
that we've got now of plateauing uh core vitals uh traction.
1:00:17
Paul Calvano
I'm I'm an optimist so I do want to call a couple of things which is it's awesome that we have more of the vitals available across the different
1:00:24
Paul Calvano
platforms. It it's it's great. I know well late to the party like it's it's amazing to see you know LCP imp um you
1:00:32
Paul Calvano
know that sort of thing showing up on on a broader set of of uh um browsers and really appreciate all the work that you
1:00:40
Paul Calvano
all did to move that forward and the community did to push that forward in interop and get that stuff done. So I think that we're making a lot of those a
1:00:47
Paul Calvano
lot of that headway. Uh albeit you know certainly we all feel like we have room for improvement there. Um, Tammy, we're
1:00:55
Paul Calvano
at time. Do you want to do you want to take us out?
1:00:57
Tammy Everts
Yeah, we are at time and I um and I have to apologize. We we didn't get to all the questions that people had and I just discovered a little nest of questions in
1:01:05
Tammy Everts
the Q&A which I was have to admit I was not paying attention to. I was tracking the main chat. So, sorry to Adnan and
1:01:12
Tammy Everts
Henry for uh not getting to your questions and to anyone else. But um I don't know. I'd love to chat again sometime. This was really really fun
1:01:20
Tammy Everts
everybody. Um, so thank you again to Paul and Alex and Annie for making the
1:01:28
Tammy Everts
time. I knew this was going to be awesome and it exceeded my expectations.
1:01:33
Tammy Everts
So, thank you. Thank you so much. Um, to anybody who wants to to get the recording, we will be sending everything
1:01:41
Tammy Everts
out um, I think at the end of the week as soon as everything's kind of available if you if you did uh, the sign up, which I guess you did because you
1:01:47
Tammy Everts
have this link, so thanks
Highlights
10:01
Performance inequality is getting worse, not better.High-end devices improve faster than low-end devices every year, so the gap between them widens with each replacement cycle — even as networks get faster overall.
24:25
Native mobile observability tooling is roughly a decade behind the web's.Without a CrUX-equivalent dataset, teams are still profiling mobile apps by manually intercepting traffic through a proxy — the same approach used for web before Chrome DevTools existed.
35:35
P75 is a floor for the whole program, not your ceiling.Core Web Vitals reports at the 75th percentile so smaller, lower-traffic sites aren't lost in noise — sites with enough volume should set their own tighter budgets at P90 or P95.
45:50
Are AI agents even "users"? Nobody's sure yet.The panel's most open question: should agent traffic even be measured with Core Web Vitals, or do older metrics like time-to-first-byte matter more when there's no rendering to observe. Annie: "did we finish optimizing for humans?"
47:42
Don't let "optimize for agents" become an excuse.The test: does a change also help real users at the margins? If so, prioritize it as part of existing performance and accessibility work — not a separate, competing track.
