Ethereum Protocol Fellowship (EPF) Cohort 7 — Applications open until May 13

AllCoreDevs - Testing #055

2025-09-29 Agenda: #1736 canonical JSON

Transcript

00:03:32
Mario Vega:Thank you so much. Good day, everyone. Welcome to ACDT number 55 to this September 29.
00:03:40
Mario Vega:We have a lot of items in the agenda today. I'm gonna share the link to the issue so we can follow along.
00:03:47
Mario Vega:This is the one… And… we can start with Fusaka.
00:03:52
Mario Vega:And the first item in the agenda is DevNet3. As far as I know, we have DevNet3 running, and we don't… no longer have DevNet 5 running, so…
00:04:04
Mario Vega:Do we have anyone from Epand Ops to give an update on DevNet3? I see there's no finalization, as I can see right now. Is… is this suspected?
00:04:15
Mario Vega:Is Piquet Indy or Barnabas in the call?
00:04:24
pk910:I think Barnabas is not here yet, but, yeah, we've just figured a new, unfinology test on DevNet3, so, that chain split, and…
00:04:34
pk910:The unfinity is kind of expected for now.
00:04:41
Mario Vega:Do you know… do you have more context on this? Do you know how much is this test gonna last?
00:04:48
Mario Vega:Or is there any action items for client teams to take care of on this, or…
00:04:57
pk910:As far as I know, the unfinity test was requested by some clients.
00:05:03
pk910:It will probably last for one or two days, and then we'll,
00:05:08
pk910:Deploy the proper clients again, and see if they can recover properly.
00:05:16
Mario Vega:Excellent, thank you.
00:05:18
Mario Vega:Let me see… Alright,
00:05:23
Mario Vega:Cool. Is there any updates from clients regarding dentistry? Any findings that want to be shared on this call? Either ELs or CLs?
00:05:32
Mario Vega:Who want to chime in on… on the… on this subnet?
00:05:48
Mario Vega:Alright, yeah, okay, since no updates, I think we can move along. Good.
00:05:54
Mario Vega:Yeah, and the next topic is the… there was the announcement, the blog post, I mean, regarding Fusaka, so we now have the schedule and activation times for Holeshki, Sepolia, and Holy, with Holeshki happening this Wednesday.
00:06:13
Mario Vega:I think… All of the release versions are now up in the blog post.
00:06:20
Mario Vega:yes, Bari, yeah, we just finished that. Do you want to update something on the DevNets?
00:06:28
Parithosh Jayanthi:Not sure what updates were already mentioned, but we have Holsky ShadowFork up as of, like, half an hour-ish. The network seems to be finalizing. We're discussing with the Aragon team why the node's not able to keep up, but it was able to do the ShadowFork, so it's likely more peer-related than anything else. And,
00:06:52
Parithosh Jayanthi:Bezoo's billing blocks, but it's billing it without transaction inclusion, so they've also been informed, and…
00:06:59
Parithosh Jayanthi:Would be great if they can have a look.
00:07:06
Barnabas:And we have also submitted, some pop-up transactions, and exits.
00:07:12
Parithosh Jayanthi:Exactly, so we should have, all the queues full at the time of the transition, which was able to catch a few bugs in the last week.
00:07:21
Parithosh Jayanthi:And the transition should happen in, like, the next half an hour, I think.
00:07:38
Mario Vega:Excellent, thank you so much.
00:07:40
Mario Vega:Should we jump into Manage Health Fork, now?
00:07:48
Barnabas:We're gonna be doing other testing setup works before we do main initial work. I think we're gonna be waiting for the minute releases before we do maintenance of work. That's gonna be at least a month away.
00:08:00
Parithosh Jayanthi:Yeah, our plan is to use the latest releases from all the clients to do the shadow fork for the immediate next testnet, try and catch all the bugs that we can.
00:08:10
Parithosh Jayanthi:And the other line of testing is that DevNet 3 is still up and running, and Banervice triggered non-finality today, so we let it non-finalizing for about 2 days before joining it back, so that we can make sure that the releases are also able to recover.
00:08:25
Parithosh Jayanthi:And parallelly sync tests are also running.
00:08:32
Mario Vega:Excellent. Thank you.
00:08:35
Mario Vega:Yeah, it makes sense to delay, mainnet Shadow Forks until we have, like, the releases for mainnet.
00:08:42
Mario Vega:Alright, cool. Oh, going back to the activations, we have Holesky on…
00:08:48
Mario Vega:on Wednesday, and I wanted to touch upon the topic of client releases. Is there any updates on client releases so far?
00:08:58
Mario Vega:Please check the blog post to see if the client version that is pointed out in the blog post is up-to-date.
00:09:05
Mario Vega:If not, is there any update from any of the clients? Are we ready for, for Wednesday?
00:09:18
pawan:I think for Lighthouse, it wasn't, listed earlier, we just had a release today.
00:09:25
pawan:So I think the blog post can be updated with the… Specific thing?
00:09:33
Parithosh Jayanthi:Yeah, we already have it, it is waiting to be approved.
00:09:40
Mario Vega:Excellent, thanks so much.
00:09:43
Mario Vega:Alright Any other updates from clients, regarding, Holeshky support, on the releases?
00:10:00
Mario Vega:Cool. If not, I think we should be ready, and we can discuss the transition on next ACDC on Thursday after it happens on Wednesday. The BPO activation happens next week, so we'll have, ACDT, to discuss, first the…
00:10:18
Mario Vega:transition into Fusaka on Holeski, and then the BPO, if there's anything that we need to fix before… before it happens next Tuesday.
00:10:30
Mario Vega:Cool. Yeah. And next thing, we have a couple of topics on, execution client testing on the east side. There's a new release, version 5.2, which is basically just small updates, to the…
00:10:47
Mario Vega:to the exception mapping, so there's not… nothing from the client size to do, as far as I know. And we have, 5.3 version of East, which is pending, and we will update the blob schedule. And with that update, we're gonna also verify the client design configuration via,
00:11:06
Mario Vega:if underscore config, and we're gonna use the execute config application in the, in East to verify the…
00:11:14
Mario Vega:the releases. If we find anything, we're gonna ping the clients,
00:11:19
Mario Vega:To see if there's any fix that needs to be done from your side.
00:11:27
Mario Vega:Alright, let me see what's next? Shadow Fork, we already discussed.
00:11:34
Mario Vega:Anything else on Shelfworks that I might have missed, or anyone wants to chime in? But I guess there's not much to discuss, since Holesk is not… sorry, since Shellfwork has not yet acted at Fusaka, we can discuss that on the next call, I guess.
00:11:51
Mario Vega:And the last topic regarding Fusaka is the Sherlock Fusaka audit contest.
00:11:57
Mario Vega:I think it's still ongoing. Is there anyone here that wants to give an update to this?
00:12:12
Justin Traglia:I can just say that it's going well, and there's, 2 more weeks left.
00:12:16
Justin Traglia:And the competition.
00:12:25
Mario Vega:Alright, that's everything for Fusaka. Anything that I missed, that we need to discuss today regarding Fusaka?
00:12:36
Mario Vega:Anyone, anything that anyone wants to bring up?
00:12:42
Mario Vega:Important findings, or anything related to client box?
00:12:56
Mario Vega:Frederick, do you like to expand on this?
00:13:02
Fredrik:I think, yeah, we already have a Fusaka directory in the, PM repo, but,
00:13:09
Fredrik:We should, make sure that… We,
00:13:15
Fredrik:we add the relevant files, for the… that's what Harry listed out to do, basically, and we should do that as we have done for
00:13:25
Fredrik:Pictra, for example, basically, create the plans for
00:13:32
Fredrik:for the different, testnets and mainnets, so that we follow the process that's been set up, so that we don't miss any specific steps in testing and, yeah, other aspects. I think Tim used
00:13:48
Fredrik:To do that, but, since he's away, so on.
00:13:54
Fredrik:Should probably take up that torch.
00:13:58
Pooja Ranjan:I actually see a folder created in Ethereum slash PM, it is for Fusaka, maybe we would want to move all the related files in this folder.
00:14:08
Fredrik:Yeah, it's just a paradass readness checklist in there right now, and then we should…
00:14:16
Pooja Ranjan:Yeah, we can have all the PRs to move all related files in this folder, and then we will add it to the network upgrade folder.
00:14:30
Mario Vega:Who wants to take on this, this task?
00:14:33
Mario Vega:Since Tim is out of the office.
00:14:40
Pooja Ranjan:So, I mean, I would recommend here that we add all the related files in Fusaka folder, and once that is done, we can move the Fusaka folder to Network Upgrade… Network Upgrade folder archive, which is right now. We can update it right over there.
00:14:57
Pooja Ranjan:But first, let's move all so anyone with related information, make a PR, and I believe Nixo will be in a position to, merge all the PR. Yeah, Stokes as well.
00:15:09
Mario Vega:Excellent, and Alex pointed out that he can start the, the PR for this. Thank you.
00:15:18
Mario Vega:Alright, let's see, alright, anything else?
00:15:25
Mario Vega:Related to Fusaka, or should we move on to the next topic?
00:15:35
Mario Vega:Cool. Alright. So, next topic is, gas limit testing updates.
00:15:42
Mario Vega:And the first subtopic is, the mainnet 60 million gas activation in the Fusaka releases. Any comments on this regarding clients? I…
00:15:53
Mario Vega:I'm out of context here, regarding whether the clients added or not the 60 million gas limit, so if anyone can chime in to…
00:16:05
Mario Vega:Any of the clients, I mean?
00:16:10
Mario Vega:Like I mentioned that GET is ready for 60 million, and it's in the release, I presume.
00:16:18
Andrew Ashikhmin:So yeah, I have a question. Do we want, it to be included into the, testnet Fusaka release, or in the mainnet Fusaka release?
00:16:30
Parithosh Jayanthi:So, testnets are already at 60 million, so it shouldn't really matter if it's included in the testnet release. I think the main intent is that it is included before the mainnet release.
00:16:47
Mario Vega:Any other clients? I see Nethermine.
00:16:54
Kamil Chodoła:Yeah, I can chime in here. So, we have a PR with 60 for NetherMind. From a testing perspective, I know that GEF have API system to make sure that 60 is working well. That's also improved quite a lot.
00:17:09
Kamil Chodoła:I get back after a longer break right now and still analyzing the data, and the outstanding thing from testing perspective is still Zen testing, which I'm discussing right now with Johan and the team, so hopefully this week we'll be able to test it and give some extra input on that.
00:17:24
Kamil Chodoła:Yeah, from NetherMind perspective, we have APR, which we'll include on my net release.
00:17:32
Mario Vega:Excellent, thank you. Yeah, and we have a… we have a topic for discussion today on compute and state benchmarking, so we can, dig deeper in that section. Any other clients? I see Bessu in the call. Do you guys have 60 million gas enabled in the test and releases?
00:17:54
Ameziane Hamlat:Yeah, so it is not on the testnet release, but, yeah, it will be on mainnet, for sure.
00:18:02
Ameziane Hamlat:For the mainnet release, I mean.
00:18:14
Mario Vega:Let's see… is Reth here? Is anyone from Reth in the call?
00:18:25
Mario Vega:Excellent. Is it in the… in the release already?
00:18:33
Mario Vega:Excellent, thank you.
00:18:35
Mario Vega:Alright, who am I missing? I don't think any other clients are here.
00:18:40
Mario Vega:Cool. Excellent. Okay.
00:18:45
Mario Vega:Thank you. Yeah, and there's, just another…
00:18:51
Mario Vega:bullet point on this is we have a proposal in the East repository.
00:18:58
Mario Vega:to update the, directory structure that we have for benchmarking. I think we can,
00:19:04
Mario Vega:discuss, but I… maybe… maybe it's out of topic for this call, but just for everyone's reference, we currently have benchmark tests in the East repository, and they now are gonna be divided in two sections, and we can discuss those sections in a bit, but basically the compute is one section, which is basically a state,
00:19:26
Mario Vega:A small state in the test, but,
00:19:29
Mario Vega:a lot of competition, and then state which includes, a bigger state for, bigger… I'm sorry, for more intensive, state testing.
00:19:42
Mario Vega:And please, follow the link, and you can see how we're gonna restructure this. It's very simple. But yeah. With that said, I think we can jump into compute-intensive benchmarking. We're gonna…
00:19:55
Mario Vega:There's just one subtopic, so please, if anyone else wants to raise another topic, another discussion topic for this, please raise your hand. But yeah, the only topic for discussion is that we're gonna have an update on the,
00:20:10
Mario Vega:on East, to update the format, the testing format for the benchmark tests. I don't think there's anything to be done from client side on this, it's just basically a better way of writing tests.
00:20:24
Mario Vega:As far as I know. So yeah, there's no action item for clients. Basically, it's gonna produce the same blockchain test and the same state tests that we have already produced, so no changes in the conception of these tests, it's just a matter of, like.
00:20:38
Mario Vega:just, making it easier to write the benchmarking tests. Let's see…
00:20:45
Mario Vega:Luis, yeah, if you want to chime in just, on this PR, just please feel free to.
00:20:56
Mario Vega:If not, we can continue. Yeah, and the next,
00:21:01
Mario Vega:topic is the state benchmarking. I think Johim, is prepared for, to give an update on this. So yeah, Johim, just feel free to…
00:21:13
jochem-brouwer:Yeah, yeah, thank you, thank you. Yeah, so I think I am bringing this topic up every week, so that's very nice. So, again, we want to create state-intense benchmarks, and this is, like, these are kinds of new tests that
00:21:27
jochem-brouwer:Which we don't really have the tooling for yet to run these tests. And what we want to do is we want to run tests against,
00:21:35
jochem-brouwer:clients with, like, a large state, for instance, mainnet state.
00:21:39
jochem-brouwer:And then we want to see and to figure out what are, like, the worst-case situations here which can happen.
00:21:45
jochem-brouwer:I'm working with Camille from Nevermind on this tooling to run these tests, and of course, if we want to run tests, we also need to have the situations which we run on.
00:21:55
jochem-brouwer:Obviously, Xen is a candidate for this, because the Xen contact has a very large storage, and what I would like clients to think a bit about is what is actually the worst-case situation in your client.
00:22:09
jochem-brouwer:So, we are thinking about state, and we could say, okay, maybe deleting state is the worst-case situation, or updating state is the worst case situation, or writing new storage slots is the worst-case situation.
00:22:22
jochem-brouwer:That, I think, completely has to do with the implementation and the optimizations which a client does, which I have no ID for at this moment.
00:22:31
jochem-brouwer:Or maybe there are even more exotic situations where, well, a client might have optimized something very well, but now has a very specific area where the behavior is now very, well, bad. And of course, we need tests for this, so if you have, like, a very specific situation.
00:22:47
jochem-brouwer:where you would like your client to get a benchmark for, then please let me know. I can write these tests, I will
00:22:54
jochem-brouwer:I would love to write, write these tests. And,
00:22:59
jochem-brouwer:Yeah, we would really want that we have these tests, because we want to update the gas limit, we want to increase this on mainnet, that's fine, but we don't really have eyes yet on, like, these worst-case situations for the state.
00:23:12
jochem-brouwer:And we can… so we are writing this tooling now to run these tests, and I think that will be done very soon. And then we can use these to actually perform these tests in an isolated environment, so we can know beforehand what, like, the areas of optimizations for clients are.
00:23:32
jochem-brouwer:So if someone has, like, an idea for extra tests or other situations, then please let me know. But, yeah, let me know that the state benchmarks are coming very soon.
00:23:46
Mario Vega:Awesome, thanks. Is there anything from the east side? How are we approaching this, the testing of this?
00:23:55
Mario Vega:I'm curious about how our… Yes. I mean…
00:23:59
Mario Vega:To paraphrase my question, I think, are we including the big state on East, or how is the big state,
00:24:07
Mario Vega:Gonna be included in the tests.
00:24:09
jochem-brouwer:Yes, this is a very good question. So, the test itself is, so specifically this XEN test, but likely also every, every other test for the state benchmarks, they will be written in EAST.
00:24:23
jochem-brouwer:And what you will do is you will use the address step.
00:24:27
jochem-brouwer:to target a specific network, so, for instance, like mainnet, then you would use the XEN contract for your Xen tests. And what you would then do, and this is… this is why we need this extra tooling, so you do not… you can obviously not put the entire pre-state into the pre-state files of East.
00:24:46
jochem-brouwer:So what you will then do is you will spin up a node.
00:24:49
jochem-brouwer:And the current tooling, what we are using, is we have basically a CL mock, which drives the chain to run these tests on, and this generates the payloads for the client to run. And then to actually run this test, we replay these payloads on top of other clients, and then we see how they perform.
00:25:08
jochem-brouwer:And these other clients, they all run from the same snapshot.
00:25:13
jochem-brouwer:So, you basically, you start the chain, you use this snapshot, and then on top of this snapshot, you run the payloads generated by these East tests.
00:25:28
Mario Vega:The tooling that you mentioned that you're missing, is that on the east side, or on the client side?
00:25:34
jochem-brouwer:This is actually, well, basically a third-party tool currently. So, this is tooling, that's currently, it is, well, okay, this tooling is currently my own custom tooling, but Metamine's guest benchmark repository is, I think, the repository which will include this thing.
00:25:50
jochem-brouwer:But personally, I would say that this is also something which we would likely want in East as well.
00:25:58
Parithosh Jayanthi:I would actually see if we can have it as a fully independent tool with a way to hook it into EAST as well as another man tool, because I can also imagine the mock CL ability is something we can use elsewhere, right?
00:26:11
jochem-brouwer:Yes, that's true. Okay, maybe as a standalone tool, that would actually be maybe a bit more logical, yeah. Interesting, yeah.
00:26:21
Kamil Chodoła:Yeah, so… I was seeing that in the beginning, it's like, IST is having the definitions of the tests, and it can contain definitions for tests which are specific for the chains, like, we often is right, for example, on mainnet.
00:26:36
Kamil Chodoła:We can have specific tests for blotnet, we can have specific tests for any testnets, if you would like to.
00:26:42
Kamil Chodoła:And then we have the site tool link, which for now probably will be GAS Benchmarks, which will be responsible for generating the payloads. Here we are using some custom version of Netarmite, which will be generating the payloads on top of that state.
00:26:57
Kamil Chodoła:And then, also, like, what Gazelle actually have right now is the execution of those tests, using some VMs which are having,
00:27:08
Kamil Chodoła:backups of all of the clients ready just to be mounted. Test will be executed, and the state will be, afterwards, reverted to the initial state.
00:27:18
Kamil Chodoła:So, like, the separate tool, but it will always have all of the definitions of those tests.
00:27:24
Kamil Chodoła:Long term, we can see if we can integrate that to EAST, but probably, as Pari mentions, it's like, it's better to have the execution part being outside for that, because this is quite heavy, and not really… like, not to be very useful for EAST to have it, rather East to be a repository of tests for that.
00:27:46
Mario Vega:Makes sense. Thanks so much.
00:27:49
Mario Vega:Alright, any questions on this topic for Joachim? From clients, I mean?
00:27:59
Andrew Ashikhmin:Yeah, so I have a question, like, when you say snapshots, what do you mean by snapshots?
00:28:07
jochem-brouwer:So, a snapshot will be any… I'm specifically meaning, if PandaOps snapshots, but this is basically, like, I think PandaOps at every… yeah, thank you, Barry. Like, every 10,000 blocks on mainnet and also on testnets, I think it's every 10,000 blocks, or maybe 100,000 blocks, they release this snapshot, so you can directly download this snapshot and, use that as your
00:28:29
jochem-brouwer:Well, basically your state's database to run the test form.
00:28:34
Andrew Ashikhmin:Okay, I see, thank you.
00:28:48
Ameziane Hamlat:Yeah, I have a quick question, related to, these tests. How close are they from
00:28:57
Ameziane Hamlat:the transactions that we see on mainnet? Because, for example, when I look to your… to the comment, I see 7,700 as a store, and
00:29:09
Ameziane Hamlat:And, on base's side, if I remember correctly, when, we were looking into it, like, the most, like, the biggest bottleneck was actually S-load.
00:29:21
Ameziane Hamlat:even, like, on the Xen contract, transactions, and, which, which, which could be, you know, weird, like, when, at, at, like, first place, when we…
00:29:31
Ameziane Hamlat:when we know that, like, there are more SS stored than S loads, but…
00:29:36
Ameziane Hamlat:The thing is that we found that most of the S-load were actually on
00:29:42
Ameziane Hamlat:were actually, zero reads, so basically slots that, don't exist, in the state. So, I was wondering how…
00:29:52
Ameziane Hamlat:Is this something that you already have in the tests, or…
00:29:57
jochem-brouwer:Yes, okay, very good question. So, I think what you meant is that you, specifically were looking at replaying mainnet transactions, so replaying the mainnet situation.
00:30:08
jochem-brouwer:Well, this is doing, basically that, but even worse. And what I mean with worse is that on mainnet, what you see for the Xen situation is that tokens are being minted, and,
00:30:23
jochem-brouwer:this has some extra computational overhead in the contract itself. So it does, like, some extra logic. What I'm trying to do is I want to write, using the existing Xen contract, as much storage to the present contract as possible. And one of the ways to do this, and I'm not sure if this is actually the worst-case situation, but this is likely
00:30:44
jochem-brouwer:Well, bad enough, is to call the approve function.
00:30:49
jochem-brouwer:So what you do there is you can approve, like, another address to spend your tokens, even if you don't have any tokens yourself.
00:30:57
jochem-brouwer:So, for this situation, like, on the mainnet transactions, you will see, like, less S-stores and maybe more S loads. The tests which you saw in the comment is specifically looking only at S-stores, so that's why you see, like, only 8S loads.
00:31:14
jochem-brouwer:I will, of course, also implement a test, which focuses on the S loads, because this could, of course, also be a bottleneck, so these specific tests are only looking at the S-store, but this tries to squeeze out as much S-stores as possible on XAM.
00:31:30
jochem-brouwer:And I will do the same for SLoad.
00:31:33
jochem-brouwer:So yeah, this is basically a worst… well, a more worse situation than the things we have seen on mainnets yet, and of course, this is with the side note that I am running these tests on an isolated node.
00:31:47
jochem-brouwer:So they are not running, like, any networking or whatever, so that's more optimal. But in terms of execution itself, this should,
00:31:56
jochem-brouwer:well, in practice, it can happen, but likely no one wants to spend their money on this. So…
00:32:03
jochem-brouwer:This might be, like, a very good approximation of the worst-case situation for S-store on mainnet.
00:32:14
jochem-brouwer:And of course, if you have, like, any other ideas for these kind of benchmarks, like, please let me know, or if you see something weird.
00:32:20
jochem-brouwer:please reach out, because I might either have done something, like, super, super dope, or there's, like, a very good explanation for it, so please reach out if you have any cool ideas on how to make these situations even worse.
00:32:37
Mario Vega:Thank you. There's a great question from Maria on the chat, whether this is only seen on the same contract, or if there's any smaller contracts. I assume it may not, maybe, that exhibit the same behavior. Do you know if that's the case?
00:32:57
jochem-brouwer:I think a client team should, chime in on this, but I think all clients use, like, some kind of… well, for the Snap protocol, you have, like, a 01 lookup time for SLoad.
00:33:08
jochem-brouwer:So, I think it does not matter if you have, like, a very big contract or a very small contract, but of course, this also depends on the implementation and the optimizations.
00:33:23
Marius van der Wijden:So, exactly what, what Joachim said. The only, the only thing,
00:33:31
Marius van der Wijden:That can be different is if you are…
00:33:34
Marius van der Wijden:if you are loading… S-loading something that is, that is cached somewhere, or if you're S-loading something that is non-existent, or if you're…
00:33:44
Marius van der Wijden:as loading, like, existing… existing data. Those are the three different, different things, but it's not… not really dependent on the size of the contract where you're… where you're exploiting from.
00:34:04
Mario Vega:And there's, another question…
00:34:07
Mario Vega:So for Bessu, are there any changes in optimization that may make this different from other clients?
00:34:16
Mario Vega:Do you have someone from Besu that could, chime in?
00:34:20
Ameziane Hamlat:I mean, it depends on the question. For example, we had a few discussions during the interop with, with some…
00:34:30
Ameziane Hamlat:folks from Wrath, So, for example, Wrath, like, the… if I remember correctly, the bottleneck was on…
00:34:39
Ameziane Hamlat:the hash calculation, like, the root hash calculation. So, on our side, we, we preload the storage slots on the, like, during, during transaction execution.
00:34:55
Ameziane Hamlat:So, like, each time we touch, one slot, key, we preload, all, like, the, the branch, the branch nodes to…
00:35:07
Ameziane Hamlat:So then we can, you know, speed up the root hash calculation.
00:35:12
Ameziane Hamlat:Yeah, so that's, like, that's the main, I think, the main… the main optimization on the… on the root hash side.
00:35:19
Ameziane Hamlat:But I don't know if the question is related to something else. But, for example, for the S… like, for, like, the S-load performance on…
00:35:30
Ameziane Hamlat:on zero leads, so this is something we're already aware of. So there is, like, there is one small limitation on the LuxDB side, so there is a bug in OroxyDB that, so basically, which is basically, triggering, an issue
00:35:50
Ameziane Hamlat:like, an exception when there is, when there is a zero read, instead of just returning whatever, you know. So we… there is a PR that is pending, but it is not matched yet.
00:36:05
Ameziane Hamlat:Yeah, so yeah, apart from that, Yeah, that's all, actually.
00:36:12
Marius van der Wijden:Yeah, so…
00:36:13
Marius van der Wijden:Just to expand on this, we also have a, we also have a problem with non-existent weeds. Well, it's not a problem, but it's, like, whenever we do
00:36:24
Marius van der Wijden:When we read something that is non-existent, we need to go through all of the in-memory diff layers, and then go to the disk layer, which is a load of work for reading something that is non-existent.
00:36:38
Marius van der Wijden:So actually, like, the existent read is…
00:36:43
Marius van der Wijden:Quite a bit faster than a non-existent REIT for us.
00:36:59
Mario Vega:Yeah, and just to echo Johann's, if there's any particular interesting ideas that you want for him to write a test for, I think it would be very good if you guys can reach out. I think this particular scenario that you're mentioning, Marius, I think it would be very nice for us to also have a
00:37:18
Mario Vega:a compute test. Maybe there's something that we can do on the compute side.
00:37:23
Mario Vega:For that. For MTS loads, I mean.
00:37:31
Mario Vega:Anything else? Any other questions for Johan, or on this topic that we want to… Race…
00:37:39
jochem-brouwer:Yeah, I just want to note, as final point, that, as you mentioned, it would be very nice if people could reach out, like, this kind of situation, I would like to see tested. But my goal is also for myself to go into these clients, to all of the clients, to see what is being optimized and what is not being optimized.
00:37:57
jochem-brouwer:And to see there, okay, what's, like, the worst case situation that, well, how can we abuse this optimization, or how can we not abuse it? But it would be really helpful if I could have some help on that, because it's, of course, much harder to figure out the code on yourself, instead of talking to someone who wrote this code, yeah.
00:38:19
Mario Vega:There's also a nice point, by Ben. If…
00:38:23
Mario Vega:We could change as a store to not check the current value for pricing. So just to paraphrase that, we need… we would need an EIP for that, right? Just to basically not…
00:38:35
Mario Vega:Having the current value to be a precondition to calculate the gas pricing for that.
00:38:43
Ben Adams:Yeah, so… So, obviously the reason it does check the current price is so it can correctly
00:38:52
Ben Adams:Add additional… Costs for state growth.
00:38:57
Ben Adams:But it's also the most expensive part of S-Store, is that.
00:39:02
Ben Adams:Searching for the current value.
00:39:05
Ben Adams:And that's… that's where, usually, checking for a value that doesn't exist comes from.
00:39:25
Mario Vega:Thank you, thank you. Cool. One last comment from, CPRS,
00:39:34
Mario Vega:All of the case… he posted a link.
00:39:37
Mario Vega:I assume this is a list?
00:39:39
Mario Vega:Yes, this is a list, of test cases that will eventually make it into East.
00:39:46
Mario Vega:Just for everyone to go over this list, because this is something that we are already working on, so if your test case is not in this list.
00:39:57
Mario Vega:please chime in on make… or make a comment, I guess, that would be the best, approach. Make a comment into this list so we can add a test, a scenario that will eventually make it into East.
00:40:13
Mario Vega:Thanks. If nothing else on this topic, we can jump into Glamster then. So, the first agenda item is that we have a new Consistent Specs release that happened last Thursday, version 1.6 Vita 0. So, Justin, do you have any… anything else to add to this?
00:40:34
Justin Traglia:Not really, I mean…
00:40:36
Justin Traglia:Yeah, there's a new release, EPBS. Devnet Zero will be based off of this as well.
00:40:44
Mario Vega:Alright, thanks. Just for clients to be aware, just to consume the tests, I assume. Yep.
00:40:53
Mario Vega:So, yeah, and jumping into specifics, so we have, block-level access list and EPPS, for Glamstrom, so we can jump into, block-level access list first, and then into EPPS, if there's any topics. So, I see the first, comment, or first bullet point, sorry, is from Rahul.
00:41:12
Mario Vega:Let me see if he's in this call… Let me see…
00:41:22
Mario Vega:Oh, yeah, sorry, there you go. Yeah, can you chime in on the blog access list? Any, any updates that you want to share?
00:41:30
raxhvl:Sure. In our last release for block-level access list, that was version 1,
00:41:36
raxhvl:We focused on simple happy, path scenarios for block access lists.
00:41:41
raxhvl:And since then, we've been working on some complex test cases.
00:41:46
raxhvl:And throughout our testing, what we found is the intersection of block access lists and some existing EIPs, such as 7702, is quite nuanced.
00:41:55
raxhvl:For instance, when you create a 7702 delegation, the target account need not be read from the state for the delegation to be created.
00:42:03
raxhvl:This is because the target account can't be read from the address directly.
00:42:07
raxhvl:So we clarified this in the IP and added a negative test case to ensure
00:42:12
raxhvl:That clients do not perform a state read for the target account when creating a delegation.
00:42:18
raxhvl:We also needed to, address some, system operations, like self-destruct and precompiles, which required some further, clarification.
00:42:27
raxhvl:Those also have been updated in the IP now.
00:42:31
raxhvl:And throughout this month, we fixed several bugs within the execution specs, and the IP has been refined to some extent to cover all our learnings and edge cases.
00:42:42
raxhvl:That we've, discovered in the past month.
00:42:45
raxhvl:We now have close to 30 test cases.
00:42:48
raxhvl:That's currently under review.
00:42:51
raxhvl:And, we should, should expect a release soon, YAM.
00:42:55
raxhvl:Also, we'd like to proactively run test cases for clients. If you're a client and have not shared a dev branch with us.
00:43:05
raxhvl:please do so. This will ensure that, we release a polished version of test.
00:43:10
raxhvl:And finally, if you're client dev that's new.
00:43:14
raxhvl:to testing, or have any questions, feel free to reach out. Be happy to help you get started.
00:43:22
Mario Vega:Excellent. Thank you so much.
00:43:24
Mario Vega:Yeah, just to echo that, please reach out to Raul or Felipe,
00:43:31
Mario Vega:So they can, either point you to the latest block-level access list release in East, or they can help you out to run the tests for you if you provide the branch to do so.
00:43:45
Mario Vega:Any questions on block-level access lists, and specifics from clients, especially?
00:43:52
Toni Wahrstaetter:I'm not from a client, but let me ju- let me just add one thing. We had the block of Access List breakout call on Wednesday, and we discussed,
00:44:00
Toni Wahrstaetter:one important topic, that might affect more people than just those that were at the call. So it's about, do we want to do a block-level access list hash versus a block-level access list root?
00:44:13
Toni Wahrstaetter:I won't rehash the whole discussion here, so please check out the Discord. I have summarized,
00:44:20
Toni Wahrstaetter:The discussion, if you scroll up a bit, or you can also rewatch the breakout call.
00:44:26
Toni Wahrstaetter:But this is definitely something… it's a small change, it's still open, so right now it's a block of access list hash, but it might make sense to move towards a block of access list route. The change would be rather small, but still, if we really want to go for that, we should decide, yeah, as early as possible.
00:44:44
Toni Wahrstaetter:So please, if you have an opinion on that.
00:44:48
Toni Wahrstaetter:yeah, reach out, join the call, or, voice your opinion on Discord. And there's also a Eve Research post by Jason about, Block Live Access List X, pre-conf.
00:45:02
Toni Wahrstaetter:That also, highlights why a route might be, beneficial for some use cases.
00:45:12
Mario Vega:Excellent. Thank you so much, Tani. Just to surface one comment from the chat, Karim mentions that there's a message in Discord that there's a missing test, for BESU that is related to block load exit list that we should check.
00:45:26
Mario Vega:So, thank you for that. Make sure that we can check that.
00:45:31
Mario Vega:Cool. And the other point is that, we have this link provided by Eve Bandelp's. It,
00:45:42
Mario Vega:the block level access list.net specification. Just please come here and see if the information is okay to you.
00:45:52
Mario Vega:I don't think we have a date, I cannot see a date here, so…
00:45:57
Mario Vega:It might be not specified.
00:46:00
Mario Vega:So yeah, there's no date, it seems, it's only slated for late, yeah.
00:46:06
Parithosh Jayanthi:There's no date yet, we're still waiting for Kurtosis conflicts to look good, for us to get a better idea of local testing that we can do, and then once there's at least 3 clients, then we're ready to spin up a DevNet.
00:46:19
Mario Vega:Excellent, excellent, thanks. Yeah. Alright.
00:46:23
Mario Vega:Moving on, the last topic today is EPVS.
00:46:28
Mario Vega:We also have a link from EidpandaOps here. Let me share it in the chat.
00:46:34
Mario Vega:Yeah, same… similar things, we have a Devness slated for the end of October, and I think we are missing the client branches, say, in this case, too.
00:46:52
Mario Vega:Cool. So yeah, on the deeper topic of EPBS, is there anything…
00:46:57
Mario Vega:Any client wants to share on this, or do we have, like, more context from someone that can share?
00:47:03
Mario Vega:That can be shared here. Sorry, about EPVS.
00:47:13
Justin Traglia:I can probably just talk more. I mean, I know clients are working on implementing everything for the first DevNet. We'll know more in two weeks whether or not DevNet Zero can be launched at the end of October.
00:47:27
Justin Traglia:Clients should provide an EPBS DevNet-0 branch for ETH PandaOps, to test with.
00:47:37
Justin Traglia:There was some testing… Things that we need to figure out, and
00:47:45
Justin Traglia:Katya asked if there are any metrics needed for EPBS? I don't know.
00:47:52
Justin Traglia:That's pretty much it. I mean, we're just working on implementing everything.
00:47:59
Mario Vega:Alright, thank you, makes sense. Yeah, if there's anyone else that knows a little bit more about… I mean, probably there's no one right now, but, just try to keep an eye on metrics, I think that's a very interesting…
00:48:12
Mario Vega:Topic that we need to, resurface at some point.
00:48:17
Mario Vega:Thank you, Katya, for the question.
00:48:22
Mario Vega:Any opinions or anything that wants to be shared from clients regarding EPVS?
00:48:40
Mario Vega:Okay, cool. Yeah, that was the last, topic in the agenda for today. Is there anything that we missed that has to be discussed today?
00:48:52
Mario Vega:Please raise your hand now.
00:48:54
Mario Vega:Before we end the call.
00:49:01
Mario Vega:If not, I think that's it for today.
00:49:04
Mario Vega:Thanks, everyone, for joining. Yeah, let's,
00:49:07
Mario Vega:Let's see, hope that the,
00:49:12
Mario Vega:The activation in Holesky goes smoothly this Wednesday, and we can discuss it on the ACDC on Thursday.

Chat Logs

00:03:47
Mario Vega:https://github.com/ethereum/pm/issues/1736
00:04:28
Justin Traglia:yeah it’s expected. Doing more non-finality testing.
00:05:07
Justin Traglia:https://discord.com/channels/595666850260713488/892088344438255616/1422209211391283260
00:05:58
Parithosh Jayanthi:Sorry just got on the call
00:06:17
Parithosh Jayanthi:Did we miss devnet updates?
00:07:12
pawan:Did we transition to fusaka on that netwwork?
00:07:26
Barnabas:https://dora.fusaka-holsf-1.ethpandaops.io
00:07:32
Gabriel Trintinalia | Besu:teku-besu, correct?
00:07:41
Barnabas:Replying to "teku-besu, correct?" check interop please
00:08:46
Jae Hee Lee:Can we also get to test chaos testing Minhyuk implemented on devnet-3?
00:08:55
Parithosh Jayanthi:Replying to "Can we also get to t..." Oh yes! That’s a good idea too
00:09:04
Parithosh Jayanthi:Cc @Barnabas we should merge it in and try
00:09:27
Barnabas:Replying to "Can we also get to t..." Yeah we can do this in the second half of the week once we recovered the current non finality
00:09:29
Fredrik:It’s being merged to the blog soon
00:09:31
felix (eest):https://blog.ethereum.org/2025/09/26/fusaka-testnet-announcement#execution-layer-holesky-sepolia--hoodi-releases
00:09:41
Jae Hee Lee:Replying to "Can we also get to t..." Thanks!
00:09:45
James He:We have released a version to support fusaka, there are ongoing issues, from a team perspective it should be fine for testate
00:09:58
James He:But for some users there are issues
00:11:09
Parithosh Jayanthi:Our todo list for holesky in case we miss something: client releases shadowfork roll our holesky nodes validate holesky snapshots spin up one of every client on our stack tracoor for holesky, forky, dora dugtrio should be updated and running update checkpointz
00:12:48
Fredrik:Perhaps creating a Fusaka dir on the pm repo?
00:13:57
Pooja Ranjan:https://github.com/ethereum/pm/tree/master/Fusaka
00:17:35
Ameziane Hamlat:In besu, it is not included in the testnets release
00:19:01
Mario Vega:https://github.com/ethereum/execution-spec-tests/issues/1926#issuecomment-3333544495
00:20:13
Mario Vega:https://github.com/ethereum/execution-spec-tests/pull/2160
00:20:27
Louis:Update to apply eip7825 for Osaka
00:27:02
jochem-brouwer:XEN tests mainnet: https://github.com/ethereum/execution-spec-tests/pull/2101 (currently hardcoded addresses not using stubs yet)
00:28:11
Parithosh Jayanthi:https://ethpandaops.io/data/snapshots/
00:32:08
Maria Silva:Any idea if the bad performance on load for best is only for Xen? Or is it also for other smaller contracts?
00:32:42
Marius van der Wijden:Sload performance should be pretty independent of contract
00:33:03
Marius van der Wijden:Sstores are dependent on contract size
00:33:58
Maria Silva:Right, so for Besu are there any changes in optimizations that may make this different from other clients?
00:33:59
Ameziane Hamlat:Replying to "Sload performance sh..." Agreed, it is more related to the pattern, accessing non existing slot keys.
00:36:56
jochem-brouwer:For these optimizations or specific situations which are good/bad in clients I want to craft specific tests to signal areas where certain clients need (or need not) to spend some focus on 😃👍
00:36:58
CPerezz:Any LSM-based client should have an issue with non-existing leaves right?
00:37:07
Justin Florentine (Besu):i'd be surrised if that wasn't the case for everyone
00:37:12
Ben Adams:Could change Sstore not to check current value for pricing 🤔
00:37:54
Marius van der Wijden:Replying to "Could change Sstore ..." I would kinda like that
00:38:15
Ben Adams:Replying to "Could change Sstore ..." Is the most expensive (short term) part of sstore
00:39:17
CPerezz:FYI, all of the cases in https://hackmd.io/9icZeLN7R0Sk5mIjKlZAHQ will eventually land in EEST. So make sure that whatever you propose isn’t already there.
00:39:30
Justin Florentine (Besu):Replying to "Could change Sstore ..." we'd be giving up a brake on state growth but...
00:39:34
jochem-brouwer:Antwoord verzenden naar "Could change Sstor..." I'd also like to explore this a bit
00:39:50
CPerezz:Bloatnet scenarios being coded basically
00:40:08
Ben Adams:Replying to "Could change Sstore ..." Idk what right answer would be; but maybe something to look into
00:40:17
CPerezz:Anything works. Just reach out Jochem me or whatever is easier!
00:40:50
Maria Silva:Replying to "Could change Sstore ..." Any idea of the speedup we would have if we took it out of the cost formula?
00:44:24
Karim T.:@raxhvl if you can check my message on discord I think we found one missing test on Besu related to BAL .
00:45:33
Mario Vega:https://notes.ethereum.org/@ethpandaops/bal-devnet-0
00:46:35
Mario Vega:https://notes.ethereum.org/@ethpandaops/epbs-devnet-0
00:46:49
Justin Traglia:Here’s a short recap of our breakout call this week: https://x.com/JustinTraglia/status/1971589863280402721
00:46:55
Karim T.:Replying to "https://notes.ethere..." We will be able to share a besu image of Besu for BAL
00:47:43
Katya:Any metrics needed for EPBS?

Summary

13 highlights · 2 decisions · 4 action itemsExperimental

fork status and schedule

  • Holešky Fusaka activation this Wednesday 2025-10-01 08:48:0000:30:09
  • Holešky BPO1 next Tuesday 2025-10-07 01:20:0000:30:09
  • Holešky ShadowFork live and finalizing; transition in ~30 minutes00:07:08

testing progress

  • DevNet3 non-finality test running for 2 days to verify recovery00:04:34
  • BAL testing: 30 test cases under review covering 7702 edge cases00:41:30
  • State benchmarks tooling ready soon; need client input on worst-case scenarios00:21:29

client updates

  • Lighthouse release today for Holešky; blog post being updated00:09:25
  • 60M gas limit: Geth/Reth in releases; Nethermind/Besu for mainnet only00:15:39
  • Besu: Zero-read SLOAD issue has pending PR fix00:35:17

epbs and bal updates

  • BAL hash vs root discussion ongoing; feedback needed on Discord00:44:24
  • ePBS DevNet0 end of October; consensus-specs v1.6.0-beta.0 released00:47:26

organizational

  • Fusaka folder in PM repo needs testnet/mainnet plans like Pectra00:13:36
  • EEST benchmark tests restructured: compute vs state-intensive directories00:20:13

Decisions

  • Mainnet shadow forks delayed until mainnet releases available00:08:10
  • Benchmark tests split into compute and state-intensive categories00:19:38

Action Items

  • Alex Stokes: Create Fusaka PM repo structure with testnet/mainnet plans00:13:57
  • All client teams: Share BAL dev branches for proactive testing00:43:01
  • All client teams: Provide ePBS DevNet-0 branches to EthPandaOps00:47:36
  • All client teams: Share worst-case state scenarios for benchmark tests00:23:25

Targets

  • Wednesday 2025-10-01 08:48:00 - Holešky Fusaka activation00:30:09
  • Tuesday 2025-10-07 01:20:00 - Holešky BPO1 activation00:30:09
  • Monday 2025-10-13 21:10:24 - Holešky BPO2 activation00:30:09
  • Tuesday 2025-10-14 07:36:00 - Sepolia Fusaka activation00:30:09
  • End of October 2025 - ePBS DevNet-0 launch00:47:26