AllCoreDevs - Testing #075

|2026-03-23| Agenda: #1976| canonical JSON

Transcript

Mario Vega:Thank you. Good morning, everyone. Welcome to ACDT number 75. Today is March 23, 2026. Yeah, let's just get it started. We have an agenda comprising almost entirely of DevNets.

Mario Vega:And we are gonna decide whether we, split into two breakouts, further into the call.

Mario Vega:Because, it might, might or might not be necessary, but we can, we can start discussing, and then we can decide on two breakouts, like we did just, sorry, last week.

Mario Vega:Yeah, let's just get started.

Mario Vega:So yeah, first item in the agenda for today is the Blob DevNet Serum.

Mario Vega:I think, Barnabas, you might be the best person here to give us an update. Let me see if he's on the call…

Barnabas:Yep. I can give a quick update. So, in Blob Design Zero, we are testing the partial cell.

Barnabas:Tough, and Nethermind and Erigon is failing to stay at the head. This seems to be just an issue in their trunk branches,

Barnabas:I'm curious if, Ericon or another mind has a…

Barnabas:They will release that we could use instead of the, trunk branches, because the trunk branches seem to be just breaking way too often.

Barnabas:So, ideally, we should have a stable release with the GitLab V3, and then we could pin it to that, and we could resync it.

Barnabas:Unless people want to dive into… what the…

Barnabas:the breaking databases are. I have… I've pinged, both… both client teams with the…

Barnabas:the log messages that I see, and I'm just, curious to hear, what's up.

Barnabas:And on the other side, we're still waiting to onboard more, CL teams, once the partial cell implementations are done, for them.

Łukasz Rozmej:We will take a look into… into what you sent us. I… I wasn't aware yet, I haven't…

Łukasz Rozmej:Noticed it yet.

Łukasz Rozmej:Aww.

Barnabas:Yeah, just let me know if you need anything else.

Mario Vega:Alright. Anything from Erigon, you guys, if you guys want to chime in, or…

Andrew Ashikhmin:Yeah, so we are, finishing our 8037 implementation. Most… we are mostly done, still, like, maybe a couple of bugs remaining, and I saw a message from… was it from Stefan, or…

Andrew Ashikhmin:About the, the assertive, so I'll take a look into that, tomorrow, probably.

Mario Vega:Oh, I think, this is for blot definite0, is that correct?

Andrew Ashikhmin:No, no, no. It's about, about, about, 8037 testing.

Mario Vega:Alright, yep.

Mario Vega:But is there anything for Blob.0 from your side, or…

Andrew Ashikhmin:Nothing.

Mario Vega:from there.

Andrew Ashikhmin:No, I haven't looked at the block.0 issue.

Andrew Ashikhmin:Okay, I'll take a look tomorrow at it, probably, as well.

Mario Vega:Sounds good. Barnabas, anything else?

Mario Vega:Notice. Or below them is zero. Okay.

Mario Vega:Alright. Yeah, let's… let's move on.

Mario Vega:I think,

Mario Vega:Last week, there were some issues regarding Ball DevNet 3, and there were problems on clients staying together, if I understood correctly, so let's just jump into Bald Devna 3 issues. Stefan, I think you're the best person here to give an update, or also Spencer?

Mario Vega:Oh.

Stefan Starflinger:Yeah, sure. So… We are getting some updates from clients in,

Stefan Starflinger:I think Bezle pushed an update today that I will look into. Erigon has been pushing a lot of updates over the weekend. I'm just looking into those right now. It seems there has been a lot of progress.

Stefan Starflinger:Nimos EL and Etherix are, very far ahead. I think they're pretty much done regarding the testing.

Stefan Starflinger:But maybe Spencer can… say a little bit more regarding the testing.

Stefan Starflinger:I think they're still with,

Stefan Starflinger:Guess there might still be issues regarding, the contract size increase.

Stefan Starflinger:I'm not sure if that's just some error mapping, or if there's still some… some tests. Bezo, I hope, will be passing all the tests with the recent push. I will… I'll fix that. Bezo had 19 failures.

Stefan Starflinger:Goethereum still has 49 failures, and Nethermind,

Stefan Starflinger:still has… and Reth, they both have still over 400 failures, so they're still actively developing.

Stefan Starflinger:But I think we're pretty close now, from the implementation, and I will be doing a lot of testing this week.

Stefan Starflinger:Over to Spencer.

spencer:Hey, yeah, I don't really have…

spencer:have that much to add. I think,

spencer:Yeah, a lot of the failures on Hive are, our exception mapping failures, so… Yeah, I guess…

spencer:over the next day, day or two, I'm definitely gonna be pushing to get all the tests passing.

spencer:Passing on Hive, yeah, there's been good progress over the past… the past week.

spencer:It's worth mentioning that

spencer:We might have another spec change, like, tiny, tiny change for, like, a single test.

spencer:but we can, deal with that for any other future spec changes to 1837.

spencer:And the next definet. So, I think, we're all in agreement to stick with what we have for now, get everything passing, and then…

spencer:future updates can happen for DevNet 4.

spencer:The bug that Stefan found on Erigon.

spencer:And we don't actually have a test for that, so…

spencer:Yeah, I'll get working on that as well.

Mario Vega:Sounds good. Any comments on DevNet 3?

Mario Vega:Either specific to 837 or anything else, from clients.

Mario Vega:Who want to chime in? Any issues with the tests?

Mario Vega:Or, concerns?

Dragan Rakita:From Redside, I run… I have run all the… the state tests, the latest 5.5… 5.5.1,

Dragan Rakita:There are a few inconsistencies, but as Spencer said, we'll move that changes to the DevNet 4?

Dragan Rakita:So… Yeah, des… Text feature seems okay, no.

Mario Vega:Alright, anything else from other clients, basu, Nimbus, or… Geth?

Mario Vega:Are we doing okay with this, or…

Mario Vega:Do you guys need help?

Mario Vega:Great.

Mario Vega:Great. If not, I think the next point in the agenda is, block load access list definite number 4.

Mario Vega:I do feel like we're still, like, trying to iron out the issues on DevNet3, but I wanted to open the floor for discussion if we want to start,

Mario Vega:Yeah, discussing, WN4, changes.

Mario Vega:Stefan, how do you feel here? Do you feel it's too soon, or should we defer this to next week, or should we start…

Mario Vega:This week.

Stefan Starflinger:Yeah, I think it would make more sense to defer from my perspective.

Stefan Starflinger:But I'm open to discussion if anyone wants.

Mario Vega:Yeah, I agree. As we…

Stefan Starflinger:can talk about, like, would we want to create a Glamsterdam Definite 1, instead of having a Battle Definite War?

Mario Vega:That's a great question. I think we should,

Mario Vega:wait until we discuss the EPVES Debit 1.

Mario Vega:Actually, we should jump into that.

Mario Vega:And then, and then discuss cards are done, definitely one.

Mario Vega:Alright. So, ePBS.Net1, do we have someone with the status on this?

Barnabas:Epv.1 is currently on hold until we can make a decision on the PTC questions.

Mario Vega:Okay, great.

Mario Vega:Alright, so, at this point, we should decide whether we want to do the breakout into separate rooms, or we want to stick into this. I feel that the only decision that has to be made for ePBS is PTC.

Mario Vega:And… from the sound of it, this looks like it's gonna be a quick one.

Mario Vega:So, should we start just discussing that, and see if we can close it out, and then jump into block-level access to this right here? Is that fine by everyone?

Mario Vega:Yep.

Mario Vega:Alright, yeah, Botus, stick it away if you want.

Potuz:Oh, I… we said that we would have clients coming with options, so I started asking clients what they prefer.

Mario Vega:Sounds good. Alright, let's start with the client's,

Mario Vega:Either any of the SEAL clients would want to kickstart the conversation of BTC?

Stefan Bratanov:Yeah, I can talk for Teko. I think we…

Stefan Bratanov:Well, from my perspective, I prefer the newest version.

Stefan Bratanov:Which is basically, computing the PTC during process floods.

Stefan Bratanov:Makes the state size a bit smaller.

Stefan Bratanov:The only thing is the additional computation process loads.

Stefan Bratanov:But I don't think it's that much of a big deal. It's a good,

Stefan Bratanov:Alternative to the previous one.

Mario Vega:Yeah.

Stefan Bratanov:The one that, Justin showed.

Mario Vega:Alright, any other clients I want to chime in?

Mario Vega:And with, preferences?

Eitan Seri-Levi:For Lighthouse, I don't… we don't have a super strong preference, but we think it might be useful to have the PTC in the state.

Eitan Seri-Levi:So that we don't have to, like, recalculate it for older states, if we ever need that for some reason. But I think it's maybe more useful to have the full, PTC, for the epoch instead of just the last one.

Eitan Seri-Levi:But yeah, we don't have, like, a super strong opinion.

Mario Vega:Any other CL clients want to chime in on PTC?

Dustin:My tendency is to prefer things in-state to not in-state for actually testing reasons, largely. I think we've seen

Dustin:Hmm.

Dustin:Well, many bugs over the years, that amount to the caching was wrong.

Dustin:And across every client, essentially. I don't… I'm not gonna single anyone out. And this both creates… by putting in a consensus, means that, like, that there will be…

Dustin:A consensus test… testable thing, and then that will be, sort of, the same across, sort of.

Dustin:Tests were tested in that regard.

Dustin:I don't have… yeah, that's sort of my bias, I guess, here.

Mario Vega:Yeah, but this?

Potuz:Yeah, for Prism, it's essentially the same thing that Dustin just said. We… we would prefer having this on the stage. It's not a strong preference. 50-20 works. 42.92, I think, is the other option, also works. It's fine for us.

Potuz:But our preferred method would be not only the previous version that has two epochs in the state, but actually add an extra one, and keep three epochs. One for the previous epoch, and the next two epochs, so that we have duties also.

Potuz:In a finalizing chain.

Potuz:we should, by just having the dependent route on the state, we can just pick any state, any head state, or any state on any branch, and it would just give us the PTC

Potuz:It will solve completely what caching is, and yes, it adds, between 256 and 384 kilobytes to the state, but the state is several hundred megabytes anyways.

Potuz:So, it avoids hashing, it avoids rehashing, you only need to compute it once in epoch,

Potuz:it solves completely the caching, and, as Dustin said, it's testable on spec test. I think its pain on space on the state is not

Potuz:There's not much punishing.

Dustin:Yeah, I would generally… I mean, to me, the size is not the argument, I mean, against that, if such an argument exists. I'm sure some people are bothered by that size, but given that, as you say, the size is several hundred megabytes and growing with every validator,

Dustin:Because, of course, that's… it's not only the active validators, then… This is…

Dustin:the relative… the ROI is pretty high on this for me, even for 256 or 384 kilobytes.

Mario Vega:Alright, mark, any other guys who wish to chime in on this? I feel like…

Mario Vega:It's a little bit contentious, so it seems to me.

nflaig:Yeah, so from Lodestar's side, I guess we are also fine with all three options, to be honest. So I was working on, the 4992PR,

nflaig:And found it a bit wasteful that we have to process every slot, and in Lodestar, we didn't even use that data, because we have the cache already.

nflaig:So the… the modifying the state each slot was not ideal, in my opinion. That's why I brought up, 50-20, which is more minimal, and you only need to update the state once.

nflaig:But I'm also in favor of the other option that Paul just mentioned, where you cash restates.

nflaig:Because this also solves, a problem, or it's not a real problem, but basically, right now, if you, you need to get the duties for the whole epoch at slot zero, and if you're the validator and you have a slot zero duty.

nflaig:You basically have very little time to prepare for that.

nflaig:I think for PTC, this would probably be fine.

nflaig:But yeah, still, I think on duties it's even good to have another epoch of look ahead.

nflaig:So I would also be fine with that change.

Mario Vega:Alright, it doesn't seem like we have a… A strong… Decision.

Mario Vega:Pathos, do you feel like, should we take this into the breakout room?

Mario Vega:And then make the decision, or…

Potuz:I think we could make a decision here. It seems that it was essentially everyone on the same PR except Techu.

Potuz:And even within Checo, I see Enrico being convinced otherwise.

Enrico Del Fante (tbenr):Yeah, I think, like, the testability argument and this lot zero timing, that has been…

Enrico Del Fante (tbenr):Improved on the block proposal.

Enrico Del Fante (tbenr):and we get something like that for BTC as well, I mean, I think, like, at the end makes sense to have

Enrico Del Fante (tbenr):Some bias toward having more data on state, and having more stable things, and testable things.

Enrico Del Fante (tbenr):That's my personal…

Mario Vega:Alright, do… are we missing any sealed lines from commenting?

Mario Vega:Alright, should we make a decision, or what is how do you feel? Or what's the…

Mario Vega:What information are we missing to make a decision?

Potuz:I think we've already made a decision, it seems.

Mario Vega:Yes, and I see your comment on the chat,

Mario Vega:What, specs and tests are you talking about?

Mario Vega:Do you have a PR for that, or…

Justin Traglia:I don't know which PROTUS is gonna update, but he said he was gonna make the changes that we just agreed upon.

Justin Traglia:Here. So updating it to be 3 epochs worth of PTC.

Justin Traglia:Values.

Justin Traglia:Yeah, to be clear, the PR's not ready yet, but it will be in a few minutes.

Mario Vega:Alright, so… steps forward should be… seems like, it's updating the PR,

Mario Vega:and then merging it, and then that's the path we're gonna follow for EPBS, Definite 1.

Mario Vega:Is that… is that correct?

Justin Traglia:Sounds correct to me.

Justin Traglia:And then, we can produce a release with this, shortly. Maybe end of this week.

Mario Vega:Excellent.

Mario Vega:Alright.

Mario Vega:And one last question from Barnabas, do we do variable PTC deadline?

Mario Vega:Just to close it out.

Potuz:No, we were never going to test benchmarking, on DevNet 1. DevNet1 is supposed to be about testing the new structures.

Potuz:So, we don't even want to be dealing with the builder API, for example, so we're gonna be just only testing on DevNet1,

Potuz:external builders, Withdrawals from builders and these kind of things.

Potuz:But not even the Bilaterabia.

Barnabas:No, I meant, do we do variable PTC deadline in.

Potuz:Yeah, so that's something that is just not a hard fork, it's not… it's more about benchmarking. I think those kind of things should be discussed, like, several weeks, months from now.

Potuz:At any rate, there's… Tony has pulled up a document.

Potuz:arguing against variable B2C. There doesn't seem to be even consensus on whether we want this or not. It's a trivial change that we can add at any time if it's

Potuz:deemed, important.

Potuz:I don't think we need to discuss this now. We can just focus on testing what

Potuz:We have a broken Definite Zero now. We can just focus on when we're gonna ship Definite 1, when we're gonna have this PR, and just continue testing the structures, not the bench.

Mario Vega:Okay, great. So, Justin, share the document in the chat, please take a look, regarding this.

Mario Vega:Cool.

Mario Vega:If nothing else, I think we can jump into block-level access lists now.

Mario Vega:Yeah, I think, Tony, you have a couple of topics that you wanted to raise, are you ready to…

Mario Vega:To go ahead with this?

Toni Wahrstätter:Yes, the agenda for, the Battle Breaker would be very…

Toni Wahrstätter:small today, so I think we can also just keep it in that call.

Toni Wahrstätter:basically asking clients how far they are with their optimizations, especially focusing on the parallel batch I.O. And we should also have some first numbers on the execution parallelization, and it would be great if we

Toni Wahrstätter:Yeah, if we get an update on that.

Toni Wahrstätter:Yeah, maybe if someone… if someone from Benchmarker is on the call, or…

Toni Wahrstätter:People test, involved in the testing.

Toni Wahrstätter:Would be great.

Toni Wahrstätter:We can't hear you, Johann.

Marius Van Der Wijden (M):Well, yeah.

Marius Van Der Wijden (M):I hope that Johan comes back with his microphone, but until then,

Marius Van Der Wijden (M):There has been only, I think, one or two test runs with the…

Marius Van Der Wijden (M):With block access lists, yet.

Marius Van Der Wijden (M):But the one that I looked at…

Marius Van Der Wijden (M):Had a lot of failures, but…

Marius Van Der Wijden (M):The tests that were not failing.

Marius Van Der Wijden (M):And those are most likely, like, unrelated failures. The tests that were not failing, they were significantly faster.

Marius Van Der Wijden (M):So, yeah, the parallel execution seems to have…

Marius Van Der Wijden (M):Pretty good benefit on the worst cases in execution, just as we…

Marius Van Der Wijden (M):assumed, so, that is very nice. I don't know if…

Marius Van Der Wijden (M):Rafael is on the call, or Johann, has this microphone off.

jochem-brouwer:Yeah, let me check. I see a green thing, so maybe you can now hear me?

Toni Wahrstätter:Yes, it works.

jochem-brouwer:Okay, great, nice. Okay, so for the benchmarks, we already have a lot of benchmarks, but for the clients, we would need, we need these flags. Especially the prioritization is now the batch I.O, so we need a flag.

jochem-brouwer:for clients to enable slash disable the batch I.O. optimization. So this batch I.O. optimization is that you read the, well, your prefetch.

jochem-brouwer:the, block access list, reads, so that you can quickly get them when they are necessary, when you are executing the, the benchmark. I see on the Ball Definite 3 spec, and also on the ball definite 2 spec.

jochem-brouwer:That's,

jochem-brouwer:Nethermind, Eragon, and REF do not have the… sorry, Ball Definitely, Nethermind and Eragon do not have the batch I.O. flag, so, we cannot,

jochem-brouwer:we cannot run the clients with this batch.io flag enabled slash disabled, so we cannot see the, well.

jochem-brouwer:the effect of this optimization, so the request for these clients, so NetherMind and Irigone, is to enable this flag to specifically

jochem-brouwer:allow this optimization. Then for, the…

jochem-brouwer:other part, this is about the filling of these tests. Camila's been working really hard on this, but just to check here, for these benchmarks, I think the only thing that we need here, or well, yeah.

jochem-brouwer:the numbers what we need here is the performance, and we don't need, like, a trace of this. I just want to check with everyone if this is correct, so we do not need, like, an opcode count, because this is necessary for the gas repricing, but we don't need this for the FX for the batch I.O, is this correct?

Toni Wahrstätter:Sorry if that was directed to me, could you quickly repeat that? I didn't get it.

jochem-brouwer:Yeah, it was more like a general question, but for the batch I.O, the thing that we need here is, like, we need a comparison between the runs of clients where the batch I.O. is enabled and the batch I.O. is disabled, and we do not need, like, an opcode count, so, like, a debug trace. Is this correct?

Toni Wahrstätter:Yes, exactly. We're mostly interested in stateful tests between enabling it and disabling it.

jochem-brouwer:Okay, great. Okay, then, Camille,

jochem-brouwer:the book here is that we need to fill these tests, and we are currently using Nethermine to do this, but there seems to be a bug or, like, a memory leak in the bug building part. So, I would suggest, or, well, yeah, let's use another client to fill these tests, so that we can start running these stateful tests also.

jochem-brouwer:Because we're currently running the complete test, but that's of course not…

jochem-brouwer:But it should not have a large effect on, on the batch I.O. numbers.

jochem-brouwer:So let's fill those on another client, like Peso or GEF, or whatever, supports filling these.

Kamil Chodoła:Yeah, I will… I will try to use any of those, and still will be debugging what's happening here, Nethermind.

Rafael Matias (skylenet):Also from my side, so we do already… so, end of last week, I was able to run Geth.

Rafael Matias (skylenet):and BESU, like, with and without optimizations, and without balls at all, like, sequential. And, yeah, I shared, like, the results with, Tony and some other folks, and you could definitely see, like, the performance, gains using balls.

Rafael Matias (skylenet):And today, Stefan also told me that Wrath also has these flags now, so I'm just right now doing a run with Wrath.

Rafael Matias (skylenet):And yeah, once it's finished, I will also share the results.

jochem-brouwer:Yeah, I want to quickly add, because we are really interested into this, specifically, this batch I.O. thing. So for the clients, we need this endpoint where this prefetch is enabled or not, and not that we disable, like, the entire block of access list optimizations without it, because then we are also

jochem-brouwer:Taking out, like, parallel execution, and other optimizations which clients might have.

jochem-brouwer:This batch I.O. is necessary because we have to decide, yeah, on the final spec of the dock access.

Rafael Matias (skylenet):Yeah, so right now it's just Nethermind and Erigon missing.

Rafael Matias (skylenet):On that front.

Marius Van Der Wijden (M):But Raphael, you're running the test on Definite2 spec right now, right?

Marius Van Der Wijden (M):Okay, when… And we expect to update that to .3 spec?

Marius Van Der Wijden (M):once DevNet3 is launched, I guess?

Stefan Starflinger:Yeah, exactly. That's… once it's stable enough, then we switch over to Definite 3 specs for the benchmarks. The idea is to still have Definite 2, just…

Stefan Starflinger:So we can do a first iteration of the benchmarks.

Stefan Starflinger:But not necessarily for the numbers to make a decision on.

jochem-brouwer:Okay, so just to iterate on the asks, so this is an ask for every GOM and for NetherMinds to add this flag to enable or disable this batch I.O. optimization.

Stefan Starflinger:Yeah, and if Nimbus EL or Aetherax also added, that would also be great.

Andrew Ashikhmin:Well, we don't have that optimization in Erigon yet, so we just have to implement it.

Dustin:Sorry, could you clarify what the… Flaggers, exactly?

Stefan Starflinger:So if you look at the spec, there are 3 different flags,

Stefan Starflinger:And the most important one is the no prefetch, or the prefetch one, where there is an optimization to read all of the state.

Stefan Starflinger:In the beginning, and we want to enable and disable that to see the performance.

Dustin:Right, and this is prefetched, like, in isolation, right? Not the other parts, right? That's…

Stefan Starflinger:Yeah, exactly, in isolation, yeah, to just trigger this one optimization.

Dustin:Right, right. Okay, understood, thanks.

Rafael Matias (skylenet):Yeah, I've seen some people have asked, like, for the link of some results, I'm just getting it, and I will share it here in the chat.

Mario Vega:Okay, Tony, is that the entirety of blog topics, or do you want to do… Discuss something else?

Toni Wahrstätter:No, I think that's it.

Toni Wahrstätter:If no one else has something to discuss, so those optimizations and the first benchmark results are basically it.

Toni Wahrstätter:So far, it looks like baseline GEF get a 3X out of the execution parallelization.

Toni Wahrstätter:And yeah, we are still working on the stateful tests to test the batch.io.

Rafael Matias (skylenet):Also worth to note the 3X we're mentioning, that was, with a limit of 4 CPUs.

Rafael Matias (skylenet):Yep.

Mario Vega:Alright, sounds good.

Mario Vega:Any other topics regarding block.access lists? There's one last topic that my client brought up in one comment on the agenda, so I wanted to jump into that.

Mario Vega:But I want to check first if there's nothing else from blog.access list that we want to discuss today.

Mario Vega:All right, cool. Like client, are you there? I think there's one comment from you.

Mario Vega:Yeah.

lightclient:Yeah. Hey guys.

lightclient:Just wanted to quickly bring up the documents that we came up with last week on the frame transaction then pool.

lightclient:I know this was one point of contention on the last All Chords at its execution.

lightclient:call.

lightclient:So… The way that we've written it is we first wrote a…

lightclient:kind of, like, broader HackMD.

lightclient:That goes over…

lightclient:the general ideas of, like, how do you build a mempool for native accounts abstraction? We talked about the… some of these, like, shared validation rules. These are things that were really pioneered by the ERC7562.

lightclient:And then we lay out a handful of strategies that we feel are, like, reasonable approaches to building the actual transaction pool.

lightclient:Then the second link is a PR to the EAP, which is a more detailed implementation of the spec for Strategy 2.

lightclient:I can quickly just, like, say the strategies. The first strategy is self-relaying.

lightclient:Oh, so that means that the…

lightclient:Consensus nodes all support users who submit AA transactions and are going to pay for their own gas.

lightclient:This is, like, you know, the standard use case.

lightclient:Today, for transactions?

lightclient:But they don't support in the transaction pool gas abstraction.

lightclient:So it's still supported by the actual changes in the protocol, but we aren't going to, in that strategy, allow people to propagate, like, bundled transactions.

lightclient:And I think that if we did strategy one, it's quite simple. The risks are fairly minimal, in many ways, similar to what we already have with 7702 transaction pool.

lightclient:But the sponsored transactions will have to go through some, like, private flows, rather than being in the public mempool.

lightclient:We feel that this is… this is a really good strategy, and it's a good fallback if we decide to do strategy two, which is kind of what our preferred, you know, preferred strategy is right now. It's a good fallback if, like, through that process, we have too much contention that we can't move forward, or we find problems that,

lightclient:Are not easily solvable.

lightclient:So, that's, you know, that gives us native cano structure of the protocol. It's, you know, pretty easy and straightforward. The second strategy is the canonical paymaster strategy, which is, you know, still supporting self-relaying, but we also support bundlers who use a canonical paymaster.

lightclient:And the reason we can use this canonical paymaster in this instance is because we canonicalize a paymaster that has certain properties which the protocol can reason about.

lightclient:And that we know that it's safe to allow many transactions to depend on for validity.

lightclient:And, you know, without, like, explaining, like, in detail, essentially what we need the canonical Paymaster to do is to

lightclient:once it receives Ether, once it has balance, that balance doesn't leave the contract unless it's paying for the execution of another contract. Or, sorry, another user. And so, like, with some of these building blocks, we can construct a pretty safe transaction pool from that.

lightclient:So that's our preferred approach right now. If we were to go the full route and do totally permissionless deployment of any kind of Paymaster logic, then we would need to do the full ERC7562.

lightclient:And, you know, that is pretty well specced out, but it does bring into…

lightclient:the protocol, like, you know, staking for paymasters, and a reputation system for paymasters. And so, we felt that while there are nice benefits of it, because you have high flexibility.

lightclient:The complexity is just too high for us to recommend going for this approach from the beginning.

lightclient:So that's… What we're thinking on the transaction pool.

lightclient:Happy to answer any questions if they exist right now.

lightclient:Also happy to have a breakout room sometime this week if people would rather have, like, a longer-form venue to discuss this stuff before we jump to ACDE on Thursday. Just, like, let me know what people are feeling.

Mario Vega:Thank you. Yeah, I agree that a breakout… even for personally, myself, I would love to join and discuss this, and…

Mario Vega:Okay.

lightclient:I'll try and schedule it for Wednesday.

lightclient:That's good with people.

Mario Vega:Awesome.

Mario Vega:Yeah, thank you so much.

Mario Vega:Yeah, thanks for sharing this. This was the last topic in the end of today. Is there any other topics that we have to discuss today anyone wants to bring up before we close the call today?

Mario Vega:Alright, if not, I think it's, time to close it. Thanks, everyone, for joining, and yeah, have a nice week.

Potuz:Hi, everyone!

jochem-brouwer:Thank you, bye.

Marius Van Der Wijden (M):What do I think you…

Łukasz Rozmej:Bye.

Chat Logs

Parithosh Jayanthi:Replying to "Potuz flexing some m..." Sometimes I use the research room whiteboard as a background to seem smarter

nixo:Replying to "why rare?" ¯\_(ツ)_/¯ it’s the differential geometry variant

Potuz:Replying to "why rare?" ah, yeah, gave manifolds last semester, that shows how often I use my office's board :)

Potuz:Replying to "why rare?" surprised you can recognize the topic :)

Marius Van Der Wijden (M):49? I have 3 locally

Łukasz Rozmej:a lot of our failures are error message missmatches - we will fix them soon, then we will see what exactly is the problem

Marius Van Der Wijden (M):I will look into it

Barnabas:I’d propose to not have a bal-devnet-4 but instead try to scope glam-devnet-0

Justin Traglia:This is the newest version, right? https://github.com/ethereum/consensus-specs/pull/5020

nflaig:but that doesn't run anything in process slots, it's in process epoch

Potuz:yeah I didn't understand Stefan's comment

Potuz:lol

Potuz:5020 is not in process slots

Enrico Del Fante (tbenr):That’s a convincing argument to me actually

Marius Van Der Wijden (M):How big would it be in state?

Dustin:256-384kB is tiny and this is an O(1) thing

Dustin:state is huge

nflaig:it same as 96 new validators from what I last checked if we have 3 epochs in state

Potuz:Nico?

Marius Van Der Wijden (M):And it churns per epoch?

Potuz:someone from Lodestar?

Barnabas:everyone is happy with every option then nobody is happy about any options lol Can we please agree to do one today on the call please?

Eitan Seri-Levi:definitely prefer having the full epoch worth (or three epochs worth) of committees instead of just the last PTC of the epoch if we’re going to do this

Potuz:Replying to "everyone is happy wi..." yeah, we'll decide

Potuz:Replying to "everyone is happy wi..." don't worry

Potuz:if we go with the epochs one then I think we can agree on making it 3 epochs and I'll modify the PR

Potuz:I'll have it ready in 5 minutes

Justin Traglia:Replying to "I'll have it ready i..." Be sure to update tests & specs for other upgrades (heze) too 😄

Barnabas:which pr is the final one then?

Potuz:Replying to "I'll have it ready i..." nah, I'll have the core changes, and will work tests after

Barnabas:can we please close the other 2 open PRs ?

NC:I think it’s the one that was closed before?

Potuz:https://github.com/ethereum/consensus-specs/pull/4979

Enrico Del Fante (tbenr):New one or change 4979 ?

Potuz:I'll update this one

Enrico Del Fante (tbenr):ok

Barnabas:and do we do variable ptc deadline?

Justin Traglia:Replying to "and do we do variabl..." In general, it doesn’t sound like there’s sufficient support for it.

Marius Van Der Wijden (M):Months??? In months we should be finishing H*

Justin Traglia:Toni’s doc: https://hackmd.io/@Nerolation/B1GKtAUcZg

Parithosh Jayanthi:Replying to "Months??? In months ..." Yeah… that was my understanding as well

terence:Replying to "Months??? In months ..." Its fine to not have it at the time of the fork though

terence:Replying to "Months??? In months ..." We can always roll it out later

Toni Wahrstätter:I agree, since it's a simple (based on clients), non-consensus critical change, it can be further postponed

Jihoon:Replying to "https://github.com/e..." FYI, I've closed #4992 and #5020 based on this decision.

jochem-brouwer:Can you hear me?

Mario Vega:Replying to "Can you hear me?" no

Ameziane Hamlat:Replying to "Can you hear me?" no

jochem-brouwer:lemme rejoin

lightclient:sry had to step away for another call

terence:Is epbs done? can we leave and go back to coding? lol

jochem-brouwer:https://notes.ethereum.org/@ethpandaops/bal-devnet-3

Mario Vega:Replying to "Is epbs done? can we..." Yes it’s done 🙂 please take a look at the PR shared

Łukasz Rozmej:Still working on it :(

Stefan Starflinger:Replying to "Can you hear me?" reth, besu, geth have it Nimbusel,ethrex and nethermind are missing the flags

Stefan Starflinger:Replying to "Can you hear me?" And erigon missing

Karim T. (matkt):Should we try with and without state root in parallel also ?

nflaig:Replying to "https://github.com/e..." https://github.com/ethereum/beacon-APIs/pull/592 need to re-add next epoch lookahead again

Karim T. (matkt):@Rafael Matias (skylenet) where we can find your result ?

Karim T. (matkt):Replying to "@Rafael Matias (skyl..." thanks

lightclient:https://hackmd.io/@matt/frame-mempool https://github.com/ethereum/EIPs/pull/11415

Rafael Matias (skylenet):Replying to "@Rafael Matias (sk..." Example of a besu comparison: https://pop-os-shiny.taile8f668.ts.net/compare?runs=1774000622_2696dbe8_besu-bal-sequential%2C1774002127_64a46368_besu-bal-optimized%2C1774005008_106db2cd_besu-bal-noprefetch&labels=instance-id&tableBase=%220%22&filter=memory_ (Might be a bit slow to load)

Barnabas:Replying to "https://hackmd.io/@m..." how is the PoC coming?

Karim T. (matkt):Replying to "@Rafael Matias (skyl..." @Rafael Matias (skylenet) A (besu-bal-sequential) do you have conflict to be sure it’s really sequential ?

Potuz:I pushed the minimum change to add 3 epochs to 4979, lint fails locally who knows why (Python's package managers sucks) And I still need to check if the computation will succeed at that stage of epoch processing (before updating the slot in the state) but the idea should be clear to align on this and just ship devnet 1

lightclient:Replying to "https://hackmd.io/@m..." it’s coming

lightclient:Replying to "https://hackmd.io/@m..." sorry have to decide if i’m going to join a nazi suicide cult first

Summary

18 highlights · 4 decisions · 4 action itemsExperimental

blob devnet updates

blob-devnet-0: Nethermind and Erigon failing to stay at head on trunk branches00:03:14
Need stable releases with GetBlobV3 instead of unstable trunk branches00:03:38

bal devnet progress

bal-devnet-3: Nimbus EL and Etherax nearly done with testing00:06:48
Besu 19 failures, Geth 49 failures, Nethermind/Reth 400+ failures remaining00:07:04
Many failures are exception mapping issues, not logic bugs00:07:51
Minor spec changes deferred to bal-devnet-4 to stabilize current implementation00:08:24

epbs specification

Consensus reached: Store 3 epochs of PTC in state (prev + next 2)00:13:15
PTC in state enables testability and eliminates caching bugs across clients00:15:45
State size increase: 256-384KB, negligible compared to hundreds of MB state00:16:44
Variable PTC deadline postponed; non-consensus, can add later if needed00:23:55

bal benchmarking

Batch I/O flag urgently needed from Nethermind and Erigon for benchmarking00:27:10
Flag must isolate prefetch optimization, not disable all BAL optimizations00:27:47
Geth showing 3X speedup from execution parallelization (limited to 4 CPUs)00:35:53
Nethermind block-building memory leak blocking stateful test generation00:30:54

network planning

Proposal: Replace bal-devnet-4 with Glamsterdam-devnet-000:11:05

account abstraction

Frame transaction mempool: Strategy 2 (canonical paymaster) preferred approach00:37:06
Strategy 1 (self-relaying only) fallback if Strategy 2 faces issues00:37:52
Full ERC-7562 permissionless paymasters deemed too complex for initial deployment00:38:22

Decisions

Store 3 epochs of PTC in consensus state (previous + next 2 epochs)00:19:31
Variable PTC deadline deferred; can add post-fork if needed00:24:27
Defer minor 8037 spec changes to bal-devnet-4 to stabilize devnet-300:08:24
ePBS devnet-1 focuses on structure testing, not builder API or benchmarking00:23:55

Action Items

Potuz: Update consensus-specs PR #4979 with 3-epoch PTC storage00:22:21
Nethermind and Erigon teams: Add batch I/O prefetch flag (isolate from other optimizations)00:27:10
Kamil: Use alternative client (Besu/Geth) for stateful test generation00:30:54
lightclient: Schedule breakout Wednesday for Frame mempool discussion00:37:28

Targets

ePBS devnet-1 spec release targeting end of week00:22:58