Mario Vega:Thank you. Welcome, everybody, to ACDT number 71. Today's February 23rd, 2026.
Transcript
Mario Vega:Yeah, we have a couple things in the agenda. Fusaka, Glamsterdam, state float. I think the main discussion point's gonna be focus on dignitary readiness, but before we go there, yeah, let's start with Fusaka.
Mario Vega:Okay, so the first item in the agenda is the Blob DevNet Zero update. I think, Barnabas, for this, you're the best person with contacts here. Would you mind to give us an update?
Barnabas:Yeah, so Blob.net Zero is testing the partial cell request, and last week we had a bit of
Barnabas:trouble. Some peers got disconnected. I haven't had
Barnabas:much time to, investigate what the actual reason for all the disconnects were. Restarting clients, just brought them back online, so it's probably not something critical.
Barnabas:I would assume that we just don't have enough, peers on the network to feed all the call-ins that we need, even though,
Barnabas:We have half of the nodes and super nodes, but yeah, I will dig into it a bit more. Right now, we are finalizing it, and we are near, 80%.
Barnabas:But, yeah, we currently also only have Lighthouse and, Prysm, so it would be very good if we could start onboarding some other clients as well. Hopefully, someone from the CR teams can give us an update on how is that coming along.
Mario Vega:So, which teams do you have now, and which ones do you need?
Barnabas:We have Lighthouse, Prysm, and we are looking to, also include Teku, Nimbus, and, Grandine, and,
Barnabas:Just every LD, basically.
Mario Vega:And just for context, for Dustin, this is for Plot DevNet0.
Barnabas:This is for the partial cell stuff.
Barnabas:Dustin.
Matthew Keil:And you forgot Lodestar, and I still love you, Carnivas, just saying. We did get, implementations going, and we're getting a lot closer. We're hopeful to be there, for this week's, call on Thursday.
Mario Vega:Thank you. Anybody else?
Mario Vega:On the rest of the CL teams.
Mario Vega:Again, for context, this is blob.0.
saulius:Yeah, so for Grandine, we are still working on that, so maybe next week or so.
Mario Vega:Okay, thanks. From Teku, is there anyone?
Mario Vega:Who wants to chime in?
Stefan Bratanov:Yeah, so for Teku, we are still working on implementation.
Stefan Bratanov:Yeah, I have spoke to Lucas about it, so…
Stefan Bratanov:Hope you'll have a better day.
Mario Vega:Sounds good. Thank you. Barnabas, I wanted to ask, so this is, partial cells also covered, right?
Mario Vega:the subject.
Barnabas:Yes.
Mario Vega:Forget Blobbit P3, is this something different?
Barnabas:No, that's the same. One is on the ER side, one is on the CR side.
Mario Vega:Do you need horses?
Barnabas:We need support from each of the ELs to support the partial styles, but everybody has that already implemented in their trunk branches, so that's all good.
Mario Vega:Excellent.
Mario Vega:Alright.
Mario Vega:Good. Anything else from that topic, then? I don't think so, right?
Mario Vega:then we can move on. Yeah, let's go into Glamsterdam. For block-level access list definite 2, I think everything's been going smoothly, but is there any events or anything that can be brought up?
Mario Vega:Again, this is for block-level access list 72.
Mario Vega:Any findings?
Barnabas:So the network is, fairly stable. We're looking to, get an update on DevNet 3 readiness.
Mario Vega:Alright, yeah, let's just jump into that.
Mario Vega:Yeah, alright, since there are no updates on DevNet 2, let's go to Devon 3. I think the main discussion point here is gonna come from, actually, the testing team. Basically, what I've, gathered so far is that, we are planning on adding EIP 8037,
Mario Vega:into .NET3, but it's… it's proving to be quite the feat, to… to get this, the tests working.
Mario Vega:So yeah, for that, I think, Spencer, you have the best, context on this. Would you like to chime in?
Spencer:Hey, yeah, sounds good. So…
Spencer:I guess just to give a high-level context, so… 1837 adds…
Spencer:Something called the cost per state byte, which is dependent on the block,
Spencer:Add the block gas limit?
Spencer:And this, for the testing framework, adds an extra kind of dimension that we haven't yet Added before, so…
Spencer:It's adding this, like, at a very high level, and it's… it's been quite tricky to…
Spencer:Update the framework to… to align with this. So… And…
Spencer:how the framework works right now, for more context, you can kind of think of it as being split into three different sections, so there's the normal tests.
Spencer:The static tests, which are, like.
Spencer:the old Ethereum tests, YAML files, that we refill, so it's another way that we fill. And then also the benchmark tests.
Spencer:So we have, kind of, these three different sections that… All interplay together,
Spencer:But yeah, so the… I guess the tricky part overall is that, yeah, the…
Spencer:We have gas costs in tests that are now dependent on the block gas limit.
Spencer:And, yeah, I think there's just a lot of files that need to be changed,
Spencer:We spoke about this briefly on Friday during the gas repricings call.
Spencer:One of the options as an alternative would be to…
Spencer:change the cost per state byte to a fork constant, and to be honest, yeah, I don't think it's…
Spencer:a good idea. I think Marius made a comment on the,
Spencer:the PM, agenda, kind of, kind of stating that it would defeat the purpose of 8037, which is true, and…
Spencer:I think, looking forward in the future, I… yeah, like, I…
Spencer:Our framework should support gas costs changing dependent on the block, the block.
Spencer:A gas limit, so… I have been working on some solutions on the framework to get this working.
Spencer:Over the weekend, but, yeah, I still need to work on it, and get that across the finish line, but…
Spencer:I think Marius also, suggested here to…
Spencer:Postpone 1837 to 11 at 4.
Spencer:Which I wouldn't be opposed to, because it would give us more time to,
Spencer:Get this working. And yeah, this so far has only been updating the framework, not writing the tests, so…
Spencer:Yeah, and for DevNet 3, I assume, like, the priority is…
Spencer:bowel optimizations, and please, please correct me if I'm wrong.
Spencer:But yeah, that's, I guess the main… the main update, from our side.
Mario Vega:Alright, thank you. Marius, would you…
Marius van der Wijden:Yes. Spencer, do you have, like, a sense of how long this would take you to implement all of this and write tests for 8037?
Spencer:So, to be honest, I would like to go all out on this until ACDE, and then maybe we can make a decision on ACDE about pushing it to DevNet 4 if the progress is slow.
Marius van der Wijden:Yep, I think that's a… that's a great compromise.
Mario Vega:The dragon?
Dragan Rakita:Could we hard-code it for the DevNet 3, and then implement the proper version for DevNet 4?
Mario Vega:What do you mean, hard-coded?
Dragan Rakita:Hark, I think the problem is… The…
Dragan Rakita:Dynamic gas for the state, it depends on the block gas limit.
Dragan Rakita:That part feels like… At least this is my, like, take from what Spencer said.
Dragan Rakita:If this is the problem, then we can hardcore that number and just do proper version on the next DevNet 4. For DevNet 3, we will still have, like, something related to AP8037.
Spencer:Works for me, but if, others…
Spencer:are less, less keen. Please speak up.
Marius van der Wijden:It… it would also work for me. I… I would prefer not… like, I would prefer to use the formula, but, we can also just put the definite hot code it.
Marius van der Wijden:I just won… I just…
Marius van der Wijden:No matter which way we do it.
Marius van der Wijden:I just would like to have…
Marius van der Wijden:tests for it before we launch the DevNet.
Marius van der Wijden:And, I don't know if this… if hard-coding it will make the tests appear… be much faster.
Dragan Rakita:Why I'm proposing this? Because the 8037 is very sensitive.
Dragan Rakita:And it related to gas is very sensitive, so there is a lot of edge cases here.
Dragan Rakita:So, in my take is, let's do as…
Dragan Rakita:As best as possible, let's do… have something that we can reuse later, and have, like, more time to implement it, to test it later.
Dragan Rakita:So, if we start now, We will have, like, a bigger time frame to test everything.
Mario Vega:It is Maria.
Maria Silva:Yeah, so I think my understanding is also, like, there's, different components to AT37 that may actually make sense to test them separately. So we do have the multidimensional metering component, which is already introducing some logic that
Maria Silva:we need to think about tests, edge cases, and so on. So we…
Maria Silva:could make sense in DevNet 3 to focus that part of the EIP, and then do the cost… dynamic cost per byte component, in DevNet 4.
Maria Silva:So yeah, I'm essentially agreeing with… with Trevor.
Mario Vega:My question for Spencer would be, do you think going first with a constant and then dynamically? It feels to me that that's gonna be…
Mario Vega:refactoring that's gonna be necessary for next DevNet, so perhaps it could be even worse?
Mario Vega:I don't know, I haven't seen the code myself, but what do you think?
Spencer:So it makes the framework changes a lot easier, which means there's, like, more time to spend on writing the tests, so I think it… I think it's a reasonable
Spencer:A reasonable idea, from my side.
Mario Vega:Yeah, okay, so I think nevertheless, we do need some EIP, PR as a basis on where to… how to… how to write the tests.
Mario Vega:How do we handle that if we're gonna do halfway?
Mario Vega:DAP AP3… 837?
Spencer:I think I have a PR open just now, so maybe I can…
Spencer:I can add the change there, and we can use that as the base, but I'm not sure on what the number should be for, cost per state byte.
Spencer:I guess you can use the formula. I can use the formula to figure out.
Spencer:For, like, maybe, like, 100, 100 million.
Maria Silva:Yeah, so I would suggest using the formula for the block limit we are targeting for DevNet 3.
Maria Silva:If it's 100, then we should use 100.
Maria Silva:Yeah, in terms of the update to the IP,
Maria Silva:So, if I understand correctly, we would now have an EIP that would have a fixed cost per byte, and then for DevNet 4, we would change again the EIP, is this correct?
Mario Vega:Yeah, that's my, my question is, where are we gonna… where is everyone gonna reference this ephemeral change from?
Mario Vega:Including tests, and including declines.
Mario Vega:Just for everyone to be on the same page for Definite Tree.
Maria Silva:Would it… would it make sense to just be a spec version?
Maria Silva:Instead of changing the EIP itself, or would that be worse?
Mario Vega:I think that's a question for client teams. Do we feel like referencing ills in this time around would be okay?
Mario Vega:If we don't have the EIP changes, because… yeah.
Mario Vega:And Dragon mentions in the chat that Devon3, how come this is a good place?
Mario Vega:Alright.
Mario Vega:Who's gonna be in charge of, doing this,
Mario Vega:this spec reference. Do we need a markdown document? That's my question. Or can we reference a PR in yields?
Mario Vega:For this?
Barnabas:We can reference, PR. Just point me to the PR, and then I will update the spec sheet.
Barnabas:And then, we can reference that.
Barnabas:And I will also highlight it, maybe on the top of the…
Barnabas:Spec sheet that this is, an edge case.
Mario Vega:Makes sense. Okay, so coming back to ills for this, Spencer, your… does your PR work for this? Can we reference it?
Mario Vega:For 103?
Spencer:Yeah, no, definitely. Sounds good.
Mario Vega:Alright, seems like we have a plan. Alright, just to,
Mario Vega:leave everything clear. We're gonna do a hard-coded cost-per-byte version for log-level access list number 3. Debnet 3, sorry. We're gonna use an ills PR for this.
Mario Vega:And then for DevNet 4, we're gonna reference the actual EIP for the implementations.
Mario Vega:In terms of when can we release the tests,
Mario Vega:Spencer, do you need more time for the changes? If… taking all of this into account, do you think we need more time for the tests anyway?
Spencer:I think I'll be able to get something out by, ACDE.
Mario Vega:Okay.
Mario Vega:Sounds good. Alright, so… tests would be out for Thursday, is that…
Mario Vega:Reasonable for everyone?
Mario Vega:Marius Barnabas? Mario?
Barnabas:Yeah, from our point of view, it's not an issue.
Mario Vega:Good.
Mario Vega:Good, perfect. Okay, it seems like,
Mario Vega:We have a… we have a plan?
Mario Vega:Does anybody have any questions?
Mario Vega:Regarding them the tree?
Spencer:Just while we're, here, I think I also mentioned, so this is, on 1837, something that, Sam Wilson,
Spencer:Brought up, basically about how tracing is handled.
Spencer:So I think in the yields PR we have right now, we've added a state gas and a state gas cost to the…
Spencer:The tracer, is this something that… El clients are happy with.
Spencer:Yeah, I'm not a client dev. I don't use tracing much, actually, so…
Marius van der Wijden:Sorry, Ken.
Parithosh Jayanthi:deal.
Parithosh Jayanthi:here.
Marius van der Wijden:That is the… 3155 change, yeah, I think that's good.
Mario Vega:Okay.
Mario Vega:Great. So, moving on, alright, let me just…
Mario Vega:touch upon briefly the other EIP that has been included in DevNet3, which is EIP7954.
Mario Vega:Any discussion points regarding this?
Mario Vega:That anyone would like to bring up?
Mario Vega:Or are we good in those, in those regards?
Mario Vega:Okay, seems like a non-issue.
Mario Vega:I'll just go ahead. The next point in the agenda is EPVS, but first, let me pause before going into the CL side of things. There's one other topic that's regarding the EL, which is, the new endpoint, the testing build block comparison number one. I'll… let me just share the link.
Mario Vega:Basically, Paratush wanted to see if we could, make this implementation mandatory for clients, starting from DevNet 4.
Mario Vega:Because it helps testing, it helps benchmarking, it helps several things.
Mario Vega:So, I would like to touch base with the client teams regarding their opinions on this new endpoint and their status.
Mario Vega:I think, also, never mind, if you can… you guys can share, I think you're… you are… you guys are the, authors of this new endpoint.
Mario Vega:So if you have any recommendations, also please feel free to chime in. Yep.
Mario Vega:So, Bari, do we have more context on this?
Parithosh Jayanthi:Yeah, there… maybe there's one design note as to why we want to add a new, new endpoint to begin with. So typically in the past, we would have done this in the debug namespace. The problem with the debug namespace is that a lot of public, RPCs, etc.
Parithosh Jayanthi:against recommendation by default, expose debug, because there's some functionality with Ethereum RPCs that cannot be achieved without exposing the debug namespace. The problem with, adding this testing API to the debug namespace is this would allow you to abuse the debug namespace to build whatever blocks you want.
Parithosh Jayanthi:In order to just keep these things separate, we wanted to introduce the testing namespace. It's also just the first RPC call in the testing namespace, and there's a lot more we can do in the future.
Parithosh Jayanthi:this RPC call would also help us a ton with, with your benchmarking, because you would then, in the future, be able to just provide a list of transactions.
Parithosh Jayanthi:And get a built block as a result. It should simplify quite a few things in quite a few places, and yeah, thanks to the Nethermind team and for Chase for writing the tests and coming up with the spec for this.
Parithosh Jayanthi:Yeah, now we just want to make sure that all the clients have an implementation so that we can move forward with it.
Mario Vega:Excellent, thank you. Client apps, Do you guys need,
Mario Vega:Anything to implement this, or do you oppose implementation?
Mario Vega:Please chime in.
Mario Vega:Alright, there's no pushback, I guess we can make this,
Mario Vega:a requirement. We can bring it up next week also, regarding this… the status of this, and how…
Mario Vega:Things are going.
Mario Vega:Alright.
Mario Vega:Perfect. Okay.
Dustin:I, I guess…
Mario Vega:Ms.
Dustin:I just wanted to, like, make one… a general… I mean, Barnabas already said this, and I already… like, I actually feel pretty strongly about the merge-then-talk-about requirement thing. I mean, if… if people are willing to stipulate that, well, it will be merged, and it looks like that's being stated, then…
Dustin:fine, that's okay. But I… I'm a little… a little tired of these kind of, YOLO's, you know, quasi-specs, which are, like, draft EIPs that get required for some DevNet. Like, please, no, no more of those. Emerge, then require.
Parithosh Jayanthi:Yeah, maybe just a direct, answer for that one. We were lacking the ability to generate the tests because we needed an implementation in order to generate the tests. It's a downside of how the RPC test gen works today.
Parithosh Jayanthi:There was a blocker, blocker's been solved, and at least as of end of last week, we have the tests, which is why it hasn't been merged yet, but we will get it merged this week, because the counterpoint is that we don't want to merge an EIP without… or, API change without tests.
Mario Vega:Yes, Camille?
Kamil Chodoła:What I just want to add is, like, this is a separate namespace. It shouldn't contradict with any feature right now, and anyone basic… any client team can basically implement it without a risk of breaking something else if done properly.
Kamil Chodoła:And as Pari also explained, there was a huge, need of doing that, because, for example, for some stateful generations of tests to unlock completely new areas of testing for benchmarking, we needed to use some hacked versions of clients, hacked versions of Nethermind to basically
Kamil Chodoła:proceed with any generation on top of existing states, and this is allowing to use any client, this is unlocking plenty of potential opportunities for better benchmarking, for better test generation. It's already proven to improve
Kamil Chodoła:by a few times the test generation on top of existing state of mainnet or blotnet. So, these are quite nice ideas to include and implement, and it will allow
Kamil Chodoła:to improve the testing capabilities of the clients in very, very various ways, like, not only just to build a block, but also to trigger the building of the invalid blocks, to help with balls, which Jocham is already exploring right now. So, a lot of
Kamil Chodoła:Potential upsides and very small negative impacts, in my opinion, at least.
Mario Vega:Okay, thank you.
Mario Vega:Justin, I see your comment on the, on the chat. Yeah, I agree. This is, this is a problem. I don't think… I mean, at least… I know you're referring to…
Mario Vega:Execution APs, specifically, but I think from the spec side.
Mario Vega:For EIP testing, ills has been…
Mario Vega:Helping us out to build the tests without any client implementation.
Mario Vega:And it's getting better?
Mario Vega:Yeah. Would you mind expanding a little bit on this, or…
Justin Florentine (Besu):I don't really want to derail things, I just wanted to point out that this happens very regularly, and we get caught in this dependency loop where we can't create tests because we don't have an implementation, and it seems very broken and backwards to me, is all. I think you get it, I think most folks on the call get it, so I won't belabor the point. I do appreciate that it's mostly an RPC issue, and that
Justin Florentine (Besu):the broader Eels test don't suffer from that problem. So, it seems to be going in the right direction, but boy, it's been a challenge for quite some time.
Mario Vega:Agreed. Okay, yeah, let's keep moving, then.
Mario Vega:Yeah, going back to EPVS, DevNet Zero, sorry.
Mario Vega:Do we have any updates?
Mario Vega:And let me see the comment from Justin in the…
Mario Vega:Alright, so Justin, you have a comment on the agenda at the moment, but this is for Fossil, right? This is.
Justin Traglia:Different from EBS.
Justin Traglia:I mean, there's just a note at the top that says that the…
Justin Traglia:Also, spikes have been renamed to EZ, but there were just some PRs that I want core devs to review that are related to EPBS.
Justin Traglia:But, we don't have to talk about that.
Mario Vega:Alright, do you need any help?
Mario Vega:Or… input from the core labs, at this point?
Justin Traglia:Oh, yeah. Reviews would be great.
Mario Vega:Yeah.
Mario Vega:And anything else, that they need to implement, other than reviews, or…
Justin Traglia:At the moment, I don't think so.
Mario Vega:Alright, and I see some comments in the chat from Nico. It seems like, Lone Star and Lighthouse do have interrupt for EPBS, and tech mentions that they can have interrupt ready towards the end of the week, or early next week.
Mario Vega:Yes.
Mario Vega:I think that's it.
Mario Vega:And Prysm will not make it in by this week, but we, they are gonna try to aim for next week. Alright.
Mario Vega:Anything else regarding the EPBS that needs to be discussed today?
Mario Vega:Barnabas, would you like to mention that?
Barnabas:Yeah, so I would propose that we have a realistic,
Barnabas:launch date for EPBS Demo Zero on the 4th of March.
Barnabas:We will do a launch no matter how many clients are ready, but ideally we have at least 3, ideally 4 ready for that day.
Mario Vega:Sounds good, and from the reading of the comments from chat, it seems like we will do have at least 3 clients for that date, so I think it should be okay.
Mario Vega:Alright.
Mario Vega:Good.
Mario Vega:Anything else for EPS? I don't think so, right?
Mario Vega:Now that we can just keep going. The next, item in the agenda is, gas limit testing.
Mario Vega:or benchmarking. Camille or anyone else, would you like to give an update on this?
Kamil Chodoła:Yeah, so mostly we are focusing right now on repricing's effort.
Kamil Chodoła:doing a lot of back-and-forth fighting with some failures during generation, but I guess right now we are on a good spot.
Kamil Chodoła:With all the tests which are required, likely generated already, and some data already collected.
Kamil Chodoła:for that purpose. The only stuff is that new tests which are running on top of state of mainnet or bloatnet are pretty long to execute, and single loop of gathering all of the data points for all of the scenarios in all required,
Kamil Chodoła:Conditions is, like, more than a day, just to capture one data point.
Kamil Chodoła:for all the clients. And so, yeah, so right now, trying to solve that and see where we can either cut the time of the execution or parallelize more on more machines. So this is being discussed right now. So this is the main effort.
Kamil Chodoła:You worked heavily also with Mario last week to optimize the generation significantly, so there are some few cool new stuff, either on ills or on Generator's side.
Kamil Chodoła:But rather than that, yeah, we are progressing
Kamil Chodoła:waiting for some inputs, additional inputs from Maria for new tests, but yeah, those landed over the weekend, and I still need to check a few issues.
Mario Vega:Sounds good. Yeah, sorry.
Maria Silva:Maybe I can just give a quick update on my side. So I'm… I am… last week, I was looking at the first, data that was coming out from the benchmarks that Camille was saying… was saying, and, yeah, I do… we… we have found
Maria Silva:a few issues on how the tests are defined, like, they are not measuring the exact
Maria Silva:variants that we needed, like, for instance, an S slow that is warm versus cold, like, there was some… some… some, some changes that we need to do to… to… to… to the test, so it's still a work in progress, so I don't think we'll have
Maria Silva:the final, or the intermediate numbers for EIP 8038, so the state access EIP, by the end of this month. But yeah, we are working on it and hoping to get numbers as fast as possible.
Mario Vega:Alright, thank you.
Mario Vega:Alright, so moving on to the next topic, state load. I think, Camille, you're also in…
Mario Vega:Topic things in this topic?
Kamil Chodoła:Yeah, so we had a major issue recently with BloatNet, due to the scale right now of the BloatNet and the ratio which we put, and some random problems, for example, with NetherMite Corrupted at an unexpected moments.
Kamil Chodoła:We had a bloat which went down, but thanks to Philip, we have it back online, and it's bloating, and as far as I know.
Kamil Chodoła:we also reached a new goal… I mean, we hit the milestone of blocking specific contracts to the certain size, which would be much bigger than the current biggest contracts on the chain.
Kamil Chodoła:So, right now, the snapshots collection is being in progress, and yeah, moving forward, we need to…
Kamil Chodoła:Discuss how we want to… how we want to blot, because seems like this,
Kamil Chodoła:This approach which we have right now with a network and bloating, and clients are going down from time to time is not sufficient, especially if we keep in mind that we want to bloat at some point to 10x or something like that.
Kamil Chodoła:So that's why, for example, Martin from our team is right now investigating some possibilities to create a custom tools, plus Johan and Tony are working on a state injection tool, as I recall. So some research around optimizing this approach is happening in parallel.
Parithosh Jayanthi:There's maybe two points to that one. The first one is we haven't started snapshots yet, because we're still waiting on Eragon to sync up.
Parithosh Jayanthi:I think Will asked a few questions earlier today, I haven't gotten to it, but, I can answer that, and once Ergon is ready, we should be ready to take a snapshot. We're gonna do two snapshots, one before Fusaka is scheduled in, in Bloodnet, and the second one is after Fusaka is scheduled, so that we have, both of those dates available.
Parithosh Jayanthi:But, to the other question of how we move forward, I actually wanted to challenge that assumption, because the reason BloatNet is failing is that
Parithosh Jayanthi:is that clients can't handle the state already. I would actually propose that we try and figure out why clients are breaking right now, rather than just getting to a 10x state, because we're already breaking. I don't really see the benefit of continuing to increase the state size.
Marius van der Wijden:Whoa.
Marius van der Wijden:Are clients really failing because of the state size, or the amount of state growth?
Parithosh Jayanthi:Well, state size plus gas limit as a combination. So, yeah, if we do 100 million gas, we're breaking, but last time I checked, I think we were breaking at 300 million gas with the state, which…
Parithosh Jayanthi:Is kind of a realistic target, right?
Marius van der Wijden:I thought we… we are currently at, like, 3 giga gas or something on…
Kamil Chodoła:Let me maybe clarify the recent weeks, what was happening. So, basically, we started increasing gas limit.
Kamil Chodoła:And a few issues which we spotted was, like, GEF was running some invalid flags at all, which were not properly applied from mainnet into BloatNet, so, like, we are facing some issues which are… actually might not be the real issue.
Kamil Chodoła:In plenty of scenarios. Nethermind is corrupting, and we cannot reproduce that locally at all.
Kamil Chodoła:So, that's why I was hinting a few times at the hardware issues rather than the software issues, but I cannot
Kamil Chodoła:prove both, actually. So, the reason I'm mentioning some different approaches toward Test 10X is because of a few reasons, like, we want to eventually reach that at some point.
Kamil Chodoła:happen in parallel while solving the issues, because I guess at some point we might have solved the issue on 3X whatsoever, and next issues might pop out at 5x.
Kamil Chodoła:So if we'll parallelize both efforts, we will be able to verify issues at 5x straight away, rather than waiting, bloating, dealing with some instabilities.
Kamil Chodoła:During that time.
Kamil Chodoła:So, it would allow us to progress a little bit faster. While I agree that this is highlighting some potential problems which we maybe should tackle early, but till now, some of those problems weren't strictly related to the issues in plants rather than configurations and some
Kamil Chodoła:Randomness which is happening.
Kamil Chodoła:During bloating.
Parithosh Jayanthi:Maybe there's one exercise to try. So right now, the approach with bloating is take all the keys, put it on one machine that's able to build blocks, and then continue on with the network.
Parithosh Jayanthi:Maybe the way to figure out exactly where we break is redistribute the keys equally, and then go to 300 million gas, see if it can actually build blocks across clients. If we're not able to do that.
Parithosh Jayanthi:then I think we have a problem already. And, to Maurice's point, yeah, we are going at Giga Gas or whatever, but that's only happening because one client is building all the blocks, not all the clients.
Parithosh Jayanthi:One or two clients, let's say.
Mario Vega:Alright, I just wanted to close, well, I don't want… I didn't want to close the topic. Camille, do you agree with these approaches?
Kamil Chodoła:Yes, sorry, I understand. So, yes,
Kamil Chodoła:Actually, Gregory, but still, I believe we could explore something, because in case we want to test on some much higher numbers, there will be a huge value in having something which could allow us to test some…
Parithosh Jayanthi:Yeah, yeah, some, like, local… local bloating approach does make sense for that.
Kamil Chodoła:Yes, this is exactly what I mean, like, local bloating, and you can always spun up the network from that point.
Kamil Chodoła:We know about some issues during syncing when the gas limit is way too high, but yeah.
Kamil Chodoła:some possibility to do that much more convenient way and much more predictable way, rather than… yeah, I guess the assumption was to have the current blot to be completed already, like, a few weeks ago, but due to some random issues, we were…
Kamil Chodoła:Suspending that for quite some time, so…
Mario Vega:I'm good.
Mario Vega:Thank you. I think that's everything for today's agenda. Do we have any other topic that anybody wants to bring up?
Mario Vega:Needs to be discussed today.
Mario Vega:Johan, yeah?
jochem-brouwer:Yeah, there's one point, and I just realized that I should have raised this before, but for the benchmarking for the block for access list, we actually need to fill these tests for Amsterdam.
jochem-brouwer:So we need the testing build block API to support, building the…
jochem-brouwer:MCLM block, so we can actually start benchmarking states-related tests for the block FX list.
Mario Vega:Yes, come in.
Kamil Chodoła:So yeah, I guess, for the Netarmine implementation, Margin already started doing… I mean, not even started, he's having some draft implementation of…
Kamil Chodoła:Testing build block with a possibility to pass a fork on top of which we want to build blocks, so hopefully this will help.
Marcin Sobczak:Yes, I have some issues with it, but it's work in progress, and yes, at the end, we want to be able to build on every fork.
Mario Vega:Sounds good.
Mario Vega:Okay.
Mario Vega:Anything else?
Mario Vega:anyone wants to bring up today?
Barnabas:I can just confirm that, Lodestar and Lighthouse are actually interoping locally on, like, glutosis test.
Barnabas:So, good job.
Mario Vega:Amazing. Great news.
Ahmad:Hey, quick. Yeah, quick question. So, for those who are…
Ahmad:I mean, not completely new, but…
Ahmad:Relatively new, who are trying to, onboard, just understand the, the tooling processes,
Ahmad:Better. There's the, the testing, like, group, channel grouping in the ETH R&D Discord server. Is that generally the, the place to go to kind of, like, research and just familiarize, someone's…
Ahmad:For someone to familiarize themselves with the, the tooling processes, like Kurtosis, Dora the Explorer, Hive, all of that. Or is there a better, place? Perhaps, like, a GitHub issue, or Doc, or some… something that's.
Mario Vega:Yeah.
Ahmad:more organized? Just curious.
Mario Vega:I think 8 R&D is the best place to… and yeah, we can take it async, just to not hold off everyone. But yeah, 8th R&D should be the place where you chime in, and someone will probably just help you out with the onboarding.
Ahmad:June.
Ahmad:Got it, got it. Okay, cool.
Mario Vega:Alright.
Mario Vega:Cool. Thanks, everyone, I think that's it for today. Yeah, thanks everyone for joining, and we'll see, everyone in ACDE this week.
Marius van der Wijden:Sorry, Okay. Ahmad? Ahmad? Ahmad?
Marius van der Wijden:Ahmed? Oh.
Ahmad:bod.
Marius van der Wijden:Ahmad? Ahmad? Okay, perfect. There's, epf.wiki, which has a bunch of information, so, if… if you are looking for information about development, or, like, the testing stuff.
Marius van der Wijden:epf.wiki.
Ahmad:Right on, right on. I appreciate it. Noted.
Mario Vega:Thank you. Excellent. Right.
Mario Vega:Okay.
Mario Vega:Thanks, everyone, again. Thanks for joining. We'll see everyone in ACDE this week.
Mario Vega:Thank you, bye-bye.
Dragan Rakita:Bye.
Parithosh Jayanthi:Aye.
Justin Traglia:Bye, everyone.
Chat Logs
00:06:01
Dustin:missed the context, this is bal-devnet-2?
00:06:10
Caleb:no
00:06:10
Parithosh Jayanthi:Blob-devnet-0
00:06:29
Barnabas:oops
00:09:59
Mario Vega:Context: EIP-8037 on BAL-Devnet-3
00:09:59
Parithosh Jayanthi:re: reth missing blocks and devnet-2, fix is here: https://github.com/paradigmxyz/reth/pull/22490
00:10:14
Parithosh Jayanthi:Replying to "re: reth missing blo..."
Not really worth bringing it up on the call afaik
00:11:01
Barnabas:Replying to "re: reth missing blo..."
do we need to wipe db?
00:11:15
Barnabas:Replying to "re: reth missing blo..."
looks like it has been merged an hour ago, so the new img should be ready to roll
00:11:17
Barnabas:Replying to "re: reth missing blo..."
cc @Jen
00:11:43
Dragan Rakita:Replying to "re: reth missing blo..."
This is on block building, didn’t use good gas limit for 7778
00:11:43
Jen:Replying to "re: reth missing blo..."
Yep PR was merged, any more action needed from our side?
00:12:08
Barnabas:Replying to "re: reth missing blo..."
Ok, so you didn’t store anything invalid in your db?
00:12:42
Dustin:if a postponement decision is made can it be made soon?
00:12:43
Dragan Rakita:Replying to "re: reth missing blo..."
That would be the case
00:12:46
Parithosh Jayanthi:Replying to "re: reth missing blo..."
Yeah afaik it was just about missing blocks, not a db corruption
00:12:50
Barnabas:Replying to "re: reth missing blo..."
6a633a4 is rolled out
00:12:58
Dustin:doing it in a week or whatever is pointless
00:13:13
Dustin:meh ok
00:16:25
Dustin:does Maria's proposal make testing easier?
00:18:10
Toni Wahrstätter:Someone remind me please, did we decide to have eth/70 on devnet-3?
00:18:24
Dragan Rakita:devnet3 hackmd is good place
00:18:29
Barnabas:Replying to "Someone remind me pl..."
yes as an optional thing
00:18:34
Parithosh Jayanthi:Replying to "Someone remind me pl..."
Eth/70 also has its own devnet
00:18:38
Dragan Rakita:Replying to "devnet3 hackmd is go..."
https://notes.ethereum.org/@ethpandaops/bal-devnet-3
00:18:55
Parithosh Jayanthi:Replying to "Someone remind me pl..."
https://dora.nft-devnet-10.ethpandaops.io/
00:19:06
Jen:Replying to "re: reth missing blo..."
Ty !
00:19:36
Toni Wahrstätter:Replying to "Someone remind me pl..."
great thanks!
00:21:22
Parithosh Jayanthi:What’s our updated timeline for devnet-3?
00:21:35
Barnabas:Replying to "What’s our updated t..."
same as before, 4th march
00:22:42
Mario Vega:https://eips.ethereum.org/EIPS/eip-7954
00:23:19
Mario Vega:https://github.com/ethereum/execution-apis/pull/747
00:25:25
Barnabas:I think first we should merge if the spec is good
00:25:30
Barnabas:then talk about impl.
00:25:56
Parithosh Jayanthi:We’ll merge it in this week
00:26:03
Justin Traglia:For ePBS, there are some open PRs I would like core devs to review. See this comment: https://github.com/ethereum/pm/issues/1932#issuecomment-3944893338
00:26:12
Marius van der Wijden:Geth has a working impl
00:27:34
Justin Florentine (Besu):this is a recurring and frustrating problem tbh
00:28:03
Justin Florentine (Besu):we need tests that are decoupled from implementations
00:29:05
Parithosh Jayanthi:We’ll get there eventually 😄 we had this refactor merged in last week too: https://github.com/ethereum/execution-apis/pull/751 it already makes the contribution to the rpc specs way easier
00:30:33
Justin Florentine (Besu):Replying to "We’ll get there even..."
awesome!
00:30:34
nflaig:we have working interop with lodestar<>lighthouse
00:30:53
Marius van der Wijden:Replying to "we have working inte..."
Epbs?
00:30:56
Parithosh Jayanthi:Replying to "We’ll get there even..."
Contributor guide will come up next to make it easier 😄
00:30:59
Marius van der Wijden:Replying to "we have working inte..."
Lets gooo
00:30:59
Stefan Bratanov:For Teku, we hope we can be interop ready towards end of week or early next week
00:31:04
Potuz:We’re sprinting, don’t bug us that we can barely listen to this call cause we are coding faster than Claude right now
00:31:21
Barnabas:Replying to "We’re sprinting, don..."
“are we there yet?”
00:31:29
Parithosh Jayanthi:Replying to "We’re sprinting, don..."
“Have you tried using 2 Claude accounts”?
00:31:29
Potuz:Prysm we won’t make it end of this week but we are definitely aiming for next one
00:31:37
James He:Started a branch for devnet 0 in case we don’t have everything merged into develop
00:32:19
Barnabas:Lets set a realistic launch date set for 4th of March as well!
00:32:29
Potuz:That sounds very good for me
00:32:37
Dragan Rakita:Replying to "We’re sprinting, don..."
Spawn subagents 😄
00:32:59
Potuz:Replying to "We’re sprinting, don..."
I don’t get paid that much
00:36:03
Parithosh Jayanthi:https://benchmarkoor.core.ethpandaops.io
00:36:26
Parithosh Jayanthi:Rafael soft launched the benchmarkoor rewrite
00:36:47
Barnabas:Replying to "Rafael soft launched..."
he gave us an update last week on acdt
00:38:36
jochem-brouwer:Is it the state or is it the gas limit of bloating per block?
00:38:43
jochem-brouwer:Yes exactly
00:38:46
Barnabas:Replying to "we have working inte..."
LFG
[Full message cannot be displayed on this version]
00:42:39
Marius van der Wijden:I agree, imo we should bloat more reliable, not faster. I don't care if it takes 2 months to get to 10x mainnet if its an automatic process
00:46:18
James He:Epf.wiki
00:46:24
jochem-brouwer:https://epf.wiki
00:46:25
g1:https://epf.wiki/#/
00:46:26
Potuz:That’s a great link to learn how to test
Summary
18 highlights
· 4 decisions · 4 action itemsExperimental
Summary
18 highlights · 4 decisions · 4 action itemsExperimentalfusaka updates
glamsterdam bal devnet 3
testing infrastructure
epbs devnet 0
gas repricing benchmarking
Decisions
- EIP-8037 included in bal-devnet-3; hardcoded at 100M gas; dynamic deferred to devnet-400:18:46
- testing_buildBlockV1 endpoint required for clients starting DevNet-400:26:24
- EPBS DevNet-0 launches March 4th with minimum 3 clients ready00:32:18
- BloatNet focus: reliable automated bloating over speed; 2 months to 10x acceptable00:40:57
Action Items
- Spencer: Publish EIP-8037 EELS PR with hardcoded cost-per-byte for DevNet-300:20:26
- Barnabas: Update DevNet-3 spec sheet referencing hardcoded 8037 EELS PR00:20:36
- Parithosh: Merge testing_buildBlockV1 execution-apis PR #747 with tests00:25:56
- CL teams: Review ePBS consensus-specs PRs (4792, 4817, 4772, 4826)00:30:34