Mario Vega:Thank you, Akash. Good day, everyone. Welcome to ACDT number 63. Today is December 8th, 2025. We have a very light agenda, but the… I think the most important discussion point is that, Fusaka Fork happened last week, and
Transcript
Mario Vega:went well. But yeah, for the first point, I would like to start with… yeah, congratulations, everyone. For the first point, I would like to touch upon the events that happened, if…
Mario Vega:anyone from any team, I think Prysm is one of the ones that wants to share more in this section. Is anyone from Prysm here? Should we start with them?
Mario Vega:If not, Barnabas, do you have, any events that happened during the fork that are important and meaningful that we should share?
Barnabas:Yeah, so Prysm attestations came in a bit late, and this ended up in a cascading effect. They had a flag to fix the issue, that was already included in their latest release in 7.0.0, and they said that they would push out a proper fix, sometime today.
Barnabas:I haven't heard anything from them since, but hoping in a new Prysm release for
Barnabas:Early this week, but ideally by end of today.
Mario Vega:Alright, thanks.
Mario Vega:Yeah, I think we can give some time from someone to present to you, if they want to expand. From any other teams, do we have any important events that are worth sharing here? Findings, or reports, bugs found?
Mario Vega:Anything?
Bharath:Yeah, I mean, I can share one thing, so…
Bharath:I don't think there's any impact in this, but, one of the things… this is through going through the Titan relays,
Bharath:GitHub, I noticed that… so we introduced a Get Payload V2 for Fusaka. I noticed they have a bug fix where we didn't… where they didn't add it… add it to the routes.
Bharath:like, basically, like, they didn't add it to their API routes. That would ideally have caused, like, payload… the API not found, but I think, luckily, we had,
Bharath:Fallbacks in both commit boost and ref boost to, like, avoid this.
Bharath:So, I just thought, like, that's something kind of notable, but I… there wasn't, like, any impact
Bharath:on that, as far as I'm aware, like…
Bharath:So, yeah, I think if that… the fallbacks weren't there, there would have been a bunch of missed slots, so…
Bharath:But, yeah, but I wanted to, like, point that out.
Mario Vega:Thank you.
Mario Vega:Was there any impact on network stability that was worth sharing, or…
Mario Vega:Wasn't you… wasn't not noticeable.
Bharath:Oh, no, on the Get Payload V2, there was an impact, because the client's MevBoost and CommitBoost both implement, like, fallback, so when we call V2 and it's not there, we fall back to, like, the V1 API.
Bharath:So, the V1 API hash is definitely… has to be implemented, otherwise the relay wouldn't be alive. So,
Bharath:So yeah, so there wasn't, like, any impact, because we also noticed no, like, missed slots in that aspect. There was no impact on that, and
Bharath:Yeah, but it was something, like, worth, like, just bringing out and, like, letting you guys know.
Mario Vega:Thank you. Thank you.
Mario Vega:Cool. Any… any other findings or comments from other client teams that is worth sharing in this meeting?
Mario Vega:Alright, I think… I think not. That's… that's actually good. I see a comment from the peer-to-peer team, no, sorry, for Barnabas,
Mario Vega:Raul, do you want to expand? I see you also commented on this.
Raúl Kripalani:Yeah, yeah, yeah. So, I just wanted to give a short update on some data we've been collecting from Mainnet in preparation for BP01, just to kind of, like, maybe give a bird's-eye view of how things are progressing, and what we've seen, and maybe some recommendations. Not necessarily for BPO1, which is coming tomorrow, but for next BPO upgrades. Is it okay if I share my screen?
Mario Vega:Yeah, absolutely.
Raúl Kripalani:Absolutely. Very cool. Let me, get this ready.
Raúl Kripalani:So, what… the link that Barnabas shared is actually from a small set of notebooks that we're putting together, and this is going to be enhanced over the next few days. We're hoping to basically get different perspectives and vantage points into the network from different, areas to kind of, like, keep tabs on how things are going. So, the first notebook that I wanted to show here is,
Raúl Kripalani:one that shows blob inclusion, so that's basically the patterns that we're seeing in terms of how many blocks, were, including how many blobs over time, and you can see that, kind of like the most popular ones, the most popular blob counts are 9, which means that we are obviously at, like, almost, saturation right now. Then you do see some, some density, some more density on 6,
Raúl Kripalani:3 and 0. Some blobs, some blobs didn't include any blobs, we'll… we'll touch on that briefly now. This is another similar view, but this is breaking it down by epoch, so you can see kind of, like, a little bit of a flame graph-ish view on how many blobs were
Raúl Kripalani:How many blocks included how many blobs?
Raúl Kripalani:In a particular epoch, and you can see some interesting patterns here, like, for example, there was an epoch that included a high rate of, of, blocks with no blobs. Sorry, I don't know why I can't zoom in here. Other graphs allow me to zoom in, but not this one. So this one, and then kind of like the next epochs actually compensated for that by stretching and including 9 blobs. So we see that elasticity in the mempool, and that elasticity in
Raúl Kripalani:inclusion, which is great. We see the same pattern here again. We're actually investigating exactly what might have happened here. They were kind of like a sequence of 9 or 7 blocks, I think, that were… that didn't include any blobs. So we're actually, you know, finding patterns here and then doing deep dives, so I'll probably have another further update to share at another point.
Raúl Kripalani:This is a pivoted view of the same thing, but this is, showing, yeah, just, just a different view, where we see that, exactly that, that pattern. So, what we see here is that.
Raúl Kripalani:Yeah, so most,
Raúl Kripalani:We… we're actually seeing blobs included, and especially kind of, like, a high rate in the… in the, edge saturation there, the max blobs.
Raúl Kripalani:Sorry, that went out. And then, these are some other views. These are, like, maybe some visual… more visual candy. This is one… actually, let me go into column propagation, because this is the first one that I wanted to point out some patterns on.
Raúl Kripalani:So what we see here is, basically a… all this data that I'm showing you is just from yesterday. These notebooks are going to be regenerated on a daily basis, and we'll probably add, more, longer periods and longer views, so that we can keep tabs on, kind of like, you know, the evolution side of things. And what we see here is, for… rows are columns.
Raúl Kripalani:And these correspond to column subnets, and then this is the passage of time, and we're actually plotting for every slot, when do we see the first column? Sorry, when do we see the first copy of each column?
Raúl Kripalani:So you can see some very interesting patterns here, where you see that there's actually a lot of similarity. If you look at the Z value here as I'm scrolling, as I'm moving my mouse down, you see that that is actually changing on a per column basis. Sorry, I just moved across slots, so you saw it change more than it does within a slot.
Raúl Kripalani:But you see there, like, the moment that we see a first column, we're basically, like, practically seeing all other… all other columns, at the same time. This data, by the way, is collected from, ETH PandaOps' Shattoo deployment. We need to qualify it a little bit better, because there's a mixture of nodes, so different nodes might be seeing different things.
Raúl Kripalani:And of course, we're plotting the min for every column here, so there might be some adjustment that's needed here. But yeah, some interesting bits is that we do have some slots where, for example, the first column, the first copy of every column was seen, kind of like.
Raúl Kripalani:200 milliseconds before the attestation deadline, and this is, like, starts to be problematic. We need to correlate this with blob count and a bunch of other things, so, so, it's again, like, this will be enhanced.
Raúl Kripalani:But yeah, you see, kind of, like, when the first column is seen, then the rest of the columns are practically seen at the same time. This is a similar graph, but more sensitive. This is actually computing the interest lot delta from the fastest column.
Raúl Kripalani:And plotting it in milliseconds. This… the timescale is… so you see the sensitivity here? Like, practically…
Raúl Kripalani:once we see the first column, all of the columns are seen, like, within 250 milliseconds, which confirms this sort of pattern here. But what I found interesting is this one. Delta normalized is just… so this is in millisecond terms, and this would be in percent terms, so basically.
Raúl Kripalani:If 0 is the slow… is the fastest column and 1 is the slowest column, then what is the pattern that we're seeing here? And you see that in general, like, we see some banding, which is very interesting, so we see that…
Raúl Kripalani:The first columns are actually seen sooner, so, and this makes sense, because clients are probably, blog builders, sorry, blog builders are probably…
Raúl Kripalani:Pushing columns sequentially through the network, starting from zero and then progressing up until 127, zero indexed. And…
Raúl Kripalani:And yeah, so we see more… so these are the first columns that we see, but then we see some banding as well, which is very interesting. We're investigating where this is coming from, because it doesn't necessarily come from, like, you know, binary, like, indices or anything like that, so it might just be that the subnets behind…
Raúl Kripalani:The meshes behind these custody subnets are actually stronger, or are better resourced, but you do see, like.
Raúl Kripalani:some… some stepping there, which is very interesting. So we're trying to, like, dig into… into that, and I think we'll have some… some data. And then the one that Barnabas,
Raúl Kripalani:deep link to is the… the blob flow. So we're… we've charted, once again, using shadow data, which is super, super helpful, which also pulls for meats here. We are seeing, what we see here is which proposers are… proposer entities are including how many… how many blobs.
Raúl Kripalani:This is a direct relationship between proposers and number of blobs, and then, relays to blob counts, so we wanted to check, is there a pattern? Are some relays including, like, you know, favoring blobs or, like, specific block counts or not?
Raúl Kripalani:we just see, kind of, like, for reference purposes, a proposer-to-entity relay, so we see, like, which proposers… proposer entities are using which relays, and then finally, I think this, like, puts everything together. This will be the most interesting one, out of here, that kind of, like.
Raúl Kripalani:connects all the dots across the board. There's really, like, the spread is pretty…
Raúl Kripalani:uniform. Not uniform, but I would say, like, there's no pattern that really stands out here, except for, one that I pointed out to the flashbots, people, which is we do see that they're, proportionally to the amount of blocks that they, that they relay.
Raúl Kripalani:they have a larger, like, they, they are above average in zero blobs, essentially. That's kind of, like, the one thing that stood out.
Raúl Kripalani:So yeah, having done that, that overview, very brief overview, I do want to go, I wanted to go back to the column propagation one.
Raúl Kripalani:Because this pattern is denoting that we're… that there's some room for optimization in dispatching quantums to the network, and I wanted to propose… what I wanted to propose to CLs here is there was an optimization that we shipped, a few months back.
Raúl Kripalani:Which is, in Gossip Tub, it is a…
Raúl Kripalani:What we called batch publish, but essentially this is a breadth-first policy for getting columns out.
Raúl Kripalani:as a builder, or as a node that happens to have those columns… that happens to have all blobs, and therefore you can source them from get blobs, and you're able to publish the message in the first place, so you don't even need to wait for the builder's propagation.
Raúl Kripalani:So, in these two cases, it's gonna help to do a breadth-first dispatch. So that is, with batch publish, we're giving 128 columns directly to Gossip Sub, and then today, most implementations, if they don't use
Raúl Kripalani:breadth-first dispatch. What they're gonna do is they're gonna prioritize, sending all 6 or 8, or 10 copies, or whatever the mesh degree is, for every column… for every column.
Raúl Kripalani:First, and then they would move on to the next column, and so on, and so on, and that could be justifying the pattern that we see here. So with best breath-first dispatch.
Raúl Kripalani:What we'd be doing is basically prioritizing sending, like, at least two copies of each column first, and then we continue deepening the replication in each… the redundancy in each subnet.
Raúl Kripalani:So that would be one of the things that we propose. This is already implemented in Go Gossip Sub, so for the Prysm team, it's going to be quite easy to adopt once they upgrade Gossip Sub.
Raúl Kripalani:I heard from Joao and others that maybe it's not needed in Lighthouse, but I wanted to double-click on that, and then wanted to check with other teams as well. Of course, this is, like, this change would not be consensus-breaking in any way, it's just a network optimization.
Raúl Kripalani:So, I'll stop sharing there, and just wait for any questions.
Mario Vega:Thank you so much. Any questions on this? I think we… since this was a little bit impromptu, we might need to revisit the topic also in ACDC, if possible, just to check if anyone is missing from this conversation, they should be aware of it on ACDC. But yeah, Pawan?
Pawan Dhananjay:Yeah, hi, quick question, did you also measure anything with… related to get blobs effectiveness? Like, are we still seeing private blobs at all, or.
Raúl Kripalani:Yeah.
Pawan Dhananjay:Yeah, just wanted to see if there's any difference between.
Raúl Kripalani:Yeah.
Pawan Dhananjay:Cop.
Raúl Kripalani:Yeah, yeah, yeah. That's a great question. Actually, we need to correlate this data with precisely presence of blobs in the mempool.
Raúl Kripalani:with missed slots, and with a bunch of other things that we have in mind, so not yet, but hopefully by today or tomorrow, we'll be having that correlation, and I'm happy to send people a link in Discord.
Mario Vega:Cool, any other questions?
Mario Vega:Alright, I think not, amazing presentation, Raul. Thank you so much for sharing.
Mario Vega:Alright, so, circling back, to Fusaka, because I see that someone from Prysm joined.
Mario Vega:Is a… is a…
Mario Vega:Who was it? James. Yeah, James from Prysm, do you have any… because we touched upon the topic of Fusaka at first, and I think Prysm would like to, if they have any updates to, the events that happened during Maynet, so,
Mario Vega:just wanted to chime in to see if you have any… any updates on that, or anyone else from Prysm, if they're listening.
James He:Yeah,
James He:I guess we're trying to do another release with a more permanent fix. Hopefully that comes out today, if not the next few days.
James He:And, POTUS is also helping write up a technical post-mortem, and we'll have, like, a more official post-mortem, as, as well.
James He:And the new release will have a backfill in it.
Mario Vega:Excellent, thank you. Thank you for the update.
Mario Vega:Alright,
Mario Vega:Alright, so yeah, with Fusaka out of the door, we have also BPO Fork number 1 happening tomorrow, in less than…
Mario Vega:Exactly 24 hours, almost.
Mario Vega:I just wanted to bring up the topic to see if anyone had any, necessary comments on this, are we good to go? Or any concerns, that can be raised?
Mario Vega:At this point.
Mario Vega:Yeah, I guess not much to say on that topic. We will monitor the chain tomorrow when the fork happens, and…
Mario Vega:Stay in the comps to see, that everything goes smoothly.
Mario Vega:Alright,
Mario Vega:With that, any other topics on Fusaka, or any other comments on Fusaka that anyone wants to share today?
Mario Vega:If not, we can go directly to Glamsterdam.
Mario Vega:Yeah, alright, so the first topic in the agenda for Amsterdam is… Amsterdam is, block-level access list. Do we have anyone in the call that would like to share some updates on this?
Stefan Starflinger:I can maybe start out giving some updates from our side.
Stefan Starflinger:So we've been running the Kurtosis testnets a lot, and I managed to get a new milestone of 13 epochs running with 4 clients, which is the most we've done so far, so that's pretty nice.
Stefan Starflinger:Otherwise, I think Nethermind has now also added the debug endpoint. I will share, kind of, the link.
Stefan Starflinger:in the chat, if clients, can, if possible, add the debug endpoint, update the debug bad blocks endpoint to add the generated block accesses, that would be great, so it's easier to debug. I think we have now Nethermine and Bezo that have it.
Stefan Starflinger:Otherwise, we've been finding still quite a few issues in the kurtosis network, so we would recommend the clients all
Stefan Starflinger:run with the EVM fuzzer still to see if there are any issues with their clients. I think that's the easiest way, kind of, to figure out if there are any inconsistencies still. Even if you pass all the tests, we're still finding some issues.
Stefan Starflinger:For example, a recent one, maybe Felipe can talk about it more also.
Stefan Starflinger:If you're on the call,
Stefan Starflinger:About, kind of, some of the tests that you've been working on?
Stefan Starflinger:And that will be in the next release.
felipe:Yeah, sure.
felipe:Yeah, we've, we've been finding, basically, gas boundary issues in…
felipe:I think you found one, Stefan.
felipe:With, self-destruct.
felipe:And so we've been working on adding, tests around,
felipe:Checking gas before account excesses, all throughout,
felipe:The spec where this is found.
felipe:And right now, we have, I would say, probably one or two new cases of this, and we already had quite a few in the last releases.
felipe:But we hope to get these reviewed…
felipe:And merged in and released, early this week.
felipe:for clients to be testing against. And anything that we're finding in kurtosis, we've been trying to write tests for as well. Although sometimes, without these
felipe:Debug endpoints, they're not as easy to find, so these would help quite a bit.
Stefan Starflinger:Yeah, and also, I think most clients now have it in their logging, when there is a disagreement that you log either just the minimal div, or just log the whole log-level access list, so it's easier in the tests and also in the kurtosis networks to see what is the difference.
Stefan Starflinger:That's been very nice. And, like, my goal would be to start a DevNet. I've actually already, accidentally started DevNet.
Stefan Starflinger:But I can't really run any load on it, because it will, basically, fall apart pretty quickly. So, we will probably start DevNet1 once, Kurtosis is running for at least 24 hours, with at least 3 clients.
Stefan Starflinger:that's kind of the benchmark where I would like to start, like, a proper definite.
Stefan Starflinger:I think that's pretty much it from… from my side. And we… One more update we've added in was… yeah, it's caught up with most of the log-level access lists, so that's pretty nice.
Stefan Starflinger:Tests.
Mario Vega:Excellent. Thanks so much. Yeah.
Mario Vega:Yeah, yeah, please, go ahead.
raxhvl:Yeah, I've been also working on some, some of the last bunch of tests.
raxhvl:Those cover system contracts.
raxhvl:So those, like, beacon Root and other system operations, trying to see if those work in isolation well, and with,
raxhvl:User contracts, so we should have those.
raxhvl:In their messages.
Mario Vega:Thank you.
Mario Vega:Alright, any comments, from…
Mario Vega:Yield apps on block level access list.
Mario Vega:If not, thanks so much, Stefan and Felipe, for the work that you guys have been putting on this, and yes, it's nice to hear that even though we are having failures, we are creating tests with it, so I think that's very helpful and very fruitful.
Mario Vega:So yeah, let's see if we can launch the number one in the following weeks.
Mario Vega:Cool.
Mario Vega:So, if nothing else from Block-level Access Lists, thanks for sharing the updates. We can move on to EPBS, and I see that Justin Jory shared a update on the comments. Would you like to, chime in?
Justin Traglia:Yeah, sure. Thanks.
Justin Traglia:I can just give a recap of what I just said then.
Justin Traglia:Yeah, we had a breakout call on Friday. It was very long, it went, like, 25 minutes over.
Justin Traglia:We talked about dynamic penalties, which, could solve the free option problem, for… yeah.
Justin Traglia:in EPBS and out of EPBS.
Justin Traglia:The non-Staked Builder API from Brath, there's a Discord thread that we can talk about that in, if you want.
Justin Traglia:And, trustless payments. We didn't make a decision about trustless payments, at the breakout call, but we're planning to make that decision.
Justin Traglia:At ACDC, this week.
Justin Traglia:So please review the arguments from, like, both sides, so that we're prepared to make a decision.
Justin Traglia:The next breakout call is December 19th, and that's also the last EPBS breakout call. We'll start doing that here at ACD… ACDT.
Justin Traglia:Starting in 2026.
Justin Traglia:And then I've been personally working on refactoring withdrawals code and the specs.
Justin Traglia:Yeah.
Justin Traglia:Do client teams have any updates?
Mario Vega:Just to surface one comment from Barnabas,
Mario Vega:last Friday on the call, we didn't hear any significant amount of argument why we should remove trustless payments.
Justin Traglia:Debatable, I don't know.
Mario Vega:Desperate.
Bharath:Yeah, I think one thing outside trustless payments, in EPP, yes, is that,
Bharath:One thing, to… to make a decision on and to decide is whether
Bharath:we will have, unstaked builders in EPBS, like, so there's been debate on,
Bharath:Whether, clients will even, like, reach out to
Bharath:Builders who are not staked, or whether will we have builders
Bharath:Right? Basically, whether… today, none of the builders are staked, right? We have off-chain entities.
Bharath:The decision of whether, like, clients are gonna, like, only support, like, stake builders versus, like.
Bharath:okay, we're gonna support… we're also gonna, like, talk to stake builders, and also gonna talk to unstake builders. I think that's probably also a decision,
Bharath:we might want to make. I don't think that'll happen in ACDC on Thursday, but I think it's just… it's a conversation, because there are some implications, like.
Bharath:we probably wouldn't have to maintain, like, the builder specs anymore. I think MevBoost would kind of, like, get out of the picture based on that decision.
Bharath:So, yeah, on my part, like.
Bharath:I have written a couple… a design for the builder API, like, I've iterated also a little bit from last Friday, from the EPBS breakout, because regardless of whether we support staked or unstaked, it's a useful data point to have on how the builder API will look like, so that we can make a decision, like, like, oh, it's too complex, it's not worth it.
Bharath:Something like that. So… so yeah, that's… that's another decision,
Bharath:we probably have to make for EPBS. I've heard a bunch of conflicting opinions, so… yeah, it's probably… as Justin mentioned, there is a thread in the Discord chat, so that's a conversation, like,
Bharath:Like, I think we should start and, like, move on, right?
Mario Vega:Thank you.
Mario Vega:Alright, and one good comment from Katya,
Mario Vega:is… there's a question about whether we should be defining some PBS metrics across clients, to gather during DevNets. I think that's… that's important.
Mario Vega:Do you have any idea on that, or is it too early to decide?
Mario Vega:Alright, maybe it started. Cool. Any other comments on EPVS?
Mario Vega:If not, we can move on. Please, join the ACDC to continue discussing this, and also the breakout call on December 19th to join the discussion.
Mario Vega:To make the decision about rotors pavements.
Mario Vega:Alright, so moving on, on the last topic of the agenda, we have the,
Mario Vega:The gas topic, or basically the stateless, sorry, the state,
Mario Vega:Benchmarking, and compute benchmarking.
Mario Vega:Do we have, any updates on this topic? Camila, I see that you are on the call. Anything to share here?
Kamil Chodoła:Yeah.
Kamil Chodoła:Hi. So, we have two things which we are working on right now. Most importantly, focusing with all our efforts right now on repricings.
Kamil Chodoła:So we have a tooling, how to tackle that. We are just making some improvements. On a Thursday, I noticed a few of the scenarios which are not behaving properly under the EST Execute, which Louis is taking a look at right now.
Kamil Chodoła:So we'll have a better approach for that.
Kamil Chodoła:Last week also, we confirmed some stateful scenarios performance, which improved on Nethermine sites, specifically, because Nethermine seems like it's a little bit slower than others because of the DP design.
Kamil Chodoła:And right now, our efforts internally are in determined arms to make a better DB design, to not be a blocker for any potential future gas increases.
Kamil Chodoła:And yeah, so… and one more thing is we have this new process for filtered, what, 100 megacity blocks for a compute testing. I've seen updates from Spencer on that, and want to include that so we have a better… more frequent updates of testing scenarios out there.
Kamil Chodoła:So, yes, we are progressing. I want to make some… make sure that maybe some further bump of 75 megas blocks is viable in short term, but doing some improvements to confirm that.
Mario Vega:Excellent. Yes, thanks. On the artifacts for…
Mario Vega:100 million gas tests. We do have a new workflow, and it should be running at the moment. Basically, this is gonna generate
Mario Vega:New benchmark tests, state, fixtures, sorry, to… To be released.
Mario Vega:On a more frequent basis, and this is just gonna be a release of 100 million gas. So, this should be available using the artifacts download, come on, come online from GitHub.
Mario Vega:I think we can share more, more details on how to access this, this,
Mario Vega:this async. But yeah, basically the idea is that we are gonna be able to make releases… not releases, not for releases, but the tests, they're gonna be available much sooner than the release cycle that we've been having so far, up until now.
Mario Vega:Cool. Thank you, Camille, for the update.
Mario Vega:And yeah, Spencer just shared,
Mario Vega:The details on this, on this update, on the, on the, on the artifacts.
Mario Vega:Excellent. Anything else on this topic that anyone wants to chime in?
Kamil Chodoła:Yes, just last thing is that for repricings, which are very tightly related to gas bumps for future, internally, nevermind, we are working on better traceability of the blocks from the past.
Kamil Chodoła:So right now, we have only RPC calls, and RPC is quite slow for tracing the historical blocks,
Kamil Chodoła:Or at least to check the… how the blocks look like, so not full traces, but for… but only opcode traces.
Kamil Chodoła:And right now, we have a Netarmind plugin, which is capable of tracing… tracing quite a huge chunk of blocks very, very fast, just to get information which opcodes were used.
Kamil Chodoła:Right now, it's under testing,
Kamil Chodoła:Right now, executing a trace for 1 million blocks of Sepolia.
Kamil Chodoła:And it will be much, much faster than MPC stuff, but it's still waiting for the result and some… fixing some issues.
Kamil Chodoła:But this might be interesting, so anyone interested in such data, let me know if you have anything you would like to learn more. Just accept the round number of upcodes per range of blocks.
Mario Vega:Excellent, thank you. Do you have a link for this, or.
Kamil Chodoła:Not yet linked, because this is, like, very, very early stage. I mean, we are right now testing it. Once PR will be published, I will send the link.
Mario Vega:Of course, thank you.
Mario Vega:Cool. Anything else anyone wants to share about the, benchmarking?
Marcin Sobczak:Yeah, we have, we have a spec for, for newer PC endpoint testing period block.
Marcin Sobczak:it will be, to… to be blocked from provided transactions, and, like, SPAC is drafted, and, we have, proof of concept implementation in our mind, and I also saw that REF has some implementations, so,
Marcin Sobczak:We are at this page, and in Nevermind, I need to fix a few minor issues, so it's generally written, but I need to fix it to work overly.
Mario Vega:Thank you. I shared the link for this, this proposal, please take a look.
Mario Vega:I think it's important that we decide
Mario Vega:on what do we want out of this, endpoint. It's also gonna help testing, because it, in theory, we should be able to even field tests using this endpoint, so it's gonna be very, very fruitful. So please chime in, if you have any ideas, especially from other EL
Mario Vega:clients,
Mario Vega:If you have, like, comments, or if you think this is gonna be… there's gonna be trouble implementing this, just please, right… right now is the correct time to… to come in and chime in into… into… into this new proposal.
Mario Vega:Alright, thank you, Marxing.
Mario Vega:Yeah, alright, I think we… with that, we finished the last topic. Do we have any other…
Mario Vega:Any other topics that anyone wants to chime in?
Mario Vega:Before we close?
spencer-tb:Hey, I just wanted to, I guess.
spencer-tb:Bring up a small, kind of, testing organizational question.
spencer-tb:I guess mostly for EL client devs, and then also DevOps, so…
spencer-tb:I guess since last week, we have the new CFI EIPs.
spencer-tb:So the way I see it now is we kind of have
spencer-tb:like, these free tracks going on for Amsterdam, and so the free pricing, block access lists, and then the new CFI VIPs. Last week,
spencer-tb:We internally spoke in our testing meeting about how we want to do test releases, and…
spencer-tb:Basically, we want to start working on the new CFI DIPs, the specs, and get some tests out, so I guess I just wanted to ask the question to…
spencer-tb:EL clients prefer that we… and DevOps prefer that we, kind of…
spencer-tb:test these new EIPs separately outside of block access lists?
spencer-tb:I think, for us, that's what we prefer, so…
spencer-tb:Just to throw the question out there.
spencer-tb:I think we should have, these separate block access list test releases, and then
spencer-tb:maybe separate test releases with the new CFI DIPs, and that would… that would encompass all of the new CFI DIPs, if that makes sense?
spencer-tb:Or if anyone has any better suggestions. Yeah, just throwing the question out there, just to…
spencer-tb:See if we can, get Glamsteram tested as efficiently as possible.
Mario Vega:Amazing. I think, yeah, Dragon?
draganrakita:I don't have a preference here, but…
draganrakita:It feels that clients are making good progress on the ball.
draganrakita:So it seems okay to test all of them with the ball.
draganrakita:But…
draganrakita:Yeah, because we are… when we are testing new IPs, we'll be testing a ball in the same place.
draganrakita:So maybe it will be a little bit better, but yeah, seeing it again, I don't have a big preference here.
draganrakita:In general, if there is no ball inside the state test, or blockchain test, oh, no, not state, but blockchain test, we will not create a compare ball at all. If there is ball, we will compare it.
draganrakita:So either way works.
Mario Vega:Got it, thank you.
Mario Vega:I think the main question is how far ahead are decline teams? I don't think much, because we just decided this last week.
Mario Vega:But yeah, the testing team, I think we can go either way. We can just make the releases on top of block-level access. In my opinion, maybe if we do this approach, it will expedite the testing, because if you are testing everything together, it's gonna end up faster in the DevNets, so maybe that's beneficial.
Mario Vega:So we could explore that, maybe,
Mario Vega:Alright, any comments from all the, all the other client teams on this?
Mario Vega:If not, we can take this to the team, and maybe we can make a decision and update you guys async.
Mario Vega:On, on the, on what we have decided.
Mario Vega:Basically, just whether the EIPs and the block load access lists are separated or joined together.
Mario Vega:And, yeah, just one comment. Seems simpler to test with access list, but that's your preference, yeah. Yeah, I personally would, would, lean…
Mario Vega:towards putting the EIPs with the black glue accesses already, just to make sure that we are testing faster.
Mario Vega:For the future devoids.
Mario Vega:Makes sense. Okay, we will go with that approach. If there's,
Mario Vega:And updates in the meantime, before next ICD, we can keep you guys updated on the async channels.
Mario Vega:Or if we find any issues, or if we have any problems with this approach.
Mario Vega:Yeah, more comments supporting this. Okay, sounds good.
Mario Vega:Alright, thanks for the input.
Mario Vega:Alright, anything else?
Mario Vega:That, anyone wants to share today, before we close up?
Mario Vega:All right. If not, I think that's the end of the call. Thanks, everyone, for joining, and happy BP01 fork tomorrow, and see you in ACD on Thursday.
Mario Vega:Bye. Thank you.
Stefan Starflinger:Bye, alright.
draganrakita:Mine.
Chat Logs
00:05:55
Barnabas:sounds like epbs fixes this 😂
00:06:02
Barnabas:hate to sound like potuz
00:06:54
Barnabas:p2p team did something great: https://ethp2p.github.io/notebooks/02-blob-flow.html
00:07:21
Raúl Kripalani:hey! actually i'd like to give an update on mainnet and Fusaka
00:07:25
Raúl Kripalani:in preparation for BPO1
00:07:26
Justin Traglia:Replying to "p2p team did somethi..."
That’s cool 🙂
00:08:19
Bharath:Replying to "sounds like epbs fix..."
I don’t think ePBS fixes this, if a staked builder doesn’t implement the builder-api, we could kinda get fucked regardless lmaoo
00:08:37
Bharath:Replying to "sounds like epbs fix..."
And by builder-api I mean the direct connection from proposer to builder
00:08:37
Barnabas:BPO1 is actually almost exactly 24h away
00:10:27
Barnabas:do we know why blob count 2/5/7/8 so infrequent?
00:10:53
James He:This isn’t something specific to fulu, but fulu definitely made it a lot worse, https://github.com/OffchainLabs/prysm/issues/16096, this also includes a work around , the workaround isn’t perfect but it’s a solution until we have something more permanent in
00:11:37
Bharath:It would be good to know how late relays are broadcasting blocks post Fusaka since that can impact data column broadcast
00:13:55
Barnabas:Replying to "sounds like epbs fix..."
staked builders would be required to have builder-api.
00:14:03
Barnabas:Replying to "sounds like epbs fix..."
otherwise they couldn’t be a builder no?
00:14:47
Pawan Dhananjay:Do we have any data for getBlobs effectiveness?
00:15:21
Pawan Dhananjay:Very cool visualizations btw 👌
00:15:44
Dustin:what is the incentive to include blobs?
00:15:51
Dustin:for FB
00:16:24
Barnabas:Replying to "what is the incentiv..."
0.004Gwei 😂
00:17:29
Justin Traglia:Replying to "for FB"
flashbots?
00:17:39
Dustin:Replying to "for FB"
Yeah
00:18:18
Raúl Kripalani:yeah sorry i just dropped this without even putting it on the agenda 😄
00:19:02
Bharath:Replying to "sounds like epbs fix..."
Not necessarily, if they have staked the amount with a 0x03 withdraw credential. You become a builder.
I guess if you don’t implement the builder-api its still fine since you can use the p2p bidpool
00:19:25
Raúl Kripalani:pls drop viz requests here
00:19:46
nflaig:Replying to "sounds like epbs fix..."
if the builder has a bug you would still have a slot with an empty execution payload
the difference is that the CL part of the block would not be missed but that's due to the decoupling of CL<>EL block, not really related to whether you have staked builders or not
00:20:30
Justin Traglia:Replying to "for FB"
Builders have an incentive to include blobs for the blob fees. Relays are just an intermediate without incentives/disincentives.
00:20:42
Raúl Kripalani:Replying to "do we know why blob ..."
not at the moment, but once we have the correlations we may see some interesting patterns. in parallel we can ask builders/relays
00:22:21
Raúl Kripalani:Replying to "for FB"
yep it might just be that the builders that work with Flashbots bias towards 0 blobs
00:22:29
Stefan Starflinger:* Update the debug endpoint (if possible) https://notes.ethereum.org/@ethpandaops/bal-devnet-0#Besu-debug-endpoint
00:22:37
Marius van der Wijden:What happened after 13 epochs?
00:23:14
Stefan Starflinger:* https://github.com/ethereum/execution-specs/pull/1846 SELFDESTRUCT GAS Boundry
00:24:00
Raúl Kripalani:Replying to "Very cool visualizat..."
thanks! source is here: https://github.com/ethp2p/notebooks feel free to open issues for visualizations you'd like to see
00:24:57
Stefan Starflinger:* devnet-0 https://dora.bal-devnet-0.ethpandaops.io/
00:25:16
Justin Traglia:In prep for the ePBS update. We had a breakout call on Friday. Talked about dynamic penalties (solves the free option problem), the non-staked builder api, and trustless payments. We’re going to make the decision on trustless payments (keep or remove) at ACDC this Thursday.
A short recap:
https://x.com/JustinTraglia/status/1997021426914668843
The next breakout call is Dec 19th:
https://github.com/ethereum/pm/issues/1835
Which happens to be the last ePBS breakout call. Starting in 2026, we would like to discuss dev updates here at ACDT.
00:26:19
Stefan Starflinger:shhh no one should know
00:26:36
Katya:Are clients interested in defining some ePBS metrics across the clients for devnets?
00:26:59
Justin Traglia:Replying to "In prep for the ePBS..."
Also, I've been working on refactoring withdrawals code in the specs. Hoping to get this merged this week.
00:28:37
Barnabas:@Justin Traglia probably worth raising, that on last friday’s call we didn’t hear any significant argument why we should remove trustless payments.
00:30:53
alex:Replying to "@Justin Traglia prob..."
ehhh 😅 . i guess that’s subjective. are you sure you’re speaking for everyone?
00:31:16
alex:Replying to "@Justin Traglia prob..."
i do see three thumbs up 👍
00:31:22
Stefan Starflinger:Replying to "@Justin Traglia prob..."
I was a little confused by calling relays and block builders the opinion of the community. Are RPC providers also community?
00:31:51
Justin Traglia:I haven’t thought about it much. Too early.
00:32:11
alex:maybe quick note that to me its not clear at all that even if trustless lands side-cars are out. to me that’s still an open discussion. if others disagree feel free to add your view.
00:32:58
alex:Replying to "@Justin Traglia prob..."
100%. community is much wider. “out of protocol block building actors” is maybe what was meant.
00:35:22
spencer-tb:PR has details for bench artefacts: https://github.com/ethereum/execution-specs/pull/1827
00:35:31
Bharath:Replying to "maybe quick note tha..."
Yeah we need to weigh out the pros and cons for staked and unstaked builders.
https://github.com/ethereum/consensus-specs/pull/4733 allows builders to specify off-protocol payments they want to make. We definitely will have a builder-api for staked builders which seems to be similar to the status quo today. It seems to me that relays can stake 32eth, become a builder and just opt out of trustless payments?
00:36:01
Bharath:Replying to "maybe quick note tha..."
My initial hand at the builder-api for unstaked builders, it seems like it could get a bit complicated for clients and relays. Complexity is always debatable tho
00:38:01
Mario Vega:https://github.com/ethereum/execution-apis/pull/710
00:40:45
Barnabas:maybe bundle similar type EIPs together?
00:40:51
Barnabas:instead of having all its own release?
00:42:39
Barnabas:I think its up to the STEEL team, what they prefer
00:42:41
Barnabas:and which is faster
00:42:56
Barnabas:if we have to spend weeks rebasing it later, then maybe better to just have everything be based on top of BALs
00:43:04
Marc:seems a bit simpler to test with BALs but no strong preference
00:43:14
Stefan Starflinger:makes sense
00:43:34
spencer-tb:Nice! Lets aim for that
00:43:40
raxhvl:Would lean w/ BAL too, would be interesting to see interaction with other EIPs
00:43:56
Stefan Starflinger:but lest aim for devnet-3 for this
00:44:10
Stefan Starflinger:2/3
00:44:34
Justin Traglia:Bye everyone
Summary
14 highlights
· 2 decisions · 4 action itemsExperimental
Summary
14 highlights · 2 decisions · 4 action itemsExperimentalfork status and schedule
critical infrastructure
testing progress
client updates
Decisions
Action Items
- Prysm team: Release Prysm fix with backfill and publish post-mortem00:20:02
- Testing team: Release gas boundary tests for self-destruct edge cases00:24:05
- P2P team: Correlate column data with blob mempool and missed slots00:18:36
- EL clients: Review testing_period_block RPC spec and provide feedback00:37:50