Parithosh Jayanthi:Yeah. So the 1st point in the agenda, can we get an update on society? That net 2,
Transcript
Parithosh Jayanthi:and what bugs have been found, or what's the current status? Maybe barnabas gives us an overview first.st
Barnabas:Yeah, sure. So the current overview is during the weekend we had a non finality due to a Taku and Netherland bug.
Barnabas:both tech and the mind bug have been since fixed, and the network is now finalizing again.
Barnabas:This morning we have had to look lots of databases there was lots of database corruptions on windowcl on. Never mind on.
Barnabas:And currently the main, the main client that is having issues is draft.
Barnabas:So tech rest, one tech rest, 2 nimbus rest, one nimbus ref. 2. They are failing to find peers.
Barnabas:So there might be some peering regarded problems.
Barnabas:and granting that the one had a corrupted database as well.
Barnabas:and that just got restarted so it should come online soon.
Parithosh Jayanthi:Could you? Maybe tell us why we had non finality first? st And why was that such a problem for the Els more than the Cls.
Barnabas:So the inefficiency was caused by Netherland. And being offline, they just had over
Barnabas:like 40% of the network. So maybe never mind, can explain what the bug was
Barnabas:and then maybe Taco can also.
FLCL:The bug was just about 4 K. Ids,
FLCL:we are improperly calculated, and we we are not connected on
FLCL:execution layer to any other peers. And yeah, probably can. The could form a a separate cluster of bears
FLCL:with a separate fork, because of that, not sure.
Enrico Del Fante (tbenr):On the Telco side
Enrico Del Fante (tbenr):we we publish published the I guess a cleanup commit on the on the Fussaka dev 2 branch. And
Enrico Del Fante (tbenr):yeah, I think it was was a a clean up P. Commit mostly, but I guess there has been some regression in that. And we
Enrico Del Fante (tbenr):when we pushed the the commit there? We were not thinking about that. Everything now is automated, and the commit was about to be published. Anyway, on Fusaka. It's good that we have the actual immediate test, that something was wrong there.
Enrico Del Fante (tbenr):But I don't have any more detail on that. We will take a look closer. Look to what has been changed there to find out what's was the problem. Yeah.
Enrico Del Fante (tbenr):that's it.
Parithosh Jayanthi:Okay, maybe one follow up for the fork. Id bug, it says, some reason we didn't really catch that in local testing, because that also sounds like it should be quite easy to reproduce.
FLCL:So it worked before, because Genesis, most likely Genesis timestamp, was the same is a timestamps for
FLCL:the initial forks. Maybe it was zeros or so.
FLCL:Not sure it. It's an old part of the code. And
FLCL:That was not changed a lot, and probably new configuration in the devnet.
FLCL:Rebuilt it. Yeah.
Parithosh Jayanthi:Okay, I'm glad we caught that. Then.
Parithosh Jayanthi:and one of this did you already look into? Why, that led to hearing issues or database corruption issues or do? Does someone from that? Or
Parithosh Jayanthi:Bezu have some update on that.
Gabriel Trintinalia | Besu:Room, business.
Barnabas:Yep, go for it, guys.
Gabriel Trintinalia | Besu:Sorry from Bezel. I think there was not related to any of the blocker changes. I may be an existing Sync issue
Gabriel Trintinalia | Besu:in bezel. With the Deb being restarted, the node is synced well and and progressing well. Now.
Parithosh Jayanthi:Okay, so on, Bezo. It was slightly unrelated in it.
Parithosh Jayanthi:There might still be some bug out there, cause.
Gabriel Trintinalia | Besu:Likely I will look into it, but I don't think it's even related to the the flocker changes.
Parithosh Jayanthi:Okay, cool. Good to know. Thank you.
Parithosh Jayanthi:Maybe Roman from reps. Have you guys looked into the peering issue yet?
Roman:we're trying to look into it. But the the problem is that is that we have trouble reproducing it locally.
Roman:So the the best approach for us right now is inspecting the logs and trying to understand what happened on the tackle boxes
Roman:under breathtack boxes.
Barnabas:It's also on Nimbus. Is there something I can do like debug mode? Or maybe you can run locally binary on the
Barnabas:notes, so you can actually reproduce it locally on the definition.
Roman:Yeah. Thanks for the offer. Let's continue the discussion telegram chat.
Roman:I'm I'm talking to Matt right now as well. So we'll look into it.
Parithosh Jayanthi:Okay. Were there any other unexplained bugs on fossified net? 2. What's still left to be done on it?
Barnabas:So the really 1st bug was the prism bug
Barnabas:that should have been caught by a spec test. I actually forgot to mention that. And this caused prism to basically fork away right away. And we were able to reproduce that
Barnabas:fairly easily with their genesis sync.
Barnabas:There's no new spectas for it, and that has been merged.
Parithosh Jayanthi:Awesome. Yeah, I think that also is a whole class of tests. Right? I don't know, Justin. Do you want to maybe talk about it. Do we want to back port? The validator changes to every fork.
Justin Traglia:We could. I'll need to look into that more.
Parithosh Jayanthi:Okay, because it seems like at least a test that's worth having irrespective, which folk we're on.
Justin Traglia:Yeah, I agree.
Justin Traglia:I don't have much other like information on this. But maybe Potus or someone else would.
Parithosh Jayanthi:I don't know who from prisms here today. But yeah, we can probably get some information from them later. Irrespective. If you're a Cl. Team. Then please have a look at the test, and it should be, I guess, part of the next release. So you will. You will be using the test
Parithosh Jayanthi:soon, anyway.
Justin Traglia:And I can.
Justin Traglia:I can quickly say that all other clients do pass the test, which was verified by like the nightly spec tests.
Parithosh Jayanthi:Awesome. That's perfect.
Parithosh Jayanthi:Anything else on Fusaka Devnet to Barnabas.
Barnabas:Maybe I can bring up the
Barnabas:spamming issue that we had. So currently, we have no spamming going on. So maybe that's why we were
Barnabas:able to recover the non penalty because there was not enough load on the network. We didn't have any transactions. We didn't have any block transactions.
Barnabas:we currently have a spammer bug that they're taking a look at right now. So hopefully, we can actually produce
Barnabas:every element finality. And hopefully, we can also recover from that.
Parithosh Jayanthi:Okay, is that also what you wanted to talk about Jacob, or is, did you want to bring up a different topic.
jochem-brouwer:Yeah, it was something related to that. Maybe I can just bring that up or.
Parithosh Jayanthi:Yeah. Go ahead.
jochem-brouwer:Yeah, okay, so what I really want to do is also some performance tests on this def net.
jochem-brouwer:because we have reintroduced or not reintroduced. We have introduced this 7,907 erp, which?
jochem-brouwer:No, it's 10 times the maximum contract size.
jochem-brouwer:And what I really want to test. There is actually calling as many contacts there as possible.
jochem-brouwer:Because, yeah, I will not go into the details. But I think it's still possible to call about 11,000 contracts with this size within a 30 million guest transaction. And I have been setting up these tests. But for that we actually need 11,000, let's say 12,000 of these large contracts. And this is actually being done by the spammer. But yeah, as you mentioned, that likely is a built in the spam because it is not doing
jochem-brouwer:deploying anything at all at this moment, and we are currently at 3,500 contracts. So we are not close to the actual state which we need for this. Well, this performance test
jochem-brouwer:I just wanted to bring up that we need these performance tests because I really want to see that if we do these kind of well, let's say, attack vectors, that we do not certainly run into like blocks, which take like 3 or 4 seconds on clients to to run. And I'm working on these attack factors. And yeah, just wanted to bring that up.
Parithosh Jayanthi:Thank you. Do you have an intuition for how long it might take to deploy also contracts and get through the test? Would would you have an some topic talking point by acde on Thursday.
jochem-brouwer:I have started this spammer at the point where we forked. And I think that was Saturday.
jochem-brouwer:Currently, this problem is not deploying anything, but I think we can say that
jochem-brouwer:well, we need. We need one block per
jochem-brouwer:per contract, and you, of course, have. Well, it actually take 2 blocks because of the pricing that we. If you deploy the 30 million gas contract in a 45 million gas block, then the gas price will rise. So let's say we need what is it? 6,000 but 8,000 blocks? Yeah, for for this to deploy? I think it is possible to deploy this by Acd.
jochem-brouwer:E. But I'm not really sure, because then this has to be fixed. This the spammer broke has to be fixed. Yeah.
Parithosh Jayanthi:Awesome and I see in chat that Pk already mentioned, that he's working on fixing spammer, so
Parithosh Jayanthi:I guess you 2 will be in touch.
jochem-brouwer:Yes. Okay. Great. Thank you. Thank you. Pk, yeah.
Parithosh Jayanthi:Great I also see Bharat on the call. Do you maybe want to give us an update on how? The Mev testing and mev workflow is going.
Bharath:Yeah, for sure.
Bharath:So yeah, I've updated like, my boost and my boost relay the flashbots, one to support Hulu.
Bharath:And the thing. And I've done a bunch of local testing there. And everything generally seems to be fine. There seem there are issues with the
Bharath:issues which I found with with respect to the relay, there are some issues
Bharath:like when we. I'm not locally, at least, we're not noticing, like blob transactions getting into the relay payload. I believe that's a issue with the Red R Builder. We've seen a bunch of error logs. I've reached out to the our builder folks on a Tg. Chat about that. With respect to mevboost so generally in with the Mev relay, pipeline, as I told, transactions without blobs
Bharath:are getting in through the mev relay for the fullu fork. So mevboost and mev boost relay generally seem positive as per my local testing, but I think, like Barnabas like. Probably we will get it on like Devnet, too, and generally see how it works there. So yeah, that's that's the thing.
Bharath:So the timeline.
Bharath:Sorry. Sorry the the timeline. There would then be making sure that the relays, including globes so figuring out that bug, and once that's done, we would be deploying the relay on Devnet 2.
Bharath:Yeah, yeah, for sure. Like, I believe the blobs thing might be an issue with our builder. So we've just reached out to them. And I think the our builder is not building blocks with blobs, for whatever reason I'll talk to them, and we'll see where the issue is. And yeah, once that's fixed
Bharath:and blobs are going through Webboost relay. I think we should be good. There.
Parithosh Jayanthi:Okay, thank you for the update. That's also looking good.
Parithosh Jayanthi:Yeah. Just to touch upon a previous point. Casey, from the prism team mentioned that we. They had stale validator indices in their proposal. Look ahead, code and previous spec tests as well as curses did not change the active valid data set. So that is the spec test that has been added.
Parithosh Jayanthi:Thanks for that. Update.
Parithosh Jayanthi:Anything else related to Fusappa Devnet 2.
Parithosh Jayanthi:I'm currently fighting with the sync test repository. I tried some local sync tests. It seems like lighthouse is taking a long time to sync. I still have to validate that. The Ci version of the sync test is currently broken, and I'll try getting it working in the next day.
Parithosh Jayanthi:But if that's done we should have sync tests on Footsapp and Evnet, too.
Parithosh Jayanthi:This.
Parithosh Jayanthi:And yeah, sorry. Go ahead.
pawan:Sorry are these sync tests with assertor? Or is it like something new? Some new tools.
Parithosh Jayanthi:It's kurtosis in a certain. So it's just a script that starts kurtosis and then uses a certain to make sure that the nodes are synced, and then we throw it in our Github ci, so we can just run it as often as we want.
Parithosh Jayanthi:and there's a ui to track sync time over time. So we can see how it gets worse as the State gets bigger and so on.
Parithosh Jayanthi:Barnabas is asking about backfill. Is there any update that Cl teams want to share about that?
pawan:We have been working on adding backfill after updating the Cgc, we haven't merged it yet, but I think we should be able to get it working soon. We have started testing it. I think.
kasey:Prism has a branch. We're working on it. I haven't touched it since the Cdc. Changes, but that shouldn't be too far off.
Parithosh Jayanthi:So anything else on fusa.net 2
Parithosh Jayanthi:great do we then want to start talking about Fusaka Devnet 3. And what's the plan there.
Parithosh Jayanthi:Do you? Wanna do you have thoughts on that, Barnabas? What's
Barnabas:It's well, the prerequisite for that is basically deciding the 7, 9 or 7 changes.
Barnabas:Do we have the values for that already set in stone, all the repricing vips.
Parithosh Jayanthi:For the repricing eip, we do have values. I think we're just confirming that the values are good for scaling beyond 60 million as well.
Parithosh Jayanthi:But for 7, 9 0 7, I would assume. That depends on Jocum's test as well as I see Marius is on the chat. He had some thoughts on indexes. Do you maybe want to bring it up? Marius?
Marius:Sorry I took modify my zoom stuff. Yeah. So the problem with indices is
Marius:like, without the without the index.
Marius:I don't think the result is is workable because we
Marius:we kind of need to look up too much data in order to answer the question whether we have enough gas to look up this much data.
Marius:And you can construct these weird fringe cases where you have like sub calls that look up
Marius:data and then run out of gas, and then you have the outer call. Call into the sub call again, and and so on.
Marius:So the I think the index is needed by all clients.
Marius:The question is whether the index should be in protocol or out of protocol.
Marius:and the problem with having it out of protocol. Is that in a stateless world?
Marius:This would mean that the witnesses could be bloated with this data.
Marius:Because you can just do export size on a on a bunch of contracts, and and then the witness had to contain all of these contracts.
Marius:I guess right now. We don't have stateless, and we don't have witnesses.
Marius:So
Marius:I don't know if we need to over index on on this this future of of of witnesses of like. But I think that is something that we need to be very mindful of when we move to stateless
Marius:is that we need to make this index be in protocol at some point.
Marius:So the question is, should we do it now.
Marius:or should we do it later on?
Parithosh Jayanthi:Yeah, Ben, do you wanna go.
Ben Adams:Yeah, and related to that is, should we charge extra for it? Because it's essentially would be
Ben Adams:adding the equivalent S load to
Ben Adams:essentially, you're not doing another database lookup to get right.
Parithosh Jayanthi:I, wanna okay, Anskar, do you wanna go.
Ansgar Dietrichs:Yeah, I mean on that point. Basically, I have a similar, I mean similar general concern as as Marius, that basically, I feel kind of uncomfortable with the index not being in protocol, I think doing it in protocol, for Fussaka at the same time, seems to me like not really an option. Maybe people disagree. But yeah. And then, as Ben was saying, actually like, also the fact that now we, if we have the index that it's not really priced is also a problem. But if you price it, then actually, you kind of have to. That means that you have to also reprice
Ansgar Dietrichs:contract loads for existing small contracts right? Because they also 1st go to the index. And the whole point of this index was that we wanted to avoid having to reprice contract loads for existing use cases because they might break. So it really feels like, no matter which which way we approach it. It just seems to create complications.
lightclient:I mean, the pricing is kind of weird, because we don't really charge for the lookup for code right now.
lightclient:kind of bundled into this account to read thing, but if you think about it like, we pay the same to read the balance as we do to load the code for the account and loading the code hashes, like, you know, 2 lookups in the database where the balance is just one.
lightclient:So I think to me it's
lightclient:I mean it. It is slightly different, like, you're not mercolizing the the code size index. So
lightclient:I'm not convinced that we really need to price it like this.
jochem-brouwer:I just want to bring up that Guillaume will likely present something at Acde which solves this problem. Yeah, I will not go into detail, but this will not reprice the existing contracts, and it will put the code size into the account rop.
jochem-brouwer:so.
Parithosh Jayanthi:That's good to know meeting. Do you wanna go.
milen | Erigon:Yeah, Hi, I just wanted to make a small comment that I was looking into this in in Aragon. And
milen | Erigon:I think I also discussed it with other people in Aragon. And we we think that we actually can load the code size without loading the code
milen | Erigon:and memory, and we actually don't need any extra index for this operation.
milen | Erigon:So we don't need to store any extra data.
milen | Erigon:So kind of like, my preference would be to sort of leave this out of protocol unless there's very
milen | Erigon:good reasons like, in terms of future direction, or why we would want to add it in protocol.
milen | Erigon:So yeah, I just wanted to highlight that. But if if you guys kind of foresee that there's any strong reasons to add it in in the try. Then I guess we'll we'll have to.
milen | Erigon:But yeah, we don't really need this index, for example.
Parithosh Jayanthi:Is there, then, any concrete thing we should be working on between now and Acde? Besides the test that your team is running.
Parithosh Jayanthi:and I guess we make a decision at Acd about how 7, 9, 1, 7 goes ahead.
jochem-brouwer:Yeah, I will make sure that I will try to get these benchmarks and these performance tests done by ace, yeah.
Parithosh Jayanthi:Okay, anything else, you guys think needs to happen before Ecd Onskar or Marius regarding the cip.
Ansgar Dietrichs:The only question because I was briefly way when Johan talked about the tests that we want to run, anyway. And does that include the test where we basically do this clever kind of running out of gas at the right moment in time to basically stress test the worst case test? Or is it only the regular test where you're actually fully load and able to pay for the full load.
jochem-brouwer:No, this is exactly the test which you say. What you do is you do a call, or should call to a contract where you want to call a target contract for this target contract, you will have exactly enough code to call into a 24 kB or less contract, but you will not have enough gas to actually pay for this this code size. Well, cost.
jochem-brouwer:So what you can then do, you will force clients to actually look at this code, and it can either be via the site index or for clients which do not have the site index. They will. They are forced to load the entire code
jochem-brouwer:to actually look up the code size so they can determine the price. And yeah, of course, this will run out of a guess, but that also means that we can still read the same amount of contacts which are like now, bigger than the 74 kB.
jochem-brouwer:And that's indeed the performance test. So likely that's what I would expect. But yeah, my attack is not working, or every client has already done this successfully, but that would mean that every client which does not have this site index has low something like 3 GB of data. So yeah, I think, yeah, this performance test. That's that's the goal of this test. Yeah.
Ansgar Dietrichs:Awesome. Yeah, no, that's exactly the test that I think we just need to run. So that sounds good. Then.
Parithosh Jayanthi:There was also a comment in the eip that Mario linked. Do you? Maybe you want to just bring it up, Mario.
Mario Vega:Yeah, of course, it's a very simple
Mario Vega:open question is whether we charge for the extra size during the transaction like entry contract. So just imagine that we we call the transaction calls into directly into large contract. We currently do not charge anything extra. So that's basically just the question of whether we should charge for it or not, and I think we should do
Mario Vega:it just. I just placed this comment into the eap for us to decide at some point that we have to either charge for it or not.
Mario Vega:Yeah, I don't know if if anybody else has thought about this
Mario Vega:or not, but I think we should. We should definitely decide on something.
jochem-brouwer:Just to comment on that. If we do this, then we have to analyze well mainnet contracts which could actually break.
jochem-brouwer:Because this means that if you currently call into a well and about existing contracts, that means that you now need more gas than, or you need more upfront gas than you have now, and that could, of course, mean that something breaks. So that is something we should definitely analyze. First, st yeah.
Roman:Sorry. Can you explain it? How?
Roman:How does it break? Because it will be only charging for the extra?
Roman:And we don't have such contracts today.
Roman:Such a big contract today.
jochem-brouwer:Okay, I guess I don't know if I understand the solution. Sorry. Yeah, I will reread this this way. Yeah.
Mario Vega:Yeah, basically, basically just charge. If
Mario Vega:the entry point of the transaction is a large contract.
Mario Vega:since there are no large contracts right now in Mainnet. I don't think there should be anything breaking.
Mario Vega:but I think this should not affect the intrinsic gas cost of the transaction. It should only affect once you enter the execution. If you don't have enough gas to pay for the last contract just out of gas immediately. That's my suggestion, at least. But I don't think it should break any existing behavior. In my opinion.
jochem-brouwer:Right? Okay, I see. But I'm not really seeing how this solves the problem, because you still you still have to load the code in order to figure out if this is a large contract. So
jochem-brouwer:yeah. But but maybe we should. We should discuss this Async. I'm not sure of it.
Mario Vega:It's basically, I think it's the same problem with without the index. If you have the index, the code size in the in the in the state tree is basically the same. The same issue for an inter
Mario Vega:intercontract call.
Mario Vega:Yeah, I don't think there's any any
Mario Vega:any extra problems to this. But I'm sure yeah, we should definitely analyze it.
Mario Vega:Yep.
Parithosh Jayanthi:Okay. So our deadline for Thursday is gonna be getting all the open questions for 7, 9 0 7 taken care of
Parithosh Jayanthi:and to make sure that we're happy with the cost set for modex. Right? And if that's done we should be ready to discuss 3 on Thursday.
Parithosh Jayanthi:Okay, there was one other question from the Eric on team, I think Andrew had asked this. Our blocks supposed to be Ilp serialized as header transactions almost withdrawals
Parithosh Jayanthi:the questions on the interrupt chat. I'm not sure. Does someone already have an answer for them? Or do we want to engage on the interrupt chat about this.
Parithosh Jayanthi:Okay, let's take it to the interrupt chat. Then andrew asked the question. About see? 4 h ago?
Parithosh Jayanthi:Great. Then there is the one sec. Let me get the link up again.
Parithosh Jayanthi:So there's a open topic that Marco pointed out about engine get blobs v. 2 from last week, and to not prohibit partial responses.
Parithosh Jayanthi:Are you on the call, Marco?
marco:Yeah, about those. Yeah, do you wanna introduce it?
marco:Yeah, sure. So a bit of context here is that
marco:for the Peerdas stuff, we use the get blobs. V. 2 Api to ask the El side if there's any work that we can reuse, and in the case that we have a full column. We can just reuse all the work from the El side. And we're golden.
marco:The problem is that in 40% of cases, if there's a private blog in that block. That means we're missing one of the cells in the column. And with the current text.
marco:that means that the El side returns null, and there's no work that we can reuse.
marco:We're pretty sure, on the networking side that we can make some like relatively low, level changes to gossip, sub.
marco:with little changes required on the clients to
marco:allow us to reuse all the
marco:partial responses, all these, like everything except the missing cell.
marco:and then for some context on data like around 40% of
marco:blocks with blobs, have it have one private blob. And so it's like, in these cases, we're really just missing that one cell.
marco:And currently, we'd have to fall back to transmitting the whole column.
marco:If we don't prohibit partial responses, we can reuse all that work, and then just propagate that one missing cell.
marco:And so the change here is just to make the text not prohibit partial responses. Kind of goes back to what the well it does go back to what the old.
marco:older texts were saying.
marco:And then my understanding of the pushback here is that we do not have this optimization. Currently, this is true. And we are paying extra for serializing
marco:this data. That is currently not useful, which which is also true.
marco:Yeah.
marco:And so I just kind of want to talk synchronously through this and and try to
marco:and end up with some some general agreements.
Parithosh Jayanthi:Yeah. Maybe to clarify 1 point that was also in the in the comment, section, so partial responses are still useful. Info, because, my assumption was also that they weren't useful, and that's why we removed them.
marco:So currently, right now, they're not useful.
marco:But we think that we like pretty sure that we can make them useful without a hard fork.
Parithosh Jayanthi:Got it. So we want to lay the groundwork for a future change that might make it more useful.
marco:Exactly. There's
marco:There's like a gossip sub change we can make without a hard fork that would make us that would make partial responses useful.
marco:And so with, if we don't prohibit the partial responses.
marco:it's likely that we could ship this gossip sub change without a hard fork and get these benefits, as we scale the blot parameter like with the Bpo, only changes.
Parithosh Jayanthi:Got it? At least, in the comment. I see that pawan had some hesitation towards merging the change. Do you maybe want to voice hesitation or.
pawan:Yeah. But I think, like the main point was that
pawan:we initially did did not have partial responses. And then I initially
pawan:did have partial responses, and then we didn't have. So I was just a little hesitant to merge this change without actually having the optimization, because, like, if El starts returning partial responses, then it is a serialization and deserialization load on both ends as the number of blobs increases. And I also felt that we could
pawan:potentially. Just add, like another
pawan:version of get blobs later on. If this optimization works out and it would like that also wouldn't sort of require hard folk per se, but
pawan:it is a little more coordination effort. But yeah, I just like the main objection was to validate the optimization before actually
pawan:changing the the behavior.
Parithosh Jayanthi:I guess the main questions gonna be if we want to include this change incrementally for, and then just ship the change proactively. But I guess mirroring Pawan's point. Do we already know enough that we know that this change is going to be useful, such that
Parithosh Jayanthi:we can proactively add it? Or do we want to do this later with a post for Safa, for example.
marco:Yeah, I I'm you know. If if we think that we could like make this kind of change
marco:after
marco:we we launched Osaka like that. That's also fine by me. I think I just don't want to
marco:be in a position where we have like this
marco:pretty obvious optimization that we can't use until Glams again.
Parithosh Jayanthi:Yeah, I think if we do it post Susaka. We'd likely just use engine get blobs. V, 3, and then that would be a engine capabilities check, and if v. 3 exists, then the Cl. Would use v. 3, and if not, it would fall back to v. 2.
Parithosh Jayanthi:So I. I don't think there's a blocker with this needing to be Infosaka. But yeah, I I think if all Cls agree that they want to put this in then it's great. But right now it at least seems like
Parithosh Jayanthi:prism and lighthouse mostly want to learn more about the optimization before agreeing.
marco:Yeah, yeah, I I don't. I don't have a strong preference here. And I'll just add what a couple more things, which is I think we can make the
marco:the El side technically doesn't have to change like, if it keeps its current behavior, it would still be spec compliant, the Cl side would have to change, and it might be able to do some optimizations where it doesn't
marco:it? It
marco:doesn't have to pay so much for deserialization. If it notices any of the values are null, so it can just like skip the blob deserialization of any of the values are null. But again. Like, yeah, I get that. This is some extra extra work.
marco:But yeah, like I no.
marco:my, my strong preference here is we can do something for this without before Glamsterdam. The specifics I leave up to you all.
Parithosh Jayanthi:Okay. Do we have any other Cl teams who wanna speak either in favor or against the change?
Parithosh Jayanthi:I think in general, it seems like the main downside is that we're spending a few more cycles doing serialization and deserialization for a potential future optimization.
Parithosh Jayanthi:I guess that's the crux right?
Parithosh Jayanthi:Done.
marco:That's my understanding.
Parithosh Jayanthi:Casey also mentioned that he's in support of Marco's proposal. But most of prism hasn't reviewed it yet.
Parithosh Jayanthi:Perhaps we can create a thread on Acd and put a deadline for tomorrow to agree. If client teams agree, then we can schedule this change, for if that works.
marco:Yeah, I guess another comment is that
marco:if we are already doing a lot of testing, now, does
marco:does it make sense to like introduce a v. 3. Version mid Fusaka. Will that also necessary? But like, require a similar level of testing that that we want to do? And would we be hesitant on rolling out that v. 3 version.
marco:do, I guess, is there prior art? There.
Parithosh Jayanthi:Yeah, I I think I'd also agree with you. I think if we're doing this change after Fusa, it might not actually get the same level of testing as we do with Fusaka. If we do it after Fusaka it'll likely be like engineer one where a client team rolls it out and then presents their results, and other client teams decide to implement it or not.
Parithosh Jayanthi:But I'd say the play here would be to create a thread on all core devs and then collect Feedback Async over the next day, and then have a deadline for merging this in tomorrow. I. It seems like tending to. Yes, makes sense over here. So unless the strong objection by tomorrow, we should put this change in for separate.
Parithosh Jayanthi:does that sound okay to everyone?
marco:Sounds good to me.
Parithosh Jayanthi:Okay, thank you.
Parithosh Jayanthi:Sunnyside lab team. Do you guys want to give a brief update on what you've been testing on Fusaka and what the next steps are.
J Sunnyside Labs:Hi, yeah, we've just started running devnets with devnet 2 spec
J Sunnyside Labs:and yeah, as we discussed, we will 1st do benchmarking of the network bandwidth.
J Sunnyside Labs:and then we will do bandwidth constraining test after. So there will be 2 parts in the test.
J Sunnyside Labs:And
J Sunnyside Labs:yeah, we've started with 50 present guest news over the weekend, and it seems quite good so far.
Parithosh Jayanthi:Awesome and I guess you guys will share a write up on the interrupt chat and then reach out to client teams.
J Sunnyside Labs:Yes, we will.
Parithosh Jayanthi:Perfect and I guess that leads well into the next topic. We should slowly be planning more offensive networks on Fusaka. So split networks trying to do deeper reorgs. I think one that would be interesting is with a proposal. Look ahead to have reorgs over one or 2 epochs. See what happens there?
Parithosh Jayanthi:Do we want to start doing these with Fussaka Devnet? 2 specs, or do we want to block on Fussaka Devnet 3, and then do it afterwards. What does the timeline look there?
Parithosh Jayanthi:Barnabas mentioned Stemnet 3. And
Parithosh Jayanthi:he has at least a few pluses.
Parithosh Jayanthi:Okay, then let's
Parithosh Jayanthi:set that as the plan so we'd focus on getting devnet 3 specs frozen asap and then focus more on breaking Devnet 3.
Parithosh Jayanthi:Cool anything else on Fusaka.
FLCL:I I probably it might be relevant to bring the topic about below Pertex. Limitation.
FLCL:Yeah, there is slight opposition of having it as part of follow up schedule
FLCL:and probably for the next devnet. We need to agree what we will do with that because it's right now implemented this
FLCL:via blob schedule. Right?
FLCL:So my proposal is still to to do this. and
FLCL:or maybe just tries bloop limit. Consensus bloop limit to 9. And
FLCL:simplifies the blob schedule. If it work for you guys. Why, it may be important. Is that? Because it seems like
FLCL:the expo is not scaled. It's so, for example, some clients decline transaction. Decline OP
FLCL:transactions. Propagation. If it has.
FLCL:if a transaction has more than 7 tops, it means that local block building will not include big transactions in the future, and
FLCL:I guess it's done to not overload the pool with heavy transactions.
FLCL:But maybe we need to do some steps towards its scaling.
FLCL:Yeah.
Parithosh Jayanthi:Yeah, there was one, maybe clarification. Anskar asked. The
Parithosh Jayanthi:yeah was the was the decision to move the per Tx blob limit to the prdos eip? Or was it to leave it in the Vpo.
Barnabas:I mean, that doesn't matter where the where it lives.
Barnabas:The the question is whether we want to actually just hard code this value? Or do we want to be making configurable? Because right now what we do is if it's not configured by the client, then we default Max, which is not right, and there was a push back for doing that.
Barnabas:So the next question is, if we shouldn't be defaulting to Max, should we be defaulting to 6,
Barnabas:and if it just default to 6, then we probably don't want to make it configurable.
Barnabas:or if you make it configurable, then we need to just default to 6 instead of defaulting to Max.
Barnabas:and we can still keep it in the Ppo.
Barnabas:Yeah, I be.
FLCL:With Craig. It's it's 9 consensus there. Mean, we
FLCL:have it already. More than 6.
FLCL:should we add this limit now and check it.
Parithosh Jayanthi:This is kind of retroactively saying, the Max, the explore, the expo blob we have today, which could be 9, is basically going to be the Max we have for the future right.
FLCL:And yeah, I guess.
Parithosh Jayanthi:Is there that in itself makes sense to me? I think we're also doing a similar change with transactions per gas.
Parithosh Jayanthi:So is it. Has there been any opposition towards this change?
Parithosh Jayanthi:Is there a discussion somewhere that we can follow this on.
Barnabas:Any discussion in the execution for
Barnabas:regarding this, but not not how many people participated.
Barnabas:I think we just need to open a new Pr. And discuss it over there.
Parithosh Jayanthi:Yeah. Also, I think Marius says. He thought they wanted to. Max at 6.
Parithosh Jayanthi:Francesco, you had your hand up. Do you want to go?
Francesco:Yeah, I mean same thing you said. I don't remember exactly when this happened. I don't know if it was on an old core dev, or on another testing call, but I'm pretty sure that
Francesco:there was like a general agreement for for 6 at some point as the the value then I mean, besides that, I don't. Personally, I don't really care if it's configurable or not like, if
Francesco:people want it to be config configurable, and it doesn't cause too much work to to change it back to configurable. That's fine. But I I really don't think we should. We should be defaulting to
Francesco:the Max, if if it's not, there isn't like a specifically configured value. And yeah, and specifically, we had
Francesco:the number 6 as something that a lot of people agreed on.
FLCL:That was very quick discussion, and I just trying to bring such an argument to that because I felt to
FLCL:to bring bring it before so if we set this to 6
FLCL:we will need to validate that new transactions on the fork border when a circuit happens
FLCL:before we allowed 9 transactions.
FLCL:If we allow just 6, we will need to validate on forks. That
FLCL:transaction pool does not contain
FLCL:big transactions now, and it will be a little bit more complex, and seems to me that 9 is better.
FLCL:number as it is the current number, and we will not need to check for a decrease
FLCL:in the future.
Parithosh Jayanthi:Ben, do you want to continue.
Ben Adams:I I just say, yeah.
Ben Adams:I'm keen to have it low. But if if we've already done the work to have it configurable. It's worth keeping that because when we hit 72, maybe 6 isn't the right number, maybe one get a little bit higher.
Ben Adams:So it'd be if we've. If we've already done that work, then we might as well keep it in.
Ben Adams:That makes sense, because as we go ahead, that that number might increase as the number of blobs increases.
Barnabas:Right. But then the argument is that
Barnabas:we still need to change the IP to specify that the Mac. That the default value, if it's not defined should be 6,
Barnabas:and not Max.
Ben Adams:Yeah, I mean, I I just mean if we if we have it in the Bpo
Ben Adams:then already, then we might as well just keep it in the vpo. And set it 6.
FLCL:They want to clarify Marius mentions 6 is a constraint for text pool but you can build a block with 9 blocks right now, right?
FLCL:It's not constrained by consensus.
Marius:We can. We can build a block with with 9, with 9 blocks, but we only allow 6 block transactions in our transaction pool.
FLCL:Yep.
FLCL:so this number that we are discussing right now. It's a constraint for like
FLCL:block building, too. I mean you will not be able to include big transactions in the future.
FLCL:Oh, is this difference considerate.
Barnabas:I think the idea was to limit the gossip limit on the outside.
Barnabas:That's why we wanted to limit it to 6.
Barnabas:So we have a predictable value. There.
FLCL:Okay.
Ben Adams:Yeah. Yeah. And also also, if we're including
Ben Adams:larger transactions that are in the transaction pool that we lose the benefit of, get blobs.
Parithosh Jayanthi:Just mimicking something that Anskar put in the chat. We apparently have already made the decision to move Max out of the Bpo and set it to 6. So if we do want to revisit this, then I think the best approach is to have a thread on either all core devs or execution dev.
Parithosh Jayanthi:and to bring it as a change to Ecde on Thursday.
Parithosh Jayanthi:But that being said, I think right now, it does make sense to leave it at 6, and for
Parithosh Jayanthi:and Barnabas does mention, we don't have the modification to the eip, so we should have that.
Parithosh Jayanthi:Yeah. But other than that, I think that would be the
Parithosh Jayanthi:central plan forward for the topic, does that sound? Okay.
Parithosh Jayanthi:okay, so we would still need an update to the eip, if that has not been made already.
Parithosh Jayanthi:Moving on to the next topic since we are kind of close to the end. Gas limit changes
Parithosh Jayanthi:quite a few plan teams, I think. Never mind. Gets best. And
Parithosh Jayanthi:Erica have all released releases with 45 million as the default gas limit
Parithosh Jayanthi:the expand Ops team has also released a blog post with how we arrived at the 45 million gas limit number, and I think Bezoo team is working on a release this week to update that default as well.
Parithosh Jayanthi:So I think this is also a good time to start asking sales to update their gas limit knowing that the performance related fixes are now put into the Els.
Parithosh Jayanthi:So yeah, I just wanted to flag that for Cr teams. I know that. Prism already has a branch with the change ready. And
Parithosh Jayanthi:yeah, I think other Cls will follow shortly after.
Parithosh Jayanthi:and relating to the gas limit topic in general. Felix, from the guest team has been looking more into Modex optimizations on gas.
Parithosh Jayanthi:There's an active chat in the Guest limit channel along with some tests and looking into it.
Parithosh Jayanthi:And yeah, the bezel team
Parithosh Jayanthi:says that they're working on testing the new release sync performance regressions, and so on, so that should be there soon.
Parithosh Jayanthi:Besides that, I think Pk. Had a question on the gas limit. Do you want to talk about it. Pk.
pk910:Yeah, just a general question on Perth Def, net one, we currently have a gas limit of one
pk910:1 million and usual spamming scenarios. Basically break chain right now. I think that was in preparation to get contracts deployed, just wanted to ask if
pk910:is still needed, or we can lower it again.
Parithosh Jayanthi:I think Joachim was the one doing the deployments. Do you wanna
Parithosh Jayanthi:talk about it, Yokom? Is it? Can we lower the gas limit again, or do you still need it to be very high?
Parithosh Jayanthi:He might have dropped off. So we might have to talk to him about this later.
Parithosh Jayanthi:Okay, is there anything else people want to talk about regarding the gas limit?
Parithosh Jayanthi:Okay, any other topics anyone wanted to bring up for for the call today.
Parithosh Jayanthi:Otherwise I think we have a plan for at least a couple of the topics for Acde, and we should be on target for freezing the scope for and then planning definitely. 3.
Parithosh Jayanthi:Thank you. Everyone for joining then and have a nice day.
pawan:Thank you.
Ameziane Hamlat:Thanks, bye.
Marius:Yeah, thank, you.
jochem-brouwer:Thank you.
Mario Vega:Thank you.
Chat Logs
00:01:31
Justin Traglia:There was also a little Prysm bug, which Lin made a spec test for (thanks!)
https://github.com/ethereum/consensus-specs/pull/4413
00:02:51
Barnabas:too much cleanup :D
00:03:09
Barnabas:Watchtower for the win :D
00:04:30
Barnabas:I don’t recall any changes
00:05:44
Barnabas:Replying to "There was also a lit..."
Right that was one of the first bugs triggered right after FULU with deposits. This was already on friday, totally forgot about that.
00:10:37
pk910:I think spamoor had a problem with that unfinality.
I'm working on it..
00:12:05
Barnabas:no pressure 😂 - we are all looking at you
00:12:44
kasey:The prysm bug was that we had stale validator indices in our proposer lookahead code. Previous spec test and kurtosis setups did not change active validator set.
00:13:39
Justin Traglia:Replying to "There was also a lit..."
Regarding standard tests for other forks, I’ve made this issue. This won’t be done in time for the next release (initiating this process in a couple hours). But we’ll be sure to get these tests into the next (beta.0) release.
00:14:48
Barnabas:any update on backfill?
00:15:27
pawan:Nice!
00:18:19
Ameziane Hamlat:Replying to "I think spamoor had ..."
FYI, last week I could run spamoor tests shared by Jochem on a besu performance devnet. So I could run the factorydeploytx scenario to deploy the ~ 12k contracts and then calltx scenario. I built spamoor from sources so the issue could be not related to the sources as I could make it run correctly.
00:18:52
Ameziane Hamlat:Replying to "I think spamoor had ..."
https://discord.com/channels/595666850260713488/1386773806407749743/1388068741597757492
00:18:54
Ansgar Dietrichs:I agree with Marius’ assessment, out-of-protocol index seems very dangerous
00:19:07
J Sunnyside Labs:A quick update from Sunnyside Labs: We’ve started testing with devnet-2 spec. We will share general details in a Discord thread and communicate with each team individually.
00:19:27
Parithosh Jayanthi:Replying to "A quick update from ..."
Do you guys want to give a brief update after the devnet-3 topic?
00:19:35
Felix:we should not do it now
00:19:41
J Sunnyside Labs:Replying to "A quick update from ..."
Sure 🙂
00:19:44
Ansgar Dietrichs:doing it now seems insane timeline-wise
00:21:39
Ansgar Dietrichs:well you’ll have to merkelize it for zk / stateless
00:23:50
Ansgar Dietrichs:the fact that we are 3 days from wanting to freeze the spec and we still have these conceptual questions really makes me believe that pulling 7907 completely is the only responsible choice
00:23:56
Mario Vega:Small comment worth looking in the EIP: https://github.com/ethereum/EIPs/pull/9910#issuecomment-2997357062
00:26:29
Roman:reth agrees
00:29:17
Ansgar Dietrichs:Replying to "reth agrees"
with pulling 7907? or something else?
00:29:29
Parithosh Jayanthi:A question about EIP-7934. Are blocks supposed to be RLP-serialized as `[header, transactions, ommers, withdrawals]`
00:30:39
Parithosh Jayanthi:https://github.com/ethereum/execution-apis/pull/671
00:33:04
Marius:I think we should do this
00:33:29
kasey:The context here is execution engine api correct?
00:34:01
Marius:The drawback is just a bit of processing time on the engine api, the advantage is that we can ship this before glamsterdam
00:34:01
FLCL:is this pubsub change documented somewhere?
00:34:42
marco:In draft: https://github.com/libp2p/specs/pull/685
00:35:47
kasey:I had the same thought as Pawan - we have exchange capabilities which would enable us to add new execution engine apis between forks, but having this at fusaka would streamline things quite a bit and make the optimization CL only.
00:36:18
FLCL:If CLs agree to bring this optimisation, let's just switch to partial
00:36:59
Francesco:If we decide to go ahead with this, can we soften the language in a way that allows returning [null, …, null] even if you have some of the blobs? (e.g. rather than [A, null, B]). Basically just leaving it up to the EL whether to return partial responses or not
00:37:05
kasey:I am in support of Marco’s proposal actually
00:37:27
kasey:(But most of prysm hasn’t reviewed it yet so I don’t want to speak for team)
00:38:59
pawan:Yeah to be clear, I like the optimization as well. Just don’t want to spend more cycles on changing the spec, testing etc
00:42:21
Barnabas:I’d do it on devnet 3
00:42:37
Barnabas:We really need to freeze the spec and no more changes
00:42:48
Barnabas:otherwise we might test something that is gonna give us misinformation
00:43:24
Marius:I think we need to start ramping up testing in parallel, so we have the infra in place fir devnet-3 specs
00:44:17
Ansgar Dietrichs:I thought we decided to move the per-tx blob limit to the peerdas EIP, out of BPO parameters, no?
00:44:49
Barnabas:Replying to "I thought we decided..."
yes but FLCL is arguing that it should not be removed
00:45:04
Ansgar Dietrichs:Replying to "I thought we decided..."
ah okay sorry, understood it the other way around
00:47:09
Marius:I thought we wanted max at 6?
00:49:03
Marius:We only allow 6 in our txpool
00:49:10
Ansgar Dietrichs:I think we made that decision on acde two weeks ago (the one I filled in for Tim): move max out of BPO and set it to 6.
So if we want to revisit, we should do that on acde. but might be too late to change for Fusaka.
00:49:22
Marius:Also you need to go through the txpool anyway for the proof changes
00:52:19
Barnabas:Replying to "I think we made that..."
we still don’t have a modification for this in eip-7892
00:52:22
Ansgar Dietrichs:I think we discussed the in-protocol vs pure in-mempool restriction for quite a while. I personally liked the in-mempool only approach, but overall most people preferred in-protocol
00:52:27
Barnabas:Replying to "I think we made that..."
unless i missed that?
00:53:16
Parithosh Jayanthi:https://ethpandaops.io/posts/gaslimit-scaling/
00:53:18
Ansgar Dietrichs:Replying to "I think we made that..."
should I have followed up on this? I stopped thinking about these topics when the call was over
00:53:31
Ben Adams:wen CLs gaslimit
00:53:33
Barnabas:Replying to "I think we made that..."
https://github.com/ethereum/EIPs/blob/master/EIPS/eip-7892.md#L124
00:53:55
Barnabas:Replying to "I think we made that..."
yeah EIP changes need to be tracked, otherwise its gonna be lost in the echo
00:53:56
Barnabas:Replying to "I think we made that..."
lol
00:54:04
Enrico Del Fante (tbenr):We will merge it soon
00:54:10
nflaig:will get it in our next release (1-2 weeks)
00:54:13
Ameziane Hamlat:On besu, we should have a release out on Wednesday. We’re in the process of testing the new release : sync, performance regression, ..etct
00:55:03
Ansgar Dietrichs:Replying to "I think we made that..."
and that’s something Tim normally does?
00:55:18
Ansgar Dietrichs:Replying to "I think we made that..."
anyway I guess now he is back, how convenient :-)
00:55:59
Barnabas:Replying to "I think we made that..."
he makes sure someone is assigned to change the spec yes