Toni Wahrstätter:Hello, everyone, and welcome to the first, EOP7928 breakout call.
Transcript
Toni Wahrstätter:I thought it makes sense, to have a breaker call around the EAP, now on a more regular basis, just to keep track of progress happening.
Toni Wahrstätter:On today's agenda, there's not that many topics, but we still have, some open points to discuss, and I would propose that we start, today's first breakout with Jared talking a bit about the implementation progress he made in GEF,
Toni Wahrstätter:Then there is Rahul, who will talk a little bit about testing progress and everything that is happening on the testing front.
Toni Wahrstätter:And then we have Chen from the EVE storage team for QuarkChain, who will present some results on their performance analysis of block-level access lists in GAF.
Toni Wahrstätter:And before we start getting into those topics, let me just quickly recap some info around the EAP.
Toni Wahrstätter:First of all, on blockaccesslist.xyz, I posted it into the chat.
Toni Wahrstätter:You should find all the important information, especially if you scroll down to the resources headline.
Toni Wahrstätter:There you find all the, specs, like the EL specs, the execution API specs, and the CL specs, as well as test cases and everything. So this is kind of the place where we try to bundle all that information.
Toni Wahrstätter:And…
Toni Wahrstätter:Yeah, everything is summarized there. If you want to submit new test cases, then Rahul will show you how to do so. There's a dedicated testing doc where we collect many test cases.
Toni Wahrstätter:But yeah, without further ado, I would say, Jared would be…
Toni Wahrstätter:pass it to you in order for you to present us some initial results you collected on GAF.
Jared Wasinger:Yeah, cool. Yeah, so I can just… just to summarize where the Geth implementation is at right now, we fully implemented the EIP, except for storage and account reads,
Jared Wasinger:… So, basically, at this point, … we're…
Jared Wasinger:we're more or less ready to… to… to do a DevNet, in the sense that the code is written and I'm reasonably sure.
Jared Wasinger:that the consensus API, the engine API changes and the minor changes should work, but I'm still verifying that on my end.
Jared Wasinger:The execution… the… so, block access list execution and, building are… are… are implemented.
Jared Wasinger:And I've done… quite a bit of work to get the performance To a reasonable, … point.
Jared Wasinger:So, I guess I'll just share my screen. I have a few slides just kind of summarizing
Jared Wasinger:The latest benchmark.
Jared Wasinger:Where… used to Google Meet, so, …
Jared Wasinger:Okay, so I'm just going to…
Jared Wasinger:Looks like I have to give it a permission here.
Jared Wasinger:I have to rejoin the call.
Toni Wahrstätter:Karim, I think this is a good point, regarding the reads, I think this is still up for discussion. So far, my feeling was that, the majority of people is still for including the reads, just because parallel
Toni Wahrstätter:I.O. might, be very valuable, especially with bigger blocks.
Toni Wahrstätter:But, yeah, this is still something we have to discuss.
Toni Wahrstätter:Okay, Jared is back.
Toni Wahrstätter:Jared, are you… are you already talking?
Toni Wahrstätter:Is this true?
Milos:You're muted.
Toni Wahrstätter:Jared, can you hear us? You're still muted.
Jared Wasinger:Yeah, yeah, yeah, oops, okay. Yeah, so, just to, …
Jared Wasinger:overview of the serial execution that Geth currently performs. We execute transactions one after the other. Simultaneously, for mutated state, we will fetch
Jared Wasinger:Intermediate trinodes in the background, so that when we go to compute the state root, It's… vast, …
Jared Wasinger:Oop, this is the wrong slide. Oops.
Jared Wasinger:Yeah, that's the next slide.
Jared Wasinger:Okay, so…
Jared Wasinger:Mmm… right, in block execution, compute the state rehash, we validate block fields and the state rehash against what's reported in the block, and then we write the mutated state to disk.
Jared Wasinger:Block access list execution, parallelizes the transaction execution and the state route calculation. So in this model.
Jared Wasinger:We…
Jared Wasinger:Don't have the advantage of being able to prefetch trinodes in the background, but when we go to commit, we can do everything in… we can commit all of the nodes, or when we go to compute the tri-root hash, and we have to fetch the intermediate tri nodes, we can do this all in parallel.
Jared Wasinger:So, it ends up being not a huge performance hit, …
Jared Wasinger:So when we execute… when we execute transactions and simultaneously calculate the state route.
Jared Wasinger:We provide an empty pre-state to the… to these, like, worker threads.
Jared Wasinger:And then they will, build… they will, …
Jared Wasinger:So they will sort of, like, …
Jared Wasinger:compute the state they need to execute against by looking at the access list. And then some of that information is also supplemented from the…
Jared Wasinger:from disk.
Jared Wasinger:Block excess construction, yeah, like I mentioned, we haven't implemented reads. I was…
Jared Wasinger:kind of under the impression we would remove them. It sounds like we'll talk about this today. For rights, we… it's pretty straightforward when we go… we… we… yeah, it's pretty straightforward.
Jared Wasinger:Alright, so, can you all see my mouse?
Toni Wahrstätter:Yes.
Jared Wasinger:Okay. So… The performance of the master branch is…
Jared Wasinger:the overhead is dominated by the state… by the execution. So these first three columns, these first three rows.
Jared Wasinger:are… if you sum them together, this is the time it takes to perform the execution. So…
Jared Wasinger:So then when we go to update, the…
Jared Wasinger:The account update here, and then the storage update here are fairly quick, because we have pre-warmed trinodes while executing transactions. But the actual transaction execution time is… is large.
Jared Wasinger:Oh, I should also mention, so this… these numbers I'm giving are a benchmark of an import of 10,000 blocks.
Jared Wasinger:It's not a huge…
Jared Wasinger:span of time, and I'm running a longer benchmark now, so I will have some updated numbers soon, but right now, this is what I have to present. So…
Jared Wasinger:Right, so with the BAL branch, on this, on this benchmark, The, … so the performances…
Jared Wasinger:Overall, the block processing speed's about 2.6 times faster.
Jared Wasinger:So the state root calculation here is our… our slowest, but the block execution…
Jared Wasinger:is, very fast. … And then, so these… these… these block preprocessing is… is…
Jared Wasinger:we can kind of ignore that, because that's just… this is a… this branch was a work in progress when I performed these benchmarks, but I think that this step
Jared Wasinger:This step, and this… the preprocessing state loading is actually just a component of the preprocessing step, so it's not, like, an additive…
Jared Wasinger:You don't add all these up to get the total, the total time it took to process the block.
Jared Wasinger:But I think that this step actually can be optimized away, and I'll be working on that at some point over the next week or two.
Jared Wasinger:…
Jared Wasinger:Right, so, I mean, yeah, this is kind of what I have right now, and I think that…
Jared Wasinger:I've been doing some… I've been running a longer benchmark, and there are definitely… …
Jared Wasinger:spikes. So here, the execution is, like, fairly reasonable, but I've definitely seen some spikes where the execution is actually, in general.
Jared Wasinger:on par with the state root calculation, so far as what I'm seeing.
Jared Wasinger:… And yeah, so I will be continuing to benchmark this and try to improve it, …
Jared Wasinger:I think right now, though, from what I've seen with the latest results, I think at least a 2.2X or slow… or 2.2X or so speed up
Jared Wasinger:seems to hold, and … I mean, I'll have to run more benchmarks to actually see what the master branch performs over this longer period of time that I'm measuring, but…
Jared Wasinger:Yeah.
Jared Wasinger:So that's… The… yeah, that's pretty much my update.
Toni Wahrstätter:Awesome, thanks.
Toni Wahrstätter:Just a quick question, in this… when you say execution, and you didn't do any parallel I.O. in this model, right? Because you said you were not using the reads yet, or the state locations.
Jared Wasinger:Well, so the I.O. is parallel because you're…
Jared Wasinger:it's all… they're executing in parallel, so I think, like, a good portion of the speedup is actually due to the parallel I.O, …
Jared Wasinger:But, yeah, it's… it's not executing with the reads, and also, …
Jared Wasinger:I mean, I can… I can definitely measure reads. It… it… it…
Jared Wasinger:it's doable. It kind of involves adding a separate component, but… It can be done.
Toni Wahrstätter:Right.
Toni Wahrstätter:Yeah, this also plays into this, what Kareem is saying in the chat.
Jared Wasinger:items.
Toni Wahrstätter:the state locations, then you can do something like batch parallel I.O,
Toni Wahrstätter:But if you don't have the read locations, then you can still do some parallel I.O, because you already parallelized the transactions.
Jared Wasinger:Yeah, just to answer this question, so I don't…
Jared Wasinger:I don't think we need to do batch…
Jared Wasinger:I don't… I'm not exactly sure what this question is, but I don't think that we need to do batch I.O. up front. I think we can just start executing
Jared Wasinger:Like, we don't need the pre-state completely loaded.
Jared Wasinger:at the start, if that's what this question is getting at, right? We can just start executing, and then load things in as we need them.
Jared Wasinger:…
Jared Wasinger:And with the… yeah, I mean, with the reads also, it's like, it doesn't really make sense… because they're not indexed by the… by where they occur in the block, I think that parallelizing loading them up front would actually be…
Jared Wasinger:I think it would be slower, personally.
Jared Wasinger:….
Toni Wahrstätter:Right. Yeah, this is a good argument. And then the question becomes… If we don't…
Toni Wahrstätter:Put the state locations in there, because they give us some… certain amount of speedup.
Toni Wahrstätter:Are there other reasons why we keep them in?
Toni Wahrstätter:And one of the reasons could be that
Toni Wahrstätter:We just, have more information in the block lab access list.
Toni Wahrstätter:We don't only have the information that
Toni Wahrstätter:changed. We know what changed, and in which way it changed, or to what it changed.
Toni Wahrstätter:And then we will also know, like, what was accessed.
Toni Wahrstätter:Yeah.
Jared Wasinger:Yeah, definitely. I would say, in that case, it would be…
Jared Wasinger:I wonder if it… if there… what the usefulness trade-off is. Like, if there's no… if there's information that storage slots were accessed in a block, but we don't know what transaction they were accessed in…
Jared Wasinger:… I… I'm… I guess I'm just wondering how that would be used.
Jared Wasinger:But….
Po:Okay, so….
Jared Wasinger:But to answer the question, batch load…
Jared Wasinger:Yeah, I mean, I can measure it. I… I…
Jared Wasinger:We already… in the benchmarks I presented, there is actually some batch loading up front of the pre-state that gets modified, and it… it adds
Jared Wasinger:I mean, maybe, like, 10% overhead overall, so….
Toni Wahrstätter:Cool, thank you.
Toni Wahrstätter:Yeah, Chen, did you want to say something, or do you want to, just continue with presenting, the work you guys did?
Po:Yeah, actually, when we are doing the benchmark, and…
Po:We have a… we use some of the… …
Po:account and storage slot, use this information to do… with this information, we can do some kind of, …
Po:This snapshot and, to accelerate the…
Po:block process as a queuing time. I, I don't know if I…
Po:I expressed my opinion correctly, yeah, because….
Toni Wahrstätter:Okay, so you, you say, the state locations would also be helpful for execution.
Po:Yeah, yeah, because, there are two options. One is, like Jared did.
Po:You've fetched the storage and account state during execution.
Po:Yeah, the second option is you'll just prefetch all the pre-blocks state, then executing, yeah. The…
Po:Benefits of the, … Prefetching the state is that you can do some
Po:Snapshot, because we know all the… Free.
Po:post-financial state. This, state, we can, kind of, do multi-layer stated reading.
Po:Yeah, we can snapshot every… about, for example, 10 transactions, and then we can…
Po:make, this snapshot. Yeah, if we want to adjust the state, we could, just, But…
Po:Do a kind of a multi-layer reading, yeah.
Po:And this will accelerate the, state rate, state rate.
Toni Wahrstätter:Okay, is that… is that part of the… of the benchmark as you're going to present?
Po:Yeah, it's part of the result, but I didn't, write… presented the routine on my presentation, yeah.
Toni Wahrstätter:Okay, no worry.
Toni Wahrstätter:Okay, yeah, it would be… would be nice if you could… if you could share, those results with… with this group.
Toni Wahrstätter:would be very interesting to look into those, because I think you also did some analysis on… …
Toni Wahrstätter:what happens if we 10x the block size, right? If we have suddenly, like, 10x the gas limit?
Toni Wahrstätter:How would that impact execution time versus I.O. time?
Toni Wahrstätter:Perfect, I see you posted it into the chat.
Toni Wahrstätter:Great.
Toni Wahrstätter:Yeah, I agree with Carl. I think we should… we should get some better benchmarks on…
Toni Wahrstätter:what do we actually get from… from Batch.io?
Jared Wasinger:Hey, Poe, I have a quick question about this, this, slide you just posted. So the speed up at the bottom, is that, versus master, or what… what is that… what is that based on?
Po:Let me rephrase your question. You mean master is the main branch of gas?
Jared Wasinger:Yeah.
Po:Yeah, yeah, the… Version is, …
Po:Yeah, in the second PPT is 1.16… it's, close to the latest version. The latest version is 1.1716.2, and, my version is 1.16.1, yeah.
Jared Wasinger:Gotcha, okay.
Toni Wahrstätter:Okay, and if I interpret it correctly, it's like the result is also like a 2X speedup, …
Toni Wahrstätter:For average cases, right?
Po:Yeah, right.
Carl Beekhuizen:I mean, I wanna just highlight again that we care about worst case, which is a 33x speedup, supposedly.
Toni Wahrstätter:Right, yeah, that's a good point, because if you use, like, 32 cores, like… As this experiment did.
Toni Wahrstätter:And we would have had, like, worst-case blocks than, …
Toni Wahrstätter:We would have gotten a 32X speedup, this is…
Toni Wahrstätter:But even, even for the average case, like, a 2X sounds already, like, more than I expected. Of course, I expected, like, big, …
Toni Wahrstätter:And performance improvement for worst cases.
Toni Wahrstätter:But, like, 2x4 average cases is already, like, very good.
Toni Wahrstätter:But yeah, it would be interesting to find out, like, if clients actually need the state locations to do something like batch I.O. up front.
Toni Wahrstätter:Or if we remove them, and…
Toni Wahrstätter:In the end, it's not only do clients need them, but do, … Should the protocol have them?
Toni Wahrstätter:Maybe there are some use cases we're not aware of yet.
Toni Wahrstätter:That might need them.
Carl Beekhuizen:And we also need to weigh that off against the…
Carl Beekhuizen:Like, cost and time of sending that over the wire.
Carl Beekhuizen:You know, they're not… they're not free to… to add, or, sorry, to have.
Toni Wahrstätter:Right.
Jared Wasinger:It's also possible that if we can batch the storage lookups
Jared Wasinger:so for red storage and red accounts, if we can do that up front, but also… but kick that off after we've started performing the state root calculation, I mean, I could…
Jared Wasinger:I could see that being an interesting… route to benchmark and Explore.
Carl Beekhuizen:Can you say a little bit more about what you mean by after the stated computation?
Jared Wasinger:So, the… If we… if we batch load
Jared Wasinger:non-mutated storage slots and accounts up front, but we haven't yet, started to compute the state root calculation. We're…
Jared Wasinger:Adding overhead up front and delaying work we could already do.
Jared Wasinger:Because, like, the mutated storage slots and accounts aren't needed to… you don't need to load them to compute the state root.
Carl Beekhuizen:Okay, I'm… Not following here, like, don't we need the post date?
Carl Beekhuizen:Like, that the root of the post state at the end of the block?
Jared Wasinger:Yeah, sure. I thought we were talking about the, like, when we say batch… I… batch…
Jared Wasinger:I.O. up front here at…
Jared Wasinger:Yeah, I'm just… I'm just saying, like, if we… we can, of course, batch the loading of the mutated accounts up front. That's the only, like, precursor… the mutated accounts and the mutated store slots up front, because that's the only precursor for performing the state root update, but then…
Jared Wasinger:… But we wouldn't also need to warm the… …
Jared Wasinger:Any storage slots or accounts that aren't mutated?
Jared Wasinger:Before… before we, in parallel, start the root computation.
Carl Beekhuizen:Yeah, I see.
Jared Wasinger:I'm kind of just thinking aloud here, but….
Carl Beekhuizen:Yeah, yeah.
Carl Beekhuizen:I mean, I think… I think a lot of the thought process comes back to… like, Dunkard did some experiments a while ago, just on, like, batch random reads.
Carl Beekhuizen:I'm also very contrived.
Carl Beekhuizen:Example, but he got something stupid, like a 50x speedup.
Carl Beekhuizen:on… Floating arbitrary.
Carl Beekhuizen:data from an SSD.
Carl Beekhuizen:Just by doing batch reads as opposed to individual reads?
Carl Beekhuizen:So I think that's where, like, a lot of the intuition of, like, oh yeah, it's probably worth just, like, fetching all the state, whether it gets mutated or not.
Carl Beekhuizen:Like, before the execution starts.
Jared Wasinger:Yeah, I mean, I…
Jared Wasinger:I definitely agree that, … I would definitely say that, like, I haven't benchmarked reads. It appears that, …
Jared Wasinger:the work from eStorage and, and Poe, Poe, or Chen, sorry, I, I, it appears that they've….
Po:Okay, you can call… Or, let's say, by…
Po:Feminine name, yeah. Always my English name.
Jared Wasinger:Okay, okay. … Yeah, I mean, I…
Jared Wasinger:It's definitely worth exploring, yeah, for sure.
Jared Wasinger:I'd like to benchmark it, and I… and I intend to.
Carl Beekhuizen:Okay, fantastic.
Carl Beekhuizen:Yeah, I think that's… that there's, like, so much here that it's like, oh yeah, there could be one trade-off here, or one trade-off in another way, and it's like, benchmarks, I think, are the only way of answering these questions.
Toni Wahrstätter:Which… which exact benchmarks are you thinking of here, Charit? So, which benchmarks would you… because this should be something, like, very sim… very simple to do, right?
Jared Wasinger:Yeah, so I would just implement the…
Jared Wasinger:I would implement reads, and then I would…
Jared Wasinger:I would benchmark them against Mainnet as a start.
Jared Wasinger:And…
Jared Wasinger:Yeah, I mean, just batch loading the red storage slots up front, and see what kind of performance… see what the results are.
Jared Wasinger:I mean, yeah, like I said, there's some interplay with, like, the state root computation, so implementing it is…
Jared Wasinger:I mean, it's not very complex, but…
Jared Wasinger:There's definitely… yeah, I mean, it, it, …
Jared Wasinger:Yeah, just benchmarking against mainnet and seeing if…
Jared Wasinger:We can fiddle around with the implementation to… To… yield performance gains?
Karim T.:I think it will also depend on the… who we implement the…
Karim T.:the block processing, because I said if we want also to compute the stage route from the beginning, we'll have also to batch…
Karim T.:Or try to read the intermediate node of the tree in order to compute the state root.
Karim T.:If you have also two batches the read.
Karim T.:So we'll have a lot of things to read at the beginning of the block. I feel that
Karim T.:It would be maybe better if we can mix…
Karim T.:between I.O. and CPU, so as I said, if we are running all of the transactions at the same time, sometimes we have I.O, and during I.O, we can run something else, any VML code or something like that, but if we do all of the batch at the beginning, we just wait at the beginning of the block to finish all of the I.O,
Karim T.:And, I don't know if it will be better or not just asking like that.
Karim T.:And thinking about that.
Toni Wahrstätter:Yeah, I think it's a fair comment. I think we only find it out if we do some benchmarks and…
Toni Wahrstätter:And then we see, in the end, it's, it's, …
Toni Wahrstätter:It's, like, the absolute time it takes to execute the whole block. Doesn't matter, if we do it up front or during execution.
Toni Wahrstätter:So yeah, would be… I think we just need some benchmarks at this point.
Toni Wahrstätter:to figure it out. Would be great if we can even get them from multiple clients to just have some more certainty on… on that decision.
Po:Yeah, extorted.
Po:Yeah.
Po:All the… generate the sales, the…
Po:Duh.
Po:Sorry, That's why I….
Toni Wahrstätter:I, I think, I think we can't really hear you, Paul.
Toni Wahrstätter:I think your mic has a problem.
Po:Okay, so is it… is it okay now?
Toni Wahrstätter:Try it again?
Po:Okay, so I think I will have to… Transfer Mac.
Toni Wahrstätter:No worry. Perfect.
Toni Wahrstätter:… right.
Toni Wahrstätter:Yeah.
Toni Wahrstätter:I think we can… we can end that topic, regarding REITs, and at this point, I think at this point, we agree we… we need to… we need some more data.
Toni Wahrstätter:To figure it out.
Toni Wahrstätter:So I would say let's move on to the next agenda point, which is Rahul, telling us something about the testing efforts going on, and also Felipe is on the call. Would be great to hear his perspective on updates from him, too.
raxhvl:Hey, so I'm gonna quickly go over some updates on the testing front.
raxhvl:Give you some updates on the progress, give you a brief overview of how the test works on the record.
raxhvl:And then, a couple of housekeeping items.
raxhvl:So, …
raxhvl:The test mainly depends on two repositories. One is EAST, which is like a Python test testing framework.
raxhvl:And we have EALS, which is, like, a Python implementation of the spec.
raxhvl:We have both of tools working now.
raxhvl:We're currently trying a basic, set of tests, about 5 test cases.
raxhvl:And reviewing how that, It's been executed.
raxhvl:Once we are confident with that, we'll do a small release where, as clients, you guys can just start integrating
raxhvl:These tests as part of your development cycle.
raxhvl:And then, we'll keep on, working on more complex tests later.
raxhvl:Yeah, so Felipe is, like, doing, … Doing the integration testing now.
raxhvl:He's on the call to, like, give you an overview of how the test works under the hood.
raxhvl:Over to you, Chandita.
felipe:Yeah, sure. So right now there's…
felipe:As Rahul mentioned, there's, there's the spec side.
felipe:Written in Python and the testing side as well. So, what I did, was some work trying to bridge, Tony's PR on the spec side with Rahul's on the testing side.
felipe:And, we got, got it working through…
felipe:There were just some changes to your PR, Tony, to just pass things through the T8N tool, the transition tool that's talking across the testing and the specs.
felipe:And so, basically I got this working, we do have,
felipe:some of the basic tests Rahul is working on.
felipe:Right now, I'm figuring out how to make the developer experience a little better for adding some complex test cases and some invalid checks.
felipe:But as it's working, you know, this is, like, a general flow of East here. Transactions are sent over to the specs to fill.
felipe:these fixtures… eels is the specs in this case. They receive the block, they will…
felipe:build the block access list, and they pass it back. So we… what we pass back to the testing right now is the hash and the header.
felipe:and the ROP encoded, … Block access list from the specs.
felipe:What we do on the testing side is we define some smaller subset of the block access list that we care about for the test.
felipe:Some of these tests will have the full subset, right? Or it'll be, like, a full check on the valve that's returned.
felipe:But sometimes we only care about certain things, and so we do perform this check, but we also rehash the full, access list and check it against the hash in the header, so we always have this double check.
felipe:On the testing side.
felipe:And, yeah, Rahul, is there one more slide?
felipe:Yes, perfect.
felipe:And so…
felipe:Here, this is basically, like, once this fixture is filled with the specs, and this is a common flow for the testing as well.
felipe:This can be consumed by clients over…
felipe:consume engine, and so this just kind of shows, the flow there as well. But that's… those are the updates on my side.
felipe:I do… I do actually have, initial implementation of the… of this, like, more complex, model structure, to start working on the invalid tests, and… and I did…
felipe:Write a couple invalid tests yesterday, but that's still a work in progress, but it does seem to be working well at the moment.
raxhvl:Thanks, Felipe.
raxhvl:Another thing that we noticed was, there are…
raxhvl:There are some creative tests, that… that comes from that.
raxhvl:But they're often lost in Discord messages and GitHub issues.
raxhvl:So we're trying to aggregate that, as part of the report itself.
raxhvl:Into a Markdown file. This is, like, a structured Markdown file, which…
raxhvl:Goes over, the goal of the test, the setup that is required, and the assertion that we're trying to make.
raxhvl:So, if you guys have any, any test cases in mind, I'd suggest.
raxhvl:Making a PR to this, this, background files.
raxhvl:And, we can probably… Use that to make discussions, give focus updates, and for other high-level
raxhvl:Activity on the test cases.
raxhvl:…
raxhvl:We also are using this Markdown file along with the test case to integrate into Hive, which is like an integration framework which talks to all the clients.
raxhvl:… To create, like, an adoption dashboard.
raxhvl:This is what it looks like.
raxhvl:So, this pulls in the data from the markdown file in a way that's easier to read.
raxhvl:… for people to understand what's happening under the hood.
raxhvl:It also fills in the Python test against each client using pipes.
raxhvl:And then shows an update, on to our matrix here.
raxhvl:This runs on a CI server, …
raxhvl:And this is how it looks. You can tap on a test to see what exactly it's testing.
raxhvl:And… yeah.
raxhvl:Obviously, there's a link to… to the background file for you guys to create a PR.
raxhvl:Yes.
raxhvl:That's all from us, thank you.
Toni Wahrstätter:Awesome.
Toni Wahrstätter:Thank you very much.
Toni Wahrstätter:Both of you.
Toni Wahrstätter:Yeah, I posted, I posted your Markdown file, the link to it, in the chat.
Toni Wahrstätter:Would be great if client devs, every time they come up with a new edge case.
Toni Wahrstätter:Cannot do a PR against it.
Toni Wahrstätter:We're already in the Discord I saw yesterday, I think, …
Toni Wahrstätter:Can't find the message right now, but I think we already stumbled across a few edge cases where certain clients,
Toni Wahrstätter:would have put that, certain address into the block lab access list, while others wouldn't have.
Toni Wahrstätter:And in the end, I think we just need a lot of those test cases, too.
Toni Wahrstätter:Yeah.
Toni Wahrstätter:Get to a clean implementation.
Toni Wahrstätter:Perfect, thank you.
Toni Wahrstätter:Right, finally, I just wanted to ask clients about progress. It would be nice if client teams could just quickly give a little bit of an update where they are, what they think, about the EAP, and if there are any blockers or anything where we could help.
Toni Wahrstätter:Maybe we can start with… with you, Jared.
Jared Wasinger:Yeah, so… we actually have our first, review call on the PR tomorrow.
Jared Wasinger:So… Hmm.
Jared Wasinger:Yeah, from my side, not much right now, but maybe that will change after tomorrow, so….
Toni Wahrstätter:Cool, thanks.
Toni Wahrstätter:Is there someone from Besu here?
Karim T.:Yes, but maybe, … You can talk, Marieu.
mirgee:Yeah, sure, so… Amen.
mirgee:So in this event, we, we have implemented a degree of protocol support for BLs.
mirgee:… …
mirgee:But we still, we still need to update the engineering PI, and it's possible that we haven't, …
mirgee:incorporated, like, the spec changes. The PR has been open for, for
mirgee:What about the month with little changes?
mirgee:And it's possible that we are mishandling some of the edge cases.
mirgee:For example, I'm aware that We are omitting the…
mirgee:system contracts from BLs completely for the moment.
mirgee:… we've been testing against the bowel analysis, report and getting good results there.
mirgee:Was that worth?
mirgee:Yeah, but, … the PR has been blocked on…
mirgee:Some issues, but, otherwise, otherwise… …
mirgee:That's… that's… I think that's… that's our status.
Toni Wahrstätter:Awesome, thank you.
Toni Wahrstätter:Is there an update from someone at Aragon?
Mark Holt:Yeah, hi, it's Mark Holt here. I'm doing the BAL implementation at Aragon.
Mark Holt:Status is, I'm just starting, so…
Mark Holt:I'm… probably by the end of this week, I'll have an estimate of,
Mark Holt:how long it'll take to get to something that's working.
Mark Holt:I think we're just in the middle of merging. We've got a… we've got a parallel execution engine that we've been working on for some time, which is just going into main.
Mark Holt:So, I think the work to support, BAL is going to be just integrating the engine API and making sure that the data structures that we've got
Mark Holt:… essentially a mappable.
Mark Holt:Which, looking at the spec, I think is a relatively straightforward process.
Mark Holt:So, I mean, if it goes well, probably a couple of weeks will be in-dev testing, I would guess. If it goes badly, probably 3 or 4 weeks.
Mark Holt:I think is where we are.
Toni Wahrstätter:Thank you.
Toni Wahrstätter:Yeah, just to directly address, Karim's comment, about, that it would be nice to have a configuration ready for the first DevNet, especially decide on things like, do we want to put reads in there or not?
Toni Wahrstätter:I would say right now the ERP is still, like, with reads, so I would still stick to having the reads included for now.
Toni Wahrstätter:But, yeah, we can still, as said, as soon as we have some benchmark… benchmarks, we can take a final decision.
Toni Wahrstätter:I hope that we get the benchmarks before we get closer to a definite, so that we have the benchmarks rather soon, so that we can decide.
Toni Wahrstätter:on that earlier.
Toni Wahrstätter:Yeah, thank you, Mark. Is there anyone from REF here?
Soubhik:Hello, we're currently…
Soubhik:working on the engine API changes. Just today, we have done the new version for payload structure.
Toni Wahrstätter:Awesome, thank you.
Toni Wahrstätter:Did you… did you start with the engine API, or which… of which part did you start there?
Soubhik:No, no. We have implemented the reads and write storage part already. We are…
Soubhik:We're currently recording both the reads and writes.
Toni Wahrstätter:Thanks.
Toni Wahrstätter:Do we have someone from Nethermind?
Marc:Yeah, hello. So, I've started prototyping, block access lists in Nethermind.
Marc:So far, I've been focusing on the, sort of, construction part, which has progressed pretty well, so the, sort of, construction and serialization.
Marc:And the way I decided to sort of approach it is to split it into two PRs, so there's the one which is, like, the construction, and then the other part is actually using the block access list, so all of the kind of parallelization aspect.
Marc:And I haven't really, …
Marc:worked on that part of it as much, and it seems like it will be significantly more, kind of complex.
Marc:But…
Marc:the way I kind of understand it from the spec, it does seem quite logical to split it into those parts, because you could almost just…
Marc:Do the construction without necessarily leveraging everything, so that does seem to give us a little bit of, …
Marc:kind of flexibility there in terms of scheduling. I'm not sure if anyone else has approached it that way. …
Marc:But yeah, that's it for now.
Toni Wahrstätter:Great, thank you.
Toni Wahrstätter:Yeah, if there's nothing else that we want to discuss today, …
Toni Wahrstätter:we can end the call. This was basically everything we had on the agenda. I note that we still need to do the benchmarking to decide upon
Toni Wahrstätter:Do we want to include the reads or not? Or not the reads, but the state locations? This is kind of…
Toni Wahrstätter:The same here.
Toni Wahrstätter:And, yeah, and then we can discuss about the first definite. I think GEF and Biso might be the first clients that…
Toni Wahrstätter:Might be ready.
Toni Wahrstätter:But yeah, as the EAP is not that big, would be cool to get a Defnet, to get to a DEFNET as soon as possible.
Jared Wasinger:So…
Jared Wasinger:Yeah, I guess, yeah, based on Carl's comment earlier, I personally kind of lean towards keeping reads in until we can demonstrate worst-case performance, which isn't…
Jared Wasinger:trivial, so we… it would be nice to keep those in until we can set up some kind of, like, attack net where we spin up a bunch of these, like.
Jared Wasinger:Worst case, blocks without reads.
Jared Wasinger:Sorry, that… I just realized that didn't make sense. I want to evaluate, like, what the worst-case performance is without having the reads before we decide whether or not to remove them.
Jared Wasinger:So, yeah.
Jared Wasinger:At least, yeah.
Toni Wahrstätter:Yep, that's fair.
Toni Wahrstätter:Perfect. Yeah, the ERP as it stands today, still includes the reads, so I think we just leave it as, as that, and…
Toni Wahrstätter:then we can still decide at some point if we want to remove them or not, but for… as a default, I would suggest we keep them for now.
Toni Wahrstätter:Perfect.
Toni Wahrstätter:Let's see….
Marc:Just….
Toni Wahrstätter:Your hand, yeah.
Marc:Yeah, I just had one quick question. So, I saw on the agenda something about RLP versus SSZ. I've gone with implementing RLP, but… and I kind of… it seemed like last ACD
Marc:that was agreed upon, but I just want to check, are we kind of all in agreement that we're going to use that?
Toni Wahrstätter:Yeah, so just to summarize on… this was on all Cardiff's call, I think, last week, the majority… I think most people were…
Toni Wahrstätter:So most people didn't have a strong opinion.
Toni Wahrstätter:But it seemed like the… the less complex way would be to just go with RLP and not use the ERP to introduce a new…
Toni Wahrstätter:Yeah.
Toni Wahrstätter:to introduce, SSC.
Toni Wahrstätter:This is kinda, also the default that is now in the EAP. I think we have some broad consensus on that.
Toni Wahrstätter:But I'm happy to discuss that, if anyone else wants to.
Toni Wahrstätter:Yeah, defend SSC.
Marc:Yeah, I mean, I think ROP makes sense to me.
Toni Wahrstätter:Yeah, there were some pros and cons for both, like, SSC with smaller, more compact proofs.
Toni Wahrstätter:might have some, advantages there too, but RLP, in the end, it's smaller, so we save, like, 10%, 5-10%, …
Toni Wahrstätter:on the size, which is negligible, but still, and it's just known, so all clients have, codebases with RLP, and it, like…
Toni Wahrstätter:That's complex.
Toni Wahrstätter:Perfect.
Toni Wahrstätter:Boom.
Toni Wahrstätter:Thank you very much, and see you in two weeks at the next breakout call.
Marc:It is, but….
Jared Wasinger:Yeah, buddy.
Chat Logs
00:04:46
Toni Wahrstätter:blockaccesslist.xyz
00:06:27
Karim T.:Do we need read, this is the question
00:08:50
Barnabas:mic dao
00:09:52
Karim T.:I think if we can run transaction in // ,we will have // io, just not batch io at the beginning
00:17:11
Karim T.:I think the same, I ‘m saying that we don’t need read because batching io seems useless as we can run transaction in //
00:18:16
Carl Beekhuizen:I think we need benchmarks on whether the batch load is faster or not in the worst case.
00:21:56
Po:https://docs.google.com/presentation/d/1dfdHgBoHl-xwO7eOEYfkRNgs5fWlZRYda1GbYTCaAKY/edit?slide=id.g382ef0951fb_0_26#slide=id.g382ef0951fb_0_26
00:27:12
Karim T.:We need also to batch io the intermediate node of the trie for the state root
00:27:50
Karim T.:So it will be too much io at the beginning if we batch also the read at the beginning
00:28:12
Karim T.:So waiting a lot of time to just batch at the beginning of the block
00:32:39
Karim T.:I don’t hear well
00:33:13
Barnabas:Replying to "mic dao"
mic dao x2
00:33:16
Po:I'll log out and change my mic
00:36:42
Carl Beekhuizen:Oner more concern on reads: if we read as we execute, the worst case block becomes one where a ton of reads happen at the end of a thread’s execution and we suddenly become blocked on one tx’s reads.
I’d guess that on average read-while-executing if faster, but not for the worst case (and I think that we might need a maliciously constructed block to see this, it might not be clear using today’s blocks as a benchmark).
00:37:14
Po:yeah
00:37:34
Po:for the worse case, it's better to prefetch
00:38:46
Carl Beekhuizen:Replying to "for the worse case, ..."
And I’m going to keep beating the-only-thing-we-care-about-is-worst-case drum
00:38:53
Toni Wahrstätter:link for the markdown file:
https://github.com/ethereum/execution-spec-tests/blob/feat/eip-7928/checklist/tests/unscheduled/eip7928_block_level_access_lists/checklist.md
00:41:19
raxhvl:https://pokebal.raxhvl.com/ - The tracker
00:41:22
Barnabas:wen devnet 0?
00:41:46
Barnabas:fork to be activated with: AmsterdamTime
00:42:27
Barnabas:Could we have clients implement bal-devnet-0 branches, and we can setup auto build/deploy.
00:43:22
felipe:@Toni Wahrstätter I've built my EELS PR on top of all of your commits. If we can make sure you can push to my repo, are you good with making this the PR we focus on for the specs implementation?
00:43:43
Toni Wahrstätter:Replying to "@Toni Wahrstätter I'..."
yeah will do that!
00:44:33
felipe:I have to pop out for another meeting. Will catch up on the rest via recording.
00:45:03
Karim T.:It will be nice to have a link with the configuration needed for the devnet and BAL format we will use (with or without read) , etc
00:47:28
Jared Wasinger:I think also that reads vs no reads depends on attack cases as well. And it’s hard to express these in standalone benchmarks because the IO perf depends on the size of the database.
00:51:34
Barnabas:ssz warriors have not shown up
00:52:16
Barnabas:can we aim for bal-devnet-0 in 3 weeks?
00:52:30
Carl Beekhuizen:Thanks all
Summary
12 highlights
· 3 action itemsExperimental
Summary
12 highlights · 3 action itemsExperimentalimplementation progress
- Geth: Full EIP-7928 implementation complete except reads; ready for devnet00:05:37
- Geth benchmarks show 2.6x average speedup, state root calc remains slowest00:06:54
- QuarkChain benchmarks show 2x average, 33x worst-case speedup with 32 cores00:21:56
- Besu: Protocol support complete, engine API and spec updates pending00:42:54
- Erigon: 2-4 weeks to devnet readiness with parallel execution engine00:44:11
- Reth: Engine API changes in progress, reads/writes already recording00:45:59
- Nethermind: Construction/serialization prototyped, parallelization more complex00:47:13