Toni Wahrstätter:Great! Welcome, everyone. Today is December the 3rd, EAP7928 breakout call number 8.
Transcript
Toni Wahrstätter:Let's get started. I just posted the agenda into the chat. Today, it's very lightweight, I would say. We have 3 topics to discuss, starting with one change in the specs and the EAP, more a clarification around how to treat, calls that send value but run out of gas.
Toni Wahrstätter:The second one is storing the block of access list in the block body versus not storing them.
Toni Wahrstätter:We discussed it already last time, but wanted to double-check with everyone.
Toni Wahrstätter:And finally, going through, optimizations needed for the benchmarks we wanted to get in order to determine if the read values are worth it to include in the block lab access list or not.
Toni Wahrstätter:Perfect, and directly… let's directly jump into the first topic, which came up as there were some differences in how client implemented
Toni Wahrstätter:The gas accounting when it comes to coal.
Toni Wahrstätter:to a recall opcode.
Toni Wahrstätter:And basically, it…
Toni Wahrstätter:Depends on when do we check, when do we do the static checks, versus when do we actually go to the
Toni Wahrstätter:state.
Toni Wahrstätter:And… I will post this, PR into our chat.
Toni Wahrstätter:One second… There it is.
Toni Wahrstätter:And I'm curious what clients think about it. So right now.
Toni Wahrstätter:What we do is we, in the specs, we now make sure that
Toni Wahrstätter:We first check all the static checks, which is memory expansion, access costs, and the transfer costs, and only if they succeed, we go to disk. And the tests showed, like, differences between
Toni Wahrstätter:I think it was between, Beso and REF versus Geth. I think Geth did something different there.
Toni Wahrstätter:And I wanted to double-check with, clients, if this is something we can agree on, how it's now defined in the EAP.
Toni Wahrstätter:I think it only needed changes for GAF, right?
Toni Wahrstätter:Is Jared on the call? Maybe Jared,
Toni Wahrstätter:Do you have some insights there, or is this something you guys are about to change?
Jared Wasinger:I just joined, what, what, what are you talking about here?
Toni Wahrstätter:Yeah, it was the topic with the call, so basically, checking all the static gas costs around, memory expansion, access cost and transfer cost before actually doing the state access.
Jared Wasinger:Yeah, actually, I'm working on that right now, and I'm hoping I can get the test coverage around that, fixed pretty soon.
Jared Wasinger:In fact, I think that there's a possibility that that might be the cause of the chain split that we see when we try to run local DevNets between Geth and Bisu, so,
Jared Wasinger:hopefully fixing this also, makes the devnets more stable. But yeah, I'll have an update on that, soon. That's, like, my top priority right now.
Toni Wahrstätter:Awesome, thanks a lot. I'm just relaying Stefan's message here in the chat. For memory expansion, Nethermind also has an inconsistency versus Ref and Besu.
Toni Wahrstätter:Someone from NetherMind who is already aware of the topic, And Nicole?
Marc:Yeah, sorry, what was the question, sorry?
Toni Wahrstätter:So it's basically, in the specs so far, it didn't matter that much when you do the state access versus when you do the static checks for memory expansion, access cost, and the transfer cost.
Toni Wahrstätter:And what came up during the testing was that, for example, if you have a call that forwards one EV,
Toni Wahrstätter:But the account only has… 0.1 EV,
Toni Wahrstätter:Then the question is, should the target address that is about to be called, should it be included in the block level access list or not?
Toni Wahrstätter:And I think Beso and Rev did not include it, which seemed right, because you can do all the static checks before actually going to disk and actually accessing this account.
Toni Wahrstätter:So, it's… So it was probably NetherMind and Kef that
Toni Wahrstätter:Had this state access before doing all the static checks.
Marc:Right, yeah, I mean, I don't have…
Marc:any opinion on it, really? Like, I'm happy to change to what Bessu and Wrath are doing.
Toni Wahrstätter:Okay, great.
Toni Wahrstätter:Maybe, Belipe, as you're also on the call, is there anything you would add to this topic? Because I…
Toni Wahrstätter:We have been…
Toni Wahrstätter:Going back and forth in the specs regarding that, so is there anything you'd like to bring up?
felipe:to make sure.
Toni Wahrstätter:We don't forget.
felipe:No, no, I think we covered it. I am a little curious, though, because… We do have…
felipe:This is a test, and another mind is… Passing all of them.
felipe:So I'm not sure… Stefan, where they have the inconsistency.
Stefan Starflinger:That's for memory expansion. I'm not sure if I shared the right link.
Stefan Starflinger:When there are different times of, gas accounting, and never mind,
Stefan Starflinger:reads the target. I think it's the target. I'm not 100% sure, though, what exactly NetherMind reads before doing the memory expansion gas accounting. The other gas accounting seem to be in line with Bazel and REST, just the memory expansion seems to be a little bit delayed.
felipe:Yes. Okay, yeah, you didn't share the right link. This is the PR that you added a test for, and I merged.
felipe:I'm pretty sure.
felipe:Yes, exactly.
Stefan Starflinger:This is… this should be the right order.
felipe:You know…
Stefan Starflinger:So this is just an inconsistency I found with memory expansion.
Stefan Starflinger:I think that's the last thing so far, and then Netherlands should be, aligned. I think you just merged this recently, so it might not be yet in the tests, because I think you have to make a release
Stefan Starflinger:I'm not sure if you've done that.
felipe:Perfect, yeah, no, I thought we were talking about the, the state access. I remember this one. This is not yet in a release.
felipe:We're waiting for… One more PR…
felipe:And then we're… I think we're also going to be able to turn on all the old Ethereum tests with BALS, and so…
felipe:I'm trying to iron out some of the inconsistencies there.
felipe:And then we can get a release out with even more tests.
felipe:That we can try to get consensus with all clients on.
Toni Wahrstätter:Awesome, this is great to hear. So, it looks like the specs are correct, and the test should also be correct in this case.
Toni Wahrstätter:Because then we are basically ready to test all the clients and make sure, we are all aligned.
Toni Wahrstätter:Awesome. Is there anything else regarding this topic that we should discuss? It's a very interesting topic, by the way, because it's… it's nothing that broke consensus so far, but can break consensus in the future, because suddenly, like, when do we access state will become part of consensus.
Toni Wahrstätter:Is there anything people would like to add to this topic?
Ben Adams:Yeah, I think it makes sense, to do the checks.
Ben Adams:To bring the checks that might cause it to go out of gas.
Ben Adams:Before the state access, because if you're gonna go out of gas.
Ben Adams:You know, don't bother accessing the state.
Ben Adams:It makes sense from a performance standpoint.
Toni Wahrstätter:Yeah. Thanks a lot for that, yeah, I agree.
Toni Wahrstätter:Also, a lot of thumbs up here from many people, so… seems like we have a very broad agreement here, regarding this topic, which is great.
Toni Wahrstätter:Okay, so then I would suggest we move on to the second agenda item.
Toni Wahrstätter:Which is, summarized as storing the block cover access list in the block body.
Toni Wahrstätter:Versus not storing them in the block body, plus defining a retention period.
Toni Wahrstätter:And there is this comment from Lukash, if you scroll down at the agenda.
Toni Wahrstätter:Lukash made a very nice, summary of the pros and cons that each option brings.
Toni Wahrstätter:Maybe, Lukash, if you want, do you want to quickly summarize your thoughts there?
Łukasz Rozmej:Yeah, so this is basically simplicity versus…
Łukasz Rozmej:UX, probably, simplicity of implementation, and of everything.
Łukasz Rozmej:So, if we keep the balls in block bodies, we risk… so, we will be generally increasing the gas limit, right? So, we will be increasing block bodies already quite, quite a lot.
Łukasz Rozmej:And if we… if we store balls there, we probably risk increasing it at least 50% more,
Łukasz Rozmej:Then… than what they would already weigh, and…
Łukasz Rozmej:I'm not sure if I trust this number, 50%, you can, Tony, chime in, later about it. I'm, like, my gut feeling is it will be even more than 50%, but…
Łukasz Rozmej:that's the number you provided, so I'm… I'm not sure if I trust it, to be honest, but
Łukasz Rozmej:That's the problem. Another thing that you basically, for those blocks, keep kind of archived data in the block history, so…
Łukasz Rozmej:That seems wasteful for me.
Łukasz Rozmej:And…
Łukasz Rozmej:If you have an archive node, you probably still need to re-index this data in a different way, so again, this is quite useful for you, so you blow up doubt even more.
Łukasz Rozmej:Okay, good point, that balls can be then used to quickly, sync archive node later, so that's a point, to just index this data.
Łukasz Rozmej:So that's the pros of kicking balls. So, basically, you get…
Łukasz Rozmej:Kind of the best of the both worlds, so pretty fast, tip.
Łukasz Rozmej:processing.
Łukasz Rozmej:But lower resource footprint.
Łukasz Rozmej:But it's… if we do that, it introduces a lot of potential issues, because…
Łukasz Rozmej:Things like, do we actually want to sing those balls at all through DevP2P?
Łukasz Rozmej:If yes, then we need to create some kind of sub-protocol for it. If not, we can just ignore it.
Łukasz Rozmej:What if we don't have balls? We need to have a fallback mechanism of processing blocks if we don't have a ball.
Łukasz Rozmej:Which is fine, because we can… we can verify the… ball hash by generating it.
Łukasz Rozmej:While processing, but it makes things more complicated, right?
Łukasz Rozmej:What else? Potentially, it reduces the speed of doing things like catching up to the head of the chain, or…
Łukasz Rozmej:Darpc, for example, right?
Łukasz Rozmej:So, there are a lot of drawbacks,
Łukasz Rozmej:that we… that this causes. I'm not sure how balls are currently distributed on, on the P2P side, but if they are in the block bodies, they are probably just automatically distributed with blocks.
Łukasz Rozmej:If they weren't in the block bodies, it would have to come as a sidecar.
Łukasz Rozmej:Which could have potential good and bad, well, bad, it's, again, more complicated, but potentially, maybe it's easier to distribute two things that are…
Łukasz Rozmej:Still relatively heavily, both body and ball, rather than just one thing that is heavier, so it might actually, in the end, potentially reduce latency.
Łukasz Rozmej:On the P2P side, so… yeah, there's a lot of things to consider, and in my opinion, it should be, like, a conscious decision. To be honest, I was under the impression that we won't do balls in the block bodies, because that was the…
Łukasz Rozmej:like, when initially discussed, and I didn't, like, that, that was, like, idea I proposed.
Łukasz Rozmej:And then I didn't, like, pay that much attention, sorry guys, to this, because Mark is leading the charge here on the Nethermind… championing the CAP on the Nevermind front, so I recently read it and thought, okay, that…
Łukasz Rozmej:I'm not sure this is the best decision. It might be, but I think it needs to be a very conscious decision, what we want to do. So, just throwing this out, are we sure we want to do that?
Łukasz Rozmej:Because it's, like, we're increasing the history footprint probably just through increasing the gas limit, let's say, by…
Łukasz Rozmej:3 to 4 times? 5 times, maybe? And then if we add balls to it, we can increase it instead of 4 or 5 times to more around even 10 times. So… do we really want to do it that big?
Łukasz Rozmej:That's my question.
Toni Wahrstätter:Yeah, thanks a lot for this summary.
Toni Wahrstätter:I remember, like, this has been a topic in Berlin, and right after Berlin.
Toni Wahrstätter:And as you already said, like, one of the strongest arguments for keeping the blocker access list in the block party was the simplicity.
Toni Wahrstätter:So, I remember, like, this was, by the time, the argument, why we ended up putting the ball into the block body, but I can totally see your arguments, and yeah, would be curious if other clients have an opinion on this already today.
mark:So, I think from an archive perspective, we're probably gonna just drop the bowels from storage.
mark:And the reason for that is we can reconstitute them.
mark:Because, effectively, we've already got the access history. We record, you know, we record that anyway, so if we need to reproduce a bowel, it's quite likely we'll just reconstitute it from our…
mark:From our access history.
mark:Because otherwise, we'll store it twice.
Łukasz Rozmej:Will that be on sync?
Łukasz Rozmej:Will you reconstitute it on Saint?
mark:No, we'll re… well, so here's the question is, is there going to be an RPC call that's basically getting the bow?
Łukasz Rozmej:Yeah, give me block body. Like, on the DevP2P.
mark:Yeah.
mark:Yeah, but that's DevP2P, not RPC.
mark:Is there an RPC call?
mark:Are people going to want to access the…
mark:block bodies historically, via the RPC.
Łukasz Rozmej:I think there is a… but I think it's a debug feature to get RLP off the block, and BAL should be there, right? But that's, like, not a normal thing to…
mark:Yeah, yeah, well, I'm just saying, that's… from our perspective, that's what we'd be thinking about, is there's a trade-off between storing them and reconstituting them.
mark:Because we'll have the information to reconstitute them.
mark:And… if…
mark:Because re-execution's actually quite cheap, right? Because we do it all the time for our PCs. So we've got this general trade-off that…
mark:If it doubles the disk storage, we'll probably drop them.
mark:for example.
mark:Because once you're keeping them historically, that's quite… that's quite a big disk budget.
Łukasz Rozmej:Yeah, exactly.
Łukasz Rozmej:Yeah, I think it's not doubling it.
Toni Wahrstätter:I think it looks like it's adding, like, 50% on top, but not 100%.
Łukasz Rozmej:But, yeah, this might not matter in the internet.
Łukasz Rozmej:This is analysis with a current gas limit, right?
Łukasz Rozmej:Ehh.
Toni Wahrstätter:No, this was already worth $45 million, so I remember, like, back then, at the very.
Łukasz Rozmej:At the beginning, we analyzed the.
Toni Wahrstätter:30 million, and there it goes 50%.
Łukasz Rozmej:Tony, I don't really care. If we are 10xing the gas limit to around 300 million, that's… analysis is, in my opinion, worthless, because the gas usage pattern might completely change.
Toni Wahrstätter:Right, but the ball size should scale sublinearly. So the ball, because it's very well aggregated, it will never scale, super linearly. So if it was 50% extra at 30 million, it cannot become 100% at 300 million. It can only get you, like, 40% on top of it.
mark:Yeah, so, just to be clear, that's 40% over the top of the cost of the block with all of its transactions in it.
mark:Yeah.
Toni Wahrstätter:Yes, and I will actually have to redo this analysis, because I said in the chat, I posted the most recent analysis in the chat.
Toni Wahrstätter:But…
mark:As you can see there, the bow size is, like.
Toni Wahrstätter:56… Kilobytes compressed, that's the median.
Toni Wahrstätter:This was not yet at 60 million, so this was still at 45 million.
Toni Wahrstätter:and… I just compared it with this chart that you get when you Google, like, eFroscan block size.
Toni Wahrstätter:And the block size there, which only considered the EL payload, was… is currently at around 120.
Toni Wahrstätter:plus kilobytes?
Toni Wahrstätter:So that's where the 50% are from. But I will definitely double-check that and provide, like, clear numbers so that we know, is it, like, closer to 50%, 100%? But right now, I would assume it's closer to 50%.
Toni Wahrstätter:But yeah, I'm not even sure if this changes something in this discussion here, because in the end, it's still, like, significant data.
Toni Wahrstätter:So yeah, curious, if we should go for the simple approach, and… Just,
Toni Wahrstätter:Keep them in the block party.
Toni Wahrstätter:And then drop them, and be able to recompute them.
draganrakita:Just to add here, I talked with the red team. Basically, we are supporting from dropping the ball from the block.
draganrakita:And for us, it is simpler solution for the ball to not be included in the block, not all the way around.
draganrakita:And, yeah, I'm echoing the thing about the database size and complexity on the P2P.
draganrakita:So…
mark:From red point of view, dropping ball from the block makes a lot of sense, and it's something that we would like to see.
mark:I mean, it is a trade-off for syncing, though, so… You know.
mark:Because if you're… if you're a syncing node.
mark:you probably want the bowels on DevP2P.
draganrakita:That's true.
mark:I'm… yeah.
mark:I'm just pointing out that from our perspective, we've effectively got the same data stored.
mark:Because we store… Accesses per transaction in our database.
mark:So…
mark:We just physically won't store the bowel, we'll reproduce it, because we've got an optimized version of it already.
mark:which I'm… I'm point… I'm not… I think we're relatively neutral for that reason, is basically what I'm saying.
mark:I think, you know, at an implementation level, we probably won't physically store the bowel in our database, but we will be able to reconstitute it if somebody asks for it.
mark:And I would expect anybody who's syncing will want the bow, because it'll speed syncing up.
Karim T.:I see a real benefit to have a block access list also during a sync, because, as I said in the chat, if you don't have to replay a block, you don't…
Karim T.:have to, to get all of the EVM4 version, so I think it's real benefit. We will not have bulk from the Genesis, but…
Karim T.:It's, it's still a good thing to not have to replay all of the block from the ball fork.
Łukasz Rozmej:Okay, guys, but what kind of syncing are we talking about? Because if we are getting new payloads, we will be getting the balls with them, right? But do we need them on the FP2P?
mark:Yeah, I think it's if you're doing dev P2P recovery.
mark:Is what I'm talking about.
mark:If you're a… if you're doing a backward download via DevP2P.
mark:It's quite useful to have the bowels.
Łukasz Rozmej:Okay, but for example, we would have to store them in our, like, especially non-archive nodes. We don't have a way of reproducing them as easily, we don't store the archive node as…
Łukasz Rozmej:as Ref and Aragon, so… we cannot guarantee them. If we want to drop them, we drop them, and we can have a retention period.
Łukasz Rozmej:But we cannot guarantee to have them.
Ben Adams:I mean, for… for syncing, so… so what would… what are the benefits? Say we have a retention period of, like, 256 blocks or something.
Ben Adams:What are the benefits of keeping them? So if you snap sync, there's no benefit to them.
Jared Wasinger:The benefit is that you don't need to heal, you can just download the range of blocks with access lists, and…
Jared Wasinger:Once you…
Ben Adams:Dude.
Ben Adams:If you've got, like, the last 256, you can't snap-sync.
Ben Adams:Further back than that, because you're too far away from the head.
Jared Wasinger:Yeah, I guess that's why the retention period would need to be…
Jared Wasinger:The suggested retention period would need to be reasonably long, right?
Ben Adams:Yeah, but only, like… 256 blocks, or something.
Ben Adams:It doesn't need to be.
mark:Yeah, that's not very long.
mark:Right.
Ben Adams:Yeah, because you won't be able to get the steak.
Łukasz Rozmej:Well, we should… we should keep them up to finalization barrier, to be honest, right?
Łukasz Rozmej:Maybe.
Ben Adams:Yeah, but that's even… that's even less. That's, like, 64 blocks.
Łukasz Rozmej:Yeah, yeah, but… no, the finalization barrier is, like, can move… Right?
draganrakita:How… how can this be enforced on the P2P? We are talking about the limit of the last 10 blocks while the syncing is happening, and we don't exactly know
draganrakita:Like, what kind of number are we talking about?
Ben Adams:Okay, well, hey, say a week.
Ben Adams:I'm just beyond that.
Ben Adams:Period. Is there any benefit?
Ben Adams:I mean, I can see an archive node, yes, but…
draganrakita:We don't need it.
Ben Adams:So they're quite rare, they're quite rare.
draganrakita:From Brett's side, we don't want Bolt to be transmitted over P2P.
draganrakita:You're just fine having it in engine, and that's it.
Łukasz Rozmej:So, so there are… there are other things, right? So, if we are definitely, like.
Łukasz Rozmej:Our node hanged, we are restarting it, we are several thousand or 10,000 blocks behind.
Łukasz Rozmej:Then, when we… okay, we won't get that from new payloads, probably.
Łukasz Rozmej:So we need to go through… DevP2P, probably, right? So we are getting the blocks. Now, the blocks are very big.
Łukasz Rozmej:So, if we are single-threadly just replaying them, it might take quite a long time to catch up, right? So that's without balls, right? So that's a problem, potentially.
Łukasz Rozmej:That the catching up process will be way slower. If we don't have a ball, and we need to reprocess something, it is way slower.
Łukasz Rozmej:Another problem is that how the balls will be now distributed. If they are going as a CL sidecar, this is now a CL change.
Łukasz Rozmej:And we need to involve more CL devs into prototyping that, into speccing that, and it increases complexity on their side. So generally, it increases complexity a lot, if we… if we drop the balls.
Łukasz Rozmej:But… yeah.
Łukasz Rozmej:That's kind of it.
Ben Adams:But we can… we can delete them from… From the block itself.
Ben Adams:And change the… RPL.
Łukasz Rozmej:Not sure if I understood.
Ben Adams:Well, the… as I understand it, we're not… we're not, generating a hash multiple times. We're not…
Ben Adams:You've got the bowel hash, which is the hash of the bell, and the header, but then you're not…
Ben Adams:You're not rehashing the entire block to get a block hash.
Ben Adams:You know what I mean, it's the sum of all the hash… it's the hash of all the hashes.
Ben Adams:So, you can just erase it from the lock at some point.
Łukasz Rozmej:Oh, okay, yeah, you could, you could.
Łukasz Rozmej:Okay, so maybe it comes from… okay, so we now have, like, a storage and networking part, but that would kind of…
draganrakita:Doesn't… couldn't ball, like, behave something like sidecar?
draganrakita:I'm not exactly sure about CL details there.
draganrakita:That shouldn't be acted like sidecars, the blobs are currently… done.
mark:Yeah, so…
mark:I guess the question is, are you talking about dropping them from the EL, or dropping them from the EL and the seal?
Ben Adams:Because the CL has the EL block as…
mark:Yeah, yeah, yeah, yeah.
draganrakita:CL will still need to have them.
Łukasz Rozmej:I would drop in both, because…
mark:Yeah, but is it…
draganrakita:Maybe. I'm not sure, I'm not sure… maybe not, my comment's not good.
mark:Yeah, because I… well, sorry, I'm just thinking this through now from a…
mark:Kaplan archive node, it'll have… so we might drop them from the Aragon archive, but end up having to keep them in the,
mark:In the… in the CL archive.
mark:Which is interesting.
Ben Adams:How… how much do CLs outside of archives normally keep? I mean, do they keep all the box forever?
mark:I don't know that… yeah, that's a good question.
Ben Adams:Well, do they already have something like Full Force?
mark:Well, I guess that's a question we should ask.
mark:So I… I don't know, because basically.
mark:how our use case is archiving, and I think we just keep everything.
mark:So… I'm not sure what everybody else does.
Ben Adams:Because I know they do the, checkpoint sync, then.
Ben Adams:But I don't know, beyond that, what they do with their books.
Łukasz Rozmej:But if CLs keep it, we are definitely more than 10x-ing the storage requirements, and even if we have a retention period of 44s, like, half a year or something, that still will be quite a big amount.
mark:Yeah, so I think it's an open question. You need to check with the CL devs.
mark:I mean, they might not have even thought about it yet.
Łukasz Rozmej:Anyone from Bezu or GeF want to chime in? I'm not sure if anyone here…
Karim T.:In Bezos' side, I think we need,
Karim T.:A minimum amount of block accesses for SnapSync.
Karim T.:as I said, I see some benefit to have block access… all of the block accessories, but…
Karim T.:It's increasing the size.
Łukasz Rozmej:So, for SnapSync, I don't see this as a problem, because I expect block access lists will be coming through new payloads, right? So you get a new payload, you start snap syncing from that new payload as your pivot for snap syncing, and you can accumulate all those balls, or apply them on the fly while you snap sync.
Łukasz Rozmej:And then you finish SnapSync, you go to the healing, verify you have everything or not, and be done, right? So…
Karim T.:If you don't have everything, you need to find a way to retrieve what is missing.
Łukasz Rozmej:If you don't have what?
Karim T.:If you don't have everything.
Karim T.:Who you guys?
Łukasz Rozmej:That would be normal healing, right? That would be normal healing.
Karim T.:Hmm.
Łukasz Rozmej:But you generally should have everything, unless your CL didn't send you the new payloads, right?
Łukasz Rozmej:At some point.
Łukasz Rozmej:Because that's how you will get Baal on the…
Łukasz Rozmej:train tip, and during the snap sync, you are following the train tip and repivoting, so…
Łukasz Rozmej:I don't see this as a problem for SnapSync, right? It can be…
Jared Wasinger:We're back… How far back will the CL give you new payloads?
Jared Wasinger:Like, if you… if you stop your EL node and then reconnect to the CL.
Łukasz Rozmej:So, if we stop EL node, we generally restart SnapSync from scratch.
Karim T.:At least that would never mind us, I don't think…
Jared Wasinger:Yeah, but I mean…
Łukasz Rozmej:Mbezu, okay.
Karim T.:No.
Jared Wasinger:Yeah, I guess I'm wondering, like, if there's a lapse in the… like, will the…
Jared Wasinger:Will the CL node give you new payloads for, like.
Łukasz Rozmej:Yeah, I don't know, but generally, if it doesn't, for some relatively small amount period of time, the healing at the end of SnapSync should just handle it, right?
Karim T.:Yeah, but…
Łukasz Rozmej:There will be a problem if that's a big period of time.
Karim T.:For Bezoo, it will change significantly, because, for example, we have a healing of the tree that is general fast, but we have the flat dB healing that can take more times.
Karim T.:several hours, so if we have block access lists, it could be really fast. If we are just missing one block access list, we'll have to do a…
Karim T.:old healing version, and it will take a lot of time. So it will be a huge difference, just for example, because we are missing one block access list.
Łukasz Rozmej:Hey, but we could have then a retention period in ELs of, let's say, a day of block access list, and that should be…
Łukasz Rozmej:Fine.
Karim T.:Yeah, if we have this minimum retention, and we have a way to retrieve that, yeah, it should be okay.
Jared Wasinger:So, I guess from Gat's side,
Jared Wasinger:we, like, I voiced before that we want…
Jared Wasinger:We would be in favor of some retention period that enables a reasonable probability of… of, of… of…
Jared Wasinger:successfully syncing and using the BALs to heal without, needing the heal phase, and also, I think, like, if we're gonna scale the gas limit to hundreds and hundreds of millions, it… there's… like, the healing phase could become a bottleneck there, so…
Jared Wasinger:keeping… so, using BALs as an avenue to, sort of, address this…
Jared Wasinger:scaling bottleneck with the syncing is important, I think. So, I mean, we…
Jared Wasinger:we would like them… well, I say we, but it's mostly just me. I think they should be, retained for some period of time, and also transmitted.
Jared Wasinger:over DevP2P.
Karim T.:I see also, but maybe it will be okay with a genu pod, but for example, if you stop your node and you restart your node after sometimes.
Karim T.:Instead of doing, backwarding and executing everything, you can also use block accesses to reach the head.
Karim T.:But maybe in this case, we'll have a genu pair outside. It'll be okay.
Jared Wasinger:Yeah, I think… I think that, my…
Jared Wasinger:like, what I envision the retention period being, like, sort of tuned for is, like, the…
Jared Wasinger:To capture enough, portion of the cases where maybe people, like, have some kind of a network event that prevents them from
Jared Wasinger:sinking… I… I…
Jared Wasinger:I think there's always gonna be… if we have a retention period at all, there's always gonna be a case where somebody
Jared Wasinger:Turns off their node, it powers it back on, and they're kind of…
Jared Wasinger:out of luck, because they're just too far behind the chain, and they would need to restart the SnapSync, but…
Karim T.:And also, yeah, backward sync, it's using P2P, so…
Karim T.:Yeah, it would be nice, for example, in order to sync, when you restart your node, having this block access list.
Łukasz Rozmej:One downside of this is actually, I think the complexity increases
Łukasz Rozmej:Quite substantially, and this might be…
Łukasz Rozmej:Not sure if I would say delaying, because we don't have,
Łukasz Rozmej:Time, but require more effort, and potentially, delivering the forks slower.
Jared Wasinger:I mean, are you saying that implementing the… the… The replacement for the…
Jared Wasinger:healing phase in addition to the… yeah, okay, so…
Łukasz Rozmej:Yeah, like, just handling all the cases of baclovastasis available, not available, having some retention period, having, the FP2P that might transmit them, might not transmit them.
Łukasz Rozmej:Having CLs potentially having a sidecar for them, right? It's complexity over complexity over complexity, having a way of process blocks with and without them.
Łukasz Rozmej:Potentially might be needed, so… a lot of complexity, right?
Jared Wasinger:Yeah, for sure. I think whatever we decide on, I mean, because we can always… if we could create a… if we could figure out a way to avoid ballooning the complexity of the current EIP further, and then kind of push the…
Jared Wasinger:The question of…
Jared Wasinger:Yeah, I mean, I think we should go with whatever the simplest option right now that keeps the future open for potentially replacing SnapSync.
Karim T.:Yeah, I think it's more, just to define if you want to keep block access lists
Karim T.:During a specific window, and after each client.
Karim T.:will do what they want. If they want to use this block access list to implement new features, new optimization, but it's not really related to the fork, we can have just the block access list consensus to say, okay, we are keeping block access list during a window.
Karim T.:a specific window, and after, client can add more optimization later, etc, so we are just…
Karim T.:Opening the door to more optimization and features, but…
Karim T.:I don't think it will delay, the… the fork.
Łukasz Rozmej:Well, the second thing is how it should be, you know…
Łukasz Rozmej:transmitted, right? Should it be with block bodies, like it is now, or should it be as a sidecar? And that complicates the CL site, right?
Ben Adams:We probably should just follow up with CLs and see what their… Retention on boxes.
Ben Adams:That was… Rather than having them trying to understand the EL payload.
Ben Adams:Cause if we… if we expire them in… block.
Ben Adams:Say, after some… some time, the… if the CL… Just keeping that.
Ben Adams:Forever.
Ben Adams:Ben.
Ben Adams:Yeah, it doesn't completely solve growth.
Toni Wahrstätter:Mmm… Dragon, you have your hand up?
draganrakita:Yeah, just wanted to add, if the ballots are needed for the snap sync, maybe adding the additional endpoint to the snap sync protocol is maybe way forward.
draganrakita:It can be like, get me the ball of this block hash.
draganrakita:Because what I'm hearing is, this…
draganrakita:It's mostly needed for those snap syncing.
draganrakita:Or the best use case for the ball that people want to use it is for the snap singing, or recovery from snap singing.
Toni Wahrstätter:Okay.
Toni Wahrstätter:And let me just quickly summarize what we just discussed. So it feels like,
Toni Wahrstätter:the status quo will just stay the same for now. We will keep the block lab access system, the block body, and the main argument is the simplicity it comes with.
Toni Wahrstätter:Because removing it from the block body would mean… would mean, like, all the things that Lucas, just iterated on, like defining retention periods.
Toni Wahrstätter:Having different paths for what to do if the ball is there or not.
Toni Wahrstätter:And then we will also check with, the CEOs what their opinion is on the topic, but of course, we should
Toni Wahrstätter:like, the CL side of this work will already be quite big and quite complex, so…
Toni Wahrstätter:If there is a way.
Toni Wahrstätter:of course, must be efficient, but if there is a way that we don't need to create new sidecasts for the CL, it will definitely be beneficial for Glamstadom.
Toni Wahrstätter:You effin'?
Ben Adams:Yeah, I just think, one thing to check with CLs is what their
Ben Adams:retention period on blocks is… is it forever? Or do they have, like, a mechanism to expire?
Ben Adams:But I do think if we keep the bells, then it's going to cause more pressure to bring up
Ben Adams:The 4-4's expiry time, so… Have… have blocks expiring much faster.
Łukasz Rozmej:think regardless.
Łukasz Rozmej:It will. This would be a very, pressure on… for Force.
Łukasz Rozmej:If we want to increase the gas limit a lot.
mark:Yeah, so I guess the question is, given the current 4-4's retention period, what's the…
mark:What's the, you know, how much more disk, in practice, do you end up needing?
mark:Right, if it's… you're saying it's 40% of the current blocks? It potentially is?
Toni Wahrstätter:Yeah, might be an optimistic guess, so if you want to be pessimistic, I would do with a 50%.
Łukasz Rozmej:Yeah, it would be good to have some kind of calculations with different block gas limits, right? What kind of things you need for, and what's the expected growth for…
Łukasz Rozmej:What, 100 megahs and… 200, 300, right? And just… Showing that.
Łukasz Rozmej:Because… The numbers might be huge.
mark:That's the simplest…
Toni Wahrstätter:Yeah, go ahead, Mark.
mark:I was gonna say, the simplest way of dealing with it is just to
mark:say, after bowels, there's a shorter retention period. I'm just thinking in terms of managing the EIP, because it does sound like there's a lot of work if you start… if we start dropping things now.
mark:Whereas, by keeping things, we're just, like, punting the problem.
mark:Basically.
Ben Adams:I assume it would just… just… rather than…
mark:Expiring.
Ben Adams:thing would be just to say the bells can be optional.
mark:That's true.
mark:Yeah, and in practice, that's optional across…
mark:dev P2P, right? Because that's the use case that you'd be using them, is when you're syncing them. When you're syncing, you may or may not get BALs.
mark:And then if you don't get a bow, you're just gonna have to do a full.
mark:execution.
Karim T.:Not adding complexity in this case, because we…
Karim T.:We need to manage to kind of, sync, depending on
Karim T.:If we have block X, this is all not swoopy-tookie.
mark:That is true, yeah.
Ben Adams:Yeah, but you still need the capability to execute a… execute a block anyway.
Ben Adams:Unless you're just ignoring processing and flying bells in the… in the main…
Ben Adams:new form of ZK.
mark:Yeah.
mark:Yeah, then you keep the bell and drop the… drop the… drop the transactions.
mark:I mean, to some extent, that's a reason for… for not doing anything at the moment, till it's kind of… you let it bed in, because you… you could theoretically end up with a set of… a class of nodes that just… just want the bowels.
mark:They don't care about the transactions.
Karim T.:But technically, yes, you need to equal translation just to validate the block, but
Karim T.:In all of the other cases, you can just use block access list.
Łukasz Rozmej:What are the case… what are the other cases? Well, you can probably assume beyond finalization barrier, you don't need to execute the block.
Karim T.:Now, before all of the block,
Karim T.:Before the… after the finalization, I'm saying.
Karim T.:So, yeah, as I said, backward sync, all of the sync, so when you need to reach the head, reach the last finalized block.
Karim T.:So… as I said, archive node, or something like that.
mark:Yeah, so as I was saying, I think archive nodes need to just store the data anyway, but they're a special case.
Łukasz Rozmej:The archive node requirements after increasing gas limit to 300 or something will be…
mark:Yeah, yeah, that's the problem. Yeah, yeah. So, I mean, there's a bigger problem here, is once you've got a massive,
mark:Once you… once we've got more states.
mark:I think, you know, it's part of what we're thinking about is what does an archive node look like after this? And it will get to a point where it's impossible, you know, it's virtually impossible to have
mark:You need an archive virtualized in order to support the data growth.
mark:Hmm.
Łukasz Rozmej:Whatever that means.
mark:It means you're not storing everything on one machine.
Łukasz Rozmej:Okay, okay, so this…
mark:you get to that.
Łukasz Rozmej:Okay.
mark:Yeah, distributed archive, yeah.
mark:I think is where… I mean, I just think that's where we're gonna end up, basically.
Łukasz Rozmej:But the latency would be really bad.
mark:It depends on your…
mark:Yeah, so that's the… that's the… that's the design space. It's like, can you… can you…
mark:can you end up with an archive node where you can dis… I mean, it's a bit like… it's a version of the stateless client problem.
mark:Right? Because you've got… You've got network latency, not just this latency, to take into account.
mark:But, yeah.
Toni Wahrstätter:So, do we agree that we keep the balls as is for now? Or… Yeah, Ben?
Ben Adams:Yeah, I think… I think the summary, since there's… there's a couple extra use cases, like, you might want to drop the transactions, keep the valve.
Ben Adams:Then… Yeah, it might be better to not make the perfect the enemy of the good.
Ben Adams:keep them in the… in the blocks for now, and then revisit in a… in another EIP.
Ben Adams:When we see how everything plays out.
Toni Wahrstätter:Yeah, this is my feeling, too. And just, to answer the question from Jagan in the chat, like, what is the as-is case? It's keeping the block of access list in the block body.
Toni Wahrstätter:And not having them as a sidecurve.
Łukasz Rozmej:So, my gut feeling is that we need a better analysis on numbers, and come back with numbers, and then see how many people yell at us that this is unacceptable numbers.
Łukasz Rozmej:In terms of storage growth.
draganrakita:How would that affect P2P?
draganrakita:Or database, or painting.
Łukasz Rozmej:Well, everything will bloat, right, by the size of the bone.
draganrakita:That's what I'm saying.
Łukasz Rozmej:That's the current design.
draganrakita:That's not that great.
Łukasz Rozmej:Yes, exactly, that's why I brought it up.
draganrakita:Yeah, yeah, I know. I, I, basically, my…
draganrakita:Cleased my mind from the Berlin, interrupt that we did, we decided that the ball was going to be dropped, and I had that mindset from that time.
draganrakita:So, yeah, I'm not sure if it's a good decision to… like…
draganrakita:To have balls inside the block body.
Łukasz Rozmej:So, in my opinion, we should postpone decision by a few weeks, gather more data, gather more projections, talk with more people, talk with CL people, what do they think, talk with,
Łukasz Rozmej:people that are more focused on… other parts of scaling, too, like, lusanskar here, probably…
Łukasz Rozmej:Not… what does he think about it?
Łukasz Rozmej:Yeah, talk with Ansgard, talk with, maybe some, also, other people that think about storage. Not sure if there are other people that are outside of this…
Łukasz Rozmej:Outside of this meeting that are focused on storage that much.
Łukasz Rozmej:But get also sections.
draganrakita:In general, the feel that I get from this call is that it's this useful for the snap syncing, or the syncing on the tip.
draganrakita:But I think it's not worth it, because it will blow up the database.
mark:100.
draganrakita:In case it will… it will…
mark:Blowing out the database is just an implementation detail.
mark:Right, because I think, effectively, what we're saying is that the RLP of the bodies that you get across DevP2P will include the bowels.
mark:Whether it bloats, I…
Łukasz Rozmej:No, it isn't a implementation detail, because basically you say each node is an archive node, regardless if you want it or not.
Łukasz Rozmej:Because if you need to serve balls through DevP2P, you need to kind of have an archive node for those blocks.
draganrakita:AI sucks.
Łukasz Rozmej:So you, like, for us, if we don't do archive node, this kind of use case just becomes extent. It doesn't work.
mark:Right, because you've got to re-execute the block to get the…
Łukasz Rozmej:Yeah, and I don't have state, I already pruned the state.
mark:Heh, right.
Łukasz Rozmej:So, our kind of validator-optimized setup Just doesn't work.
draganrakita:To be honest, I was a little surprised that we store balls in the block, and…
draganrakita:That was the reason, what Lucas said.
Łukasz Rozmej:Well, it's… the AIP says that both are part of the block body.
Łukasz Rozmej:Currently. That's the current…
draganrakita:I just thought that that'd need to be updated. Okay.
mark:If you… yeah, but if you've got a validator-optimized node, surely you can't serve the blocks, either.
Łukasz Rozmej:Well, depends on the…
mark:Because if you prune the blocks, you can't deliver them.
Łukasz Rozmej:Yeah, it depends on the lever optimization, but, you know, 44s, dictate us how much
Łukasz Rozmej:We need to serve, right?
mark:Yeah, yeah, so that's what I was saying, I think the question is, in a 4-4s retention period.
mark:How much practically more disk space does that add to a minimal node?
Łukasz Rozmej:Well, 50% on the blog bodies.
Ben Adams:Even if the sales are storing.
Ben Adams:It's…
Łukasz Rozmej:Oh, CLs don't… so you're unstarting it in non-archive node, right? So, it's fine for them on non-archive node.
Łukasz Rozmej:So don't worry about that, that's only an archive note question for them.
Łukasz Rozmej:If they are storing it.
mark:It's not pissed.
Łukasz Rozmej:Yeah, so I think everyone wants to more or less drop them.
Łukasz Rozmej:But… It complicates stuff, potentially.
Toni Wahrstätter:And dropping them comes necessarily with taking them out of the block body, right?
Łukasz Rozmej:Well…
Łukasz Rozmej:either that or saying block body is optional… has optional balls, right? In certain circumstances, right? Because this block body is kind of…
Łukasz Rozmej:what comes from CL, block body is what we transmit through DevP2P, so…
Toni Wahrstätter:And also, like, dropping them, which means we have to put them in a sidecar, right?
Łukasz Rozmej:Maybe? Yeah, probably.
Ben Adams:Not, not necessarily. We just erase them from the block.
Ben Adams:You know.
Ben Adams:After a retention period, go back to the block, load it, remove the bow, save it.
Łukasz Rozmej:But then you have, kind of, two block bodies formats, with balls or with opals. Yeah. Block bodies can live… can live…
Łukasz Rozmej:In both states, right? Which is also might be fine, but…
Łukasz Rozmej:trade-offs, right? Also, question mark, if for CLs it wouldn't be more beneficial to transmit block body and bulk separately for latency.
Łukasz Rozmej:I don't know.
Toni Wahrstätter:Yeah, this was what I was thinking of initially. Like, you… we keep them in the block body, and you can still erase them.
Toni Wahrstätter:Because then we would not need to activate the CO to have a new set car.
Łukasz Rozmej:That's…
Łukasz Rozmej:that's doable, just needs to be specified, right? Right now, I didn't see anything from it in the AP, I think, so…
Toni Wahrstätter:Right, yeah, this doesn't exist yet. So, how would… what would you say… how would we best specify this? So, we would basically say there is a…
Toni Wahrstätter:A given retention period.
Toni Wahrstätter:And after that, you can… you can delete them from…
Łukasz Rozmej:period, blog bot, these could be, like, transmitted through DevP2P without it, right?
Łukasz Rozmej:Something like that.
Łukasz Rozmej:And we need.
Toni Wahrstätter:Is this…
Łukasz Rozmej:potentially mark that there is a null there, so there will be probably a null pointer where Bowles is, right?
Łukasz Rozmej:Just to…
Toni Wahrstätter:Is this something other clients are fun with, too?
draganrakita:I need to track, but seems okay.
Karim T.:Yeah, it's a good trade-off.
Milos:I have a question,
Milos:Would… if we would switch, like, if we would make a progress on the 4-4s and do some kind of, like, a rolling window, 4-4s, would that solve all these problems?
Milos:And we can keep it in a body in that case?
Ben Adams:I would put pressure on making that rolling window very, sure.
Ben Adams:Shorter.
Milos:I mean, why does it need to be specifically short? Any kind of, like… currently, the 4-4s dictates you have to keep it since merge.
Milos:So even something that is, like.
Milos:I don't know, a couple of months, or even a year, would definitely significantly help with,
Milos:with this, and sure, you can go shorter, but I don't think you need to be super short in any kind of form.
Łukasz Rozmej:I think at the moment we're keeping since the merge.
Ben Adams:Is that right?
Milos:No, I'm thinking, actually, going to some rolling window where we keep only the last, I don't know, since the last,
Milos:maybe… I don't know, 3 months, or how much is,
Milos:subjectivity period, or something like that. So, going to, like, last couple of months of worth of
Milos:Bodies and receipts and balls.
Milos:And keep from…
Łukasz Rozmej:From my perspective, one, yeah, it could be a workaround, right? You still need to keep it, you still don't really need those balls, really, in practice, in any way, because they can be regenerated from those blocks.
Łukasz Rozmej:And archive nodes still need to either double the data, because they still need to keep it properly, or they will actually still remove it internally and regenerate it, so… which they will probably do. So, it's…
Łukasz Rozmej:like… Trying to ignore the problem where, like, say it's an implementation detail problem, which… It's fine.
Łukasz Rozmej:But… yeah, I'm not sure.
draganrakita:Regenerating would be a problem, in general, because you can DDoS the P2P, or drop, or not respond a lot of P2P requests, because it's not just, like, fetching something from the database, it is execute in that block. If there's few blocks, that would be fine.
mark:Depends how you… depends how you store the data.
Łukasz Rozmej:Yeah, you could probably fetch it and just compile it from the fetch data, or something.
draganrakita:You're still for the ball, at least from…
draganrakita:I'm not sure about… I'm not sure if Aragon stores the read.
Łukasz Rozmej:I… yeah, that's exactly…
draganrakita:Pro.
Łukasz Rozmej:I wanted to say that the Ridley's is the problem, because the right
Łukasz Rozmej:Yeah. So it's actually, question mark, do we have read symbols? Because if we do, then, yeah, the only thing is re-executing, and if we don't, then…
Łukasz Rozmej:Then you can probably compile it from the archive data.
draganrakita:Then you're basically forcing everybody to have, like, archive kind of note.
Łukasz Rozmej:Exactly. Yeah, that's not that great. There was a reason why it's dropping, yeah.
draganrakita:It opens, like, all of these…
mark:Yeah, so I… yeah, I agree. If you start doing this stuff with bowels, which is why I sort of said at the beginning we're neutral, but other people probably aren't, because
mark:If you do start having to do this, it's a big piece of work.
mark:That's the problem, basically.
mark:Because you have… you end up having to have incremental data… an incremental data store.
mark:Which is not an insignificant amount of work.
mark:And it feels like dropping them, given the status quo, because presumably everybody at the moment can process
mark:Blocks without bowels.
mark:Right. So the implementation cost is quite small.
Łukasz Rozmej:Well, the problem is that we are currently processing small blocks without bowels.
mark:Yeah, yeah.
Łukasz Rozmej:And we'll be processing huge blocks with all balls. So, for example, I'm currently implementing block STM-like processing regardless of balls, and I expect we will be processing them relatively quickly, but yeah, it still complicates things, right?
mark:Yeah.
Łukasz Rozmej:Can be huge.
draganrakita:It's even bigger than the archive node, because we have additional reads.
Toni Wahrstätter:Yeah, this seems like a topic that we definitely want to discuss, next time, on the next, breakout call in two weeks again.
Toni Wahrstätter:I would suggest we… Don't change the EAP for now, and get more information, especially on the size.
Toni Wahrstätter:But also, if it feels like we will definitely have to go the path, of having the balls as a sidecar.
Toni Wahrstätter:Then, of course, the better, the earlier the better.
Toni Wahrstätter:we change something and involve the CL.
Ben Adams:I'm just…
Toni Wahrstätter:So someone…
Ben Adams:Just to…
Toni Wahrstätter:Sure, go ahead.
Ben Adams:I'm, like, not entirely serious.
Ben Adams:Oh, but it could be.
Ben Adams:Reserve one… one blob per block for…
Ben Adams:One, yeah, one blob per block for,
Ben Adams:about… we already have a mechanism to do a sidecar.
Toni Wahrstätter:Yeah, I mean, technically, it would work like this, right? You would just create a new sidecar, and then the ball would come in that sidecar, probably need a new
Toni Wahrstätter:Gossip topic?
Ben Adams:Yeah, but it could be… it could actually be a lot… a blob.
Łukasz Rozmej:But then you have to differentiate, etc, probably.
Łukasz Rozmej:Messy.
Milos:Thank you.
Milos:can be a blob, because if you do, like, a peer does, and clients don't download entire blogs now, you break that.
Toni Wahrstätter:Yeah, it would not be, like, a real blob where you do, like, KCG stuff with it. It would just be a blob of data that is, like.
Toni Wahrstätter:Transmitted separately from the block body.
Toni Wahrstätter:Is there any to-do for the EAP right now, until we resolve that? Should I clarify anything in the EAP already? Like, regarding…
Toni Wahrstätter:Because it feels like there is also some confusion in the chat around what does it actually mean if we keep the ball in the block body.
Toni Wahrstätter:can we just erase it out there?
Toni Wahrstätter:After some retention period, or is that not even possible?
Ben Adams:Yeah, because the… the block hash is the hash of the hash in header, isn't it? So, it's not… you don't… you don't hash all the hashes and then rehash the RLP.
Ben Adams:So you should be able to erase it. And then, when you re-execute the block.
Ben Adams:You generate the bell anyway, so that you can verify that… It's correct.
Ben Adams:And it hashes to the bowel hash.
Ben Adams:So, you know, there's already… you already regenerated it, so you could delete it.
Toni Wahrstätter:Right, yeah, I get that.
Toni Wahrstätter:Yeah, this seems like, to me, at least for now, like, the…
Toni Wahrstätter:the best solution we have, right? And…
Toni Wahrstätter:If, like, if… and the second best would then be the… the sidecarve.
Toni Wahrstätter:Which, of course, would come with a lot of complexity, because I remember, like, this was a discussion at the beginning.
Toni Wahrstätter:Ball in the block body versus ball in the side curve.
Toni Wahrstätter:And…
Toni Wahrstätter:By the time selders were involved, and many people, also networking people, were all for keeping them in the block body for simplicity.
Ben Adams:Just to follow up on your question, I don't think you should change the EIP at this stage.
Toni Wahrstätter:Okay, yep.
Toni Wahrstätter:Yeah, we're already at the very end of this call anyway, we're already over time,
Toni Wahrstätter:I will definitely put that topic on agenda for the next breakout call in two weeks again. Is there any…
Toni Wahrstätter:Last words that people… Wanna say anything else we should, we should, add to that?
mark:Yeah, so the only thing is, I'll… between now and the next call, I'm going to look at how much we care about reads.
mark:Because I think we should probably include the, do we want reeds in the bow in that overall conversation?
Toni Wahrstätter:Yeah, this was also on the agenda for today, and to make… to summarize it very quick, what we need to benchmark the reads is basically the batch I.O. implementation.
Toni Wahrstätter:So before, we don't know how much we get from reads, we wanted to test it on actual client implementations, and for that, we need two optimizations, the parallel execution and the parallel I.O, because then we can create, like, worst-case blocks and test how much faster are we actually, verifying those worst-case blocks.
mark:Yeah, so…
Toni Wahrstätter:Having the wreaths and don't having…
mark:Yeah, I didn't want to have the call. I'm just saying what came out of this conversation for me was these two questions are actually a bit related, certainly from an Aragon implementation point of view, because that's a good point. I don't think we're recording reads.
mark:which I hadn't… I'd forgotten about the read stuff, so…
mark:I need to do some research, basically.
Toni Wahrstätter:Okay, anything else we need to discuss? Otherwise, I would say we end the call, we're over time anyway.
raxhvl:Yeah, I'll quickly go over the comment that Ben had for using ZAR for…
raxhvl:storing nuns. I think so ZAR, essentially.
Toni Wahrstätter:Now, let's do that next, let's clarify that async.
Toni Wahrstätter:Because this was already a discussion point at some point, I would say we clarified that, I think.
raxhvl:Alright, Sean.
Toni Wahrstätter:Because we're already 10 minutes over time.
Łukasz Rozmej:Hurry!
Toni Wahrstätter:Then fi… Thanks, everyone. No, it was… it was a great discussion. Thanks, everyone, and…
Toni Wahrstätter:See you at the next breakout call in two weeks.
draganrakita:Bye.
Stefan Starflinger:Thank you. Bye.
Łukasz Rozmej:Bye.
Chat Logs
00:03:45
Łukasz Rozmej:famous last words...
00:04:54
Toni Wahrstaetter:https://github.com/ethereum/EIPs/pull/10866
00:06:47
Stefan Starflinger:for memory expansion nethermind also has a inconsistency vs reth and besu https://github.com/ethereum/execution-specs/pull/1825
00:08:43
Ben Adams:Makes sense to delay disk access
00:08:58
Ben Adams:If going to fail anyway
00:10:04
Stefan Starflinger:https://github.com/ethereum/execution-specs/pull/1825
00:14:31
Karim T.:BAL in the history could be a fast way to sync an archive node after BAL
00:15:07
Toni Wahrstaetter:This was the latest BAL size analysis but lacks a comparison to the block size:
https://github.com/nerolation/eth-bal-analysis/blob/main/reports/eip-7928-analysis.md
I assumed the block size is 120 KiB (based on a dashboard from etherscan).
But can double-check that number
00:17:15
Ben Adams:puts more pressure/incentive to drop blocks via 4444 to always keep BALs so historic blocks would become less available?
00:20:39
Karim T.:I think that block datas are going to grow enormously anyway. I think 4444 might be the solution, as Ben said
00:23:21
Karim T.:The fact that with BAL we can synchronize an archive node more quickly is a real plus. Archive nodes will become increasingly difficult to resynchronize, and having this capability can, for example, allow synchronization of an archive node without needing all versions of the EVM fork .
00:31:08
Ben Adams:BALs in blobs :)
00:43:01
Jared Wasinger:Yeah we want to bound the healing phase
00:43:44
Karim T.:I see also regarding backward sync , not need to replay block . So snap protocol I don’t know , but specific P2P message why not
00:45:13
Karim T.:No need to keep evm history if we have all BALs ^^ the fix to legacy code
00:45:36
Stefan Starflinger:Storage is cheaper than RAM now 🙃
00:50:45
draganrakita:What is as is?
00:51:00
Toni Wahrstaetter:keep them in the block body, no sidecar
00:52:10
Ben Adams:Still think we need to check with CLs to see what they do with historic blocks. If they keep them all then is x2
00:54:28
Ben Adams:A poor archive node (just storage) as don't have the initial state
00:56:20
draganrakita:Dropping them simplifies the things for Reth
00:57:22
raxhvl:But the block hash gets invalidated if we remove bal later?
00:57:50
Ben Adams:No because block hash is hash of the hashes
00:58:04
Karim T.:I think keep them with a retention period is the good tradeoff
00:58:08
Ben Adams:Don't rehash the entire block as rlp?
00:58:51
Stefan Starflinger:Replying to "No because block has..."
Yeah, this is something that confused me also, I was under the assumption that anything that is specified in the block body would be included in the block hash calculation
00:59:25
Karim T.:So if we replay an old block we need to accept a block without BAL but if we want to execute a recent block we should fail if BAL is missing ?
01:00:21
Toni Wahrstaetter:Replying to "So if we replay an o..."
yeah that's my understanding too
01:00:46
Karim T.:Replying to "So if we replay an o..."
So we need to have a specific logic to change the rule depending on where you are in the chain
01:00:50
Ben Adams:Replying to "No because block has..."
but you get the BAL back from executing block (as you need to recreate the BAL to verify BAL hash)
01:01:00
Karim T.:Replying to "So if we replay an o..."
Seems tricky
01:01:03
Jared Wasinger:Replying to "So if we replay an o..."
If you get the block from the CL, it has the sidecar (BAL)
01:01:09
Jared Wasinger:Replying to "So if we replay an o..."
And the sidecar isn’t persisted
01:01:24
Karim T.:Replying to "So if we replay an o..."
Hum yes ok
01:01:25
raxhvl:Replying to "No because block has..."
Right.
01:01:26
Jared Wasinger:Replying to "So if we replay an o..."
That’s my understanding of dropping BALs from the block. Works exactly like blob txs
01:01:29
Stefan Starflinger:Replying to "No because block has..."
yeah 100%, this makes it a bit tricky to understand
01:01:39
Toni Wahrstaetter:Replying to "So if we replay an o..."
my understanding is we don't need a sidecar in this case
01:02:09
Ben Adams:Replying to "So if we replay an o..."
Blob BALs :)
01:02:50
Ben Adams:Replying to "So if we replay an o..."
1 blob reserved per block
Though I'm not entirely serious
01:03:29
Stefan Starflinger:Replying to "So if we replay an o..."
system Blobs
01:04:29
Stefan Starflinger:Replying to "So if we replay an o..."
blob without size limits :D
01:07:23 Łukasz Rozmej: famous last words...
told you it won't be "quick discussion" :P
01:06:06
Łukasz Rozmej:we didn't even get to topic number 3
01:08:22
Karim T.:Still working on batch reading IO, we have // working still need to clean the PR
Summary
10 highlights
· 4 action itemsExperimental
Summary
10 highlights · 4 action itemsExperimentalgas accounting changes
storage and retention
- BALs add ~50% to block body size at current gas limit00:13:14
- Increased gas limit will significantly increase history footprint; 4444 pressure00:20:39
- Archive nodes may drop BALs from storage, reconstitute from access history00:45:07
- Block bodies can have optional BALs; nodes delete after retention period00:56:27
Decisions
Action Items
- Jared/Geth team: Complete test coverage for call opcode gas accounting00:06:25
- Toni: Redo BAL size analysis at 60M+ gas limits with clear numbers00:21:29
- All teams: Consult CL teams on block retention periods and BAL storage approach00:52:10
- Mark: Research importance of including reads in BALs for next call01:07:07
Targets
- Next breakout call in 2 weeks - revisit BAL storage with additional data01:06:02