Toni Wahrstätter:Great, thank you very much. Hello, everyone. Welcome to the 10th EIP-7928 breakout call. Today is January 14th.
Transcript
Toni Wahrstätter:And yeah, let's directly get started. We have…
Toni Wahrstätter:quite some items on today's agenda, and I would say we directly get into the first one, which includes DevNet 1 updates.
Toni Wahrstätter:Would be great if Stefan is on the call, I see he is. Maybe, Stefan, could you give us maybe some quick update on DevNet 1?
Stefan Starflinger:Yeah, sure. So, Happy New Year, also.
Stefan Starflinger:DevNet1 is going pretty strong currently. There were a few hiccups, a few syncing problems, but, we managed to figure pretty much most issues out.
Stefan Starflinger:Right? Currently, I think we're still on holiday.
Stefan Starflinger:They're still working on figuring out their syncing issue, but otherwise, Nethermind is pretty stable. Let me change my audio.
Stefan Starflinger:Can you hear me better now?
Stefan Starflinger:Yeah, it's better.
Toni Wahrstätter:It was okay, but now it's better.
Stefan Starflinger:Okay, great. So, NetherMind has managed to fix most of the inconsistencies, and is doing pretty well now.
Stefan Starflinger:We're running the EVM fuzzer, and we're running Uniswap transactions consistently, and it seems there are no errors that we can find with these two strategies anymore.
Stefan Starflinger:And most clients seem to have solved the issue.
Stefan Starflinger:That they were facing over the holidays, so that's great.
Stefan Starflinger:Now I think we're ready for, talking about DevNet 2.
Stefan Starflinger:And I think we can leave the definite one behind once Reth manages to sync up again.
Toni Wahrstätter:Awesome. Thank you very much for the update. Is there anything from client teams that someone would like to add regarding DevNet 1?
Toni Wahrstätter:Or maybe Rahul or Felipe, you guys have, great insights into testing, anything that you guys would like to…
Toni Wahrstätter:mention regarding DevNet 1?
felipe:I think only… that… you know, a couple of the issues that arose from DevNet 1.
felipe:We're used to create tests, but these were not, there was no release
felipe:that targeted DevNet1 with these tests, but these test cases are now included in the next test release.
Toni Wahrstätter:Awesome. Thank you very much.
Toni Wahrstätter:Great, yeah. The second agenda point would… would already be definite 2.
Toni Wahrstätter:I'm not sure if everyone has seen it, on…
Toni Wahrstätter:Tuesdays, Mondays, I forgot. Testing call.
Toni Wahrstätter:4 additional ELPs were slated for DevNet 2 for the BAL DevNet 2.
Toni Wahrstätter:I think it's the slot num app code, it's,
Toni Wahrstätter:EVE transfers emitting a lock, and two more EIPs, I forgot now.
Toni Wahrstätter:What do you think? Is this something…
Toni Wahrstätter:we should already put on Defnet 2, or do clients feel it would be smarter to not yet add any EIPs to DevNet2? Of course, considering that, at some point we need to add some more EIPs, just to…
Toni Wahrstätter:Make sure we… we stay in time.
Karim T. (matkt):I think it's too early, because we wanted to use DevNet2 to verify if the batch reading is useful or not. We also… we will also add separation between block access list and body, etc. So…
Karim T. (matkt):In my opinion, I would prefer to have the DevNet2 only focusing on block access list, and maybe the next one
Karim T. (matkt):We'll be, able to…
Karim T. (matkt):to integrate more EIPs, but my feeling is maybe it's too soon to add additional EIPs that are not related to block access lists in DevNet 2, but…
Jared Wasinger:Yeah, I lean towards agreeing with that, from the Geth's side.
Jared Wasinger:There's a possibility we could make the…
Jared Wasinger:the deadline, or the intended start date, but we still haven't even, to my knowledge, nobody has started implementing, the EIP for the…
Jared Wasinger:ETH transfers on log, and I'm personally not very familiar with that, not sure what the complexity is, so it's hard for me to, give a…
Jared Wasinger:Firm thumbs up to the, that date.
Toni Wahrstätter:Yeah, I can, I can definitely see that perspective, too.
Toni Wahrstätter:Maybe a question to, Stefan. Like, how…
Toni Wahrstätter:set in stone is the date of the 21st of January, versus could we just…
Toni Wahrstätter:delay DevNet 2, or what would that mean for you guys?
Stefan Starflinger:Nothing necessarily is set in stone. We're happy to adjust, that we can move block-level accesses further as quickly as possible.
Stefan Starflinger:Depending on how the clients, are progressing. We can also have a definite 3 as well, so I think we just need to agree on, exactly how we want, and what we want to have in the definites, such that we can move fast.
Toni Wahrstätter:Right, yeah. As Karim said, we wanted, definitely two to definitely include, like, batch reading.
Toni Wahrstätter:At least for a few clients, such that we're able to benchmark
Toni Wahrstätter:the state locations, and finally decide if we want to keep the state locations slash the reads in the block lab access list or not. How far are clients with that? Is there a client team that has already implemented batch reading?
Toni Wahrstätter:Using the state locations from the bul.
Karim T. (matkt):On Besu, we have a first implementation. I wanted to do some tests next week to be sure it's working correctly.
Karim T. (matkt):But yeah, on Bezu, we have a first PR, and maybe we'll be able to merge it next week, or at the end of the next week, or something like that.
Toni Wahrstätter:Okay, yeah, this is great. Is there another client team that has already looked into batch reading?
Toni Wahrstätter:Okay, I interpret that as a no. How far are clients in general with the batch reading? Is this something that clients are working on right now?
Toni Wahrstätter:Just to give you some context for…
Toni Wahrstätter:Some of the repricing work that…
Toni Wahrstätter:or some of the repricing EIPs that have been CFI'd for Glamsterdam.
Toni Wahrstätter:They're still waiting for benchmarks, and be… before, we don't figure out, like.
Toni Wahrstätter:how much we actually get from block accesses, it's difficult for them to do the benchmarking, which means we… the boss is basically blocking some of the repricing work.
Toni Wahrstätter:So this is, like, why… It's maybe pressuring that we get the…
Toni Wahrstätter:BatchIO implemented in order for us to figure out
Toni Wahrstätter:How the final part design will look like.
Toni Wahrstätter:Is there a client team that,
Toni Wahrstätter:is planning to start, implementing that soon, apart from Besu.
Jared Wasinger:I…
Jared Wasinger:So, I can just give a, like, a very brief summary of what this roadmap looks like for Geth. Right now, we're just trying to get the existing, like, a…
Jared Wasinger:most of the BAL, changes, with the exception of the performance-related changes in… merged into the master branch,
Jared Wasinger:And… yeah, I mean, I intend to look into it,
Jared Wasinger:I'm hoping we can get this merging process done in the next week or so.
Jared Wasinger:But, yeah, we haven't… I haven't, made any headway on that.
Toni Wahrstätter:Okay, thank you. Also, just to relay the message from Mark, from NetherMind is also saying.
Toni Wahrstätter:Netherland hasn't looked into it yet, but they were more focused on the parallel execution.
Toni Wahrstätter:Yeah, and Karim is saying, seems like 21 for DevNet will be too soon. That's my… my feeling, and that's… my feeling too, especially if we…
Toni Wahrstätter:want to get the benchmarks for DevNet 2, we might want to at least wait until at least one client team has implemented the batch I.O, such that we can
Toni Wahrstätter:Yeah, get the benchmarks.
Karim T. (matkt):Also, for DevNet 2, we are working… we are trying to do the separation between block access list and body, and we wanted to have more clarification regarding how the test will work. More exactly, if we remove block access list from the block.
Karim T. (matkt):Who will, do the import in the test? In the Hive test?
felipe:Yeah, I… I assume you're talking about the RLP tests, correct?
Karim T. (matkt):Yes.
felipe:Yeah, this is something that's worth discussing with the client teams if… If we have a separate…
felipe:file with the bell, just as we do with the blocks.
felipe:Yeah, we'll have to discuss and figure something out that works for everyone.
felipe:Because, yeah, from Glamsterdam on, this is going to change the entirety of the RLP tests.
Karim T. (matkt):Yeah, because if we want to have maybe that for the next DevNet,
Karim T. (matkt):Huh, just to… we need to find a way to test that.
Karim T. (matkt):Yeah, that…
felipe:The engine API still has, like, Consume Engine is still testable, is this correct?
felipe:Is the… sorry, is the issue only with the RLP tests, or…
Karim T. (matkt):Oh, yeah, yeah, yeah, yeah.
felipe:Yeah. Yeah, so…
felipe:So we wanted to get the test release out so at least Consume Engine can be tested.
felipe:If… okay, so if… if we have a separate file that has the VAL as an ROP,
felipe:Is this a good way to do it? How do client teams feel about this?
Karim T. (matkt):I don't know if we want to…
Karim T. (matkt):to talk about that, no, but just, yeah, just wanted to raise that,
Karim T. (matkt):Because today, Miroslavi did the PR, and, he asked this question, so…
Karim T. (matkt):I wanted to be sure we… We're syncing both up.
Karim T. (matkt):Yeah, we… we can definitely figure this out async as well.
felipe:But it'll be important to… to get this answered, for sure.
Karim T. (matkt):Thank you.
Toni Wahrstätter:Right.
Toni Wahrstätter:Just to answer Banaba's question here in the chat.
Toni Wahrstätter:So, what is… the state locations are useful for two things. First of all, we want to find out if… so the batch reading is useful for two things. We want to find out if the state locations are worth it, because the state locations are, like, 50% of the ball size.
Toni Wahrstätter:And removing them would, of course, improve bulb propagation and stuff. So, we have to have some certainty that the
Toni Wahrstätter:knowing all the state locations that are not even changed is actually worth it to have in the ball. So this is the… the main thing we try to achieve here. And second.
Toni Wahrstätter:Some of the repricing work is blocked, in a sense that
Toni Wahrstätter:it's not even clear which repricings EAP are needed, because, of course, if block access lists help you a lot with, fetching data from the state.
Toni Wahrstätter:Then we might not meet aggressive state repricings, as they are proposed in some of the
Toni Wahrstätter:state repricing EIPs that are up for discussion.
Toni Wahrstätter:So yeah, I was talking with Maria, who is, very involved in that effort, and she was, saying that
Toni Wahrstätter:That this is the case for them, that they can't really get, it's not only about final numbers, but
Toni Wahrstätter:basically get any number to even know is a certain repricing EAP, very much needed, or can it be delayed, for example?
Toni Wahrstätter:Just as some context.
Toni Wahrstätter:Regarding the replayable Baal benchmark question that I see here from Lukash, let me check if Jocham is actually on the call, because Jocham has some great insights into
Toni Wahrstätter:that topic. Yochma, you on the call-in could give us some… Some details on ballot benchmarking.
jochem-brouwer:Yes, so what is the… okay, the question is, like, how do we actually want to test this?
Toni Wahrstätter:Exactly, yeah.
jochem-brouwer:Do we have some good replayable ball benchmarks? That's the question, right?
Toni Wahrstätter:Yes, exactly. Like, how would we test the parallelization and that everything is working?
jochem-brouwer:Yes, yes. So, what we do here is, what we want to do is…
jochem-brouwer:If you want to parallelize transactions now.
jochem-brouwer:What clients do is optimistically execute transactions in parallel.
jochem-brouwer:So what we want to do for the benchmarking effort here is we want to create tests which cannot be parallelized without the block-level access list.
jochem-brouwer:So how do you do that? Well, a very simple example would be…
jochem-brouwer:in the first transaction, you would keep hashing something, and then you would store the hash result in the storage slot. Then in the second transaction, you would read the hash from the storage slot, and you would keep hashing it again, and you would store it again in the storage slot.
jochem-brouwer:So what is the end result of this storage slot? Well, it's basically being a slot which has been hashed, like, over multiple transactions.
jochem-brouwer:So, if you would execute this optimistically, without block-level access lists.
jochem-brouwer:then every transaction would point to the same… or would end up with the same storage slot, so let's say we would store that key 0, then every transaction would end up with the same hash. But, obviously, in this case, we have to keep hashing this item, so without the block-level access, we cannot do this in parallel.
jochem-brouwer:Because if we have the block-level access list, then we can actually read the storage slots
jochem-brouwer:from the block-level access list, and we can start hashing from there, and therefore we can execute these things in parallel. And we can also do this for, like, anything which has to do with storage, so we'll break up these tests in tests which cannot be executed in parallel without the block-level access list.
jochem-brouwer:If that makes, sense.
Łukasz Rozmej:Yeah, but, if I can, say something,
Łukasz Rozmej:That doesn't really tell us much about the…
Łukasz Rozmej:Red addresses, red slots, right?
Łukasz Rozmej:Should we include it or not? Because this is a bit different.
Łukasz Rozmej:problem.
Łukasz Rozmej:Also… if we, like, run those kind of benchmarks on a relatively empty state, That's…
Łukasz Rozmej:also pretty easy, right? Because empty state is fast to read, or almost empty state is fast to read, while if the state is big, it's slower to read.
Łukasz Rozmej:So… Yeah, I don't think it's that simple.
jochem-brouwer:Yeah, okay, so for this example, I took, like, a hashtag,
jochem-brouwer:Can someone inform me, like, about these slots? Because I think, Tony, this is about adding the slot keys which are being read to the block level accesses, right?
Toni Wahrstätter:Exactly, yeah, and the addresses that are accessed but not changed.
jochem-brouwer:Right, and the question is, do we want to add these slots, because it adds, like, a lot of data, or do we not want this, and we want to test in the benchmarks if this is, well, what the benefits of it will be, right?
Toni Wahrstätter:Exactly, yeah. Right now, they are part of the ERP, and they're part of the specs, and…
Toni Wahrstätter:So far, clients have signaled that it might be worth to have those, but of course, we… I think in Buenos Aires, we said we want to actually wait for final numbers before we lock that in.
jochem-brouwer:Yes, okay, so I'm just thinking out loud, then.
jochem-brouwer:So to test this, we would basically…
jochem-brouwer:Okay, yeah, first, Lucas, because you also mentioned, like, are we going to benchmark this on empty state? No, we will not do that, because if you would benchmark it on empty states, you are absolutely right.
jochem-brouwer:you can basically cache the entire Merkle tree into your memory, and then it's obviously, like, very fast. So, if we'll benchmark this, we will benchmark this on a mainnet fork, or maybe a load net, so that's, like, two times the size of a mainnet to actually simulate
jochem-brouwer:Well, big-sized networks, and also take care of, like, the lookup time of these, of this data.
jochem-brouwer:And then for the benchmarking itself, for what I would want to see, it's like the impact of reading these storage slots.
jochem-brouwer:So, in that case, I think we can come up with, like, a variant also of, like, a hash chain, where we also have to read multiple storage slots first, and then also write to them.
jochem-brouwer:You're right, it's not… As simple as, like, writing a hash chain, but,
jochem-brouwer:Yeah, we can do that, we can do that. I will, I will think about this, but we,
jochem-brouwer:Maybe already have some countries.
Łukasz Rozmej:Generally, yes, but what I want to say is that it's hard to reliably benchmark those kind of things, because you basically have to create some snapshots for each client.
Łukasz Rozmej:To be able to replay the same blocks over and over, right? Etc. And create those blocks, so it's not trivial, that's what I want to say.
jochem-brouwer:Yes, I completely agree. It's also, like, hard to measure this, but as part of the cash benchmarking effort, the Nethermind team,
Łukasz Rozmej:It's, like, working on this gas benchmark tool.
jochem-brouwer:And… What is doing… what people are doing there, is… That's… we,
jochem-brouwer:Try to keep, like, the cash out, we can,
jochem-brouwer:If we would benchmark these things, we would also read, we would, like, start, keep starting and stopping these slides to actually get, like, these reliable results.
Łukasz Rozmej:Yeah, I know, I know, we have this kind of things, but, yeah, I wanted to actually ping that out, that you should maybe talk with Camille, etc, but okay, we're already covered.
jochem-brouwer:Yes, yes, yes, exactly, yeah, Camille indeed working on this, yeah. So that will indeed be, yeah, how to measure these, these kind of things, yeah.
jochem-brouwer:And then, of course, we need, like, these benchmarks to specifically test, like, what's, like, the benefit of these state reads and state flights.
jochem-brouwer:I will think about this. It should be not super hard, but also not super easy.
Toni Wahrstätter:Thank you very much.
jochem-brouwer:Yes.
Toni Wahrstätter:This is great, and especially, yeah, I think it's a good idea to have it on a shadow fork of mainnet, and then also test it on BloatNet, just to make sure that things don't look very different as soon as the state increases.
Toni Wahrstätter:I see Barnabas has his hand up. Please go ahead.
Barnabas:Yeah, so you guys talked a bunch about repricing and how it will affect future repricing EIPs, and you need benchmarking, but in terms of the scope of DevNet 2, none of this matters. We're introducing four EIPs, none of them are,
Barnabas:none of them require benchmarking, basically. I feel like, the ball.net, should work on, implementation.
Barnabas:featuredness, and making sure all the EIPs that own touch the ELs, are done and implemented by every single EL team, and then once it's coming to the actual repricing values, that should be a parallel discussion, that's not really related to DevNet progression.
Barnabas:So, like, we shouldn't be blocking DevNet2 just because we don't have the, exact numbers for, repressing EIP.
Barnabas:Especially because there's no repressing EIP proposed for .NET2.
Barnabas:have the spec sheet, that Stefan made.
Barnabas:for the DevNet2 scope.
Barnabas:Yeah, as you can see, it's,
Barnabas:It's basically new instructions, the ETH transfer emit, a log, a new opcode, and, excluded refunds from log S.
Barnabas:But none of these are benchmark-related questions.
Barnabas:So, that's why I suggested that we could probably launch DevNet2 next week, because these are all,
Barnabas:fairly small implementation, VIPs.
Toni Wahrstätter:Yeah, this is true, like, all those four EIPs won't affect Block Live Access List, so nothing new for the Block Live Access List specs happening there. It's more like
Toni Wahrstätter:for example, tomorrow we will, again, CFI, DFI a few EIPs in the All Core Devs call.
Toni Wahrstätter:And it's hard to make that decision without knowing how much we actually need to change the state pricing, for example. So it's not really about those smaller EIPs, it's more like
Toni Wahrstätter:yeah, giving us earlier information such that we can take a good decision on what EAP to CFI and which one not.
Toni Wahrstätter:Because currently, it's not super clear how much block-level access list will help with state-related stuff, and if it helps a lot, the repricing might not be needed, or might be less aggressive. If it's not…
Toni Wahrstätter:that impactful for a state, then we need a potentially more aggressive repricing. So it's more about,
Toni Wahrstätter:Yeah, about the repricing work that is still undecided.
Barnabas:But this is, again, a totally different parallel discussion. What the value of the repressing should be can be decided, by the benchmarking guys, but in terms of progression, we want to have all the EIPs that are CFI'd implemented, because that is the whole point of, having CFI'd.
Barnabas:So, we implement the EIP that was proposed for the change, and then we can evaluate if the given values that were CFI'd are correct or not in a much later DevNet.
Barnabas:But for now, I would just propose that we implement what is, what has been proposed and accepted as a CFI candidate, and then if those values
Barnabas:deemed to be incorrect, then we can adjust them, and we can roll it out for the next following that.
Barnabas:Especially for the, where it's just a single… changing a single value in a repressing ERP.
Barnabas:Fairly easy.
Toni Wahrstätter:Yeah, fair enough. I very much agree with you. So, I don't think the repricing should block us at all when it comes to launching Ball Defnets, I would say
Toni Wahrstätter:it's definitely worth it to wait, if it only… if it's only a few more days, to wait until at least one client has implemented the parallel batch I.O, because this will already give us, like, very, good numbers for, not only for the repricing work, but also for block lab access lists itself.
Toni Wahrstätter:Deciding if we keep the state locations or not.
Toni Wahrstätter:But apart from that, I totally agree. Like, the repricing work, is not blocking us here.
Barnabas:But there's also the parallel I.O. stuff is not a consensus change, so it can be just rolled out. Let's say we launch MNT2 on Wednesday, it could be rolled out on Thursday, if one client is ready there.
Toni Wahrstätter:Yeah, I'm fine with that too, yeah. I was just thinking that we… okay. Okay, this is good to know, so we can launch DevNet 2 on the 21st, and if clients finish BatchIO implementation after that, we could still integrate it, right?
Barnabas:Yeah, we just need to make sure the ball is rolling forward and not getting blocked on stuff that is not an actual blocker.
Toni Wahrstätter:Perfect.
Toni Wahrstätter:Yeah, then the question is, of course, like, will clients be able to, implement those four EIPs until the 21st, which, Karim and Jared were, yeah, pessimistic about?
Toni Wahrstätter:We can… We can, of course, delay the definite to get those four EIPs ready.
Toni Wahrstätter:I guess that's what we are.
Toni Wahrstätter:about to do, or is there?
Toni Wahrstätter:Or what do you think, Karim or Jared,
Toni Wahrstätter:What do you think about this?
Karim T. (matkt):I think I would prefer to delay a bit, honestly. We still have some stuff to finish.
Karim T. (matkt):For example, separation of the local faces and body is not completely merged.
Karim T. (matkt):And,
Karim T. (matkt):RPC method, we just started to work on it today. I don't know, maybe it will be ready for 21.
Karim T. (matkt):But, for me, it seems to be a really… smaller…
Karim T. (matkt):Period to finish everything correctly, but…
Karim T. (matkt):I don't know what the other clients think, but…
Jared Wasinger:Yeah, I agree with that. I would prefer to delay it.
Toni Wahrstätter:What was the last thing? You're for delaying it, or for not delaying it, and then we roll out?
Toni Wahrstätter:Any changes later into the dev.
Toni Wahrstätter:I guess you said delaying it, right?
Toni Wahrstätter:Okay, yeah.
Toni Wahrstätter:Yeah, I can totally see that.
Toni Wahrstätter:I would say, we can coordinate, on that async, especially, also with Stefan, what Stefan thinks about this, and see where we're at on the 21st next week.
Toni Wahrstätter:But, yeah, I think it's good for now.
Toni Wahrstätter:We can keep the 21st for now, and then see where we are in a week, and if it takes a few more days, we will just delay the launch of DevNetly2.
Jared Wasinger:Yeah, so like I, said earlier, I'm not,
Jared Wasinger:I'm not the one on the death side implementing the other EIPs that are being bundled into this, so it's hard for me to…
Jared Wasinger:give a firm commitment. I do think if the DevNet only… if DevNet2 only included the, the decoupling changes that were made,
Jared Wasinger:I think it would be… Perhaps a reasonable timeline?
Jared Wasinger:But yeah, I'm also for,
Jared Wasinger:I think as it's planned right now, it's… very…
Jared Wasinger:possible that we, I won't be able to make that deadline.
Toni Wahrstätter:Perfect. I think, everything's clear now.
Toni Wahrstätter:Then let's stick to that, we will…
Toni Wahrstätter:see where we are on the 21st, and if it takes a few more days, we will just delay DevNet too, such that we can have those four EIPs implemented on top of the bar Defnet.
Karim T. (matkt):Just, not sure how long it will take to have this EAP site, but maybe we should ask people that are working on it.
Toni Wahrstätter:Yeah, I quickly talked with Marios yesterday about those four EIPs, so Marios was saying the complexity coming from those EIPs is rather low. I think the ETH transfers is the most complex one, and the other ones are…
Toni Wahrstätter:Okay-ish?
Toni Wahrstätter:But of course, this is, only one data point. Yeah, we will have to see.
Karim T. (matkt):Because if it's just stealing too much, maybe it's just better to start without that, and…
Toni Wahrstätter:Yeah, I agree.
Toni Wahrstätter:yeah, let's see where we are next week. We have more clarity, I would say, and then we can take a decision on that.
Toni Wahrstätter:Great.
Toni Wahrstätter:Yeah, and this already brings us to the… to the next agenda item, which is what changed, with block access lists recently.
Toni Wahrstätter:As already said, the block lab access list was moved out of the execution block, and is now separate. It's still in the execution payload, so passed from the CL to the EL via the execution payload, so this didn't change, but it's not, part of the block body anymore.
Toni Wahrstätter:And this should definitely be something we roll out with DevNet2 already.
Toni Wahrstätter:Same for…
Toni Wahrstätter:What else do we have on the agenda here? Same for the JSON RPC methods. That would also be part of the scope of DefNet2 already. So, to summarize, moving the block-level access list out of the block, and the JSON RPC method.
Toni Wahrstätter:Those would be the changes, compared to DevNet 1.
Toni Wahrstätter:Is there any other change that I forgot now?
Toni Wahrstätter:Don't think so.
Toni Wahrstätter:Perfect.
Toni Wahrstätter:Yeah, looking at the agenda, then there's a… let me actually post the agenda quickly in the chat, such that everyone can have a look.
Toni Wahrstätter:As you can see on the agenda.
Toni Wahrstätter:Under specs, updates, and Clarification, there was a comment by LightClient that we should further clarify what we mean with access. This has already been a topic last week and, two weeks ago, or a few weeks ago in the last Block Live Access List call.
Toni Wahrstätter:Like, when do we actually put an account onto the block-level access list versus…
Toni Wahrstätter:Versus not, and it has to do with…
Toni Wahrstätter:doing all the static gas checks up front, and only when we actually need to go to state, we put the block level access list on… We put the account on the block level access list. And since this has never been
Toni Wahrstätter:Clearly specified, so clients could do different things here.
Toni Wahrstätter:we saw some issues with testing. I think most of them are resolved, but as you can see in the EAP, there is a new section further clarifying what we mean with access for each different opcode, and when we actually
Toni Wahrstätter:put the account into the block lab access list. So if you haven't seen that yet, please check that out. I think clients are already doing it correctly by now, because otherwise we wouldn't pass those tests.
Toni Wahrstätter:But just as a heads up.
Toni Wahrstätter:Great.
Toni Wahrstätter:Then, let's move on to the next item, which is testing updates.
Toni Wahrstätter:We already talked a bit about testing now, but Felipe, could you give us a quick update? Also.
Toni Wahrstätter:potentially focusing on DevNet2 already. Is there anything we should, consider for testing?
felipe:Yeah, yeah, we talked about the…
felipe:these consume RLP changes that are going to need to happen. I already… I created a thread in the block file-level access list channel.
felipe:That we should start discussing that, I think.
felipe:It'll be nice to get input from clients on that… on that front.
felipe:I'm best to…
felipe:to go about that new, consumer interface for ROP. But as far as a test update,
felipe:Related to this… This, this update.
felipe:For the gas boundaries.
felipe:There was an issue.
felipe:on the… on the DevNet that, was related to this for self-destruct, and we had included all of the gas boundaries for… for the call-off codes already.
felipe:In the tests, and this latest Test release has,
felipe:all… I believe all, the gas boundaries and all sorts of cases for self-destruct as well, because this was related to the issue. So that coverage got increased greatly,
felipe:And we also turned on all of the old Ethereum tests.
felipe:They used to live inside the Ethereum tests repo. They now live in the execution specs repo as static tests.
felipe:And these were all filled for Glamsterdam in this last release as well, and so we have greatly increased the test coverage, since the last release in this… in this one.
Toni Wahrstätter:Awesome, thanks for the update. Is there anything… any questions someone has, client teams have?
Toni Wahrstätter:Okay, I just thought a question by…
Toni Wahrstätter:BirdNet, do we need the nchain getBAs by hash V1, nChain get balls by range? For definite 2.
Toni Wahrstätter:And this is a good question.
Toni Wahrstätter:What do clients think about this? Do we already want to include that for DevNet2?
Toni Wahrstätter:I would default, by default, say yes, but,
Toni Wahrstätter:Happy to hear, other opinions.
Jared Wasinger:Can you just… shortly explain how those are used by the CL?
Toni Wahrstätter:Yes, so… Get lost by hashing one. Let me quickly think about it.
Toni Wahrstätter:So, recently, this is different, but recently, I've created a PR regarding DevP2P, because we need DevP2P for the syncing of block-level access lists. For example, if your node falls behind and you want to request the block-level access lists independently from the block, or if you just want to
Toni Wahrstätter:Use it to update your state.
Toni Wahrstätter:Regarding engine… get pulse by hash. This was basically,
Toni Wahrstätter:Doing the same as we do for blocks.
Toni Wahrstätter:So you would basically use it as you use it for blocks today. It would give you the same, and since blocks and block access lists were decoupled.
Toni Wahrstätter:We would need this… this function to actually allow… allow you to get the block level access list, in addition to the block.
Jared Wasinger:Okay, maybe I'm misinformed. I thought the… that the BAL was still part of the…
Jared Wasinger:execution payload, okay, I'll… I'll… I'll take a look at the, proposed changes.
Toni Wahrstätter:Great, yeah.
Toni Wahrstätter:Yeah, I would also have to double-check that. Maybe we don't even need them?
Toni Wahrstätter:DevNetly something to double-check.
Toni Wahrstätter:Someone has an opinion on that?
Toni Wahrstätter:on the engine API methods.
Stefan Starflinger:Is the implementation very different from implementing the RPC methods?
Stefan Starflinger:For the engine methods?
Toni Wahrstätter:It's definitely different, yeah.
Toni Wahrstätter:So, the CL is not storing the block lab access lists, right? The CL will just, like…
Toni Wahrstätter:store the block-level access list hash, the block-level access list root, just as they do with transactions, such that we don't have
Toni Wahrstätter:duplicated data.
Toni Wahrstätter:So, where you would need those engine API methods is if a node requests,
Toni Wahrstätter:the execution payload from you, so this, a CL node would request the execution payload from another CL,
Toni Wahrstätter:And the CL doesn't store the ball, so the CL would need a way to request the block level access list from the EL, and this is where
Toni Wahrstätter:Those functions were needed.
Stefan Starflinger:But from an implementation point of view, the ethG doc accesses lists by hash should be pretty similar to the engine get, values by hash implementation, just a different handler.
Toni Wahrstätter:Yeah, I can see that. That's true.
Toni Wahrstätter:I don't have an insight into how clients implemented it, but I can definitely see that those are very similar.
Jared Wasinger:So, but, like, the… aren't the… the BALs are still in the execution payload?
Jared Wasinger:that is…
Toni Wahrstätter:Right, yeah.
Jared Wasinger:to the CL, so they have it through there.
Jared Wasinger:And…
Toni Wahrstätter:They don't store them, though.
Jared Wasinger:isn't… Hmm.
Toni Wahrstätter:So what the CL does with transactions, transactions are stored on the EL, and the CL will only store the execution payload with the transaction's root.
Toni Wahrstätter:Right? And the same will happen with the balls. So because this is, like, too much data to be duplicated, the CL will only store that block-level access list root in the execution payload, and if the CL actually needs the full block-level access list, the CL would need to request it from the EL.
Jared Wasinger:Okay, Yeah, okay, so… so it's kind of similar to blobs, then?
Jared Wasinger:Er, yeah, not really, but… Hmm.
Jared Wasinger:Yeah, I mean, from… on the implementation side, it seems…
Jared Wasinger:it's pretty straightforward. I mean, we… we have the block access lists sitting around, so it's just a matter of…
Jared Wasinger:You know, loading them up and transmitting them, but, yeah.
Karim T. (matkt):Do we have a CL using that for the moment? If we implement it for DevNet2, is this API will be used, or will it just be there?
Toni Wahrstätter:I don't think so, I don't think we would use it for DevNet 2 already,
Toni Wahrstätter:Maybe… Stefan, do you think we would need it already?
Toni Wahrstätter:I don't think so, personally.
Stefan Starflinger:I'm not sure if we would, use it yet, because, yeah, like, the CRS
Stefan Starflinger:probably won't have that implemented, but I can talk to Lodestar. Maybe they, can do something about it.
Toni Wahrstätter:So I… Awesome, yeah.
Toni Wahrstätter:Yeah, go ahead.
Jared Wasinger:I guess the source of my confusion here is that, okay, so the access list is in the execution payload, but the CL…
Jared Wasinger:Doesn't store it, but under which… circumstances would the CL… need…
Jared Wasinger:A need to query the block access list again for a payload it had already received and discarded the block access list from.
Toni Wahrstätter:For syncing. I think the CL, if you want to help other node syncing, you would want to give them the full execution payload, including their transactions, and including their block lab access list.
Jared Wasinger:But isn't that… I thought we were… that… that's what…
Jared Wasinger:is… but, oh, okay, I see. So, that kind of…
Jared Wasinger:Presumes that, recomputing it locally is too… like, these nodes syncing are somewhat close to the tip.
Jared Wasinger:And recomputing the access list locally would be too slow to…
Jared Wasinger:sync, or maybe they're not full nodes, yeah, I don't know.
Toni Wahrstätter:Yeah, the seal cannot really recompute them, so the only thing that you could do is, if you don't store it on the EL, you could recompute it on the EL and then return it to the seal.
Toni Wahrstätter:This is, I guess, a different topic then.
Toni Wahrstätter:The use case I'm thinking of, if you fall, like, a few blocks ahead, behind, and you want to catch up again, then you need some…
Toni Wahrstätter:You need someone to feed you the blocks, and they would then include the full execution payload, including the full block access list.
Jared Wasinger:Gotcha. Yeah, that makes sense.
Toni Wahrstätter:Okay, perfect.
Toni Wahrstätter:M?
Toni Wahrstätter:We can also… we can also continue that discussion async to make sure we are all on the same page here.
Toni Wahrstätter:Perfect. Anything else regarding that topic before we move on to the next agenda item?
Toni Wahrstätter:Perfect, then let's move on with, balls and RPC providers. So, Josh from OP Labs, is… joins the call today.
Toni Wahrstätter:Josh, could you give us a quick, intro into what are you trying to achieve there?
Josh | OP Labs:Yeah, thank you. So, the basic context is that many RPC providers, and really anyone who
Josh | OP Labs:Operates a lot of…
Josh | OP Labs:nodes in their infrastructure may not need to re… with VALs, may not need to verify
Josh | OP Labs:the full block, if they can outsource that verification to another node in their infrastructure. So, the idea is that
Josh | OP Labs:Only a single node would actually need to re-execute the transactions at all.
Josh | OP Labs:And the rest can just act as dumb replicas, applying vowels as they receive them.
Josh | OP Labs:As long as they can outsource the verification to this other node. So, you know.
Josh | OP Labs:haven't thought too far about implementation yet, but the idea is that maybe the EL would just have a mode where it would just apply the bowel instead of doing the full re-execution.
Josh | OP Labs:And we were curious if, any Owen Klein teams had.
Josh | OP Labs:thought about this, if there's any, like, hard blockers to this idea that make it infeasible or not useful.
Josh | OP Labs:Yeah, and if this might be considered as something to be implemented in the future.
Josh | OP Labs:Yeah, great question.
Toni Wahrstätter:This is… this is almost like the execution-less, state updates, basically. Not executing, using the bowel to make sure your state is in sync with the… with the chain.
Toni Wahrstätter:I would just forward this to client. Have clients implemented anything around that, or thought of it?
Jared Wasinger:So are you talking about using bowels for, like, sort of, like, a stateless…
Jared Wasinger:Execution? I mean, I guess I'm… I don't fully understand, because you… you need the pre-state to do the execution. The information contained in the bowel alone isn't…
Jared Wasinger:Enough.
Josh | OP Labs:So, perhaps I'm misunderstanding, but…
Josh | OP Labs:the way I understand the ballots, essentially, includes enough information to update the pre-state to the post-state.
Josh | OP Labs:let me know if that's… if that's not correct. But if it does, then I think as long as we can apply the… the bowel, then re-execution shouldn't be necessary.
Jared Wasinger:Yeah, that's correct.
Toni Wahrstätter:Yeah, as I understood, Josh, it's not like… it's not about statelessness, it's more about
Toni Wahrstätter:running a node without executing. Basically, you would outsource execution to another node that you trust. The node will tell you this block is valid, which also means, at this point, you can trust the bul. And at this point, you can then just apply the ball to your
Toni Wahrstätter:pre-state, you get the post state, and you would run a very cheap RPC provider, right? Because you could run an RPC service that doesn't need any execution at all.
Toni Wahrstätter:And this is one of the things you can do with Bulse. We had it, at some point, we discussed it under the optimizations.
Toni Wahrstätter:I guess clients haven't really implemented that yet, but would be curious if… if there is a client team that has thought about it, or might be doing so.
Karim T. (matkt):In Besu, we have something, but it's not using block access list, it's using,
Karim T. (matkt):what we have in Bonsai, we have Trilog, it's something similar to Block Access. We… we have the…
Karim T. (matkt):It's a different modification we are doing in a block, and the idea of fleets to have a captain that is executing a block and sending this trailer to followers that are just applying states.
Karim T. (matkt):So it should be something similar to…
Karim T. (matkt):What you want to do is block access list.
Karim T. (matkt):Right.
Toni Wahrstätter:It sounds like it… sounds like you could just plug in block level access list into that workflow, and you're fine.
Karim T. (matkt):I think just in case of a reorg, for example, in Trilog, we have the value before the modification and the value after the modification, like that, if we have a reorg, we just have to
Karim T. (matkt):to roll back the trilog, and if we want just to import any block, we just have to roll forward the trilog, so as it's a B-directional diff, we can roll back and manage the reorg. With block access, we don't have that, so I think if we apply a block assist, we need to be sure that we not have a reorg.
Karim T. (matkt):Because if we have a rug, I think it would be a problem.
Toni Wahrstätter:I see, yeah. I can see that. So…
Toni Wahrstätter:In theory, you would also need to cache somehow the values that were,
Toni Wahrstätter:on state before applying a block-level access list, right?
Karim T. (matkt):I think yes, but regarding how we are working in Bezu, it's needed, maybe those are clients that have different storage formats, maybe it's not important, but for Bezu, it was important to have this value before the modification also.
Toni Wahrstätter:Thank you.
Toni Wahrstätter:Yeah, just relaying what Jared is saying in the chat. Jared is also saying this is very much feasible, and…
Toni Wahrstätter:can be done.
Toni Wahrstätter:Need to be worked out, as it's dangerous for it's not here.
Toni Wahrstätter:It's definitely more feasible for running an RPC service than doing something like this for testers or for validators. It would definitely cut the costs for RPC services, if they don't need to engage in any execution at all.
Toni Wahrstätter:So yeah, definitely something, to keep on our radar.
Josh | OP Labs:Right, thank you, appreciate the feedback.
Jared Wasinger:It's possibly that something that might be worth adding under, like, a protected… more protected…
Jared Wasinger:RPC namespace, maybe, like, admin or something, but yeah, it's…
Jared Wasinger:It's… yeah, it's very straightforward to do this.
Josh | OP Labs:That's very encouraging to hear. Thank you.
Toni Wahrstätter:Great, this was, the last agenda item we had for today. Is there anything else that we should discuss before we wrap up?
Josh | OP Labs:I guess, one last thing on this topic, is there a good place where we can continue this conversation? I know, Jared has already offered the idea of, like, potentially having an RPC method for this sort of thing. What's the best place to continue this discussion to make sure we can get follow-up?
Toni Wahrstätter:I would propose the Block Live Access List Discord channel for that, for now, at least.
Toni Wahrstätter:As soon as we get more concrete, I would,
Toni Wahrstätter:yeah, directly talk with the client teams, how far they are. This is not something that is, like.
Toni Wahrstätter:blocking the EAP from being shipped, so if clients, don't implement it until Glamsterdam, nothing bad would happen. But it's definitely one of those optimizations that are very much,
Toni Wahrstätter:That sound like they're very much worth it, and… Are worth to explore.
Toni Wahrstätter:So I would, for now, propose that Box Live Access to this Discord channel, And otherwise,
Toni Wahrstätter:the client's, discord channel.
Josh | OP Labs:Sounds great, thank you.
Toni Wahrstätter:Yeah, and also Banabas had a good idea here with the RPC standard call. So there's also an RPC standardization call that would be also a great venue for those topics. Agree.
Toni Wahrstätter:Thank you.
Toni Wahrstätter:Awesome. Any more comments before we wrap up? Anything I forgot that we should definitely mention?
Toni Wahrstätter:Then, to just quickly summarize, we will, we plan to launch the DevNet 2 on the 21st of January. We will see, as the date approaches, how far we are with the four EIPs that we want to put on top of the block of access list, DevNet.
Toni Wahrstätter:We will also see how far we are with the batch I.O. optimization. That would be very great, good to have in order to,
Toni Wahrstätter:Yeah, determine if we want to actually keep the state locations in the bar or not.
Toni Wahrstätter:And DevNet2 will then already include the JSON RPC methods, And also… The…
Toni Wahrstätter:the decoupling of the execution block from the block lab access list, storing the block lab access list separately.
Toni Wahrstätter:Awesome.
Toni Wahrstätter:If there is nothing else, I would say we can wrap up.
Toni Wahrstätter:An integral at this point.
Toni Wahrstätter:Perfect.
Stefan Starflinger:Ben, have a good day, everyone. Thank you very much.
Jared Wasinger:Yep, bye.
Marc:But…
felipe:Alright, thank you.
Chat Logs
00:03:49
jochem-brouwer:Agenda: https://github.com/ethereum/pm/issues/1857
00:03:55
Karim T. (matkt):The sound is not very good
00:04:11
Karim T. (matkt):yes
00:06:24
Stefan Starflinger:summarized here: https://notes.ethereum.org/@ethpandaops/bal-devnet-2
00:08:47
Boma’s iPhone :I have a question on pr 726 will Geth change the naming like to eth_get… instead of debug so to match the spec
00:10:23
Marc:on nethermind side haven’t worked on batch read, focused on parallel execution and state application atm
00:11:29
Barnabas:is this batch IO stuff defined in a new PR or its a “client implementation detail” and there is no spec for it?
00:11:55
Toni Wahrstätter:Replying to "is this batch IO stu..."
More a client implementation/optimization
00:12:24
Karim T. (matkt):It seems 21 for devnet will be too soon
00:12:55
bhartnett:For Nimbus EL we have just started work on parallel execution. Batch reads will likely come after that.
00:13:10
Barnabas:are we 100% blocked on those benchmarks for the other EIPs?
00:13:46
Dragan Rakita:Replying to "are we 100% blocked ..."
Good question, imo we are not, but not sure what others think?
00:13:57
Jared Wasinger:Replying to "I have a question on..."
Yes. I will include that change in the next devnet
00:14:01
Toni Wahrstätter:Replying to "are we 100% blocked ..."
For final numbers, yeah.
And we want to figure out if the state locations are actually worth it
00:14:07
Łukasz Rozmej:to be honest we need parallel execution and batch reading to assess it properly as parallel execution is somewhat parallel reading
00:14:31
Barnabas:Replying to "are we 100% blocked ..."
I don’t think we care about final numbers on devnet 2
00:15:05
Barnabas:Replying to "are we 100% blocked ..."
we should focus on implementing the numbers proposed in the EIPs, then if the values will have to be adjusted based on some benchmarking results we should make a change to the EIP, and roll out the change in the following devnet.
00:15:31
jochem-brouwer:How does the client import it? I think for RLP tests there is a block import flag? So this would add extra functionality because it would change the interface
00:15:39
Łukasz Rozmej:Do we have some good replayable BAL's benchmarks?
00:15:44
Barnabas:Replying to "are we 100% blocked ..."
BALs without parallel io shouldn’t be blocking any other EIPs to be included. If we want the final numbers for devnet 2 then we might need to wait weeks if not months. I don’t think its reasonable to get the proper benchmark results in a week.
00:16:34
Karim T. (matkt):Replying to "How does the client ..."
Yes it’s block import using flag . So we will have to change that
00:17:38
Karim T. (matkt):Do we need also state root in background as state root computation will also do some read in order to have intermediate trie node and impact IO
00:18:21
Łukasz Rozmej:Replying to "to be honest we need..."
it is useful but we could probably measure it separately, so not crucial
00:19:05
felipe:Replying to "How does the clien..."
Yes I think the interface will have to change. We will have to figure out the best way to do this for clients.
00:21:46
Karim T. (matkt):We need to use snapshot mainnet state and run this transaction on top of that
00:22:49
Karim T. (matkt):I think mainnet state is better we see different behavior on bloat net so it will not show us the behavior on mainnet
00:23:23
Stefan Starflinger:Whats the difference of having these benchmarks on devnet-1 vs devnet-2?
00:23:33
Barnabas:Replying to "Whats the difference..."
nothing 😄
00:23:38
Barnabas:Replying to "Whats the difference..."
Thats why my hand is raised
00:23:43
Stefan Starflinger:Replying to "Whats the difference..."
we can leave devnet-1 running for benchmarking
00:24:11
Toni Wahrstätter:Replying to "Whats the difference..."
We need the optimizations for the benchmarks which aren't implemented yet
00:24:18
Barnabas:Replying to "Whats the difference..."
we don’t
00:24:24
Barnabas:Replying to "Whats the difference..."
not for devnet 2
00:24:39
Barnabas:Replying to "Whats the difference..."
none of the proposed EIPs for the next devnet are repricing EIPs
00:26:04
Barnabas:https://notes.ethereum.org/@ethpandaops/bal-devnet-2
00:26:42
Stefan Starflinger:How do clients feel about merging the optimizations into bal-devnet-1?
00:27:32
Łukasz Rozmej:CFI is not SFI
00:27:39
Karim T. (matkt):Replying to "How do clients feel ..."
Not a problem for Besu side, we already have // and state root in background on the devnet-1 branch
00:29:09
Stefan Starflinger:Also if we keep bal-devnet-1 for optimizations testing is easier as the BAL is still in the block body
00:30:25
Marc:note on 7843 some small CL changes are needed so maybe would make sense to add in later devnet
00:30:31
Barnabas:well get to it
00:30:53
Barnabas:Replying to "note on 7843 some sm..."
let me check with lodestar if they would be happy to add it
00:31:32
Marc:Replying to "note on 7843 some sm…"
cool, I am also working on implementing and clarifying a few of the details around engine api changes
00:32:17
Stefan Starflinger:sounds good
00:32:56
jochem-brouwer:Antwoord verzenden naar "Do we have some go..."
For the storage slot bench test, proposal would be: txs split up where we SLOAD(0), where then loop storage is laid out in such way that each slot points to the next one e.g. 1->2->3->4->5. Loop does a lot of SLOADs, checks if gas left > some value (50k), if not the SSTORE(0, last slot), we init the loop as SLOAD(0)
Final value depends on how much SLOADs were done during these txs. All SLOADs are unique. Bench done on mainnet fork / bloatnet (2x mainnet size). Bench with both storage slots added to BAL and removed. Output is difference in BAL size, and the execution speed. Would give a good measurement to how worth this is.
Will not work if clients have some kind of greedy mechanism to load the storage of a contract from disk (since address is in BAL) and put it in cache
00:33:40
Jared Wasinger:What performance are clients seeing with BALs vs main net?
00:34:20
Jared Wasinger:Assuming people are replaying main net blocks to benchmark their bal impls
00:36:02
Toni Wahrstätter:https://github.com/ethereum/pm/issues/1857
00:37:46
Karim T. (matkt):Replying to "What performance are..."
Wanted to do some tests using a shadow fork of mainnet with tx pool mirroring but I was waiting for some tools. I think Barnabas Is working on it
00:37:47
Barnabas:I would really like to urge teams to have a dedicated person for performance improvements and a dedicated person for implementing glamsterdam features. These two improvements need to happen parallel and one can’t be a blocker for the other. If you are working on parallel io improvements, you shouldn’t be looking at SLOTNUM opcode implemention and vice versa.
We have too many EIPs to have a single dedicated person for perf & EIP implementation.
00:38:22
Jared Wasinger:Replying to "I would really like ..."
In the case of BALs, the two can’t really be decoupled
00:39:36
bhartnett:Do we need engine_getBALsByHashV1 and engine_getBALsByRangeV1 for devnet 2?
00:40:57
Barnabas:Replying to "I would really like ..."
In that case you probably need another person working on the other EIPs.
00:42:14
Karim T. (matkt):Yes the same I think I missed something regarding this APIs
00:45:32
Stefan Starflinger:Replying to "Do we need engine_ge..."
is this call used for syncing?
00:48:08
Stefan Starflinger:Replying to "Do we need engine_ge..."
The BAL is expected to be provided until the weak subjectivity period
00:52:10
Barnabas:Replying to "Do we need engine_ge..."
should we add these methods into https://github.com/ethereum/execution-apis/blob/main/src/engine/amsterdam.md ?
00:52:30
Barnabas:Replying to "Do we need engine_ge..."
I can’t seem to find a PR for this
00:52:56
Jared Wasinger:Very feasible to do this
00:53:08
Barnabas:Replying to "Do we need engine_ge..."
ah 727
00:53:10
Toni Wahrstätter:Replying to "Do we need engine_ge..."
There's a pr yeah
00:53:17
Toni Wahrstätter:Replying to "Do we need engine_ge..."
Ah great, you found it
00:54:18
Jared Wasinger:Replying to "Very feasible to do ..."
Iiuc the idea is to have an npc method to have a node blindly apply a BAL on top of the state without actually executing
00:54:56
Jared Wasinger:Replying to "Very feasible to do ..."
It can be done, but the specifics need to be worked out as it’s a dangerous functionality
00:55:41
Karim T. (matkt):If you want more info https://consensys.io/blog/besu-fleet-the-future-of-rpc-scaling
00:56:46
Barnabas:rpc standard call maybe?
00:59:23
jochem-brouwer:Great timing :)
00:59:27
jochem-brouwer:Bye all! :)
01:02:07
Fireflies.ai Notetaker Gustavo:Gustavo invited Fireflies.ai here to record & take notes. View Security & Privacy info: https://fireflies.ai/policy
Type:
'/ff leave' - Remove Fireflies
View Realtime notes here: https://app.fireflies.ai/live/01KESFHA5Z1RVTD9AV2P3PCGVR?ref=live_chat