Thomas Coratger:So, let's get started. I think that we have a couple of things to discuss today regarding the issue. So, it's post-quantum interrupt number 37.
Transcript
Thomas Coratger:So let's start with the client updates, Leem, first.
Gajinder Singh:Thank you, Jamal.
Gajinder Singh:So, I think we have moved to multiple subnet testing, and where we have a separate aggregator
Gajinder Singh:for each of our clients, and there, we are focusing on, optimization for Leem, so that Leem can be a good aggregator. Right now, we… we start lagging behind, and,
Gajinder Singh:That is where we are focusing and optimizing all our aggregator flows and all the gossip and interval flows, so that we have minimal
Gajinder Singh:shared locking across various threads, and we can parallelize as much as possible. So this is our focus, as well as to continue further doing improvements and testing on multiple subnets for different clients.
Thomas Coratger:The Grace and Prysm.
Shariq Naiyer:Sweet. We've been…
Shariq Naiyer:On the devnet 4… on the devnet 3 front, we have retired devnet 3 code, so all the feature flags for devnet 3 are removed, and all the scaffolding for devnet 5 is in the code, so… so we can start building out devnet 5 as it's being spec'd. On the devnet 4 front,
Shariq Naiyer:We have added the… as we are moving towards multiple subnets, we have added the attestation committee account into the CLI. I believe that's how all other clients are doing multiple subnets.
Shariq Naiyer:We were running into some issues as Partha raised with a timeout, when running on multiple subnets, so we increased the, lip2P timeout to 10 seconds, which… we earlier had it at 2 seconds, which worked fine for, single subnets, but we might be running into issues, on multiple subnets, so we have raised it to 10 seconds.
Shariq Naiyer:Apart from that, we… when… when starting… when starting off after Genesis and trying to sync with other clients, we're running into some issues, I think…
Shariq Naiyer:I think we're aggregating when we shouldn't, and we're getting a panic from Lean Multisig, so that's something we're fixing right now.
Shariq Naiyer:So that, that's mostly on the client, client side. We have, we've set up some internal tooling to run, multi…
Shariq Naiyer:to run, devnets with multiple subnets, so, I've been running… I've been running one on my Mac, but there's limited resources on my Mac, so I've only been running… running Ream, Leem, and Grandine. And it's seeming to work pretty good, but soon, we are gonna move that over to VPS, and, we too will start running devnets, so the…
Shariq Naiyer:the burden of learning doveness doesn't fall on a single person or two people. We would like to contribute to that as well.
Shariq Naiyer:As for testing…
Shariq Naiyer:Maybe I can share my screen here. I don't know if anyone else wants to present this, but let me… okay. So…
Shariq Naiyer:Sure.
Shariq Naiyer:Okay, so this is, this is what we have got going for testing.
Shariq Naiyer:I hope you guys can see my screen.
Thomas Coratger:Yep.
Shariq Naiyer:Okay, so… One thing I saw,
Shariq Naiyer:Is that… I think… I think multiple client… I think the client filtering on this page is not correct yet. I think, different clients are showing each other's tests. But that's something we'll fix. But basically, we have,
Shariq Naiyer:We have a wide variety of tests here. If you go by Suite, we have client interop tests. So this is something, like, I'm really stoked about, because you could go here, you could load this. We're gonna fix some filtering and some naming here, but as you can see, you could sort by this…
Shariq Naiyer:Oh, by feeling vibe.
Shariq Naiyer:So if you see here, you can see that Zeeem… Leem doesn't interop with Lintern.
Shariq Naiyer:Leem doesn't interop with Kuleen, Zeem doesn't interrupt with, Kuleen again, and Leem doesn't interop with Lintern. But Leem interrupts fine with Grandine, Guen, and Eats Lambda, and Leem.
Shariq Naiyer:So, this… I think this is very nice to know,
Shariq Naiyer:Although, although I'm not gonna say anything about any clients, because I encourage all the clients to see how their clients are set up in these tests, to point out any issues that we have while setting up these tests.
Shariq Naiyer:But, as we can see, you can now narrow down which clients interop with each other. And if a client doesn't interop with a certain percentage of clients, I think running those clients in the devnets is not a good idea.
Shariq Naiyer:So, so that's what we have on the testing side. There are other tests, as we can see…
Shariq Naiyer:Okay, I'm not gonna go super long on this, but as you guys can see, we have some… everyone has access to this, everyone can contribute to this, it's on neutral ground, please go ahead, contribute, please go ahead, check it out. Any issues, let us know.
Kolby ML:Yeah, and just one more thing. So, we do have,
Kolby ML:quite a few different test suites. For example, we have a request-response one, where we're basically just, like, mocking responses to a client, and we have a gossip sub one, which does the same. Those should be fairly helpful for, like.
Kolby ML:for clients finding, like, lower level issues. The client interop one and the, checkpoint sync one are a bit more, like, high level. Do these work at a high level? But for debugging, looking at the other suites might be more helpful.
Kolby ML:And then, hopefully later this week, we should have a lean spec test assets integration.
Kolby ML:There'll likely be a requirement from Teams to implement some kind of, like, a harness in their client, like, we can link into. But once that's done, that'll be a really nice interface for, like.
Kolby ML:Every time, like, for example, Tomas adds a new test to the spec,
Kolby ML:As fast as it can, it will just, like, run through all the clients, and we'll be able to really easily, like, visually see it, which will be super cool.
Kolby ML:Cause right now, every client, normally, I'm assuming we need to, like, update that themselves, generate the tests themselves, so to have, like, one view where you could just, like, see that as the tests, come online in the spec, will be really cool.
Shariq Naiyer:Anything, thank God.
Shariq Naiyer:Yeah, I think that's all from our side.
Shariq Naiyer:We will keep updating this, and we'll keep presenting this in a section on PQ and Trop, but yeah.
Thomas Coratger:Great results.
Thomas Coratger:That's amazing. Thanks a lot for the presentation. Clean?
Ruslan Tushov:Hello.
Ruslan Tushov:thanks for integrating Kuleen into the Hive. We made some changes for Kuleen to make it easier to integrate using Docker.
Ruslan Tushov:And we will look deeper into the results to find issues with lean. Also, we made a fork of lean multisig with
Ruslan Tushov:debug feature activated using environment variable, which allows us to control
Ruslan Tushov:it doesn't aggregate or make real aggregation, but instead it sleeps for the specified time and produces a proof of specified size. So…
Ruslan Tushov:It should be possible to use this feature to run the test on low… resource machines.
Ruslan Tushov:Because segregation wouldn't consume too much resources.
Ruslan Tushov:Thank you. That's all.
T. Wambsgans:Sorry, could you please repeat, the purpose of the feature you implemented, please?
Ruslan Tushov:You know, we forked Lean Multisik.
Ruslan Tushov:and, specified in Kuleen, and as an example, other clients, Using this fork.
Ruslan Tushov:We patched the functions verify and aggregate so that they
Ruslan Tushov:don't perform real aggregation, but instead just return some bytes of specified size.
Ruslan Tushov:specified time.
T. Wambsgans:Thank you, guys.
Ruslan Tushov:I can share a link again in the chat.
Shariq Naiyer:And that's for your internal testing, I'm assuming.
Ruslan Tushov:Yeah, just for testing, because, as I said, when I was testing on the server, it was too slow for… to run many aggregations on the same machine, and,
Ruslan Tushov:If my personal computer is busy, it also…
Ruslan Tushov:Performance drops and finality slows down.
Ruslan Tushov:But using this… Stuff. It helps to run the test.
Thomas Coratger:Great.
Thomas Coratger:Thank you.
Thomas Coratger:London?
Mihir Faujdar:Yep. Thanks, Thomas.
Mihir Faujdar:Hi, everyone. So yeah, basically, since last week, been working on rewriting the SC library.
Mihir Faujdar:and the lip P2P library, with a better design. More recently, I've been looking at implementing optimizations for aggregation verification, and improving how locks are used, when Lintern receives HTTP requests for,
Mihir Faujdar:justification, finalization, when Lintern is a light aggregator.
Mihir Faujdar:This is to address the issues that were raised by Partha in the Telegram channel. On the specification side, I have submitted a PR to add more 3 test vectors, to improve test coverage. Currently, the PR is only really based on PR682, but,
Mihir Faujdar:if PR638 is also merged, I'm happy to rebase as well, or if there's any other feedback, I can adjust the PR. And also, yeah, it's been… it's really great to see the Hive tool from the Reeve team, as our focus will be to
Mihir Faujdar:improve upon the current test failures, happening for Lintern. So yeah, that's our update. Thanks.
Thomas Coratger:Great, thanks a lot for the PR on LeanSpec. I will merge it first, before the other ones about the multiforcings, but I will discuss that just after.
Thomas Coratger:East Lambda?
Pablo Deymo | Lambda:Hello.
Pablo Deymo | Lambda:We made a PR on LeanSpec.
Pablo Deymo | Lambda:That was… that this already emerged. We… changing the checkpoint scene less the latest justified and finalized, to fall on the beacon chain, spec. That's to improve,
Pablo Deymo | Lambda:safety of this, function, CheckMeSync.
Pablo Deymo | Lambda:We also changed our safe target computation to take only… take into account only new votes.
Pablo Deymo | Lambda:Following the LINSPECTR680.
Pablo Deymo | Lambda:And designated the station feature… future slot bound, following the 682 PRO Flyn spec.
Pablo Deymo | Lambda:And on the other side of things, we added some new laws.
Pablo Deymo | Lambda:To debug the number of peers we are connected to.
Pablo Deymo | Lambda:And we were testing devnet4 run, And, like, stress testing it.
Pablo Deymo | Lambda:To find issues.
Pablo Deymo | Lambda:So… That's our update.
Thomas Coratger:Thanks a lot. Lighthouse?
Thomas Coratger:Okay, grinding.
Mercy Boma Naps-Nkari:Good afternoon. So, we are currently debugging an out-of-memory issue caused by machine attacks running at CPU limits, so we found out that Tokyo, isn't really a good solution for handling CPU-intensive tasks, so we are switching to a more dedicated executor.
Mercy Boma Naps-Nkari:And, we're hoping by the end of today, or tomorrow, we should be done with it, and then give an update.
Mercy Boma Naps-Nkari:If we didn't see any other chain tags that needs to be moved to the dedicated executor.
Mercy Boma Naps-Nkari:Thank you.
Thomas Coratger:Great. Thanks a lot.
Thomas Coratger:Guin?
Thomas Coratger:No one from Guin Nimbus.
Thomas Coratger:Okay,
Thomas Coratger:So, yeah, I have a couple of other discussions that we should handle during this call. I will share my screen.
Thomas Coratger:In order so that we have a view of the… of the issue that we… I published yesterday.
Thomas Coratger:on GitHub for this call,
Thomas Coratger:So, yeah, just about, devnet 4.
Thomas Coratger:I have summarized, like, the findings that you guys published yesterday on the devnet4 channel, where you launched a bunch of subnets, and face stuff, and all of this. Okay, so…
Thomas Coratger:That is great, like, to find this debugging system.
Thomas Coratger:I have published, like, such a message in the conversation right now, but please, if you find something that's…
Thomas Coratger:is due to a lack of test coverage, or is due to a bug maybe in the spec or something like that, please submit an issue directly on the spec with a bug report or any logs that you have.
Thomas Coratger:So that we can analyze it, and have test coverage for the spec. I am not sure this is the case right now for the…
Thomas Coratger:issue that we have. I don't know if this is an issue, a debugging issue between the clients, or if this is due to maybe a lack of test coverage. Not sure, but if this is the case.
Thomas Coratger:please let us know with even, like, a row, log report directly in LeanSpec, that would be helpful.
Thomas Coratger:So, concerning the spec of devnet 4, as mentioned, we have had a lot of new test vectors, and also, like, interrupt debugging stuff.
Thomas Coratger:If I go to LeanSpec right now, we have a much richer, suite of tests.
Thomas Coratger:there.
Thomas Coratger:So before we had, like, just fork choice and chain stuff, but now we have also networking, even post-quantum and one. I am not sure that you should get all of them, depending if you…
Thomas Coratger:rely on an existing library. For example, if you rely on a Plonky3 library, maybe just post it on test.
Thomas Coratger:are not that much important, or you can add them, but maybe not priority. But I have tried to add a lot of tests, even for SSZ, for networking, for state transition, for syncing, so that we can have all of the bases.
Thomas Coratger:so, yeah.
Thomas Coratger:about
Thomas Coratger:devnet 5. Let us discuss that a bit. So, I have summarized there mostly the message that I sent on Sunday.
Thomas Coratger:So, just to summarize where we are today, so, we have this proposal that is a HackMD for devnet 5, submitted by the Zoom team.
Thomas Coratger:So right now, so we have, like, these post-quantum hash-based signatures, they are large, and they dominate the proof.
Thomas Coratger:For devnet 4, sorry, we applied, like, this on the per annotation data basis, capped at a maximum of 8 at Ascension data. So this means that the worst case scenario is 8 aggregated proofs plus 1 block signature.
Thomas Coratger:But what we want to add for devnet 5 is to fold all of the 8 per attestation data aggregates and the block signature together into a single proof that we will post on the block.
Thomas Coratger:On-chain, so that we get closer and closer to the final setup, where we will have just single proof on-chain.
Thomas Coratger:And just… then, if we verify the block, it's, it is equivalent, it will be, equivalent to verifying just one stock that proves everything.
Thomas Coratger:So this is, of course, like, a large size reduction, because we… we go from 8 to 1.
Thomas Coratger:But there is an important stuff, is that we are losing information by doing that. I am just doing a remainder so that everyone is on the same page.
Thomas Coratger:Because the fact that we fold something A, so a proof A and a proof B, into a single proof AB, this destroys, technically, the information A and B,
Thomas Coratger:Because they are separate objects, and so this breaks block production, because the next proposer will not have anymore the information of the… of the previous block.
Thomas Coratger:On the attestation data, because you will just see a single proof on-chain.
Thomas Coratger:So, what we mentioned as a solution for this is to have a tree.
Thomas Coratger:As the leaves of the tree, we could have, like, the individual proofs, then we could have intermediate aggregates, and finally the block proof as the root of the tree.
Thomas Coratger:And this way, we have, like, the smallest instance that should be published, being the root.
Thomas Coratger:And…
Thomas Coratger:This has, like, no real usability property in the next blocks, because we have no information coming from that. But if we publish root plus per attestation data.
Thomas Coratger:Proposer can extend things and see information from previous block.
Thomas Coratger:And so, I wanted to discuss a couple of points in this talk right now. So, how do we handle propagation intermediates? We discussed in the chat about potentially, like, a side gossip channel.
Thomas Coratger:where we can… so, on the main one, we publish the routes, and on the side, Gossip Channel, we just publish the attestation data intermediaries for the proposer who wants to access this data for future block.
Thomas Coratger:So that is one thing. Then also the recursion cost, because
Thomas Coratger:As we will do these recursion things with the tree, we will have, of course, more recursion and more proving cost, because we will have this tree. And so, we need to be sure that this will not impact peer-to-peer and stuff like this, and that this will be okay in terms of, stuff.
Thomas Coratger:I think that Mercy in the chat proposed a mitigation for this.
Thomas Coratger:That is to do something in parallel, because if we have 8 leaves for the tree, we can reduce that by doing 4 things in parallel.
Thomas Coratger:Two… four things in parallel with the.
Anshal:or can I interrupt you in just a minute? So, I think, like, this is, like, an older design, and based on the discussion that we had today on the channel, it is… it seems like Emile says that it's possible to disintegrate these proofs.
Anshal:Maybe we can't get A and B back. Turns out, like, it is possible to get A and B back, which solves most of our, issues that I had described in that doc. Apologies, I couldn't update that doc, but yeah, I think, since, like, we are able to disintegrate the aggregated… multi-message aggregated proof, I think,
Anshal:We'll be able to, we'll be able to, like, produce blocks, even when we get a fork of it, and we have, like, a cumulative proof.
Anshal:out of that. So, it is like a.
Thomas Coratger:Yeah, this is this one, right?
Anshal:Okay, okay, yeah.
Thomas Coratger:Yeah, yeah, I guess this is this one. This is, like, the latest discussion. Maybe Emil can confirm that, but my understanding from the latest discussion that we had is that we are able to recover the stuff, correct, Emil?
Anshal:Yes.
T. Wambsgans:Yes.
Anshal:able to recover.
T. Wambsgans:It's explicit.
Anshal:So I was saying that if we are able to recover it, we won't have to do this partial formation and the binary solution that I had earlier proposed, and that way we won't enter into the propagation issue that we had the discussion. She has also suggested some proofs around, workarounds around that.
Anshal:It was funny.
Thomas Coratger:Just to check the… Just email, like, to check that we are on the same page.
Thomas Coratger:So…
Thomas Coratger:you… you said in your message that we have, like, we can decouple the message into different types. Type 1 is, the same message slot things.
Thomas Coratger:And Type 2 is the fact of bundling N different type 1 proofs, each covering different message slot stuff.
Thomas Coratger:And so what you said inside your message is that, for example, if I have a proof of message 0, slot 0, and message 1, slot 1,
Thomas Coratger:we can recover.
Thomas Coratger:two separate proofs that are proof of message 0, slot 0, and proof of message 1, slot 1. Is it what you said?
T. Wambsgans:Exactly, correct.
Thomas Coratger:Yeah. And so, with this, We will not need the binary tree stuff that we initially thought about, correct?
T. Wambsgans:About this, I don't know, but, maybe Angela can answer, but…
Anshal:Yeah, we won't need it. It was, like, just a primitive solution that,
Thomas Coratger:Yeah.
Thomas Coratger:Okay.
Anshal:we had thought about, but, I think what Email has suggested makes our lives easier, so…
T. Wambsgans:Okay, and perfected. And if it's, like, if it seems to work for everyone,
T. Wambsgans:We can try to have an implementation ready for next week with this type 1, type 2, I believe. It's probably not very hard to do on Lin VM right now.
Thomas Coratger:Okay, this is perfect. So…
Thomas Coratger:Then, if we go to this solution, but you said also in your message that there is, like, up to 2 slots worth of attestation data. Is it still correct, Emil?
T. Wambsgans:I believe with this solution, we can make it work with an arbitrary number of pairs of slots and messages, I believe.
Thomas Coratger:Okay, okay, okay, okay.
Thomas Coratger:Okay, so perfect. So, if we go to this solution, this will just let us with this one, and so we can just remove everything related to the tree and stuff, because we don't need the tree anymore for this.
Thomas Coratger:So okay, so that is perfect, and we can work on this, for next week.
Thomas Coratger:The second item that I have seen popping up in the chat is the fact of adding a PQ heartbeat.
Thomas Coratger:as well, in parallel to this feature in the next devnets, which is devnet 5. So, if I summarize it properly, this is an item from the roadmap, and so if we look at the roadmap.
Thomas Coratger:that is here, we should have, like, the PQ heartbeat, that is there.
Thomas Coratger:Oops.
Thomas Coratger:Okay, so, yeah, PQ, RBIT is something… oops.
Thomas Coratger:is there, so it's in Glamsterdam fork.
Thomas Coratger:And so, if I summarize it, the PQ heartbeat is that we have a PQ,
Thomas Coratger:we have PQ attestation, but we don't use, we doesn't use aggregation yet.
Thomas Coratger:Because in the new fast finality contentious rules, we will have, like, a decouple protocol with, one aspect being, like, the hard bits.
Thomas Coratger:So, continuously producing block, even under some adversary condition, and a finality gadget on top of it to produce finality.
Thomas Coratger:And so, in the store map, the first step of this was to introduce, like, just the post-quantum Earthbeat on top of everything else.
Thomas Coratger:And so this is something that we would love to test in devnet 5, probably.
Thomas Coratger:So I think that, Jan and, Gadjinder, you proposed this, so maybe you want to say a word about this, but I think that in the spec, this is just equivalent to adding, like, the goldfish, gadgets that is, like, the…
Thomas Coratger:the protocols that we want to use on mainnet, and just run that with PQ heartbeats, even under many faults or crashes or stuff.
Thomas Coratger:I guess this is what we want there.
Gajinder Singh:Yeah, so before Jan speaks, I just want to say something on this, so that maybe Jan can correct if my understanding is wrong. So, PQ Heartbeat is basically an availability protocol, and it doesn't need to be in PQ, but since we are doing PQ, and that is something that we want
Gajinder Singh:to, contribute to the main, beacon consensus as well, so this is something, I think, that we can take up as a focus. So, in that sense, it is an availability protocol, as per my understanding, and what
Gajinder Singh:It will replace in, in our lean consensus is that, the way we calculate,
Gajinder Singh:the SafeTarget, because SafeTarget right now uses new aggregated payloads, and instead of new aggregated payloads, basically, we will be able to use these,
Gajinder Singh:PQ votes that will be basically, be based on, our committee, which will be randomly selected on every slot.
Gajinder Singh:And on the basis of that, we… since these payloads will not be aggregated, doesn't need to be aggregated, and since they are not even packed in the block, we can just quickly use this
Gajinder Singh:to calculate the safe target and parallelize some of the competition. So, this is my understanding, and I hope,
Gajinder Singh:Jan can further add on to it and clarify If there is some gap.
Yann Vonlanthen:Yeah, hi everyone.
Yann Vonlanthen:Yeah, I think that… yeah, you both said it very well. I think, from my perspective, the…
Yann Vonlanthen:Or the reason why we thought it could be a good idea is just… To simplify, hopefully, like.
Yann Vonlanthen:you know, instead of… I mean, it could be a feature, but it could also just be, like, a simplification.
Yann Vonlanthen:Of the consensus protocol.
Yann Vonlanthen:Ideally. But I mean, yeah, you'd have to tell me how much it simplifies things.
Yann Vonlanthen:But essentially, the way I see it,
Yann Vonlanthen:If we have… if you have only the, you know, this heartbeat as consensus.
Yann Vonlanthen:You don't have to, yeah, you don't have periods, like, the periods of non-finality, they still give you
Yann Vonlanthen:Yeah, the heartbeat can deal with that.
Yann Vonlanthen:You don't have to, you know, restart your… your devnets.
Yann Vonlanthen:It's designed to deal with that.
Yann Vonlanthen:And yeah, it could be… I guess I wanted to ask as well, what type of…
Yann Vonlanthen:Of heartbeats, you know, quote-unquote, are you using now?
Yann Vonlanthen:Like, I guess I don't even know what FreeSF Mini
Yann Vonlanthen:specifies, but my understanding is that you already have some sort of heartbeat, right? So…
Yann Vonlanthen:Maybe it could be…
Yann Vonlanthen:Yeah, either keeping this one, or changing it to Goldfish, so that it would be closer to what we envisioned for… for mainnet.
Gajinder Singh:So, we don't explicitly have a heartbeat, but basically, I think what 3SF does is… is using, new votes, new aggregated payloads, as the availability protocol, so they are acting like
Gajinder Singh:heartbeat as well. But yes, so they… so they are like, okay, once… once that estration happens, then the aggregator aggregate it, and then, it gets… it is available to…
Gajinder Singh:to all the validators to use it to calculate, the safe target, so…
Gajinder Singh:But in my understanding, we… a validator will not have to wait
Gajinder Singh:For aggregated new payloads to come in as a heartbeat, and we will have this as an independent thing that can be used to figure out, okay, this is the safe target that you can vote on.
Yann Vonlanthen:Right, yeah, in that case, it would be, like,
Yann Vonlanthen:An additional votes type, added on top that is not… not aggregated.
Thomas Coratger:Yeah, definitely. We can add Goldfield for this,
Thomas Coratger:So, okay, so on the devnet 5 staff, I think that we can, play on two different staff with, this PQ heartbeat stuff with Goldfish.
Thomas Coratger:And with the fact of meta-aggregating the proof into a single proof for the block, so we can work on these two items in parallel for devnet 5.
Thomas Coratger:I will try to ingest that and create small sub-issues in the spec, so that we can progress on this front in order to implement that.
Thomas Coratger:In the spec.
Thomas Coratger:Okay, so that is about, devnet 5 things. Does anyone have something else to add about devnet 5, potential stuff?
Thomas Coratger:Question… Concerns?
Thomas Coratger:Anything?
Thomas Coratger:Okay, great. So, about the testing infrastructure, this is perfect. This has already been presented by the RIM team, so as we discussed.
Thomas Coratger:like, we wanted to present and to display nicely the tests that we have in the lean spec with the different clients, so that we can have a nice visualization of
Thomas Coratger:which tests are passed by which clients, and we have this Lean Roadmap Hive website that is live, and where all of these test vectors will be integrated soon, so that is a nice new.
Thomas Coratger:So right now, we have a bunch of tests, but not the ones from the inspect test vectors, but they will be integrated soon, like this has been just presented by the Ring team, so we had this demo feedback and stuff, so…
Thomas Coratger:please, be sure, like, to comment. We have a dedicated testing channel on Telegram, so if you want to have comments, concerns, or stuff.
Thomas Coratger:Don't hesitate.
Thomas Coratger:Then, like, the thing about a lean spec purely, we are doing the multi-fork architecture stuff.
Thomas Coratger:So, we already have, like, a PR that is there. We worked on this internally, and so this is really for review from your side, so please guide, like, when you have a bit of time, check this out.
Thomas Coratger:And let me know if you have any concern, question, or stuff. Basically, as we discussed last week, we created, like, a specific L star folder where we put all of the consensus-critical stuff.
Thomas Coratger:And we play nicely with this, inherence framework so that we can have, like, the fork protocol and, like, subsequent forks, depending on,
Thomas Coratger:on L-star folk in an inherent manner.
Thomas Coratger:So, we worked on this internally with the Steel team and Content U-Spec team.
Thomas Coratger:They are pretty happy with it right now, so please check it out and let me know. I will merge it as soon as you guys approve. This is not a change of spec or something like this, just,
Thomas Coratger:Moving files, but nothing changed in the spec. No comments changed, and nothing specs changed.
Thomas Coratger:And yeah, we opened two, new, issues.
Thomas Coratger:That has been surfaced by, Mercy. We had, like, some service duty stuff that was, like, a problem, and we have a missing block-by-range feature, for the syncing.
Thomas Coratger:So yeah, we are… both of these issues are already assigned and currently work in progress.
Thomas Coratger:So, that is about, the spec.
Thomas Coratger:So, again, like, when you have time, please check this out. Then we have also this tracking issue that has been created by Leo from the Steel team, with a bunch of following stage, in order to go to something fully, fully
Thomas Coratger:generic.
Thomas Coratger:We will try to start on this just after it has been merged, just after the first one has been merged, so that we can decouple that into small issues.
Thomas Coratger:And tackle them, step by step.
Thomas Coratger:So yeah, that is one thing. Maybe, like, Emil, you want to add, like, the progress you made on LeanVM? Just before, like, we merged a bunch of PRs on Plonky3, including, like,
Thomas Coratger:wheel, that is now fully integrated into Plonky3.
Thomas Coratger:So that,
Thomas Coratger:External project has full control over it, so we can have more external contributors, more optimization, more testing, more tooling, and more ease into the code.
Thomas Coratger:So, we integrated that, and with some follow-up PRs coming. Right now, we are working a bit on the ZK, so the zero knowledge property integration inside Wheel, so that's… it's maybe, like, Emil, you want to add some stuff about LeanVM.
T. Wambsgans:Yeah, we made some good performance progress, and in particular, excellent performance improvements, with the help of Barna, who landed a Pierre, that implements a new memory allocator.
T. Wambsgans:And which brings, like, 25% improvement on, like, small, head snares,
T. Wambsgans:environmental servers, which is very good. That's it, thanks.
Thomas Coratger:Great, maybe some stuff about metrics, then? Just about metrics, right now we have, like, this visualization website that is a work in progress for the testing, where we want to display all of the testing.
Thomas Coratger:I think that once this is set up and fully available, the next step
Thomas Coratger:could be to have equivalent websites, to display metrics, so that we can track the metrics, devnet by devnet, and not just having them shared within the Telegram channel, but also, like, maybe a live, nice website where we can check the metrics.
Thomas Coratger:And see what is happening in devnet after devnet, but maybe an update from Katya or Pata about the metrics.
Katya:Hello, everyone. So, from the metrics side, our Lean Grafana has been updated to version 13, so you might notice some changes. There were a tab added in the new version, so currently.
Katya:I'm working on the… some updates of the dashboard. You might have noticed that
Katya:Docker images info and resources, they migrated to the specific tabs on the same dashboard. I will share the screen in the chat after the call.
Katya:We're also planning, to add,
Katya:New metric, for… for…
Katya:monitoring the intervals, which was helpful, for Leem team, and, I will… I will share the draft PR in the tooling as well, so, just for discussion.
Katya:Regarding, your,
Katya:suggestion, Thomas, about, yeah, we can build it on top of, for example, metrics, repo, or separately, yeah, sure.
Katya:Oh, I don't mind, we can discuss. Thank you.
Katya:Probably Parsa will share the 10th for updates.
Thomas Coratger:Maria, I'm not sure we can hear you well.
Parthasarathy Ramanujam:Am I audible now?
Thomas Coratger:Yeah, now I'm much better.
Parthasarathy Ramanujam:Yeah, I was saying, following, the testing that was done yesterday with two subnet, devnet, it,
Parthasarathy Ramanujam:I had to toggle each client to switch over from a regular validator to an aggregator. The chain did progress well, but we did discover quite a few issues, which I reported on the Telegram channel.
Parthasarathy Ramanujam:I'll continue to do that, so you would, be probably seeing, clients that were,
Parthasarathy Ramanujam:probably started off as a regular validator, suddenly switching over as an aggregator to, I mean, to facilitate or aid what is the current status of a particular client. I've updated lean,
Parthasarathy Ramanujam:a lean point to report the correct status, so you would be able to view that there. I don't think this can be captured in Grafana right away. That would only report what was the status of the client at the time of the Genesis or start.
Parthasarathy Ramanujam:So, yeah, I'll keep you guys posted on how the testing goes.
Thomas Coratger:Okay, great.
Thomas Coratger:Thanks, thanks a lot.
Thomas Coratger:do we have something else to discuss? Some stuff? So, if I summarize what we have just said today, for the next week, we will debug devnet for stuff, maybe add new tests and everything like this.
Thomas Coratger:probably, like, the multi-fork PR will be merged on LeanSpec, and in the meantime, we will create some small issues to tackle devnet 5 feature.
Thomas Coratger:Yamasi.
Mercy Boma Naps-Nkari:Yeah, sorry, I'm sorry if I'm dragging the conversation backward. There was a question by you on how,
Mercy Boma Naps-Nkari:this please prove how long it takes. I don't know if that has been answered.
Thomas Coratger:So…
T. Wambsgans:what I mean?
Thomas Coratger:I…
T. Wambsgans:I think it would be, like… And…
T. Wambsgans:it should be okay. Think of it as similar to, currently, like, recursive verification, and when you want to split, you would basically have
T. Wambsgans:to verify recursively one LeanVM proof.
T. Wambsgans:pair Type 1 signature that you want to extract from the type 2 signature.
T. Wambsgans:So… My intuition is that this multi-signature, Setup is not to baseline.
T. Wambsgans:And, like, if there is some… if it's, like, 2 weeks slower than the… Baseline,
T. Wambsgans:set up.
T. Wambsgans:Is it okay, or should we have optimal performance, In this multi-message scenario.
T. Wambsgans:Don't do…
Thomas Coratger:For me, it looks okay. I don't know if, like, other clients have some stuff to add, but for me, it sounds okay. We don't need, like…
T. Wambsgans:Yeah, and anyway, it's see, like, at least it works, we can make it, we make it work for the next 5, and if it turns out we need more performance and more granular iteration, we can improve on devnet 6, but at least for the.
Thomas Coratger:Yeah.
T. Wambsgans:Probably the simplest way to start.
Thomas Coratger:Yeah, definitely.
Mercy Boma Naps-Nkari:Okay, so this is something we should actually, like, keep in mind in case of any regression, or… so that we'll be able to track it.
T. Wambsgans:it shouldn't make any downgrade performance on the standard Type 1 signatures. Like, if you keep it with single slot, single message, it will be as fast as today.
T. Wambsgans:Beautiful.
Mercy Boma Naps-Nkari:Okay, thank you. That clarifies it.
Thomas Coratger:Boomer.
Thomas Coratger:Great. Do we have any other updates, comments, or discussion?
Thomas Coratger:And yeah, I have seen, like, in the chat that we have this discussion with O about, like, the observatory, lean stuff. I remember that this was presented, and this seems nice because this is a daily job, so…
Thomas Coratger:Yeah, okay, so for now we can use that as a visualization tool. This seems super nice.
Parthasarathy Ramanujam:Can I also have a live check? Just wondering if all teams are in agreement that they will look at Grafana logs for devnet runs daily, rather than all of us having to, I mean, share these as a zip file on Telegram groups?
Parthasarathy Ramanujam:Is everyone okay with that?
Thomas Coratger:Yeah, you mean, like, you mean that people check this observatory tool instead of you checking… sending the zip file?
Parthasarathy Ramanujam:That's correct, yeah.
Thomas Coratger:Yeah, yeah, yeah.
Katya:Not exactly Observatory, but, Grafana Locky Locks.
Parthasarathy Ramanujam:step, huh?
Thomas Coratger:Yeah.
Thomas Coratger:Do we have the link for this?
Shariq Naiyer:Yeah, so it's at metrics.leanroadmap.org, and I think everyone… The client teams have an account and password, for that.
Thomas Coratger:again.
Thomas Coratger:Okay, great.
Katya:So, Grafana will have an update soon where you will be able to download the logs, so I will
Katya:As soon as they release it, I will test it and notify you that you can download logs yourselves.
Katya:In case you need to download.
Thomas Coratger:Right? Anything else?
Thomas Coratger:Then we can wrap up. Thanks a lot, everyone, for all the comment and discussion. This has been super helpful, and see you next week.
Thomas Coratger:Bye-bye.
Gajinder Singh:That way?
Yann Vonlanthen:Never mind.
Mihir Faujdar:Thanks, everyone. Bye.
Anshal:Thanks, Andrew.
Shariq Naiyer:Have a good one, guys.
Chat Logs
00:11:42
Gajinder Singh:this is very useful, great work!
00:22:22
Gajinder Singh:in 2 subnet devnet system, zeam and ethlambda are aggregators on separate subnets. - just a small nit
00:23:02
Gajinder Singh:aim is to have each client as an aggregator on one subnet - but multiple subnets in parallel
which should be next step
00:29:20
Gajinder Singh:yes with "deconstruction" things look good to me to repack attestations
00:33:19
unnawut:Do we have some rough idea the time it takes to split a proof? Is it similar to the original aggregation time?
00:34:23
Mercy Boma Naps-Nkari:Replying to "Do we have some roug..."
Benchmark should also be helpful
00:38:24
Parthasarathy Ramanujam:So the chain will not be stalled in case an aggregator of a subnet is down?
00:38:59
Gajinder Singh:Replying to "So the chain will no..."
justification will stall but not the targets
00:39:11
Parthasarathy Ramanujam:Replying to "So the chain will no..."
Ah ok. Noted.
00:41:02
Yann Vonlanthen:Replying to "So the chain will no..."
Exactly, running only the PQ heartbeat would remove justification / finalization, instead the heartbeat gives a confirmation signal, that works even if a majority of clients is offline.
00:43:18
Gajinder Singh:Replying to "So the chain will no..."
we will put up the spec PR for this
00:46:39
Gajinder Singh:Replying to "So the chain will no..."
this will solve the target regression issue quite a bit cc @Anshal @Parthasarathy Ramanujam
00:46:44
unnawut:@Thomas Coratger In addition to grafana, is this something you might be looking for? https://observatory.leanroadmap.org/
00:48:18
Thomas Coratger:Répondre à "@Thomas Coratger I..."
Yeah exactly what we want, is it constantly maintained and up to date with latest devnets? I remember you presented this but not sure if this was an experiment or something designed to be maintained long term
00:48:58
T. Wambsgans:+ and implement type1 type2 in leanMultisig
00:49:08
Thomas Coratger:A réagi à "+ and implement ty..." avec ❤️
00:49:31
unnawut:Replying to "@Thomas Coratger In ..."
Yeah there’s a daily job that crawls data from the prometheus instance that Katya/Partha setup
00:50:33
Gajinder Singh:Replying to "@Thomas Coratger In ..."
we/Katya/Partha is also working to have zclawz (our bot) put AI summaries of current devnet status
00:52:53
T. Wambsgans:I have to go, bye everybody
00:53:16
Parthasarathy Ramanujam:metrics.leanroadmap.org