Ethereum Protocol Fellowship (EPF) Cohort 7 — Applications open until May 13

PQ Interop #035

2026-04-15 Agenda: #2013 canonical JSON

Transcript

00:13:20
Thomas Coratger:So, yeah, Will cannot make it this week, so let's start.
00:13:28
Thomas Coratger:Let's start first with client updates.
00:13:37
Gajinder Singh:Hi, Ramal, thank you. And let's start with our update. We were able to achieve Interop for devnet 4 over the weekend.
00:13:47
Gajinder Singh:And I believe, we… Zoom is working well with Lantern and EthLambda, and over multiple subnets as well. So, that's fantastic news. On…
00:14:01
Gajinder Singh:on, on this front, Zim was mostly, busy in fixing the issues.
00:14:08
Gajinder Singh:That we encountered, while doing the interop.
00:14:11
Gajinder Singh:And we are working on stability and optimizations.
00:14:17
Gajinder Singh:Which… there are some few low-hanging optimizations that we are going for, and then we'll dig deep into the performance, issues that might… that are coming up, or that might come up.
00:14:30
Gajinder Singh:And, basically, our aim is to make sure that, devnet 4…
00:14:37
Gajinder Singh:Onwards, Zee becomes one of the most stable clients, participating in the network.
00:14:44
Gajinder Singh:And, that is what our focus is on devnet 5, me, we,
00:14:51
Gajinder Singh:we, so I and Anchilla are basically…
00:14:54
Gajinder Singh:discussing on what could be the devnet 5 spec. Basically, what we really want to do is aggregate all the signatures into one signature, but as we have seen in some spec discussions and PRs and rewards, that
00:15:11
Gajinder Singh:That we will need to repack at a station. So, so we will basically need to figure out a way where we can
00:15:21
Gajinder Singh:reutilize… The signatures, in repacking of the attestations, even if we have a single aggregated signature.
00:15:31
Gajinder Singh:For some particular validators, because let's say your node was not able to… was not able to receive the aggregated attestations for some particular validators, and you got them in a… in a block.
00:15:44
Gajinder Singh:And now, if you are a proposer, so you'll need to rebuild it and…
00:15:49
Gajinder Singh:You will need to repack, which is exactly the case that we discussed for the latest,
00:15:59
Gajinder Singh:justified updates that we talked about in devnet for Channel itself. So, we are basically looking into that and,
00:16:09
Gajinder Singh:determining what is the best way to go forward, and basically discussing with Amel
00:16:15
Gajinder Singh:And you, over what should be the interface.
00:16:18
Gajinder Singh:for… Block-level aggregation of all the signatures.
00:16:25
Gajinder Singh:Yeah, and then our aim is to also produce a shadow-compatible build, so we did
00:16:32
Gajinder Singh:some, shadow-related,
00:16:36
Gajinder Singh:fixes, in the sense that to support Shadow, to support running on Shadow, and once we have a stable devnet for
00:16:45
Gajinder Singh:a much more stable devnet for running,
00:16:49
Gajinder Singh:codebase and images, we will basically then try to do shadow simulations with Camel.
00:17:02
Thomas Coratger:Okay, cool, let us talk about, like, the…
00:17:07
Thomas Coratger:the block syncs for devnet… for devnet 5, just after, maybe?
00:17:12
Thomas Coratger:or maybe next week, let's see, like, as we are running on stability for devnet 4.
00:17:19
Thomas Coratger:Maybe this call is not the best one to talk about it, but maybe the next one, but let's see.
00:17:27
Thomas Coratger:maybe with Emil, who is in VM discussion. Let's continue first on the client updates.
00:17:36
Shariq Naiyer:So, yeah, how you doing, guys? We've been working on getting devnet 4 up to spec, and we have been testing against, currently, we have been testing against Zim and Grandine, and we seem to be fin- we seem to be finalizing quite well.
00:17:52
Shariq Naiyer:And, on the other hand, we have seen all the spec tests that have been added to the spec, and we're updating our spec test suite to, match, to match the devnet 4 spec.
00:18:04
Shariq Naiyer:And our main focus is going to be devnet4 stability and, and testing, and, and just ensuring that we can, we can finalize at, pretty high slots.
00:18:17
Shariq Naiyer:On a different note, we've also been working on a testing framework. We've been working on Hive. We've been integrating some of the clients on Hive.
00:18:27
Shariq Naiyer:I think we have Zim, we're slowly working on Lantern, adding Lantern, we have Reem, and we have ETH Lambda. Derek would like to give a quick, a quick, a quick spot on it, just a quick show of, tool on it, at the end, towards the end of the call.
00:18:46
Thomas Coratger:Yeah, perfect. Let us do that just after, like, client updates.
00:18:56
Ruslan Tushov:Oh, we've… implemented devnet4, however, we didn't implement case separation yet, so…
00:19:06
Ruslan Tushov:During testing, we used two same key players, and didn't test separate subnets yet.
00:19:13
Ruslan Tushov:Also, when running interop tests, I noticed that, there is some Yes.
00:19:23
Ruslan Tushov:slow penalty, and as I mentioned in chat, it seems that recourse fabrication it can be slow.
00:19:34
Ruslan Tushov:I've attached a link to the… in the chat. If you can measure it on your machine, I would add its results to the README, too.
00:19:48
Ruslan Tushov:Thank you, that's all.
00:19:52
Thomas Coratger:Great, thank you, Lantern.
00:19:59
Mihir Faujdar:Hi, everyone. So yeah, I've basically been preparing for devnet 4. We have implemented the specs and have been able to interop with Grandine in our local testing, but also good to see that we can interop with Zim.
00:20:14
Mihir Faujdar:So that's great to see.
00:20:17
Mihir Faujdar:As well as being working on adding tests, the new test vectors that have been added in, lean specs, just to ensure that we're compliant with all of them.
00:20:28
Mihir Faujdar:I guess on the metric side, just waiting for the devnet4 metric to be merged, before adding support for it. Yep, that's your update, thanks.
00:20:42
Thomas Coratger:Great, thank you. It's Lambda.
00:20:49
Pablo Deymo | Lambda:We finished working on the devnet4 support.
00:20:53
Pablo Deymo | Lambda:The last week?
00:20:54
Pablo Deymo | Lambda:This last week, we implemented the station count cap per block.
00:20:58
Pablo Deymo | Lambda:Proof aggregation changes from the spec.
00:21:02
Pablo Deymo | Lambda:We integrated the new SSZ and the STF standard intervention test from the spec.
00:21:10
Pablo Deymo | Lambda:And we also updated to the latest version of lean spec Development 4 Branch.
00:21:18
Pablo Deymo | Lambda:And we were running multiple interoperability devnets to debug issues.
00:21:25
Pablo Deymo | Lambda:With some of our clients.
00:21:27
Pablo Deymo | Lambda:And, we also started an internal initiative
00:21:32
Pablo Deymo | Lambda:We work… started working on formalizing the state transition function and four-choice algorithm in lean for language.
00:21:41
Pablo Deymo | Lambda:So we made a PR in our repo, adding the…
00:21:46
Pablo Deymo | Lambda:Porting the is justified… shutifiable after function.
00:21:53
Pablo Deymo | Lambda:In… in lean form.
00:21:55
Pablo Deymo | Lambda:So, that's our update. Thanks.
00:22:07
Shaaibu Suleiman | gean:Yeah, hello.
00:22:09
Shaaibu Suleiman | gean:Hello, yeah, hello.
00:22:11
Shaaibu Suleiman | gean:Yeah, so, we've been working on… still working on, support for devnet 4, basically.
00:22:19
Shaaibu Suleiman | gean:We've, implemented, I think basically the support for dual keys, basically for, attestation.
00:22:27
Shaaibu Suleiman | gean:and also proposal, and also, recursive aggregation. We've, provided support
00:22:35
Shaaibu Suleiman | gean:for our latest updates, been made, I think basically for lean, multi-seq.
00:22:43
Shaaibu Suleiman | gean:And also support for annotated, validators. You know, config for, validators.
00:22:51
Shaaibu Suleiman | gean:And, we've tested, we've tested, the implementation we have so far.
00:22:57
Shaaibu Suleiman | gean:against, Zim and It Lambda, and it has been, coming up, pretty well. Locally, though, we've been doing that test, locally.
00:23:08
Shaaibu Suleiman | gean:And I think our PR for devnet 4 only in Quick Start was, was merged through that.
00:23:15
Shaaibu Suleiman | gean:We could also participate, in the multi-client, interlock run for Defnet 4. So yeah, basically, that's it from us. Yeah. Thank you.
00:23:30
Thomas Coratger:Great, thanks a lot.
00:23:33
Thomas Coratger:Next, Lighthouse?
00:23:42
Thomas Coratger:Next… Grandine?
00:23:48
Mercy Boma Naps-Nkari:Good afternoon. So, excuse me. On the grounding side, we've implemented devnet4, and we're running locally to actually see
00:23:57
Mercy Boma Naps-Nkari:If we have good enough stability for them to forwards,
00:24:01
Mercy Boma Naps-Nkari:It's flamba, lantern, and tin, which everything has been me.
00:24:05
Mercy Boma Naps-Nkari:Going all well, so…
00:24:08
Mercy Boma Naps-Nkari:about the metrics, once the metrics PR get managed, we'll have our metrics ready also.
00:24:15
Mercy Boma Naps-Nkari:Thank you.
00:24:19
Thomas Coratger:Cool, thank you, Nimbus?
00:24:28
Thomas Coratger:Okay, did I miss anyone?
00:24:37
Thomas Coratger:Okay, so let's go for spec and research update. So…
00:24:45
Thomas Coratger:On this side, basically, on the spec.
00:24:49
Thomas Coratger:We have implemented a lot of tests in the past weeks.
00:24:54
Thomas Coratger:No spec change, basically, but a lot of new tests have been added, especially test vectors for you to be able to consume.
00:25:04
Thomas Coratger:And to check with. I have added a lot of good first issues for external contributors to participate as well in this effort to diversify a bit the…
00:25:16
Thomas Coratger:The contributors that we have, and not only me and a couple of people pushing tests, but also, like, the broader community.
00:25:25
Thomas Coratger:So we started by adding a bunch of very edge case, situations.
00:25:32
Thomas Coratger:Even if some of them should never happen, for me, this is cool to have tests for this to ensure that we are all on the same page with all of the behavior of the code.
00:25:45
Thomas Coratger:Then I have started yesterday to add much more basic test vectors that we didn't have before, for example, for SSD implementation, and now for networking stuff.
00:25:57
Thomas Coratger:to ensure that, even on these very basic primitives, we are on the same page. If this is not the case, this is not at all a problem. We should fix either on the client side, or on the spec side.
00:26:12
Thomas Coratger:This is not a problem, this is just a good news, because we'll be able to catch bugs much more easily.
00:26:20
Thomas Coratger:Next, we should work on some kind of visualization tool to have a list of the client and the tests they pass and they don't pass, so that we can have, like, a clear visualization. I know, Marci, that you will send me a message about this. Thanks a lot for…
00:26:39
Thomas Coratger:for taking this, like, for thinking about this. I will answer you, today.
00:26:46
Thomas Coratger:That is, on the… on the pure spec update, we will continue like this to try to reach a stability point and add more tests.
00:26:55
Thomas Coratger:For now, I am a bit concerned about the duration of the tests, especially the test vectors. I think this is mainly due to the
00:27:04
Thomas Coratger:recursive aggregation that we have added in devnet 4.
00:27:07
Thomas Coratger:causing much more aggregation to happen, and so much more time than we had before. We multiply something by 2 or 3, the duration of the test vectors, and this could become a problem.
00:27:20
Thomas Coratger:If we have much more tests, I don't want to inflate the duration that much. I should check, maybe.
00:27:29
Thomas Coratger:Then I think that we have, like, the production test vectors.
00:27:34
Thomas Coratger:on main not passing, but I think this may be due to not regenerate… due to not… to us not regenerating the production keys after the latest post agent update. I think this is…
00:27:49
Thomas Coratger:Probably the root cause.
00:27:51
Thomas Coratger:Because, test configuration of the test vectors pass… properly.
00:28:00
Thomas Coratger:So that is one thing.
00:28:02
Thomas Coratger:That is on the spec side. On more, like, research and cryptography side,
00:28:10
Thomas Coratger:We are making very good progress on PromQ3 and LeanVM. Emil will talk about LeanVM just after, but on the PromQ3 side of things, we are implementing much more things. We're in… is in the process of being fully integrated.
00:28:27
Thomas Coratger:We have also some SEM check optimization in the pipeline.
00:28:33
Thomas Coratger:with at least two additional PRs to be merged in Plonkey 3, and probably some upcoming one with a new paper coming on ePrint in the coming days.
00:28:44
Thomas Coratger:So that is basically… it's about the research, yeah.
00:28:51
Thomas Coratger:Emil, if you want to talk about LeanVM.
00:28:55
T. Wambsgans:Hello, thank you. Yes, simply, some… some check simplifications, basically, batching
00:29:02
T. Wambsgans:previously that there was one sum check for each Airtable, and now we batched them.
00:29:08
T. Wambsgans:And so, in some sense, the lean VM now is only, like, one commitment with wear, one GKI error for logup, one… some check for error validity, and one we're opening, and that's it. So basically, it becomes simpler conceptually, and also more recreation-friendly. And maybe, I don't know, about the…
00:29:31
T. Wambsgans:The, the fact that the…
00:29:33
T. Wambsgans:recursive aggregation is slow. I think that's indeed a problem for the tests. What do you think? Maybe, couldn't we, like, run to server
00:29:42
T. Wambsgans:to have faster continuous integration than the default GitHub one, maybe, I don't know. And if we have a dedicated server.
00:29:52
T. Wambsgans:one valuable aspect would be to be able to have AVX512, which is not available on GitHub. And typically, for a lean VM, I know that sometimes there are some commits that breaks AVX512, and since I'm not on this machine, it's hard to detect it, so having,
00:30:12
T. Wambsgans:a fast, continuous integration on all the targets would probably be valuable for the spec, for lean VM, for everyone. So maybe we should invest some time and resources into it, probably.
00:30:29
Thomas Coratger:Yeah, yeah, sounds good. Yeah, definitely, yeah, that should be an option. There are, like, definitely different options.
00:30:37
Thomas Coratger:Either these, or maybe for some…
00:30:43
Thomas Coratger:For some specific case where we don't want to test precisely the aggregation, but we want to test some other properties, we could just mock the aggregation and have, like, some random values, because this is a test where we don't want to test aggregation, and just reserve
00:31:01
Thomas Coratger:the aggregation for a couple of tests, where we want to test aggregation precisely, that could be also an option.
00:31:10
T. Wambsgans:That makes sense. Actually, like, we could say that you only have
00:31:16
T. Wambsgans:A few tests is, as you said, a few tests On the spec, and,
00:31:23
T. Wambsgans:on the lean VM side, it's okay if the CI is slower, so we could have much more tests here, probably, yeah.
00:31:37
Thomas Coratger:On the RIM side, you mentioned that you wanted maybe to show us some tooling integration or something? Maybe this is the right one?
00:31:48
Shariq Naiyer:Yeah, so, one thing is, we were discussing visualization of the spec test in all the clients. We wanna use the same… we're trying to get this visualize, the tool that Derek is going to show, to, to do that visualization as well. I'm gonna pass over this to Derek right now, and he can present, what we are working on.
00:32:09
Derek Sorken:For sure, hello? Just make sure I can be heard while I just had a fresh install of Zoom. I think it's saying I can be heard. Perfect. Yeah, so, I'm just gonna go ahead and share my screen, if that's alright, but basically…
00:32:22
Derek Sorken:It's a visualization… at the end of it, it's a visualization tool, but…
00:32:31
Derek Sorken:Just for a, maybe a high-level overview for anyone who isn't familiar with Hive, essentially, it's a…
00:32:38
Derek Sorken:A test harness, and it's… the way that it's ran is,
00:32:44
Derek Sorken:It's done through Docker, and so at the beginning of the test, for example, we'll have, like, a test to run with a client, Zim, as we're running Zim, and…
00:32:54
Derek Sorken:What we're trying to do in this test.
00:32:57
Derek Sorken:is test, client RPC compatibility.
00:33:00
Derek Sorken:And so, essentially, all it is at the end of the day is, a bunch of lambda functions for different tests.
00:33:08
Derek Sorken:And so it'll go through…
00:33:10
Derek Sorken:a bunch of different tests, to test different functionalities for, in this case, fork choice, checkpoints, and state for RPC.
00:33:21
Derek Sorken:And, what it'll do is run through those tests, and for each test, it will start up a Docker image of Zim, run through the Zim test, and then…
00:33:33
Derek Sorken:a workspace, so it'll dump all of the logs, and so we can see, like, here's our Zim logs, and then we have Reem, Lantern, and, ETH Lambda.
00:33:46
Derek Sorken:And right now, we are in the process of getting a Lantern devnet 4, but just for the sake of demonstration,
00:33:55
Derek Sorken:We can see the visualization tool.
00:33:58
Derek Sorken:Move this out of the way.
00:34:01
Derek Sorken:There we go. Okay, so… For example, we can look at…
00:34:05
Derek Sorken:the, Lantern run, and so we can see RPC compatibility and Lantern devnet 3.
00:34:10
Derek Sorken:And we can see, right now, when we run through it, there's two fails, and we can see why.
00:34:17
Derek Sorken:We can see all of the tests here, and view all the tests, and if nothing happens in the test, then we'll just say no log output. But for each test, we can actually view all of the…
00:34:28
Derek Sorken:the Docker logs, and so if we want to see, for example, the health healthy, just to check if the RPC Health Endpoint is working properly.
00:34:35
Derek Sorken:We can check, see…
00:34:37
Derek Sorken:the whole startup process in Lantern, gone back, and see, of course, it returned it, and the test went well, and…
00:34:45
Derek Sorken:When something goes wrong. So, for example, right now, this test was just picking up.
00:34:49
Derek Sorken:There's a OX prefix in the log, and so it happened to not return.
00:34:54
Derek Sorken:an OX prefix, and if…
00:34:56
Derek Sorken:If we needed more in-detail information about it, we could, of course, go to the Lantern log and view through that. But this is just done through HiveView, which is an existing tool in Hive.
00:35:08
Derek Sorken:And the idea is that…
00:35:11
Derek Sorken:Right now, I've just done individual tests for these first few, just to show that, okay, we can look at the individual tests, and some of them right now are failing. So, for example, the one, I believe, in ETH Lambda, I'll have to take a look at the test itself.
00:35:24
Derek Sorken:I think it's just a race condition that this shouldn't be failing, but I'll have to fix that later on.
00:35:29
Derek Sorken:But, for example, the one, if we look at, like, Zim, I believe that's one of the same problems, but it'll just pick up inconsistencies, and so, like, right now, in the, health endpoint, instead of returning, like, the lean RPC, it'll turn… right now, it returns, like, Zim API, which I suppose we could…
00:35:48
Derek Sorken:make more lenient, depending on how consistent we want that to be.
00:35:54
Derek Sorken:But at the end of the day, the idea is we run all of the clients all at once for each suite, and so if you go back to the code… here we go,
00:36:04
Derek Sorken:there is, suites that you can specify, and so right now, we have the RPC compatibility suite that we've specified as a tool, and as we build onto Hive, this can be more like interoperability tests, and so I believe, Colby will be working on, like, right now in Reem, we have a matrix CI test, where it'll go through
00:36:23
Derek Sorken:Reams compatibility with each client, but ideally in Hive, then we make it so that each client runs against each other client, to see if it can finalize, and, like, stuff like checkpoint sync, and late compatibility.
00:36:36
Derek Sorken:But… Just for, like, a…
00:36:40
Thomas Coratger:Just maybe a stupid question, but in the test that you have shown in the screen.
00:36:47
Thomas Coratger:where they come from? Like, natural to understand, like.
00:36:53
Thomas Coratger:These are, like, specific,
00:36:56
Thomas Coratger:specs that you define yourself, because I think these are not spec tests or something.
00:37:01
Derek Sorken:Yeah, so these specific tests, were built on just going through the behavior, like, either explicit or implicit behavior from
00:37:11
Derek Sorken:lean spec, so I guess an example of that would be, lean spec, what it wants to do is return, like, OX, prefixed roots, and so that's one of the tests, is literally just, if I can find… all the way at the top here…
00:37:31
Derek Sorken:One of the tests is just, does it hex encode the root? Does it have that… Ox…
00:37:37
Derek Sorken:encoded root, should be OX prefix. Just implicit behavior.
00:37:42
Thomas Coratger:And… Maybe this should be tests that we add, as well as test vectors inside the spec, no?
00:37:48
Derek Sorken:Yeah, for sure, we could eventually migrate this to, like, test vectors inside the spec as well. This was maybe more so just done as, like, proof of concept and something that could be done either way, but yeah, it could migrate either way, where we also, I believe, would want to take…
00:38:04
Thomas Coratger:In this case, in your tooling right now, if we do that, if we migrate everything as test vectors inside the spec, so that all of the client teams can consume, and then in your tooling that you present there, you will just test, basically, all of the test vectors of the spec, correct?
00:38:22
Derek Sorken:Yeah, this would go for testing everything, and then making it easily visualizable, and ideally, what we'd want to do is,
00:38:30
Derek Sorken:just have some sort of long-running high-vis instance that kind of just goes through each test, and as clients update their…
00:38:37
Derek Sorken:their image, it will then be like an immediate feedback to say, okay, does this image do anything to change the test results?
00:38:45
Derek Sorken:But ideally, what we do is, for each run, in each suite, we'd run all clients all at once, so this is a run, for example, where I ran, in the… there's a command here. I ran Xem, ETH, Lambda, and REAMS, comma separated in the, command line here.
00:39:02
Derek Sorken:But what it'll do is then…
00:39:05
Derek Sorken:run, for example, all we have right now is the RPC compatibility Suite, but it'll run through each one individually, it'll test them against the RPC compatibility Suite, so see, we can see, like, each RPC compatibility test, ETH Lambda, you scroll down later, and then it's Reem, and a little bit lower is Zem.
00:39:22
Derek Sorken:And then as we add a different suite, which would just be added right in here, we could add, like, the matrix test, for example, where we just take, like, Xem against ream, two zems, 1 ream, and then two reams, one Zem, and so on.
00:39:36
Derek Sorken:And… and for each client, we could do that, and so it could really cover all the interoperability bases, just to make sure that
00:39:43
Derek Sorken:As, like, a basic start.
00:39:45
Derek Sorken:everyone can finalize with everyone else, in every combination. That's, like, every basic combination. Obviously, we could make them more complex combinations, but there's probably diminishing returns as to how long those tests would take to run.
00:39:59
Derek Sorken:As opposed to just actually doing, like, a full devnet run. But yeah, that's… that's kind of the idea of, of Hive.
00:40:09
Derek Sorken:If there's any questions as well, I could answer those.
00:40:19
Thomas Coratger:That's great. Like, personally, I love it. I don't know about other things, but… I love it.
00:40:27
Derek Sorken:Perfect, sounds great. Yeah, we'll be continuing to update as time goes on. I believe this week we'll try to get the, kind of, the matrix tests to go on, and perhaps we could present later on to kind of give an idea of what that looks like as well, but yeah, that sounds good.
00:40:43
Shariq Naiyer:And, and when he says…
00:40:46
Shariq Naiyer:Someone who's speaking. I don't know if I'm speaking.
00:40:50
Gajinder Singh:Yeah, yeah, so, yeah, this is great, and we will aggressively integrate with Hive.
00:40:55
Gajinder Singh:And make sure that, you know.
00:40:59
Gajinder Singh:This is sort of one of the steps that everyone needs to achieve when they come before and come to an interop point.
00:41:09
Gajinder Singh:So, I think this is an excellent step, and hopefully all teams will contribute to writing tests for Hive.
00:41:18
Derek Sorken:Awesome, sounds great. Oh, one thing I should also mention, I know that it was…
00:41:22
Derek Sorken:maybe a concern last interrupt call, to make sure that any test… so, for example, right now in our individual tests, there are cases that we want to test that are, post-genesis, and so…
00:41:36
Derek Sorken:the best way that I've found to make this most consistent is what we'll actually do is spin up the lean spec client itself, and so…
00:41:45
Derek Sorken:certain post-Genesis tests right now will actually test against the given client that we're testing. So, for example, we're testing Zim, or something like that. Zim would show up, and we'll insert it in here as Zim, but we'll actually run it against the lean spec client, so…
00:42:01
Derek Sorken:To get, like, this example's just like a…
00:42:05
Derek Sorken:Maybe a very, like, niche edge case, but,
00:42:09
Derek Sorken:should there be a difference in how the checkpoint shape is Genesis versus post-Genesis, but essentially what we'll do is spin up the lean spec client with 3 validators, so it can get to a point where there's some finalization achieved, and then we'll checkpoint sync the given tested client.
00:42:26
Derek Sorken:To then get that new checkpoint that's finalized. And this way, it's as unbiased as possible to just run specifically against lean spec for very, like, in-depth.
00:42:38
Derek Sorken:Specific spec tests, I think as we go on, we can…
00:42:44
Derek Sorken:there'll probably be more applications of when this would be a good use of using the lean spec client itself.
00:42:51
Derek Sorken:But yeah, I think that's probably also important to note.
00:42:55
Gajinder Singh:So, for most of the tests, lean spec client should be the source of truth, but maybe in some scenarios.
00:43:01
Gajinder Singh:Lean spec, client might not be,
00:43:05
Gajinder Singh:optimized enough to produce some particular test, and in that case, I guess, we'll figure out, how to go about and, run that test.
00:43:16
Gajinder Singh:But, most likely, I think test will be a series of hard-coded RPC calls.
00:43:22
Gajinder Singh:In that sense, I think we'll be able to sort of circumvent,
00:43:30
Gajinder Singh:the issues that lean spec client might not be production.
00:43:34
Gajinder Singh:Grade, but, yeah, I sort of… because what I know is that,
00:43:41
Gajinder Singh:Currently, Hive uses GET as the backend to… Orchestrate everything.
00:43:49
Gajinder Singh:But hey, we want that, that lean spec should be that, source of truth.
00:43:55
Derek Sorken:Yeah, yeah, 100%, I agree. So yeah, we can…
00:43:59
Derek Sorken:In Hive, it'll show runtimes, and that is one thing, like, it takes a little while to spin up, but I think that's something that we'll tackle as we go to see if we can kind of get things more optimized, but I do agree that, yeah, lean spec is ideal as source of truth, but if it is insufficient, then, yeah.
00:44:17
Thomas Coratger:Yeah, but in any case, this is what I respond to the message of Colby just in the chat, but in any case, I am sure that sometime you will find or discover some tests that are not in lean spec, for example, for the peer-to-peer behavior, or for the syncing, or for some risk.
00:44:37
Thomas Coratger:Rec, RESP, Exchange, and stuff like this, so please, in this case, just…
00:44:43
Thomas Coratger:Send me a link with those tests that… so that we should add them as well in the spec.
00:44:49
Thomas Coratger:So that all of the other client teams can benefit from this test.
00:44:53
Thomas Coratger:So, this is totally possible that you will discover tests, that we should add in the spec.
00:44:59
Derek Sorken:For sure, yeah, that sounds good.
00:45:03
Derek Sorken:Alright, I'll go ahead and end the screen share.
00:45:12
Thomas Coratger:Yeah, so thanks, thanks a lot for this presentation, this is, this is amazing.
00:45:19
Thomas Coratger:now, maybe, about metrics, Katya, if you want to add some stuff, anything?
00:45:27
Katya:Yes, hello, everyone. So, PR is open. I'm working currently on the dashboard update itself. If you have any comments, please take a look into this PR, so if
00:45:39
Katya:does it cover all these… all we need for, measuring devnet4? So that would be great if someone
00:45:47
Katya:just start to look into it, and how it is… it can be applied to their code, so maybe we'll have some more updates for this. If not, I will… to the maximum, I will merge it on Friday, maybe.
00:46:03
Katya:I mean, not… not later than Friday, so this week,
00:46:08
Katya:If we cover something interesting, we can add to this.
00:46:13
Katya:And regarding running devnet4, maybe Parse, you will share the updates?
00:46:19
Parthasarathy Ramanujam:Yeah, thanks, Kathy, I can do that. I mean, we've had a stable run of 3 clients with a single subnet running yesterday. Today, I managed to get, I mean, multiple subnets, I think 2 subnets running with 3 clients, and then after a while.
00:46:37
Parthasarathy Ramanujam:it's stalled, I'll probably,
00:46:40
Parthasarathy Ramanujam:Include the additional clients today, and then share the logs as soon as we have something working.
00:46:46
Parthasarathy Ramanujam:That's true.
00:46:55
Thomas Coratger:Anything else that anyone wants to add?
00:47:03
Thomas Coratger:In any case, like, as a summary, I think that we are in a stability period, let us say, where, from what I understood, and from what I have in mind, all of the client teams want to
00:47:15
Thomas Coratger:stabilize on devnet 4, trying to debug, add more tests. This is also our status right now in the lead spec, where we are trying to stabilize pretty much everything.
00:47:27
Thomas Coratger:And had a lot of Cisco rage.
00:47:29
Thomas Coratger:Yeah, and on the R&D front, we are constantly improving.
00:47:34
Thomas Coratger:The cryptography.
00:47:36
Thomas Coratger:to be noted that beginning of May, we will have this cryptography event in Rome, so we will have
00:47:44
Thomas Coratger:hopefully… a bunch of events there, and so more news to share about new cryptography and R&D development.
00:47:53
Thomas Coratger:So, yeah, that's it.
00:47:56
Thomas Coratger:If you have nothing else, we can wrap up.
00:48:04
Thomas Coratger:Thanks a lot, everyone.
00:48:06
Thomas Coratger:See you next week. Bye-bye.
00:48:12
Shariq Naiyer:Have a good one, guys.

Chat Logs

00:20:11
Ruslan Tushov:https://github.com/turuslan/lean-interop-test/tree/rust-xmss-time/rust-xmss-time
00:23:35
Ruslan Tushov:Replying to "https://github.com/t..." Sorry, need to go. Feel free to write direct message or mention me in telegram
00:36:49
Gajinder Singh:this is fantastic! already been a fan of hive, time to double down :)
00:38:31
kolby:https://github.com/ethereum/hive
00:38:49
Mihir Faujdar:Replying to "Here is the repo" Thanks!
00:40:09
kolby:For a brief scope of tests we will add are - intergration of leanSpec tests - p2p behavior, - syncing, - req/resp exchanges, - RPC compatibility, and - other protocol-focused scenarios.
00:40:42
alan xu:What’s the diff between hive and kurtosis?
00:41:26
kolby:hive is a black box testing tool an intergration test framework so to speak, where as kurtosis is like a devnet assertion tool
00:41:31
Katya:Replying to "What’s the diff betw..." Kurtosis is for spinning up nodes, hive haas scenarios to test
00:41:38
Thomas Coratger:Répondre à "For a brief scope ..." please once you have something let me know with a link towards the tests so that I can add them to the spec as well

Summary

14 highlights · 3 action itemsExperimental

client updates

  • Lodestar achieved devnet 4 interop with Lantern and ETH Lambda across multiple subnets00:13:47
  • Signature aggregation redesign needed for devnet 5 to support attestation repacking00:15:11
  • Lantern finalizing well with Lodestar and Grandine on devnet 400:17:52
  • Reem reports slow proof verification performance; benchmarking requested from other teams00:19:22

testing progress

  • Hive test framework integrating Lodestar, Lantern, Reem, ETH Lambda for RPC compatibility00:32:38
  • Hive will test all client pairs for finalization and checkpoint sync compatibility00:41:28
  • Devnet 4 running stable with 3 clients, single subnet; multi-subnet testing in progress00:46:27

specification updates

  • Extensive spec test vectors added for edge cases; good first issues opened for contributors00:24:50
  • Test duration increased 2-3x due to recursive aggregation overhead in devnet 400:26:12
  • Production test vectors failing; likely need regeneration after post-quantum update00:27:34

research and development

  • Post-quantum (Plonky3) integration progressing; SUM check optimizations coming00:28:00
  • LeanVM now uses single commitment with batched SUM checks; simpler and recursion-friendly00:29:02
  • Cryptography event scheduled for Rome in early May00:47:36

organizational

  • Metrics PR to be merged by Friday this week for devnet 4 dashboard00:45:27

Decisions

  • Devnet 5 will aggregate signatures into one per block; repacking mechanism TBD00:15:11
  • Hive will test all client pair combinations for interoperability validation00:41:28
  • Lean spec client will be source of truth for Hive test scenarios00:42:26

Action Items

  • All client teams: Benchmark proof verification performance and report results00:19:22
  • Reem team and Thomas: Add Hive test cases as spec test vectors for broader client consumption00:38:31
  • Hive contributors: Share networking/p2p/RPC test cases for inclusion in lean spec00:40:09

Targets

  • Friday this week - Merge devnet 4 metrics PR00:45:27
  • Early May - Cryptography event in Rome00:47:36