Citadel Dispatch

Share on social media:

I sat down with Miles Suter, Rockstardev, and Lisa Neigut on March 18th 2022 at the Bitcoin Takeover event held at Bitcoin Commons in Austin during SXSW to discuss lightning development and the three main lightning implementations: LND, CLN, and LDK.

Video: https://bitcointv.com/w/1jRpeX7VrQkzRcoqha6yso

support the show: https://citadeldispatch.com/contribute

twitch: https://twitch.tv/citadeldispatch  

bitcointv: https://bitcointv.com/video-channels/citadeldispatch/videos

podcast: https://www.podpage.com/citadeldispatch  

telegram: https://t.me/citadeldispatch

stream sats to the show: https://www.fountain.fm/

join the chat: https://matrix.to/#/#citadel:bitcoin.kyoto

[00:00:10] Unknown:

People will talk about how development in Bitcoin is slow,   and that we,   you know, it's it's really opposite to the mantra of move fast and bake break things, but it's also,   permissionless.   And and these three people   are all building applications on top of Bitcoin in a permissionless way each on the Lightning Network. So,   we've got Lisa Niget. Lisa is a c lightning protocol engineer   at Blockstream.   Before she was a back end engineer at Square or Block, formerly known as Square. And she's also,   the chief   explaining officer, I believe at   the best education Bitcoin education company in the world called, base 58. And,   Lisa is a graduate of the University of Texas here in Austin, and it's really special. Lisa's just down the road,   in Houston. And, and thank you so much for, for coming up here to Austin to, to share what you're building on lightning today. We've got rockstar dev,   otherwise known as uncle,   to many of you,   rockstar,   in addition to being a core contributor to,   BTC pay server, which is an open source,   payments processor,   is also the VP of engineering at strike. So Jack Mallers was up here before talking about,   global monetary interoperability.  

Rockstar, we're talking about more, what they're actually, actively developing at strike and how they think about developing on top of lightning.   And then we've also got Miles Suter,   who   is a product lead on the Bitcoin side at, the Cash App, which is a subsidiary of block or formerly known as Square. It's still just not,   it's going to get there. Okay. It's going to get there. He's also a contributor, spiral BC BTC,   and,   cash app recently,   launched,   the ability for their users to access lightning. So,   this will be, you know, a great talk about one of, one of the ways that,   a lot of people are working to, to build and scale Bitcoin   and help make it easier to move Bitcoin around the world.  

So with that, I'll bring you guys up and, and Matt O'Dell will lead the discussion.   So one of our mics is gone. At least you wanna stay here.   Matt. Oh, yeah. You over here. Matt, you're there.  

[00:02:31] Unknown:

Mic. I should use that mic. Under his feet though. Is it mic?  

[00:02:39] Unknown:

We good on mics? We're good on mics. We're good on mics? We're good on mics. We're good on mics? We're good on mics? We're good on mics? We're good on mics. Okay. Awesome. Yeah.   So   we're very fortunate to have,   all these great people up here with us,   today to talk about lightning and the future of lightning.   What's really interesting, and Parker kind of,   touched on it,   is   on when when when we're talking about Bitcoin, when we're talking about on chain Bitcoin, the protocol,   right now it's dominated by a single implementation.   But with lightning,   we started with a multi implementation approach. And the idea is that you have multiple different teams that are working on lightning and that they're all cross compatible with each other.  

So we have Lisa here. She's working on sea lightning.   We have rock star who,   at strike they're building on, LND.   And then we have miles at cash app,   who they're working on LDK,   which,   their open source team, Spiral, is taking the lead on that, and they've integrated that into Cash App. So we have   3 3 of the implementations. I think there what are there? 6 implementations   now?  

[00:03:52] Unknown:

It's always changing now. Yeah. Yeah. Maybe even less than that. Four major. Yeah. I would say. Majors. Yeah.  

[00:03:59] Unknown:

So   recently there's been a lot of drama with,   cross implementation work.   Are we going to talk about that or is that too juicy?   How do you guys, how do you guys feel about that? Do you think like, is, is a multi implementation world of lightning actually going to   come to fruition, or is it going to just be dominated by a single implementation? No. This is a question for Rockstar. Right? Let's jump right in. Well Let's go. Yeah. Yeah. Most disagreeable topic.  

[00:04:29] Unknown:

My view, especially when it comes to lightning is, yeah, it's it's it's a good thing, man. It's a good thing especially because,   when it comes to   base blockchain and Bitcoin Core, you do not want a lot of disagreement. You want consensus. You want building slow. But when it comes to Lightning and what Lightning enables us,   you want teams   pushing the envelope, pushing the   the spec, which is what you see now with Bolt 12 and lnurl. I mean, lnurl to me, I I'm a huge fan of Fiat Jeff, and, what he did with lnurl   pushed the spec. Right? It opened up new use cases,   new opportunities.  

You know, you can do things that you couldn't do before. So for me, the more implementations we have and the the more market   gets to decide, yeah, what wins in the long term, like, I'm all for that. Yeah.  

[00:05:33] Unknown:

Yeah. I, I tend to agree as well.   I think that there you get different things out of, the different implementations. And when we started LBK, we really wanted to create   an implementation that's much more modular, much more customizable,   and,   one that's   doesn't have, like, a a a VC backed company behind it. So, I mean, I wish all the implementations well, and everybody's driving it forward in its own way. But we thought it was really important to have one that didn't have to, like, find a way to make money. It could just be sponsored for free   at blocks. So that was kind of a core part,   of why we kicked off,   LDK   and, also that it's written in Rust. It seems to be the hot thing on the streets these days.  

But, yeah,   Cash App was the first implementation to use it, and it's still quite it's still quite new. We had Matt Corral flushing it out and filling out some of the gaps as we're building out our product and doing it kinda hand in hand, which was a really, really cool experience.   We started   Spiral, known as Square Crypto,   over 2 years ago with no expectation of   collaboration or,   like, using them. The we spun it up just so they could benefit the entire ecosystem.   But as we started to look into Lightning, as they were getting closer to being able to ship something to production,   the timing really aligned nicely.  

And so just having that expertise   and being able to lean on them and work as a team in the same Slack channel,   theoretically in the same building if we weren't all remote,   like, yeah, it was it was a really, really cool thing to see. And I think,   that cross pollination is something we're really excited about at Ploc.  

[00:07:18] Unknown:

Yeah. And as someone so I've been involved in the lightning spec process since late 2018,   which is, like, starting to become a while now.   I have to say that, like, having Matt Corallo and the rest of the spiral team become   more involved in the spec process over the last couple of years has been a huge,   benefit, I think, to the spec and to the things that we focus on adding and improving in the lightning network. So,   I think, Matt, like, your your first,   I think your question was, like, do the inner does it in our operation, does it work? And I think the reality is that, yeah, it does work.   Right? There are, like, a couple different lightning implementations and they do all, for the most part, play well together. You can run them. You can open channels between them. And so, I think that, like, if you wanna just kinda, like, you know, draw between them. And so, I think that, like, if you wanna just kinda, like, you know, draw a line in the sand and say, did it work? Was it successful?  

I think, like, yes. I think it was very successful. And we have, like, a bunch of big companies now that are able to,   that are sponsoring   and really growing, I think, practices of engineers   that have deep experience in exactly how lightning works and are, like, distributed. Right? They have different sources of funding.   They kinda have different mandates   as organizations   around what they're focused on.   Like, one thing that I feel like I've seen come out of the spiral team, especially through Matt's, like, leadership there is, a real focus on, like, what is the lightning experience look like on the mobile   on mobile apps. Right? So,   kinda like, what is the light client lightning experience and, like, the spiral team. And also to be fair, the ASIN, the Claire team, they're based out of France.  

They don't get as much representation in American,   bit coin situations because they are located in Europe. But, they have also, along with Spiral, really been pushing forward the mobile use case and what we need to add to the protocol,   across the board to make it more possible for mobile users to use. So yeah. I think there's, you know, I think it's a distributed protocol   2 crypto project that exists in entire space.   And it it is that, because we tried to do the spec process and we put a lot of time, energy, and effort across a bunch of different teams to, like, spin up different implementations, etcetera. So yeah. I I I think it's been a success so far. Yeah. I mean, I I apologize a little bit for just coming off swinging.  

[00:09:44] Unknown:

Let's go. I'm kind of can you explain what that spec process looks like,  

[00:09:50] Unknown:

Lisa? This is me. Yeah. Yes. I can explain what the process the process looks like. So the general idea so for those of you who aren't familiar with Spark processes, and I would assume most of us are not. I wasn't until I started working on Lightning.   The whole idea between behind having a specification.   So I like the way I like to describe it is there's, like, kind of, like, almost, like, tiers of, like, difficulty in open source software.   Tier 1 is, like, you write some software, and then you open source it. So writing software and, like, having a working thing is really difficult, and shipping that is, like, I think its own level of, like, a difficult thing.   Thing about a spec, I would say, is kinda one level more difficult. And that you're not just writing software, you're also writing a whole set of instructions   about how to   reimplement that software you just wrote. So it's, like, very detailed   instructions about how to recreate the thing that you've just written and published.  

But you do it in a group setting, so it's not just you alone writing code. You talk to a bunch of others. So you write you can write, like, a you write the instructions kinda by yourself or whatever, but then you send it to a committee   of other people.   And you have to convince one of them to also go through these, like, written instructions   and reimplement what you've done on their own, probably, in a different language,   from the written instructions that you've written down.   And then, you take the 2 different software open most, you know, these are all open source softwares, so it's multiple open source projects.   You take these 2. You point them at each other, and, like, okay. Do they interoperate?  

And if it does work, then once that's happened, once both teams have successfully implemented   this, like, written down from the written down description of the spec of, like, the process the software should follow, etcetera. Once you've got 2 teams in lightning, so there's, like, 4 implementations.   The current process is once you have 2 teams that have implemented just, like, written down instruction set, the other side's verified, hey, it looks good. We've gone through bit of a review process both on the code as well as, like, the written instruction set. Right? Like, in English.   At that point, we say, okay. This back is, like, considered part of the protocol. So it's like I would say it's, like, much more rigorous than just writing code and shipping it and getting building traction and getting users, etcetera. Like, you you have to do all that on top of this, but there's also this sort of rigorous,   peer review, peer process that spec stuff for lightning goes through.  



[00:12:12] Unknown:

Yeah. And, 4 implementations   are l and d, c lightning,   eclair, and then now LDK. Right? Right. Eclair was what we were talking about earlier, async. Yeah. Async. Yeah. Because for me, when we talk about specs, maybe that's where your first question comes in. They're like, are we at the point where   the consensus on a spec level in advance is possible? And I I would say, yeah, we're getting there where it like like it won't be possible.   But   What does that mean?   It means that you will start having   features implemented that if they get mainstream adoptions,   then   other implementations   will add   it instead of first agreeing on spec and then implementing.  

So but but for me,   that's a natural process that you see   in in any anything that becomes popular. And lightning now I mean, there there were there was this meme, what, like, 2 years ago, lightning doesn't exist, whatnot. Like, lightning nowadays is not processing. Like, it it's getting to the point where it's, like, 100 of 1,000,000 of dollars are settled across lightning.   Like it is in in production.   So just   the whole ecosystem,   the product is maturing, and we are getting to that point where, like you had browser wars. So   I don't think we'll have wars with different implementations,   but I do see the system. Not yet at least. No.  

I see the system maturing, and then we will have other protocols. Like, I know now CoinJoin,   CoinJoin is picking up those wars, but there is also, like, a coin pool protocol as a as a   spec is out. It's a proposal. So to me, the more   alternative layer 2 protocols we have, the the the better, and then some of the drama will move there. And,   yeah, the system will mature.  

[00:14:06] Unknown:

In addition to, like, spec, spec compatibility, I think I think there's an under kind of served area here in terms of product compatibility as well. Building on Bitcoin is hard just in and of itself. Building on lightning's   even harder just because of the complexity,   involved in the the safety like, the the need to be utmost secure throughout that whole process.   That said, there's there's just basic wallet compatibility stuff for everybody building wallets right now that needs to get needs to get this act together. And, hopefully,   I've I've put together a matrix   of, like, which wallets can send to other wallets, who defaults to 0 amount invoices,   who can't send to 0 amount invoices. And there's it's just a real big mess. And it feels like if all of us wallets could talk to each other and kinda get on the same page and work a little bit more in unison, that could add a lot more cohesion to the ecosystem.  

And so that's just kind of like a basic thing on a product level before   diving into even some of these deeper,   spec level things.  

[00:15:02] Unknown:

Hey. Let's do it right now, man. What it is? Like, BIP 21 with lightning  

[00:15:07] Unknown:

query string? Like I I'm all about BIP 21. So let's,   more to come there. You wanna explain BIP 21 to the audience? So BIP 21 essentially   lets you create it.   You can have a one shared QR code with,   combining an on chain and a lightning invoice. Essentially, it looks like a lightning invoice, but it has a fallback to an on chain,   wall address in there. I think that moving towards the standard,   will do a lot for just, like, increasing,   user adoption and making normies not have to think, like, am I sending on chain? Am I sending on lightning? Is lightning bitcoin? Like, what the hell is even going on here? And so if you have one QR code, if I scan with a lightning only wallet okay. Cool. I'll I'll take the fastest cheapest option. I'll use lightning. If I scan with a on chain only wallet, okay, I don't recognize that that lightning QR code, I'll fall back on that on chain address, send it on chain.   And so in your bank account, you might   it's like your debit account. You might move funds in there via a variety of different ways. And you don't really care. Like, is this an ACH? Is this, like, over the wire? You just see money in, money out. And I think as a community, I think the north star we should be working towards   is, just moving Bitcoin.  

And, hopefully, as we scale and as we get more users, we're using the faster, cheaper, more efficient layers.   But if we're going to onboard a 1,000,000,000 people, I think we need to make it really simpler like, much, much simpler. And, ultimately,   hope hopefully, in due time, I think, we can abstract away, like, the need to call out Lightning at all. It's just moving Bitcoin.  

[00:16:45] Unknown:

Yeah. Because I did bit 21 with Lightning inquiry string and BTC based server. Like, I coded that. But I know, Lisa, if we are agreeing here, then I need to get that in strike. You know? So what's your opinion on that one?  

[00:16:59] Unknown:

I'm a be honest. This is the first time hearing about BIP 21   With lightning. With lightning. Yes.   My understanding is that lightning invoices, you can add a fallback address on chain. So sounds like I mean,   from my knowledge of the current invoicing system on lightning,   seems like we're probably running to an issue there where existing Bitcoin wallets can't parse lightning invoices is a problem. Right? So the data that you want is that use case was considered when they wrote the lightning spec. But the problem is that the wallet implementers   do not offer a way to parse that data, so they have invented another spec called BIP 20 1 that they were getting  

[00:17:43] Unknown:

wallet people to implement. BIP 21 is is like the oldest   standard, then it's backwards compatible. But it's what you're saying is when, bolt bolt 11 was done, there was fallback address for on chain address. Mhmm. Like, this is the other way way around, and it's, like, the most backwards compatible.   So   there is this now conversation, like, okay. What is unified QR code? And are we   in agreement on feature set that it offers? Because when when we are talking about bolt 12 now, there is, like, a refund option and so on and so forth.   So I don't know. I need to see if I'm going with Miles, bit 21  

[00:18:21] Unknown:

lightning, or I'm going full back. Yes. It's I mean, obviously, there's gotta be a lot of community discussion here and make sure all the wallets are compatible. My understanding is that there's a number of, like, Coinbase   withdrawing from Coinbase   to a unified QR code will will break because   they haven't implemented that correctly. But it should just be a small   small change for a lot of these,  

[00:18:42] Unknown:

consumer wallets to get bit 21 working properly. Does Coinbase support withdrawal with Lightning? No. Okay. No. I see. I see. Okay. So there's, like, no chance that they would understand, like, the bolt 11 thing either. It's interesting. I mean, I think I think what you guys are highlighting is that I think there's a lot of work still to be done in the wallet space. I don't think   I don't think that, like, as an industry, we've quite figured out exactly   Like, some of the most exciting stuff yet to be built is, like, this, like, interop and, like, getting the experience exactly   hammered out at the wallet level. And,   I think, you know, I work on the spec layer, which is just some extent, like, a few layers beneath the wallet stuff. And I think having a lack of, like, a centralized, like, at least forum for specs on all that stuff, I think you can definitely tell through kind of all the, I wouldn't call it confusion, but,   all of the competing, like, specs little, like, suggestions of how to move forward.  

The fact that there's not, like, a consortium kinda like a consortium might be I don't know if that's a good word or a bad word.   But, like, you know, at least, like, the wind lightning started as a protocol. Like, there were we were everyone was there together writing the spec together and agreed that that's what they were working towards as a group. Whereas, I think on the wallet side, there hasn't been kinda that, like, cohesive   clarity of, like, exactly what   it should look like. Does that make sense? And, like, you're trying to figure that out. I don't think anyone has, like, the right answer.   But it is a little bit like that x kcd thing where you had a problem with, like, the previous specs. You made a new one. So there were 14 ways to do it. Now there's, like, 15 sort of thing. And I I do think we see that happening a lot on the wallet side.  



[00:20:27] Unknown:

Yeah. That's that's why I'm thankful to people like Parker who organized these events to get these conversations started because while we are transferring   100 of 1,000,000 of dollars across Lightning, there is still work to be done. So  

[00:20:43] Unknown:

I do wanna I I was talking about how hard it was to work on lightning a second ago, but I do wanna just, like, give a huge shout out to everybody working on this, especially Jeff from spiral. I just saw you over there. I didn't mean to leave you earlier.   But, like, 2 I remember 2 or 3 years ago at this point maybe, we're doing the lightning torch and trying to, like, stand up a node and get it all to work. And, man, it was just, like, show   liquidity? Yeah. It's, like, what what is inbound liquidity? Are you kidding me?   And so   seeing the slow but but steady progress   and seeing this   starting at the start of 2021, I think, but really progressing even more recently,   it's just gotten so much more usable, so much more reliable,  

[00:21:24] Unknown:

and it's a big testament to all the people working super hard on it. I remember I was sending a payment to Miles, and I guess it was early 2019,   and we were trying to figure out lightning. And I sent him the 1st payment. It didn't work. Then we, like, did some channel management stuff, and then I sent him the 2nd payment. It worked. We're like, yes, we got it. Then I went to send them the 3rd payment and we ran out of liquidity and it didn't work. We're like, what the hell is going on here?   It does feel like it's been a long time.   And,   and the whole space has, has moved forward tremendously.  

So I mean, with the bib 21 thing specifically, since it just got brought up, I mean, at the end of the day, you're talking about the wallets on the receiving side, being able to display this new QR code,   but it can also be solved on the opposite side with the sender. I mean, we see a moon wallet with 2 use taking the initiative there. I mean, you scan either side and it works   at what point,   at what point does it just become tech debt to say, like, I need to support copay wallet   or, you know, some wallet that no one's used in 5 years because know, it doesn't know what a lightning invoices.   Is that, like, is that an actual thing that Bitcoiners should be thinking about? Or are we are we supposed to support, you know, every   every wallet under the sun?  



[00:22:40] Unknown:

Whenever co pay is mentioned, like, I I, like,   congrats to BitPay for for taking, like, the wallet that was most advanced and running it into ground. Like, what is it? 0 innovation.   Yeah.   I think it will be a natural process.   Like, it's   you can only plan so much in advance,   and   that's what I what Lisa was saying.   Whole lightning spec, the the work done there was amazing. It was like all all the,   you know,   what it what what does Bolt stand for? It's like improvement.   Base is a lightning technology. Base is a lightning. There we go.   And Now we got the basics down. Yeah. Basics. No. You need to get it out. Listen, man. I'm VP of engineering now. I'm not into small details.  

But it's like   all those improvements, like, they were agreed upon, and then it was like hard work of implementing it. And now we are there. I don't expect that we are going to have another chance to, like, all, like, duke it out. But   that shouldn't we shouldn't look at that as a big problem. It's just okay. Let it go and grow organically and   let it happen.  

[00:23:53] Unknown:

Yeah. I think there's oh, man. I forgot what I was gonna say. Sorry.  

[00:23:57] Unknown:

No. I I got you with Bolt.  

[00:24:01] Unknown:

So correct me if I'm wrong, but on the Bolt 21 piece, I believe it's, like, every wallet should support this. Just some don't. And it's a very quick, easy change to support it. But as you're saying, it's,  

[00:24:12] Unknown:

like, the most the most possible backwards compatible. Is Bip 21 the one that Bipay was trying to push forward? They got a bunch of   oh, that was 70?   70. 71. Right?  

[00:24:25] Unknown:

Oh, so sorry. I remember what I wanted to say. The yeah. So, I mean, I think I think going back to, like, spec process in the bit 21 conversation and, like, clearly, this bit thing, bit 21 is a massive improvement to user experience   and that, like, Miles was saying, it sort of helps abstract away, like, the,   implementation details of how you're moving Bitcoin around, which I think is definitely, like, a very worthy goal to get to from a user perspective.   But, you know, like, all of these processes,   like, every spec is only as good as, like, the participation   you get into it. Right? And so, I mean, that's good for lightning as well. Like, we really need, like, everyone who's who wants to adhere to the the spec process. Like, we need you need, like, everyone, like, in the community participating. Right? So there's, like, a little bit of, like, you know, people have to, like, opt in to participation.  

And when people stop participating in the process for whatever reason, maybe they're busy. Maybe they've got, like, you know, business reasons. They're, like, more pressing, like, material concerns of where they're spending their dev time, etcetera.   Maybe they've got some competing reason why they don't wanna make it happen, whatever. But,   you know, really, it's like everyone chipping in and participating and get excited about, like, the possibility of interoperation   that really moves, like, all this, like, entire, you know, network forward.  

[00:25:42] Unknown:

Yeah. And I I always now when I'm speaking, I say to   prospective developers that want to get more involved. Like, maybe it's what Lisa is saying, this problem, that problem, but maybe you are already doing good work and it's just not   publicized well enough. So if there are such people in audience,   yeah, talking about you, man,   like,   reach out after this talk and let let's see how to work together because, yeah, this is open source. Let's, yeah, let's collaborate.  

[00:26:10] Unknown:

And and I wanna extend the invitation far beyond just engineers as well. A lot of this   envisioned ideal user experience discussion is happening in the Bitcoin design community.   And, like, we if you're a designer, hop on in there and share your input. If you're just a normal user, hop on in there and share your input and help us kinda come to consensus. You don't have to be an engineer to have an opinion.   There's all sorts of stakeholders in this environment,   and certain people have strengths in certain areas.   These these 2 are deep in the engineering scene.  

I   I don't know what my strengths are exactly, but, compared to them. But,   but, like, there's a there's a million ways to contribute. And maybe you're just maybe you just send the 100 lightning payments every day. But as a result of that, you know all the wallets, you know where the incompatibilities   are, and you can help drive something forward that maybe then that somebody else picks up and, ultimately gets all the different wallets and the different implementations   aligned and just build a a much better global lightning experience.  

[00:27:14] Unknown:

I I mean, this has been a great conversation, guys. We're almost out of time. I figure we'll wrap up with final thoughts. Rockstar, final thoughts.  

[00:27:24] Unknown:

Just glad to be with all of you here and that we're, you know,   participating in building history. Like, when we look back on this event in a year or 2, it will be like it happened here. Some,   agreements were made here. Somewhat had a   moment.   So just looking forward to this journey with you. Let's go. Marty,  

[00:27:46] Unknown:

we will win. Thanks, rock star. Lisa, final thoughts.  

[00:27:50] Unknown:

Okay. So I think Lightning is awesome. I think Lightning has a lot of potential to really prove itself as being, like, the true heart of the Web 3 movement in terms of decentralization   and, like, self sovereignty.   And I'm really excited about it. Also gonna take 2 seconds to show the fact that base 58, my Bitcoin transaction protocol class, just open registration for our next class. So if you are really interested in finding out more, getting involved in how the Bitcoin protocol works, we do a 6 weeks class, and we'd really love to have you in our class. So check out our website at base58.info.  

[00:28:21] Unknown:

Thanks, Lisa. Really appreciate your work with Base 58.   Miles, final thoughts. Prometheans versus Malthusians.   Don't forget it.   Thanks, guys.