Citadel Dispatch

Share on social media:

Justin and Brad are contributors to the Fedimint open source project. Fedimint is an open protocol providing easy to use, private, programmable, and offline bitcoin payments using bitcoin powered federated chaumian ecash.

Justin on Nostr: https://primal.net/p/nprofile1qqspg8fq209jj56663d2n6r9ehkyjffy7rkqqejfdwvtwzva426avkqxtxxuv
Fedimint Website: https://fedimint.org/
Fedimint on X: https://x.com/fedimint
The E-Cash App: https://ecash.love/
Fedimint Observer: https://observer.fedimint.org/
Bitcoin Mints: https://bitcoinmints.com/
Harbor Desktop App: https://harbor.cash/
Iroh: https://www.iroh.computer/

EPISODE: 178
BLOCK: 914982
PRICE: 863 sats per dollar

(00:03:11) Happy Bitcoin Tuesday

(00:04:13) Fedimint and Cashu

(00:10:49) Fedimint's On-Chain and Lightning Integration

(00:18:46) Challenges and Solutions in Fedimint's Development

(00:39:08) Fedimint Operator Setup and Challenges

(00:51:00) Backup and Recovery in Fedimint

(01:02:02) Future Developments and Community Engagement

Video: https://primal.net/e/nevent1qqsvswtcudcf2j55ac4yrphsm37ep9acp27ctap9dt8zvy6p5wzywgcrax3wu

more info on the show: https://citadeldispatch.com
learn more about me: https://odell.xyz

[00:00:03] Unknown:

Outskirts of Addis Ababa, thousands of servers work on solving complex mathematical challenges   that'll be rewarded with Bitcoins.  

[00:00:12] Unknown:

Zero.   I will just mount to think This mining  

[00:00:15] Unknown:

is energy intensive.   Here, they're using surplus electricity   from Ethiopia's dams.  

[00:00:22] Unknown:

The Bitcoin mining has a great advantage.   The capability to consume   huge amount of energy to deploy fast and to do it very close to the source, like, for example, here near the substation.   After many years, when the distribution grid deployed, other industries come. Probably, the power will be redistributed.  

[00:00:42] Unknown:

Ethiopia's recently inaugurated Grand Ethiopian Renaissance Dam or GERD will more than double the country's electricity output.   But Ethiopia doesn't yet have the distribution network to take this power to the half of the population that doesn't have electricity.   The idea is that the fees paid by the Bitcoin miners to use the surplus   energy, some $220,000,000   worth in the last twelve months, will go towards funding the expansion of the grid.   The government had a target of 2030   for universal access to electricity,   but now says meeting that will be extremely challenging.  

Until the people have full access and, the economy start to consume all this, power, we are using Bitcoin for transition. It's not, permanent,   endeavor. So they're not getting preference over domestic industry or No way. No way. Yeah. No way.   The cost of mining 1 Bitcoin in Ethiopia is about $20,000   when hardware, staffing, and other costs added along with the cheap electricity.   But that Bitcoin   then sells for around a $114,000   at current prices.  

[00:01:51] Unknown:

It's very profitable because 80%   of the operating expenses for a Bitcoin miner,   is related to electricity.   And in parts of the European Union, they're looking at 15 or 20¢ kilowatt hour. So to come to Ethiopia and get 3¢,   or sub,  

[00:02:10] Unknown:

4¢, I think is extremely lucrative. We'll see. For now, the price of Bitcoin is climbing and has become less volatile.   As more countries, notably The US, become more receptive to digital assets like Bitcoin, then Ethiopia's appeal as a crypto mining hub is likely to grow.   Bernard Smith, Al Jazeera, Addis Ababa.  

[00:03:11] ODELL:

Happy Bitcoin Tuesday, Freaks. It's your host, Odell, here for another CIL dispatch,   the interactive live show   focused on actual Bitcoin and Freedom Tech discussion.   As always, dispatch is brought to you without ads or sponsors. It is supported   by viewers like you sending Bitcoin donations   to the show.   All relevant links are at citadeldispatch.com.   The largest Zap we got   last week was from ride or die freak Trazyn. That's t r a z y n.   And they said   oh, I lost it.   Thanks again for the awesome rip.   Cheers. I'm glad you found it helpful.   The easiest way to support the show is   through your favorite Nostra app.  

I'm particularly fond of Primal. I've been helping build it out. You can search Primal in your favorite App Store   and simply search Citadel.   Freaks, we have a great show lined up today. We'll be talking about the Fedimint open source project.   I got two of,   two prolific Fedimint   contributors here today. We have Justin. How's it going, Justin?   Hello. Nice to be here. Thanks for having me on. Great to have you. And we have Brad. What's going on, Brad? Hey. Thanks for having us. This will this will be fun.   Cheers to that.  

So, I mean, we've talked   I've had a I've had a couple shows on Fediment   very early on,   and I've had a cup I've had many shows relatively recently on Cashew.   Both Cashew and Fedimint to the freaks that are not aware share the same,   overarching principle of using Charming eCash with Bitcoin.   But then there are   significant differences in actual implementation.   So I think a interesting place to start   is just in general,   And you guys have made a lot of progress, which is, I think, why it's important that we're having this show. And, also, last but not least,   just for people to be clear, there is a   venture backed company called Fetti,   which plans to   open source their front end client   for Fediment, which is a open source project that is not VC backed and is available to the public as free use software,   that you can use any client as the front end for. But, anyway, I think a great place to start here is why should people why should people care about Fedimint?  

Why does the project matter?  

[00:05:55] Justin:

Yeah. Sure. I can start with that.   So I think I think Fedimint offers a   nice set of trade offs for, like, using Bitcoin,   different from what   maybe other projects have.   Doesn't replace, you know, on chain self custody Bitcoin,   or even probably running your own,   Lightning node, but it does offer, like, a really easy and private option for,   holding and spending Bitcoin.   And that's kinda down to the, like you mentioned, the the Xiaomi and e cash cash part,   and then also integrating with Lightning because,   it's important. Like, Lightning is kind of the glue for the rest of the the Bitcoin ecosystem,   and Lightning's already already pretty private too. So,   yeah, I think it's just a nice nice set of trade offs.  



[00:06:45] Brad:

Yeah. I definitely echo all that Justin just said there.   Maybe zooming out a touch. I bet a lot of freaks have some knowledge on Chomping and eCatchments. I may hand off this conversation to some folks that are not so well versed in the history of Chow Me and eCash Mints.   The way to think about it in a nutshell and why would you wanna engage with one of these systems is you have this system   that you can   deposit funds into. You can send Bitcoin to in the case of Fedimint,   multisig wallet, and then you can do some cool stuff. You might get some eCash notes that don't have an account identifier or simply notes with the denomination that you can swap out. You can swap those notes out for either another on chain payments or a payment over the Lightning Network.   And if that's interesting to you, then the system might actually be pretty interesting to you in general.  

And then one of the biggest differences between something like Cashew and Fedimint   is that Cashew took a beautiful approach for simplicity, a single server that doesn't have to concern itself with a bunch of consensus mechanisms of how do you figure out, like, what is the state of the blockchain? Like, what is the state of my internal database?   Where Fedimint   tries to distribute that. That's the federated part of the the Fedimint.   It splits that across multiple peers, sometimes referred to as guardians,   and then you have the ability to tolerate some faults. So let's say one server goes down and you have three out of four total servers running, the federation just continues.  

We could go into a lot of those guts there, but, yeah, at at a high level, that's kinda how I like to think about the system.  

[00:08:21] ODELL:

Yeah. I mean, it also means   I I like I like the idea of thinking about it as fault tolerance.   I mean, a lot of people think about it from a point of, like,   rug pull resistance   from, like, a malicious actor.   But in these types of systems, it doesn't even have to be a malicious actor. Right? It could just be   someone's server went down or they messed up backups or something to that effect.   Yeah. Which is actually probably much more common in the space.   Yeah.  

[00:08:50] Justin:

The upgrade Oops. Upgrade oops. The upgrade case is the most common. Right? You know, we're upgrading the software, and you wanna be able to upgrade with with no downtime. And,   yeah, with the federation sort of built in, you can do that very, very easily. Just upgrade one node at a time.  

[00:09:08] ODELL:

Yeah. I mean, when I think about   when Charmaine and ECASH in general, whether it's Fedimint or Cashew,   I mean, I think I think it's we're gonna be diving into, like, deep technical specifics. But the cool part to me   is that from a user point of view,   it is actually   it it can be and it's already close to being   the easiest way to use Bitcoin in a very powerful way. It's programmable. It's offline. It's seamless. It can be seamless UX,   and it has strong privacy guarantees.   And, historically, when you look at   trying to get the most amount of users to use something   in a privacy   in a private way or freedom oriented way,   it doesn't really matter how many podcasts you do or tweets you send or education you try and do. What really gets more users   using technology in more powerful ways is when it's the easier cheaper option.  

And we've seen the project I always highlight is signal.   And and the reason my 90 year old grandmother is sending encrypted messages with me is because it's a seamless, easy way for her to get baby pictures.   She's not thinking about any of the technical specifics. It's just a pretty app that works and is reliable and is easy to use. And when I when I think about,   you know, onboarding   the next   next billion people onto Bitcoin or something and you want them to use it in a powerful freedom oriented way,   we're we're gonna need something like that. And I think I think Chong and Ecash plays a key part here.  

It'll probably be a combination between Cashew and Fetterman.   I will say Cashew has had a lot more momentum lately.   But you guys have seemed to have   been really putting the work in, so I wanna really dive into   to to what has gotten done,   on the the Fedimint side. But real quick, the other big differentiation   since we're, like, kinda starting off the bat with differentiations between cashew and   FEDAMENT is Cashew made a very distinct decision early on,   to kinda be lightning native.   Well, FEDIMENT   is almost it's, like, built on top, and I'm a little   over my skis here, so I'm gonna need your guys' help here. But it's it's to me, it's like it's on chain native   with,   like, Lightning gateways. Can we talk about that difference in like, how how does that affect things in practice?  



[00:11:46] Brad:

Sure. I'd love to dive into the on chain side and then maybe hand it off to Justin for the Lightning gateway side. Awesome.   So easy.   So when you set up the federation,   let's just use the canonical example of a three out of four federation. So when I say three out of four, that means that it only requires three of the four members to continue operating in order for everything to run smoothly.   But when you do this initial setup, DKG distributed key generation, you get a bunch of different private key material that gets generated.   Part of this setup process will generate,   a   pay to witness script hash   threshold   where you say three out of four of these keys need to be combined in order to move funds on chain. And so you have this static descriptor that's like, hey. This is the descriptor for this federation. So It's basically an on chain multisig. Right? It's an on chain yeah. That's a fancy way to say it's just an on chain multisig. The canonical example is three out of four. You could have five out of seven that continues to scale.  

But the the it's really just that in a nutshell. So funds do get sent into this multisig.   We don't have a ton of time to turn this into a Balaji style podcast. We also have to add them to Please no. Yeah. I'm happy to if folks want to and if you would've skipped your haircut now, but totally understand if you don't.   But the in a nutshell, it's really cool how you do a deposit on chain. I'll just do a quick tangent into there.   You say, well, that's the descriptor for this federation. I would love to send some funds into this federation, but I'm not gonna just send it to that address. What I'm gonna do is I'm gonna tweak that. I'm gonna generate another key pair on my side as a client, and then I'm gonna tweak their descriptor using   this public key. And then I'm gonna go ahead and send funds to this address that's been tweaked. So the federation actually has no clue that this address exists because it's just a tweet,   a tweet descriptor using some crazy random private key, public key pair.  

And then once the client observes, oh, cool. My funds have been sent on chain,   I wanna prove to the federation   that I actually sent those funds, and I would love some e cash in return. So the client does this all on your behalf. It monitors the chain, and it's like, sweet. My funds   showed up at this address that I generated.   Let me go ahead and construct a proof to the federation that says, take a look at this Fedimint transaction.   This is my claim that I tweaked your descriptor using this key pair. I'm actually gonna sign this FEDIMIT transaction transaction   with the private key   used in this key pair. Therefore, you can verify, like, the signature and make sure the tweak all matches. And if that all works, go ahead and give me some ecash, please. So that's the like, in a nutshell, how   a deposit works with Envademy. So, yeah, that's completely on chain native, and then there's it's a much more simple process in order to do a withdrawal where you just say, hey. I got some eCash. Would you please   satisfy an on chain transaction that sends to this address that I'm requesting? And as long as it's valid eCash, you're good to go. So I'll I'll go ahead.  



[00:14:48] ODELL:

I mean,   from my correct me if I'm wrong, but from my perspective,   this choice of of of making it on chain native,   to me, adds a lot of user benefit. It makes it easier to load the wallet at first. It makes it easier to withdraw if you wanna if you wanna leave. Right? Because,   I've I received from, like, a let's just use a standard,   you know, users buying Bitcoin on Coinbase. They have on chain Bitcoin.   It's like, how do they get it into   into something that is lightning only? Right? That which is what Cashew is.   So they first need to find some kind of intermediary wallet.  

I mean, I guess Coinbase might offer withdrawals on lightning now, but a lot of places don't. Yeah. And you don't know if there'll be liquidity. So then first, they have to get into a lightning wallet, and then they have to do that. And then on the reverse, when they're leaving,   when they wanna leave when they wanna leave a mint,   they need to make sure that they have somewhere they can send it that has inbound   Lightning liquidity,   from that Mint operator, which is easier said than done. And as small amounts, it works pretty well,   but there's a lot of UX headache that happens there. So you have the benefits to the user by going on chain native.  

But then from   a protocol operator standpoint, it adds a ton of complexity.   Right? Like, that's the trade off, basically.  

[00:16:14] Brad:

Hey. What are my coins, and what coins am I gonna use to satisfy this? What's your request? Now we have consensus. It's like, oh, shoot. Review of the law looks different than mine. Review of the law. Yeah. Well, cash is just like a single you're just like running a lightning node. It's like relatively straightforward.  

[00:16:29] ODELL:

Mhmm.  

[00:16:30] Justin:

Yeah. For the   on the lightning side for for Fedimint, basically,   the decision comes down to, like, the on chain part is sort of complex enough with, like Brad said, making sure the consensus   is,   working between all the nodes.   If you were to sort of, like, make that lightning native, you would have to essentially build,   like, a federated lightning node,   which I think there's been sort of some interest in doing,   but it's it's really hard. Right? You have to have you said you have to put all that, like, the lightning logic   into a federation, which is which is difficult.  

So I don't wanna I don't wanna say that, like, Fetamin's not lightning native. It's just not,   like you said, the backing of the Mint. The backing of the Mint is always,   the on   chain the on chain, like, multisig.   So the the approach that we took or that, you know, Eric took in the beginning was,   essentially, the Lightning support is kind of an extension.   You can think of, like, Fedimans at, like, these gateways   that provide the Lightning functionality   as an extension to the Lightning Network.   So the gateway is kind of like the last   hop   in, like, in like, if you're sending, like, an HTLC.  

So these gateways will essentially do the translation.   They'll do swaps essentially.   So you're sending some ecash out of a federation.   You will at first lock up some ecash in an outgoing contract.   And you can think of this contract like an HTLC.   It has a,   a hash, and you can claim that ecash with a preimage, And it also has a time out.   And so you then, essentially, you tell a gateway,   hey, gateway.   You can have this e cache   as long as you pay this lightning invoice for me.   And that and, you know, that process is atomic. Essentially. The once the gateway pays the invoice, it comes back and gets the e cash for,   that outgoing contract to get paid.  

So the yeah. As far as the trust model goes, the the gateways don't have to be don't have to be trusted.   Very similar to running a a Lightning node.   But the gateways do have to trust the federation,   because you're obviously dealing in ECash at that point.   And so the, you know,   the the gateway needs to trust the federation not to not to rug it as well.  

[00:18:54] ODELL:

And the gateway doesn't have to be one of the guardians.   It doesn't have to be one of the Fedimin operators. Correct.  

[00:19:01] Justin:

And you can actually as a gateway,   you can connect to multiple federations.   And so we   we do some shortcuts where,   say you're a gateway   and you're connected to two different federations.   If you wanna pay,   from one federation to the other,   the gateway actually won't even do the payment over the Lightning Network and will instead just do an e cash swap.  

[00:19:28] ODELL:

They're basically like a bridge at that point.  

[00:19:30] Justin:

Yeah. Exactly.  

[00:19:33] ODELL:

I mean, the the original dream was that, basically,   gateway operators would be,   you know, a relative free market that you'd have, like, multiple gateways for for each Fediment. Is that still implied? Or  

[00:19:49] Justin:

Yeah.   The the tricky point you get into is that   gateways kind of provide, like, the quality of service for the Lightning payments.   And so you run into this scenario where if you have, like, a truly, like, oh, anyone can provide,   like, liquidity for the Lightning Network, you can you have this attack where someone would just, like,  

[00:20:10] ODELL:

a very bad Lightning node, you know, connects and then all of a sudden you It's like a denial of service attack. Like, the users are trying to send a lightning payment or receive a lightning payment and continues to fail because Exactly. You're sitting there as a nonfunctional gateway. Yeah. So we have we have a concept of, like, vetted gateways where,  

[00:20:29] Justin:

anyone can run one, but in order to get sort of, like, approved by the federation, the the the Guardian numbers will,   will vet you and say, hey. You know, this   we think that this gate will will provide,  

[00:20:44] ODELL:

a good quality of service. And how does that work in practice?  

[00:20:49] Justin:

Just the the operators need to get in touch to   it's just like a a setting, a config setting, essentially. And so what, like, a user's client  

[00:20:58] ODELL:

that when a user is sending or receiving a lightning payment, they're not, like, choosing a gateway. The app is automatically choosing for them? Yeah. Correct.  

[00:21:06] Justin:

There the app can make sort of some   right now, it's not exposed   as far as, like, when you're sending a lightning payment.   Later, but we have a we have a new Fedimint, wallet,   that we launched on Friday, actually. That was my next question. And, we expose it through there. We we don't actually we don't let users choose yet, but we show, like, here are the gateways that are connected, and here's their lightning node, and here's what fees they charge and things like that. Okay. So let's talk  

[00:21:36] ODELL:

let's talk about this wallet first,   and then so let's we'll start with the user side. Right? Like, the user facing stuff. And then I wanna kinda dive into   what it looks like currently from, like, the Fedimint operator side and, like, what their software stack looks like and what that burden looks like. Yeah.   So what is this wallet that you launched?  

[00:21:59] Justin:

Yeah.   So we're calling it the eCash app.  

[00:22:03] ODELL:

Really, really creative name.  

[00:22:05] Justin:

Shout out to Justin Moon on that. That was his idea.   Yeah. We have so it's, it's a Fedimint wallet.   We have lightning on chain and e like, full e cash support. So you can kinda choose which way you   interact with,   with payments.   Let's see what we have lightning address.   It's only for Android right now.   So you can go to either   Zapstore   or,   ecash.love  

[00:22:34] ODELL:

to download it is our is our domain. Oh, ecash.love   is a great domain   name for, like, a non   non top tier suffix. Yeah. Yeah.  

[00:22:45] Justin:

So we have, evolved that. We tried to integrate some Nostra features. So we have,   Nostra WalletConnect.   We have NIP 87. So that's, like,   that's Cashew as well where you can use Nostra to, like,   discover federations.   There's kind of this problem when you first download the app. You know? Is that the rating thing?   That the ratings part of it, we don't have ratings in there yet. That's you know, we'd like to add that. But,   the   we we use the announcement part where you're kinda announcing, like, hey. This is a federation you can you can join. Like, that's what the front end for it  

[00:23:21] ODELL:

that's,   bitcoinmints.com.   Right? Isn't it 57?  

[00:23:25] Justin:

I believe so. Yeah.  

[00:23:27] ODELL:

Yeah. So it's it's like a discovery mechanism. These are the mints available.   And then ideally, you're also doing kind of like a web of trust.   Odell likes this FEDIMENT.  

[00:23:38] Justin:

Yeah.   Yeah, we don't have the recommendations in there, but I think that's a cool idea.   And it's kinda, yeah, it's the way of sort of bootstrapping it. If you download it, you don't have a federation to join. You can join one of those.  

[00:23:50] ODELL:

I mean, that's the biggest   one of the biggest UX hurdles is.   I'm I've talked about this in the past. I just had Eggie on,   Yeah. CashU contributor,   working on CDK and and pub.cash.   And, like, the biggest hurdle is the app providers,   I and I respect it, don't wanna pick a mint for you. So, like, every user's first question   is, like, what what do I choose? Right. Which mint do I pick? And so trying to mitigate that without   defaulting to, like, one big mint is,   will will probably always be a problem, but  

[00:24:29] Justin:

it's we can smooth it out. We can make it a little better. Yeah. And in in the app, we have things like you can view the UTXOs   because   everything's on chain. So we have, like, links to if you try try to join a federation, they'll give you a little preview, and they'll show you how many guardians there are. And then, That's cool. Where the UTXOs are on the in the blockchain.   Probably talk about so we also have a backup recovery,   which is kind of our, it's built into the feminine protocol,   and then the app our app supports it as well. So we have, like, a pretty familiar, like,   BIP 39, like, seed phrase,   that all your e cash is derived from. So you just,   it's kind of a nice experience. I mean, you you say the seed phrase, and then you can use that to get your your e cash back.  

The one caveat is   in at least at the phenoment protocol layer,   you need to remember, like, which mints you've joined,   because your your ecash is specific to a particular mint.   But for what we did in the app is that we actually   encrypt that information and and save it as a replaceable event on on Master.   So you're   we essentially derive a Master key from your   c backup,   encrypt which means you've joined, save it to save it to, and then   can retrieve that information later when you enter your seed phrase.   So it gives, like, a nice experience where you just enter your seed phrase, and then all your e cache comes back immediately.  

[00:26:07] ODELL:

Yeah. So we have that that The alternative is you would need, like, an you'd be needing to run, like, an additional server to be able to do that kind of functionality. Right? Or just, like, remember, like, remember, oh, you know, my e cash is with,  

[00:26:21] Justin:

Bitcoin principles or whatever, and then you'd have to, like, manually recover that way.   This is kind of a nice little shortcut.   And, yeah, I mentioned, a few of the other nerdy sort of, like, sediment specific features. We have the the UTXOs   of the federation,   the gateways. So you can kind of view the gateways for each federation.   And then we also have, like, a note breakdown.   I don't know how many,   like, Ecash wallets in general expose that, but, like, in,   all Ecash is, like, fixed denomination notes. Right? And you're you're it's similar to, like like a paper wallet.   Right. And it's similar to kinda like a UTXO   as far as,   when you make a payment, the the wallet is sort of doing,   like, note selection, like, analogous to coin selection.  

So we we expose that. We expose a view. We don't expose, like,   note selection yet. That'd be something we'd we'd even wanna add at some point. I mean, why would we why would the user even need that?  

[00:27:23] ODELL:

Just  

[00:27:26] Justin:

as a as a convenience.   One one feature we'd want to maybe add is, like, having the ability to, like, print   just in general in impediment, like, print paper e cash.   Got it. And if if you have to, like,   you kinda have to, like, reverse engineer the algorithm if you wanna know   how many notes it's gonna like, how big the QR code's gonna be based on the algorithm for the the note selection.   So just having those selection for that would would be a little bit easier. But,   and, yeah, the the idea is to kinda just give a bit more, like, transparency   and as far as,   like, here's what your wallet's actually doing. Here's how the notes are stored. Here's what the gateways are doing.  

Not all of it is maybe, like,   not all users will maybe even know what that info is for, but, like, if you if you want, you can you can go find it in our in our app. So,   yeah, that's that's mostly that's most of the features I'd say. Yeah. Like I said, ecash.com. What are your thoughts on, like,  

[00:28:29] ODELL:

so, like, when an ecash wallet has, like, a lot of,   lot of tokens in it, a lot of individual tokens in it, it, like, causes all sorts of performance issues.   Is there, like,   is there a negative to, like,   an easy consolidate button that just   kinda, like,   takes a thousand tokens and turns it into   50 or something?  

[00:28:56] Brad:

Yeah. Potentially.   We could talk about some of those trade offs.   So if you have fees on the mint, which no mints out there are collecting mint based fees at the moment, but in the future, you can imagine a world where they will,   that kinda sucks. You're paying money to consolidate. It's like, I don't really wanna pay money. Just I still have the same amount of money with a little bit less. So I think that's one of the major drawbacks. You also do kinda have, like, a very   obvious timing footprint of, like, here's all the eGash notes that I care about  

[00:29:25] ODELL:

spinning Right. Now. So stuff like that is not ideal. But But that's why, I mean, that's why manual consolidation   has a bunch of   timing risks, but, like, if there's an easy consolidate button   that kinda I don't know. Like, did it over time or something.   I I I there's look. And the fee thing makes sense to me, but also, like,   we consolidate   on Bitcoin on chain,   and in fact, to save on fees in the long term,   which I think would probably be   if you were deliberate about it, would be a similar scenario now because, I mean, mint fees could increase over time.  

[00:30:10] Brad:

Not yet. It's a tricky technical problem.  

[00:30:13] ODELL:

Fair enough. Future, they would be able to trick so they can't. So at launch, once you launch a Fedimint, you're not changing your fee structure. As of right now. Yeah.  

[00:30:24] Justin:

It's again, that's where we get into the tricky consensus issues.   We sort of have to deal with a lot of similar,   like, software problems that Bitcoin has to deal with because we're a consensus system.   And, you know, that, you know, that's   part of what makes us go a bit a bit slower, like, maybe in in comparison to Cashier because there's those challenges.   But, yeah, that's  

[00:30:49] Brad:

I do wanna circle back to your point, though, Matt. I think it's excellent to consider just like the trade offs. Like, yeah, it might be like a a really small fee if you're paying, like, a SaaS Yeah. Or something.   Then now all of a sudden, you don't have to if you wanna, you know, go spend a $100 worth of e cash, like, you're not gonna have to have a giant string of text or have an animated QR code that needs to scan, like, a 150  

[00:31:11] ODELL:

little parts. Because you know where you see it.   It's because of Nasr.   Like, if I in my day to day transactions,   you know, okay. People are sending me $5, $10, $20, whatever. I'm going to I mean, I wish my farmer's market accepted eCash, but I'm going to the farmer's market. I'm paying $20 for steak or whatever. But, like, on Nostr, they're sending me 42 sats or 20 sats, and then you end up with, like, thousands   thousands of tiny, tiny,   amounts   that always have to be constant. It's just like a constant challenge.  

[00:31:46] Justin:

Yeah. Our our algorithm right now does try   when you transact, it does try to consolidate, but, yeah, you you you still run into that situation. So,  

[00:31:56] ODELL:

And,   specifically, I think   and, you know, obviously, I'm more bullish on Nostra than most people.   And, you know, it's still an open question of of how successful Nostra will be.   But, specifically, both Fedimin and Cashew kinda solve that   first problem   in a very elegant way, which is, like, new user comes to Nostra and wants immediately to receive Bitcoin. Doesn't have a lightning node.   Obviously, on chain is not practical,   for that situation because they're receiving 42 sets.   So I think if if   if these projects are successful, that's gonna be it's like, it's either we're not successful   or that's gonna be a big problem.  

So just just to point it out early is my point. Yeah. It's a good point.   Okay. I'll just let go of Greg. Quick 100%.  

[00:32:50] Brad:

Sweet. On the the previous topic of how do you know what Mint   do you wanna join? Like, is there any rating system? So that's not currently in the eCash app, but there is a separate system, a different website you can go to, observer.fediment.org.   So this is an entirely separate system that Eric Sirian, started to build out. I chipped away,   making some contributions on as well. It's a really cool way to take to connect to a mist.   And without getting too much into the nitty gritty,   you can essentially scan the entire history of that mint and then observe everything that is observable   for that mint from a third party perspective.  

If anybody is curious about that topic in-depth, I gave a talk at BTC plus plus Berlin last year that's on YouTube that you could take a look at the types of things that you can observe.   But this also gives you a really nice way to explore, like, oh, look at the number of transactions that we're seeing on each of these different mints. What's the total number of assets and liabilities held on these mints?   Then what's the rating system? So that's another cool resource that's not currently in the eCash app. Observer.fediment.org.  

[00:33:55] ODELL:

Correct. Oh, yeah. I'll put all these links there. I mean, I also think bitcoinmints.com   is great because it has the reviews.   That's obviously not FEDIMENT only. It also has cashew mints in there. And that does bring up an interesting point,   that because FEDIMENT is on chain native, you do get that additional data point of   how much Bitcoin is actually in the multisig.  

[00:34:20] Brad:

Totally. And on that point too,   part of the consensus mechanism, whenever there's something being updated in the system,   it will do a full audit. So, hey. Let's take a look at all of the assets and all of the liabilities I have across this Mint and make sure that my liabilities do not exceed   the assets. If there is an issue with the accounting there, I'm panicking and everything shuts down.   So that's something that the operator doesn't really know that they're doing. But each time that there's any type of update, there's actually a full audit that's run on them. And then other users, if they're curious, can essentially perform But then we're you're trusting the operator   in live load drive. All at the end of the day, we'll go how much e cash is out there. Yeah. Yeah. Exactly. So you can   you   people can play games. They can fork the software, and if they're clever, can do naughty things.  

Where reputation systems come in handy. And, again, it's it's too going back to kinda, like, why would anybody wanna use this system and you explore the trade offs, that's a trade off you're uncomfortable with. I I wouldn't touch it. Like, that's totally Yeah. I mean, there's there's a lot of focus on the idea of just, like,  

[00:35:25] ODELL:

one and done rug pull.   But, the, like, the scarier trade off with both Cashew and Fediment   is, like, the   silent inflation,   you know, like, the slow the slow rug pull. You're running, like, a fractional reserve for a while. Yeah. The office space protocol.  

[00:35:46] Justin:

Yeah. That's kinda where the go. Like, Kyle, I had that idea with,   like, the automated bank runs,   where you have, like, the window.   I that idea is cool. I'm not aware of anyone working on that, but that's that would be kind of the mitigation on the,   the silent or the slow run pull. But  

[00:36:04] ODELL:

No. But it still doesn't   I mean, like, it's better than all these things are better than nothing.   And I think you put all the pieces together,   and you have a relatively,   particularly if you're using this as, like, a, using these as as spending wallets, right, and not, like, life savings, which is how I think most people should approach it. It's like once you hit a certain amount. I think Zeus does a really cool job with this. Actually, like, building it into their UX. Once you hit a certain amount, then it goes out to self custody,   Mhmm. Whether that's lightning. I ideally, I mean, on the in a Fedimint side, I think it'd be really cool   if it just, like,   in in the eCash app or whatever the front end app is. Like, once you hit a certain amount,   it it prompts the user to put in an on chain address and just sends out to an on chain address. And if the user needs to reload it, they can always just send it back,   maybe even build an on chain wallet directly into the app and just   you can kinda just make it one click, just goes into the on chain self custody portion.  

I think that is pretty cool way of reducing risk.   But, like, even his e it's long winded, but even his, like, Cashew Auditor thing, the round robin,   it kinda doesn't you can, like, fake it out. It's just checking to see if if lightning payments are going through in a reliable way.   So you can fake it out for a bit while you're doing things. And then, also, second of all, like, it definitely does not stop like the one and done rug where you can have, like, a cashew mint that is awesome.   It could be awesome for five years,   and then it could hit a certain amount. And then the operator's like, nice. You know, I'm gonna go take a trip to a Visa, and he's just out. And, like, there's no way for you to stop that with the auditor thing. Yeah. Totally agree. Yeah. I am not, I've never considered a way to solve those problems. They may be Yeah. Solvable. And Just just mitigations.  

That's all. Yeah. You can add a bunch of little pieces that, like, educates the user, allows them to make more educated decisions, reduce their risk.   And I really like the idea of, like,   like, building it into UX flow, like, the balancing of, like, how much is in   e cash versus how much is in on chain or lightning,   self custody. I think that is   I I I think that could be, like, the holy grail of kind of the balancing act here. If if you if you do it in a seamless way for the user where they're not even really   kind of considering how it's happening behind the scenes, they're just like, okay. I got my checking account, and I got my and maybe that's not even a good analogy anymore because I don't know if Gen z even knows what bank account is. Like, I have my checking account and I have my savings account,   and it's two separate pools of money with different different trade off balance.  

Okay. Let's go to the operator side because I think the biggest issue I mean, Fedimint has, let's be honest, has had multiple issues in terms of organic adoption.   I think partially has been the front end side.   But I think probably, which is great to see another Federman focused   front end with your Ecash app get released.   And the immunity guys are working on it for a while on the front end side. They got burnt out,   pivoted to AI,   which is actually not the worst thing in the world   for us as users because I really I really like Maple now.   So,   all power to Tony on that. Two of them actually, they're making Harbor Wallet, which is gonna be a Federman front end. I think it's out right now. I don't know. I haven't tested it. Yep.  



[00:39:36] Justin:

I think they had a one o release.   Yeah. Yeah.  

[00:39:40] ODELL:

But have you tested it?  

[00:39:42] Brad:

I have not. I have a friend that did, though. Yeah. So part of the release process testing that I do, I help coordinate, make sure that every time we create a major release that we try to test as many scenarios as we possibly can. Part of that is downstream clients being able to easily integrate with any changes. So, yeah, definitely spin up, couple VMs and have, dedicated ones just for building harbor and making sure the desktop client works well.  

[00:40:06] Justin:

Yeah. Yeah. It's harbor dot cache. I tested harbor before the one o release.  

[00:40:11] ODELL:

Yeah. So that's Paul and Ben from the former Mutiny team, I believe. I've been working on Harbor   Yep. While Tony pivoted to AI with, Marks.   But, anyway yeah. Go on, Ben.  

[00:40:25] Brad:

Alright. Let's let's go ahead and,   address some of the pain points   and then kind of the the narrative, the arc of how we kinda got to some of the most recent changes. Yeah. It Okay. Yeah. What on the operator side? Right? So, definitely heard your feedback early on with Fedimint before you had even joined the project and us working at, North American Centralized Exchange being like, how cool that there's a   a system that isn't doing KYC and all these fun things, and you can deposit   some funds out of a a system like this.   But, anyway, the the   setup of a federation   was tricky. Let's talk about what a setup of a federation looked like just a couple months ago. Shit. It's fucking horrible.  

Yeah. It's in, like, definitely no like, it's it's an iterative process. Software is really cool. You start with a thing and you're like, oh, crap. Let's see if we can, like, shape this marble into something a bit more beautiful.   And the initial setup process is tricky. So, like, in general,   you have, like, a client server model is one way to think about this.   Each of the peers, the guardians that operate this, are essentially operating a server that needs to be reachable on the wider Internet.   That's not great. You need a static IP address. Okay. Now you have a firewall. You have to make sure that inbound connections are allowed within your firewall. Okay. Let's go ahead and go to a domain name registrar and get a domain name. Oops. I chose a .xyz   domain name.  

Okay. Now we have this this,   hard coded domain name   that will map my how how to connect to my specific server who like, traditional   web software, we've been doing this for decades. Like, that is the common pattern, but that is not a very self sovereign pattern of of setting up stuff.   And then there's also headache with the actual setup flow. Once you actually get everything set up and you try to do the the,   DKG with your other peers to do the setup ceremony,   lot of back and forth that's kinda clunky.   So there's there's definitely some pain points. Heard the concerns, and we worked quite a bit to iterate on that, make it a much smoother process.   So,   let's also kinda fast forward a couple months after we launched. So and when I say launch, it's like a main net, supported version, which was officially 0.2.one,   which was roughly, like, February   2024.  

We'll fast forward about three months to May, and then   you see certain,   folks that interact with federation saying that, hey.   I'm like I'm like, something's   off. Like, we're we're down. Like, I can't like, one of our peers is offline.   And then someone what's what's going on there? Why is the peer offline? And then you do some investigation, and it's like, oh my goodness.   Somebody was able to file   some kind of abuse complaint   without any verification   and a dot xyz domain, the registrar is like, yeah. We don't we don't care. We'll just take it down.   Uh-oh. Like, all of a sudden, this previous assumption that it would take, you know, almost like a state level attack to bring down a domain name   is trivial,   and that's a big bummer. So you're like, okay. Well, domain name   is not a good solution, so we have some quick workarounds to try to get everybody up up and running again, which includes just, essentially, here's an update system for this new,   d n like, here's a new domain name which you can reach my server at. Let's go ahead and broadcast this, and people figure that out.  

Is the the quick fix in a nutshell,   but that's not a long term viable solution, and that hasn't solved any of the original problems   when it comes to, like, okay. This is kinda hard to set up, but, also, like, what if you use Starlink? Starlink is behind something called a c g NAT. C g NATs, you share a single IP address   with everybody in your region. And so sorry, buddy. You're not gonna be able to have a static IP address that can be reachable,   on the Internet.   So what do you do for all of these different type of scenarios?   Well, enter a really cool project called IRO.  

I won't give too much historical background, but, essentially, IRO was born out of the IPFS,   ecosystem,   which is Filecoin, IPFS.   Really interesting   experiments going on there. They were bumping into some boundaries of, like, look. If we wanna make it easier to establish, like, peer to peer connections   anywhere on the Internet, like, we're hitting limits of what we can do without breaking IPFS.   So we're gonna go ahead and, like, freeze this existing project, spin out,   and start, like,   simplifying   and focusing on how do we make it easy to connect to other people on the Internet. And the approach that they ended up landing on   is just a public and private key pair. So instead of using a domain name to look up how do I connect to the server, it's just a public key. That's really cool. They call that a node ID. And then there's some other technologies that they we could go deep on, but to keep it high level, essentially, it's like instead of, like, being blocked because you're behind a CG NAT or any kind of fancy firewall,   how do we make it so we could still do some type of hole punching to establish a direct connection   for different computers on the Internet?  

Voila, they released this thing called Aira that, for now, we'll just say, like, magically solves these solutions, and we can go into those details. So longer term solution for Fedimin is, like, how about we switch out our networking stack or at least make a parallel networking stack where we instead of doing just, TCP and TLS connections   that use traditional DNS and all the headaches we just discussed,   what if we just integrate Iro? And so when you set up Iro,   you get an Iro   key pair, and then Iro automatically   handles publishing that key pair that makes it so your node becomes discoverable.  

Then anybody behind other connections should be able to establish a connection to you. So I'll I'll pause there. I can go into more details, or we can kinda you pull up questions if   if I Yeah. I mean, specifically,  

[00:46:24] ODELL:

just to provide I think that was a great rundown. Thank you for that. I mean, specifically, there was two large pediments,   that   effectively had an unintentional   rug because their domains got shut down,   and there was a there was a reliance on   on those fixed domains.   And, specifically, it was the dot x y z registrar   that just, like, immediately honored whatever random spam list had put them on there.   And it caused a whole slew of issues,   both on the operator and the user side of trying to recover from that gracefully.   This what you're explaining to me is, like, to me, the the holy grail   of getting something   like this to work   is making it as easy as possible for someone to effectively   self   host, a guardian server. Like, in I mean, we we talked about the reliance on   DNS being an issue with, like, the dot x y z registrar.  

But even more so, like, you also need to   have, you know, a VPS provider, and you need to have a server in the cloud or whatnot. And that creates all sorts of friction, and it also creates all sorts of,   threat model issues for the actual operators in terms of how to operate it privately. So   correct me if I'm wrong. Does this mean   that we're close to someone just being able to, like, run a Guardian on a start nine at home privately?  

[00:47:55] Justin:

I don't or it's not very close. You can do it today. Yeah. Totally.   We have a we have an s nine p k package that you can side load on   a on a start nine. And then on Umbrel, we're in the App Store right now.   So you can install   it. And, yeah, the   the, you know, Brad's description of, like, the details is awesome. But from a dart like, a operator perspective, you don't need to know,   like, any of that. You just run it. And,   the actual setup ceremony, you're exchanging these strings.   And the the strings are essentially serialized versions of, like, how you connect to your other peers.   And once everyone has the connection to vote, you just hit launch and it that's when it does the DKG.  

And if it's successful, like, you're good to go.  

[00:48:43] ODELL:

So the, yeah, the operating part is much easier. You can write it down. Like, copying so you're launching on your starting on or your Umbrel, then you're copying pasting strings between the guardians through signal or whatever. Yep. And then you press launch. Yep. That's it. Yeah.  

[00:48:57] Brad:

That's what I mean. Yeah. I don't I'm apologies. I'm not on Twitter. But if you do pull up the Fedimint Twitter account, there will be a screen recording that I made where,   there's four separate Chrome windows. I'm operating all of these on remote home servers   that, I'm using a mixture of start nine and Docker to do the setup ceremony.   I do fast forward through the boring parts of, like, the copy and pasting and but you do get a a gist that if everybody's online and available to communicate these things, you get a set up process done in about a minute or two.   Obviously, that's, like, optimistic case. There's, you know, communication and coordination, all that stuff. I didn't anticipate it takes a wee bit longer. But, yeah, it's it's a dead simple process. Yeah. And and and that too, we made a few changes to the,  

[00:49:43] Justin:

the actual DKG process,   and the UI is all new as well.   UI server side rendered, so it's also a lot simpler.   So it kinda yeah. I wrote the new new UI and the DKG changes made made setup just   so much easier.   I remember the first time, like, we did it, like, on UniNet dev setup. Like, we had four of us that were,   two on different continents, one behind the Starlink, and it just it just worked.   It's pretty amazing.  

[00:50:14] ODELL:

That's awesome.  

[00:50:15] Brad:

I well, congratulations, guys. Oh, thank you, Steve. Fantastic.   I would like to circle back to a statement you made Matt about, it's kinda like the holy grail. I do not wanna oversell IRO.   Right?   Kinda we're getting close. My job.   Yeah. It's not I'm not an iro dev. I'm an iro consumer and curious, excited fan.   But my goodness isn't that cool freedom tech to allow peer to peer connections pretty much anywhere on the Internet.   You could imagine all different kinds of use cases, not even just for, you know, Johnny and Nikesh Minh, but anything that's interesting to connect to different computers across the Internet.   It's dead simple. It's a couple lines of code to get that started,   and you're off to the races. I would And I assume that's an open source project itself. Open source project. And they're also as far again, I sound like I'm a a Iroshill. I'm definitely not paid by Iro.  

There's nothing going on there. But,   they're a beautiful open source project in terms of their communication style. They have a lovely YouTube channel with excellent explainer videos that are very well   polished. Their documentation   is is excellent.   Their thought process   on road maps and making realistic expectations and adapting when they hit roadblocks, which is one thing we could discuss because we were   we were hoping to wait until they reached their 1.0 release where a lot of these networking protocols   have reached stability, and there won't be any breaking changes that are introduced in the IRO.  

We were anticipating that would happen, like, June or July roughly of this year,   and then, you know, software. So stuff gets complicated, and it takes a lot longer than you'd expect. But they had a very clear,   like, look on o dot 35.   We will keep these stable, and then we're gonna do this canary series for o dot 90 that works our way up to this official final stable release.   Don't don't play with that if you don't wanna be building on quicksand. Stay on o dot 35, and then you can upgrade to the official. So just in terms of, like, a open source project, communicating things beautifully to the community is yeah. They they they check a lot of boxes, so they're they're doing great. So, again, it's not the,   not the magic bullet. It's it's, you know, not a panacea, but it's a very cool consourced project I am heavily Yeah. We we had most of this stuff for on the sentiment side. We had,  

[00:52:33] Justin:

I wrote pretty much fully integrated on our o seven release, which was back in, like, April.   But as Brad said, we kind of waited,   some time to   see what they were doing as far as their breaking change. They were gonna do this breaking change,   and we actually ended up   integrating,   both versions so that we're we'll be, like if they do because they're running some infrastructure that,   these relays that do the direct connections.   And we didn't want to, like,   deploy,   you know, an IRA version and then they would, like, rub their version and say, oh, we're not supporting that protocol anymore.   So we're running we're actually running our own relay as well. I think we do we default to their relays, but you can point it at ours too.  



[00:53:20] Brad:

So we do So we actually it it does everything in parallel. So you are saying, like,   we're about to get into the nitty gritty. Matt, do you think it's worth getting into the nitty gritty? Yes. Sure. Nitty gritty. The freaks the freaks love it. It's not a good dispatch unless you don't understand   over 50% of it. Oh, we could go so deep on so many things. It's gonna be fun.   So let's take a look at the perspective of what happens when an IRO node launches.   So this is gonna start it's gonna be like, okay. What am I what's my goal here? Am I trying to accept connections, or am I gonna be trying to send connections? And then let's go ahead and bind this thing. It's really just, like, one or two lines of code. And then when you click when you do the bind,   method, there's a bunch of magic that happens. So when the application starts up, it's gonna be like, okay. What relays   am I aware of? What are those domain names that I can go reach out and and ping and see who responds to me fast enough? That's gonna be called your home relay. So   anybody here, I'm gonna do just a quick tangent into something that's called Tailscale.  

I I'm not a Tailscale shill either.   Definitely, like, a VC   backed you know, it's a start up. You could get RUB, the free tier, all that good stuff. But a lot of the networking,   tricks that are played within Niro are also used within Tailscale. So it's like, how do I SSH to a home server when I'm on the other side of the planet? You can do something like Tailscale.   But, anyway, the,   the fundamental design is very similar where it reaches out to a home server, and then this home server   starts publishing details of, okay. This is how you can look up this public key   and figure out how to dial it. And there's a bunch of additional magic. You could say, go ahead and publish some details to the mainline BitTorrent DHT.  

Okay. Let's go ahead and use some other services that are essentially just like glorified maps that map node ID to some way that you can dial me. So when your client starts up, it's like, okay. Let's see if I can check all of these different,   the the different address books, essentially, like, for this node ID. Let's see if I can figure out where this public key exists. So it reaches out to that home server, and the home server is like, I got good news for you. Here's how you establish a direct peer to peer connection.   If that doesn't work out, which it,   their goal is 90 to 95%   of all connections will end up in a direct peer to peer connection. But let's say you still wanna send packets   and you can't establish that, direct peer to peer connection, it will still be relaying encrypted data across that relay. So if it knows the origin and the destination, however, it doesn't know the contents of the message that are being sent back and forth.  

So that's how you get this, like, very beautiful automatic,   peer to peer networking system. If that relay goes down?   Beautiful question. That's a central point of failure. I'm like, you know what? That would actually suck if we do all this work to integrate IRO. You can all of a sudden. Happens to their infra. So we also run parallel infra, and anybody can run parallel infra. So any other,   orgs out there, and I would anticipate that as IRO gains in popularity,  

[00:56:30] ODELL:

different people will be like, look. We will operate relays. Relays And then clients will gracefully switch between them on failure or something?  

[00:56:37] Justin:

Yeah.   There's the discovery there's the discovery mechanism.   So there's sort of two late the the relay is what helps set up the direct connection,   and then the discovery part is how you sort of find the relay. So,   Ira has   I believe that the default is DNS discovery,   but you can also use the PCAR,   discovery method or the mainline,   BitTorrent DHT.   So, yeah, that that's kinda how you get around that.  

[00:57:08] ODELL:

That's awesome.   I mean, I wanna pull it back for a second on the Fetterman operator side.   This arrow stuff seems awesome. And by the way, Freaks, the Freaks joining us in live chat, thank you guys for joining us in live chat. You guys make this show   special and unique,   and I love seeing a vibrant live chat. It's one of the reasons we prioritize,   live streaming within the Primal apps.   If you wanna join the live chat, you can do that through zap.stream.   He has an iOS app as well or on web   or any of the Primal apps, Android,   iOS,   web.  

There's a few other Napster apps that also support the live streaming.   Napster is one, and,   Amethyst is another.   Hopefully, we will see more.   And, especially, huge shout out to the freaks   who have zapped the show.   One of them who zapped lethally was how I found out how Ira was spelled, I r o h. He zapped 2,100   sats.   But I see a bunch of other zaps from other people here, including Greg Gee with the top zap of 2121.   So thank you, sir.   I wanna get back to the operator side.   So when this DNS issue happened,   some of the Federman operators, all of a sudden,   we're in, like, a they were in they were in, like, a failure mode. Right? And they needed to recover.  

So if I'm if if someone's running as,   a sediment on the start nine,   what does that backup process look like?   And what does the restore process look like? Let's Yeah. You know, let the start nine   burns up. Right? Let's the thing catches on fire.   So how did I back it up, and how do I restore it? And if I was lucky to have to  

[00:59:02] Brad:

play around with a bunch of Start nine VMs and kick the tires to see all the different ways you can do this. It's a really cool system they have. So as part of the Start nine dashboard itself, there is an integrated backup and restore functionality in there. So if you wanna back up your federation,   all you have to do within the start nine specifically is go to that. You back up. You select the. So if you just do a start nine backup, it includes the.   Yeah. If you're selecting all the applications by default. And then   on restore,   there's a nice little trick that we play where and it's not a trick, but it's a DB snapshot. So you could   I'm gonna   at the the risk of blasting, it's not actually a blockchain, but you could almost conceptually think of some of the sequence of events,   within the federation as a separate chain. And the reason why it's not a chain is because there's no cryptographic hash that's pointing to the previous block, so,   it's be incorrect to to make that statement. But, anyway,   there's we call these things sessions, and so you have a Genesys session. Maybe the federation's been running for two years. It's gonna have a ton of sessions.  

If you just need to do a normal restore, you're actually going to parse through all of those sessions again, and you're gonna reach out to all your other peers that you're connecting to, the other guardians in the federation. And you're like, can you give me the session data? And you're just gobbling that up and parsing it and then getting back to TIP, essentially. You're like sync you're syncing.   You're syncing. Yep. You're doing an IBD, essentially, on the federation.   What's nice about, specifically, the start line integration   is we can go ahead and say, when you do backup,   just save the state of,   our our directory, our data directory. And then when you do the restore, go ahead and consider the most recent snapshot in our database, and we'll sync from that instead of needing to go back from Genesys. So then the backup restore process is significantly faster. However, it has a fail safe, within the Guardian dashboard itself. It's not necessarily a fail safe, but the default pattern across all the different deployments   is you can just download,   tar of your data directory. And so long as you have that file, you will be able to restore your your single Guardian,   and reconnect all your peers and then sync from Genesys.  



[01:01:15] ODELL:

Got it.  

[01:01:18] Brad:

So that's,   that is the backup restore process. Doesn't also, you're kinda dovetailing into,   with the dot x y z domain failures that we're in a failure mode.   That'd be a little bit different than specifically this backup and restore process.   That Guardian could have just, you know, backed up and restored their their specific FEDMID d, which is the daemon   that runs the the Guardian software.   That wouldn't have changed anything. You need to define some other workaround in order to say, like, alright. I've updated my specific domain name, and here's how you can reach me now. And then that got the system going.   However, not   it's a a nice quick solution, but, a long term elegant solution. Viro  

[01:02:01] ODELL:

is is this elegant solution Yeah. For that part. Well, I know that one of those there was two Fedimans that had the x y z issue,   and one of them   like, the funds have still not moved on chain,   because of   backup   backup failures.  

[01:02:19] Brad:

Well, that is interesting. That is,  

[01:02:22] ODELL:

news Like, it was it was not your traditional rug. It was like the operators actually   rug themselves.   Like, also Like, it they donated to the entire Bitcoin community with,  

[01:02:36] Justin:

Bitcoin stuck in a multisig due to failure. So conflicts the config file that Brad mentioned is important,   and that's exposed in the UI now. So, yeah, making sure it's kinda that is your   that's the private key for the,  

[01:02:49] Brad:

for the Bitcoin that's held on, Shane. So you need I mean, but for, like, less technical users, the one click start nine backup sounds like a fantastic solution. Yeah. Please go with that, Roush. If you Yeah. Don't know how to do a reverse tunnel over SSH, please just use the start nine. Yeah. And then it's for you. Also, just real quick on that note, in general, if there was a federation where you noticed that style of failure,   as long as you have the if if you have that config backup,   there's something that we built called the recovery tool. So since this is just a multisig wallet on chain, technically, that Bitcoin's not locked. There is a way to Right. Create a valid Bitcoin transaction to move those funds out of the multisig,   and,   that is something that   if you go to GitHub and look at or reach out on Discord, I'm sure that somebody would be able to assist you with that process.  

And that's yeah. Those funds funds are not lost, but, obviously, the whole system of, like, here's some  

[01:03:47] ODELL:

ecash notes that I was hoping to not tell you. These were my ecash notes. You have two pieces. Right? You have, first of all, like, the recovery of the actual funds on chain, and then second of all, the actual database of who owns what. Right? Yeah.  

[01:04:02] Justin:

Exactly. Exactly. Yeah.   The I guess one other thing to highlight with the Aira thing in regards to backup is that   because Aira kinda provides this your your note is identified by the node ID,   and it's not the DNS name or it's not the IP address or anything, that can actually   like, as long as you have the you have the same node ID, you can sort of move your Guardian.   So you can cut like, if you wanted to,   you   could take down your guardian, restore it on another machine. It will cut you that classic file, it will come up with the same   public key and you can just keep keep running like that. So it provides, like, easier migration,   technique. One of the thing I wanted   to mention on that is,   you know, we've been talking about start nine and, Umbrel, and we also have a Docker image if that's where you're more comfortable,   or Docker Compose,   file if you wanna run that.  

But, like, because Ira's pretty cool. Like, you can get because it it's peer to peer, you can kinda run it on anything.   So I have a project,   that actually   moves the Guardian into a phone. I made a little Flutter app that you can run a Guardian, like, on an Android. It's like an APK that you can install   and run a Guardian from your phone. So it would give you the same UI as what's in,   start nine in Umbrel.   And, yeah, for you know, I probably wouldn't recommend anyone doing that for production right now.   It's kinda early. But,   the idea would be, like, you could buy, like, a cheap phone or something,   stick it in a corner   on a charger,   and,   run a Guardian just from your phone. Because you don't really need to look at it that much, like, once you set it. Right. You would keep it, like, always on connected to Wi Fi. It wouldn't be, like, your daily driver phone that you're, like, walking around with. Yeah. And with with iRO,   as That's incredible.  



[01:05:59] ODELL:

I I think it's kind of fun. The  

[01:06:01] Justin:

I mean, you could imagine, like, a mobile mint where   the whole mint is run off, say, four phones or something, and they're moving around or whatever. They're on, like, USB power banks and, like, the Serengeti or some shit. Yeah. It's very cyberpunk. But the What, what is the what is the, like, performance requirements?  

[01:06:19] ODELL:

Is is it   very performant? Or  

[01:06:23] Justin:

For the for the phone or just sediment in general? Yeah. Like, how should you have a phone how should you have a device can you run a Guardian on? That's I don't know the numbers off the top of my head, but it's not possible. Some inspects on our if you go to our, GitHub repo, you can take a look at that for the Docker setup.  

[01:06:38] Brad:

I wouldn't   other beast. I don't even know how I Yeah. Would translate that and have to just go kick the tires on, like, the Google Pixel five or something.  

[01:06:48] ODELL:

But, it's actually What would be the limiter? It's not storage.   Right?  

[01:06:56] Justin:

Yeah. And compute wise, it doesn't use much,   memory. I think it's around   it's pretty small memory wise too. It might end up being storage at some point because  

[01:07:06] ODELL:

As the bit gets more and more used. You have more  

[01:07:11] Justin:

history.   Yeah. I I I don't have   numbers off the top of my head. But I think on the on the phone part,   like, Ira is really what makes that possible, and it it uses, like, the   due to that, like, relay thing that Brad mentioned,   it can sort of seamlessly,   like, fall back from,   like, a direct connection   to   over the relay.   So if you're say if, you know, say your phone's moving around or something and you the network conditions   change, Ira actually kind of it is supposed to adapt to that.  

[01:07:45] Brad:

So another cool thing about Fast Ira. Yeah. And I just pulled up our min specs. You   like, this is bare minimum for a three or four. This doesn't translate to a five or seven or anything else. There's more complexity, all that good stuff. But one gig of memory, 10 gigs of disk, and one CPU   is bare minimum to operate a guardian,   which is not much. Like, obviously, like I said It's not bad at all. Yeah. Pretty minimal.   And then, also, we need to consider, you know, a nice   old Android that already has a UPS   integrated with it and all these other lovely things. Like, it starts to look kind of attractive.  



[01:08:24] ODELL:

Love it.   I mean, while I have you, Brad,   have you looked into, like, hole punch at all, like, Keith's stuff?  

[01:08:34] Brad:

Oh, I've looked into hole punching the technique to establish peer to peer connections, but not hole punch specifically that you're referring to.  

[01:08:41] ODELL:

Fair enough. I don't think it's open source yet, but it seems to be trying to solve a similar   thing as Tailscale and IRO.  

[01:08:50] Justin:

Yeah. The technique is the same, essentially.   IRO is just a library. It's it's a, you know, open source Rust library that we use. So technically, it's the same. Yeah.  

[01:09:01] Brad:

I yeah. Go on. I was gonna say for more context for the freaks,   hole punching is referring to to you have a router that you're on right now on your Internet. Everybody's probably listening.   Well, I don't wanna make too many assumptions for mobile users. But if you're at your home network, you have a home router,   and your router has something inside of it called the NAT, a a network address translator. It's essentially like an internal mapping of   a, IP address internally to your LAN   to something that is used outside of your LAN. So,   when you pull that +1 921681   whatever, and you're like, oh, that's my IP address. It's like, yeah. That's your IP address inside of your local area network. That's not your IP address outside.  

So all of the fancy stuff with hole punching   is saying like, okay, world. What do I actually look like from the outside world perspective? Because if you can figure that out, then you can figure out how to dial specifically an IP address with a port to reach this computer. And so there's a bunch of magic. There's a bunch of specs that have been worked on for decades at this point to to try to solve that problem. And it still is like, there's a lot of complexities   across different home routers just in the area of NATs that make this thing a very complex problem and unsolvable for a certain subset of NATs, but probably too much in the weeds. But, yeah, hole punching in general is just solving what is my internal IP address actually look like to the outside rule. Yeah. Tor Tor does not actually.  

[01:10:26] Justin:

Yeah.  

[01:10:29] ODELL:

Yeah.   I,   first of all, I,   wanted to shout out that in the Primal Live chat, we have both Cali, the father of Cashew,   saying FEDIMENT rocks. You guys are legends. And we have Eric Sirion, the father of FEDIMENT,   saying you want at least 500 mega   megabytes of memory,   and that CPU isn't much of a limiter. Right. Unless you wanna have multiple transactions. So that's cool, that they're both joining us in live chat. And then separately, Bitcoin Ranger zap 10,000 sets   and use   the default   zap note of great post. Thumbs up. Thank you, sir.   I,   and then I've also I see a question here about signal being backdoored, which is completely random to our conversation.  

But if you want my opinion on it, I don't think signal's backdoored.   I think it's it offers you reasonable security guarantees with very good UX. And I think the way they usually compromise signal is they compromise one of the phones of someone who's participating in signal, and then they can read anything that's on your phone, including your signal messages.   So I think signal's a good option for secure messaging. Simplex is interesting. I think it's still pretty early.   Is kinda interesting. You can obviously   I I I there's a project called White Noise that is very, very early that is not reliable yet that will hopefully   provide a good option there.  

The number one thing you can choose, by the way, when you're if if you actually care about secure messaging is using that   auto delete feature,   on whatever   whatever client you're using, whatever app you're using. You know, delete after one week or delete after four weeks.   Have that set. So if someone's phone gets compromised,   it only has one week of history or four weeks of history or whatever. That's the single biggest improvement you can make,   from my point of view.   Obviously, the person you're talking to can still take a screenshot or a picture or something. So it's not protecting you from them, but it's protecting you for some external malicious actor.  

[01:12:42] Brad:

And, sir, Matt, I almost interrupted you there. This is also pretty scary when it comes if you have a smartwatch that connects to your device, and all of a sudden your threat model isn't just your phone. It's your watch that's connected. Watch do you have? It's a Garmin.  

[01:12:56] ODELL:

The Garmins are the better like, out of all of them is probably one of the better ones. That company, like,   is sneaky,   relatively stay true to their roots   It's in terms of   of user privacy and security.  

[01:13:11] Brad:

Totally agree.   Once these get kinda old in life cycle and the auto updates the security fixes stop rolling, and you're like, oh, man. Is somebody I'm a Casio, man. I don't I don't wear I don't wear a smartwatch. No smartwatch just for me.  

[01:13:25] ODELL:

Yeah. But if you're gonna get a smartwatch,   Garmin is sneaky, like, one of the best American tech companies.   I'm a bit of a Garmin fanboy. Also, like, if you don't wanna use Google Maps anymore in your car, just get one of those, like, $400   Garmin units   completely offline. Stick it on your dash.   Works like a charm.   Okay. We got completely sidetracked. This has been a fantastic conversation. I'm pretty excited. You guys got me bullish.   You guys have made real progress.  

[01:13:59] Brad:

If if Thank you. Yeah. Go kick the tires if you start a nine or an Umbrel or you know how to do docker. This is super simple to set up. If you actually wanted to play with Mutiny Net before you have Main Net funds, also super simple to to set up. Shout out Ben and everybody that does MeetMeNet stuff. I like testing on main net.   It's it's fun. Game. Yeah. But,   yeah, that's   go go experiment. Go play with the Ecash app. We'd love to hear your feedback. We're very open to any bug reports. Best way for them to provide feedback?   GitHub issues are great. We host multiple calls throughout the week. So if if, like, this I know this is a big lift if you wanted to join a call, but we're always available almost, like, office hours for,   our weekly Monday dev call. All of this is on a public channel. I record it and upload it to bitcointv.com   if you're ever curious. I'm sure some AIs are gonna be parsing all of that, and that'll be very interesting in the future.  

But the   we also have, Tuesday review call where we go typically deep dive on a a PR review topic.   Sometimes it turns into just like a generic deep dive discussion on some technical aspect of Fedimint.   Thursdays, we also have another time chunked out for deep dives. So all of those were available. We're on Discord. You can always,   hop in Discord, send a message. If you have a question, somebody will get back to you.   But then yeah. Also, good old fashioned GitHub issues is a really good way to get our attention.  

[01:15:27] Justin:

Yeah. I'll make one one more comment about the gateways.   So with all the Ira stuff we talked about on the guardians,   we haven't yet done that with the gateways. So the gateways still have a   similar requirement to what we had before with having a VPS and a public IP and a domain and stuff.   That's something we're working on. We'd like   to get gateways over iro as well.   We already have a LDK based gateway.   And so the kinda the goal there is to,   yeah, make it so others can easily run gateways to it. I do like, using LDK or we have LND support too. So, like, the end goal is, say, on a start nine, you could   run a gateway against your existing LND node or,   with an integrated LDK node, and provide liquidity that way. So,   that's coming. That'll be that'll take a little bit of time to to do, though. So That's awesome.  



[01:16:28] ODELL:

I mean, that reminds me. So you said the gateways need to be approved by the Guardian.   How does that actually Guardians, how does that actually work in   practice? You gotta you gotta, you know,   three or four Guardian set up. You wanna improve a gateway.   Is that a GUI thing or is that are people doing it in the interface? Is that a command line thing? Like, how's that working? Yeah. It's in the it's in the UI. There's a Okay. For if you run a guardian dashboard, there's a little box to add a  

[01:16:59] Justin:

add a gateway there. And,   yeah, I don't wanna get into details on that. We have we're we're sort of migrating our lightning protocol right now to a second version.   But you actually don't need to be vetted. There's there is a way to,   do it without being vetted, but those are sort of deep deprioritized because of the issue I mentioned before. So So all this is kinda that lightning gateway stuff is gonna change. It's kinda gonna get deprecated and be a new method anyway. It's, yeah. We've we're we're deploying it out now.   The detail like, the Ecash app actually has support for the new version.  

Fetti does not yet. They'll   hopefully, upgrade it at some point.   But, yeah, the if you if you like you like you said, if you deploy Guardian,   there's a little box in there for for adding a gateway. So and I'll I'll mention, we actually do run a,   like, a Fedimint official gateway.   Again, the the trust model is just that we need to trust   the Guardian operators.   So if you have a federation that you're you're running and you want some liquidity, you can reach out on on Discord, and we'll we'll try to get you connected and set up using the official one. Yeah.  

[01:18:12] ODELL:

I see Archer secured cuss   consulting   zapped   12,345   sets. 12, three, 4, five sets for answering a signal question. So   thank you,   thank you for supporting the show, sir.   Okay. That all makes sense to me. Guys, this has been great. I would love,   you know, maybe in six months or so to do a catch up on where the project stands,   have you guys back on, make it a little bit more of a recurring thing. I think that'd be a lot of fun. I mean, you guys seem to be making a lot of progress. A lot of things are moving, shaking.  

[01:18:48] Brad:

That would be awesome. Would love to do frequent check ins so that we is kind of abreast of all the some,   term we use, and I'm sure some folks are familiar with.   At previous start ups I've worked at, there's been a lot of sizzle and not a lot of steak.   I'd say Fedimint, we got a lot of steak. There ain't as much sizzle as there could be. So, hopefully, these efforts will will keep everybody kind of abreast of all the technical updates that we've been doing to make it easier to set up and interact with these things.  

[01:19:19] ODELL:

Love it. Well, great great chat, gentlemen.   Before we wrap,   do you wanna hit the freaks with some final thoughts?   I guess we'll start with Justin.  

[01:19:29] Justin:

Yeah. Final thoughts.   Go to ecash.love   to download the ecash app. Give it a try. Give us some feedbacks,   and give a shout out to the Minneapolis Bitcoiners.   We actually have a meetup coming up tomorrow, so I'll I'll be there if anyone wants to   to chat.   It's at the O'Shaughnessy distillery in in Minneapolis.   Great venue.  

[01:19:51] ODELL:

That distillery is awesome, and our family is awesome. They're big winners.  

[01:19:55] Justin:

It's, we we have, like, a private room for the meetup, which is amazing.   Yeah. They've been very generous to us. So,  

[01:20:05] ODELL:

yeah. It should be a lot of fun. Thanks for having me on. Yeah. You guys, your meetup might have the best location in the country, and I say that as a cofounder of Bitcoin Park.   So enjoy the meetup. And I saw someone ask when Odell   I brought great shame to my family that I have not been yet, but I I will make it happen,   sometime soon. Brad, final thoughts.  

[01:20:25] Brad:

Freedom money is cool. Freedom tech is cool. Being out in nature is grace. Go enjoy your life, people.  

[01:20:32] ODELL:

Love it.   Guys, I'm gonna put all the relevant links that we talked about in the show notes.   Next week, we have Seth for privacy   and Kate Walla joining again on Friday the twenty sixth at eighteen hundred UTC.   All relevant links are still dispatch.com.   If you don't have sats to spare to support the show, sharing with friends and family really does go a long way.   Leaving reviews, subscribing on your favorite platform.   We're available in all the podcast apps.   I see Lethally just zapped 21,000 sets. Thank you, sir. He says great rip. Awesome project.   Guys, pleasure.  

Keep crushing it, and,   I'm gonna play around with with some of the things you guys built, and I'll give you feedback myself. And don't hesitate to reach out if I could be helpful in any way. Cool. Thank you.   Love it. Much love, freaks. Stay humble, Stack Sats. Peace.