07 September 2021
CD37: building software from source code with @achow101 and @craigraw
EPISODE: 0.3.7
BLOCK: 699517
PRICE: 2170 sats per dollar
TOPICS: building software from source code, reproducible builds, reducing trust, coin selection, coin control, coinjoin
@achow101: https://twitter.com/achow101
@craigraw: https://twitter.com/craigraw
streamed live every tuesday:
https://citadeldispatch.com
twitch: https://twitch.tv/citadeldispatch​
bitcointv: https://bitcointv.com/video-channels/citadeldispatch/videos
podcast: https://anchor.fm/citadeldispatch​
telegram: https://t.me/citadeldispatch​
support the show: https://tippin.me/@odell
stream sats to the show: https://www.fountain.fm/
join the chat: http://citadel.chat/
Levels since May ahead of El Salvador's official adoption of Bitcoin as legal currency that starts today. The move makes it the first country in the world to officially put Bitcoin on its balance sheet and hold it in reserves. Here to break down, this test case, essentially, with us is CNBC's Mackenzie Sigalos. Mackenzie, great to have you on. I know you've been doing a lot of reporting around this. But the fact that we did see Bitcoin jump ahead of this and now it's under a little bit of pressure today, I mean, this has, in many ways, been kind of the bull case longer term around Bitcoin. The fact that you could actually use it to to transact for everyday items. Right?
[00:00:39] Unknown:
Right. And I think that that's one of the major things that people are looking for today on the ground in El Salvador. You know, citizens can now from today, technically use Bitcoin to buy virtually anything. So we're talking a cup of coffee, a haircut, they can also pay their taxes in Bitcoin. And that's why you did see this price run up in the last few weeks. Like, Bitcoin's price was touching highs that it hadn't seen since May. And then yesterday, the government announced that it was adding almost $21,000,000 worth of Bitcoin to its balance sheet and that's where you saw this, like, this huge bump up in the price. And, yes, it has come off of those highs, but I think people are watching to see how rollout goes. And and and we're starting to see that. You know, from midnight, I was on the phone with people who were on the ground in San Salvador, and they were actively downloading the wallet that the government has offered to all citizens.
So it's been really interesting to see it all happen in real time.
[00:01:33] Unknown:
I I I realize we're essentially only hours into this rollout, but do we know from a tech from a technical standpoint how those downloads are are going right now? And I ask that given the fact that, what, only about a third of Salvadorans actually use the Internet?
[00:01:49] Unknown:
Right. And and, you know, that stat has been called into question just because it's referring more to these wired lines or fiber optics. But almost everybody has a cell phone in hand and that's really the level of connectivity that you need to be a part of this Bitcoin economy. And in terms of how successful the rollout has been so far so, I mean, was this a flip that switched at midnight? No, it wasn't. But, you know, people were actively communicating with one another. The president was helping to troubleshoot IT issues. And so at around 2 AM, that's when we first saw the wallet go live. And it's something called the Chivo Wallet, which is Salvador and slang for cool. And so the government is is really pushing people to become a part of this economy, and the way that they're doing that is by saying that everyone who signs up for this wallet gets $30. And I mean, that's no small sum in a country where, you know, the average minimum wage is $365.
And so from 2 AM, it hasn't been a totally smooth rollout, but we have seen successful downloads. We have seen people receiving that $30 into their wallet.
[00:02:53] Unknown:
Mackenzie, from people who are, you know, intently watching this from the kind of Bitcoin camp, let's say, the people who really believe in it, what would represent success either in terms of volume of transactions or, you know, just the adoption or displacement of other types of payments? What are we what what are our benchmark indicators?
[00:03:14] Unknown:
I mean, you said a few of them there. I mean, I think mainstream adoption. I I think that one side of this, you know, going beyond using Bitcoin as as a currency, as a form of, you know, exchanging exchanging value. People see it as a way to to help, Salvador and save money. And, you know, it's a largely unbanked population, and so this gives them a way to build this culture around saving money. And and I think that that's one thing that a lot of crypto insiders have well, Bitcoin insiders, because this is really a Bitcoin project. And what Bitcoin insiders have spoken to me about is that they're excited about moving people from this government wallet, which is custodial, to their own wallet where they hold the private key and they have 100% ownership over their wealth. And that's, you know, that's something that a lot of these people haven't experienced before, and I think that that's what they're most excited about.
[00:04:04] Unknown:
Mackenzie, El Salvador has been looked at as a test case for a while. Are people talking about which country might be next?
[00:04:12] Unknown:
You know, Panama was watching, but I think I think that a lot of countries are are keeping an eye on how things play out in El Salvador. And, you know, not everybody is happy about it. The IMF and the World Bank have have expressed concerns about what this rollout means. So I I think that there are a lot of people on on either side. You know, there are a lot of big supporters, but, you know, you have a large portion of the population that also is is confused about what this Bitcoin rollout means. They're inherently skeptical. So there are quite a few barriers to entry. So it'll be very telling how this plays out in the next few weeks, in the next few months as we as we get a better sense of what mainstream adoption really looks like.
[00:05:27] Unknown:
Happy Bitcoin Tuesday, freaks. It's your boy, Matodel, here for another Citadel Dispatch, the interactive live show about Bitcoin distributed systems privacy and open source software. You may have noticed that we took a break last week from dispatch. It is the end of the summer, and I'm having a little bit of trouble, lining up solid guests and topics. So I will take breaks from here from here on, if I don't think we have a solid topic lined up, trying to keep, the conversation on dispatch high signal, not trying to waste your time. So that's why we had a break last week.
But I'm very excited for this conversation. This is Citadel dispatch 37. Our focus is gonna be on building software from source, and reproducing those build processes and why that is important. Before we get started, I want to do a brief shout out to all the ride or die in the live chat, whether that's through Twitch, YouTube, or Twitter. You guys make this show very special, very unique. So thank you all for joining us once again. And another big shout out to the freaks who support the show and keep it ad free and sponsor free. I think that's the way to align incentives as as well as possible going forward, and it's something that we're trying to champion here at dispatch. I never wanna have ads or sponsors.
So that's only possible because the freaks, continue to support the show. The easiest way to do that is through podcasting 2 point o apps. If you go to new podcast apps.com and pick an app from that list, you'll be able to just search Citadel dispatch to pull up the Citadel dispatch feed, load it up with stats, and you can stream stats directly to my wallet to support the show as you're listening. It's a really cool experience. After I upload it to the RSS feeds, I see the sats rolling, and it's really fucking cool. You can also support the show at sail dispatch.com.
There's links to, donate via lightning or via PayNIM. My PayNIM is very easy to remember. It is Odell, and you can support the show with merch. I have, if you go to sill dispatch.com/stack, you can get hats. We have flasks courtesy of, Quincello, and we have pins and magnets courtesy of BTC pins. So that's still dispatch.com/stack. Currently, my hats are on back order. I use Richardson hats only, which are made in the USA. They're super high quality. I am a stubborn person, and I refuse to switch brands. So I appreciate your patience as you wait for me to get them back in stock. They should be in stock shortly. So with all that said, I'm really excited to have this conversation.
We have Craig Rall joining us for, I believe, the 3rd time, maintainer of Sparrow Wallet. How's it going, Craig?
[00:08:28] Unknown:
What's up, Matt?
[00:08:30] Unknown:
Glad to be back. Joining us again. It's always good. And we have Andrew Chow, Bitcoin core dev. He's never been on dispatch, but we did speak, I think, 2 years ago on, tales from the crypt. How's it going, Andrew?
[00:08:45] Unknown:
Hey. Doing well. Thanks for having me.
[00:08:49] Unknown:
Yeah. It's a the pleasure is all mine. Thank you for joining us, Andrew. So the main topic of the conversation today is building our software from source. Recently, NVK, who has been on the show many times, launched a new project, bitcoin binary.org, to basically try and normalize the process of verifying that source code matches the binaries that people are installing, the actual install files that people install. And it kinda brought this up at least into the attention of people on Bitcoin Twitter. And Andrew jumped on the occasion and started trying to, basically reproduce the builds, of a bunch of popular projects and ran into some issues, and that's how this conversation came about. But before we jump all the way into there, I think a good place to start is, Andrew, why should people care about building their software from source, and why is it important?
[00:10:00] Unknown:
So when when you download software from the Internet, you can't be sure that what you're downloading, doesn't contain any malware. Right? And so the way to be sure that the software that you're downloading is, you know, exactly what you intend to be using, you build it from source. The problem is not everyone knows how to build it from source. We don't expect everyone to build all their software from source, because it's kind of a pain, and and, you know, that's it's a very technical thing. So, the the question then becomes, how can we how can normal users trust that the software they download from the Internet is actually built from the source code that the developers publish?
And we do that through reproducible builds. So the the way it works is that multiple people can build the same source code and they arrive at the exact same binary that the developer publishes on, you know, on their website or whatever. And, so then people can see, okay. There are, you know, 20 people who have built the same thing, and and, you know, at least, hopefully some of them have reviewed the source code to make sure there's no malware in there, and so we can reasonably trust that the developer is not slipping in some, like, hidden, hidden backdoors or hidden malware into the software that they actually publish.
So
[00:11:33] Unknown:
in your in your opinion, the ideal way for people to run software is they should be building everything from source, but that's not a it's not a reachable goal. The expectation is not everyone will do that. Correct?
[00:11:47] Unknown:
Yeah. And there is, there's also kind of a bootstrap problem. So to build everything from source, you have to have a compiler. To have a compiler that is open source, you need to build the compiler. So what do you use to build the compiler? This goes down many layers and is a very complicated topic.
[00:12:11] Unknown:
So so, basically, the idea is here is we're trying to reduce, we're trying to reduce trust in an individual single point of failure. Right. So Right?
[00:12:25] Unknown:
Yeah. So instead of just, you know, trusting that the developer blindly trusting the developer to have done the right thing, it's now possible to verify that they've done the right thing, and then users can trust a wider group that has, you know, less incentive to be malicious.
[00:12:43] Unknown:
So I I guess, like, the place to really start here in this conversation is why open source is important in the first place. Right?
[00:12:50] Unknown:
Yeah. Open source is important because you can look at the code and, you know, make sure it's not doing something weird.
[00:13:00] Unknown:
Right. So you have the code. Everyone can look at the code. The ideal trust situation is that you're analyzing every piece of code yourself. You know what the hell is going on with the code, and you're making sure nothing crazy is happening. And then you're building it yourself so nothing gets interjected in the in the middle process. And then I guess the next the the next achievable goal after there is you have a binary that is assigned binary, so you can verify that it hasn't changed from when the person distributed it. And but the question then becomes, does that install file differ at all from the code that was published previously?
And that's where reproducible builds come in. Right? Yep. So the idea there is even if you can't build it yourself and you can't you can't analyze the code yourself, you can safely know that, you know, 10 people I trust all were able to to to build the same exact binary, and they all ideally, I guess, sign it, so then you know that it's you're not just trusting the developer who shipped the code and signed it. You're you're distributing that trust among a group of individuals that were able to reproduce the build. Right? Mhmm. And this is not just about a malicious developer.
Spectre had a, for example, Spectre Desktop, a project that a fantastic project, had a issue or they had a scare. It turned out to not actually have happened, but they had a scare where they released, they released a binary that they thought and that they signed as well, but that they thought the build process got contaminated by some kind of virus on the build computer. And in in that situation, because the build wasn't reproducible, everyone was just blindly following whatever the signed binary was that was released. So it wasn't even necessarily the developer who was being malicious. It was just the computer that he used to build the software, could have had a virus on it.
[00:15:24] Unknown:
Yeah. That's that's also one of the things that reproducible builds help with. You know, if if, a very powerful malicious actor, you know, compromises the developer's computer, they can't with reproducible builds, if they insert malicious code into the binary, it would still be caught by everyone else who is building. They would they would see that the binaries don't match.
[00:15:51] Unknown:
So let's start with so how does Bitcoin Core handle this issue?
[00:15:57] Unknown:
So Bitcoin Core, we are we have we currently have 2 systems. 1 is called Gideon. This is what we're we've been using, for the past 10 years, something like that. Very long time. And we're transitioning to a new system called Geeks. But both of them work on approximately the same principle, and that is that we have a build environment that is separate from the the machine that's it that is it's running on, so it's kind of a a container, or a virtual machine. And in that environment, we we make sure that the dependencies are all the same, so everyone will have the same dependencies, and from there we just do a compile with some extra flags in there.
And the the end result is that about 5 to 10 people do the builds, before release, and we compare everyone's the hashes of all the binaries that are produced and make sure that they all match. So for every release, there will there are several people who have built the exact same binaries before the release is actually published. The the there is a a may very big difference between Gideon and Geeks. So Gideon uses actually uses virtual machines. Geeks, is a is actually a software project from, the GNU Foundation, or I think that's what it is.
And its whole thing is to reproducibly build all open source software.
[00:17:43] Unknown:
Right. So Bitcoin Core is arguably I mean, probably not even arguably, is is the most important piece of software that we have in the Bitcoin space. And it's software only. There's no hardware involved. It's not a mobile piece of software. So it is the the priority is obviously security as much as possible. We hit we hit roadblocks in terms of these concerns when you start introducing hardware wallets and you start introducing mobile wallets. With mobile wallets, especially if you get it through, like, the app stores, There there's obviously you're adding another middleman involved in that process.
I don't know.
[00:18:44] Unknown:
I'm trying to think where I wanna go with this. So so Well, there there are roadblocks, but I don't think I think they can all be solved in a similar way that Core does it. The main the main problem that I've seen with these other software is this thing called code signing. So a code signing is is the, the developer has a private key and they sign the binary that they publish and they attach that signature to it. And so when you run the software on your computer, your operating system verifies the signature first, to to make sure that, you know, the developer actually published that software, I guess.
And so for every major operating system, except Linux, because Linux isn't a unified operating system, There's code signing, that that applies to Windows, Mac, iOS, Android. Like, they all have they all basically require code sign binaries. And then hardware wallets also do code signing, so firmware is code signed, and the bootloader verifies that the firmware has a valid signature on it. This is a problem for a minor problem for reproducible builds because only the private key is supposed to be private. Only one person is supposed to hold that. So how do you how do you make sure that everyone who is reproducing the build can arrive at the same published binary that has a signature on it?
The way that Bitcoin Core does it is that we have a 2 step process, so we do, we build everything, everyone builds non code signed binaries. After there are several matching ones, the people who have the code signing keys sign the binaries and publish just the signature, and then everyone, again, does a build where they are just attaching the signature. They download the signature and attach it to the binary that they built, and then that's what gets published. What a lot
[00:20:55] Unknown:
of So you're able to verify everything but the signature?
[00:20:59] Unknown:
Yeah. So you can't, you know, reproducibly make the signature, but you can verify that it is a signature. It goes in the place where signatures go, and, you know, if you treat it as a signature, it is a valid signature. And so then it just becomes, you know, a small binary blob that gets tacked on at the end of of the binary. So so that's what Bitcoin Core does for dealing with the co signature problem. It seems like most other software aren't doing that. They, instead, what they're doing is that you you download the published binary, you remove the signature, and then compare the the strip signature strip binary to what was built.
There it's approximately the same, in terms of it it moves like the signature step from the, building side to the verifying side. But but in the end, you can verify that those are reproducible.
[00:22:06] Unknown:
So the reason we have these signatures in the first place, right, is is because it's it's strictly a net benefit for the average user. Right? The average user is is basically at least guaranteeing, at the very least, that the software is released from the developer they expect it to, if their hardware wallet is it knowing that it's not released by someone who doesn't own that private key, at least?
[00:22:33] Unknown:
Yes and no. So when it comes to hardware wallets, keys are burned on at fact at the factory. Right. So so in that case, yes, they it is a a benefit because you're there's only, like, 1 or 2 keys I can sign, and and they are hard coded in in read only memory on the device itself. And and so in that aspect, yeah, that that's a benefit. When it comes to OSes, like, you know, Windows, Mac, iOS, and Android, the key isn't present on on the the computer you're installing it on because that would be ridiculous to have every developer's keys somehow installed at the factory.
That doesn't really work. So what they do is, this certificate authority structure, this is how, like, HTTPS certificates certificates work, where there is some root trust that sign root trusted authority that signs a certificate. And the problem there is that it's really easy to get a certificate, and so malware have had, like, valid code signatures, and and so it doesn't quite help out. Like, a malicious actor can compromise
[00:23:55] Unknown:
that process. That's, like, inherently centralized process of checking. Well, there's that and also,
[00:24:01] Unknown:
like, the u there the user could download a binary that is code signed, but with someone else's key, and it it's not going to be immediately obvious that they downloaded something malicious because that someone else could have a name, like so, like, for example, Bitcoin Core signed with a key that says, Bitcoin Core Code Signing LLC, but, a malware author could probably obtain a key to the with the name, you know, Bitcoin Core Signing LLC. Right? It's almost the same thing, but, the the certificate authorities will issue both keys. So it's not quite helpful in that case because your average user probably is not gonna catch that.
[00:24:58] Unknown:
Gotcha. Gotcha. That makes sense. Okay. So I we have Craig here. You know, Craig is the maintainer of the Sparrow Wallet project, a fantastic user focused wallet. And, I mean, he's try I don't wanna speak for him too much, but he he wants all of his builds to be reproducible. Craig, you wanna jump in here? You I know you have specific questions. I I feel like Sure. I I feel like this is a this topic is pretty obviously not, you know, in my comfort zone, so I I would love to to hear a conversation between the 2 of you.
[00:25:42] Unknown:
Sure. So I think the first thing worth worth saying is just being quite modest, and when he talks about, the the, you know, what what Coincore have achieved in terms of the the reducibility of of of the builds, you know, he certainly gonna be involved in the net. And the first thing I wanted to say is that it's it's really difficult, to get this process right right. Typically, any application of reasonable cell size has very complex build process. It's it's it just it sort of has to be that that way about modern apps. They have a lot have lots of hearts go into them then. And when you're trying to package them up them up into some kind of a binary, you really do, go through a lot a lot of different depths in terms of doing so. So so, you know, all of those different steps, if if there's one element in them that reduces some degree of randomness.
Let's say there's a list of things in that have been defined and the the order of those things not been defined. Then it may may work fine on a whole number of different systems systems. Soon as you get to one one system that that happens to reorder the list list for some random, you know, completely unrelated reason, then your reproducibility goal is then failed. So, you know, it it it tends to be this binary goal, which is a very difficult one, because you can put put many, many hours hours in. If you don't get all the way to the end, you know, it's a 0 or or a 1. And that makes it it's, you know, a little challenge.
So, you know, I I think that's, from the new view of of kind of being in a much smaller team. You know, I really, really admire what Bitcoin Corp have have done. And Andrew describes the sort of 10 or so people that are on, for every every release to every release and core and build it themselves. You know, it's it's it's it's an amazing build process that you go through. I actually did my my first build build this afternoon, and, you know, there's been a huge amount of effort that has gone into making sure that that was all different from systems, and it works in the reliable level way that it's so, you know, I I kind of I just thought it would be useful to take a step step back and, sort of appreciate what Bitcoin, Coin Corp, have gone through in order to get to this point point. And Andrew mentioned now on the second iteration of of this this this this kind of pros having moved from the Gitean build system to the Gix one.
And and the matrix between those 2 is that is that Gitean one tended to to depend, much more on on sort of queries that were trusted with with the system that we wouldn't think too much about about, but, ultimately, you do you do have that this binary is doing what it's meant to do. So Git's is is for the bootstrap, kind of way of doing things, than Gitian was. And that's, I think, a big difference kind of moving forward. So that's, I think, a a a good kind of of point to to start start with. From the point of view of of of, you know, other, you know, Bitcoin projects trying to implement you might not have the army of volunteers that with Bitcoin has.
I think that there there needs to be a degree of of can be, you know, work towards a series of intermediate vehicles. So we sort of we might not be able to build and reproduce the final panel by but how can we get to intermediate gear ports, along the way that we can check? Because, sure, that's not the final binary that people run, and that's not as good as being able to reproduce the final binary. But it certainly makes that more difficult. And that's and that's really what we're trying to aim for for here. Right? Trying to look at different threat models and trying to to reduce them.
So I I guess I I kind of wanted to ask Andrew's view view on, you know, given, the new need for, I think, all Bitcoin projects to try to drive towards this goal, and the reality of it is tends tends to be a fair binary goal in some sense. What is, what what are what are your thoughts, Andrew, on in terms of of able to try and set some immediate media media along that path?
[00:30:31] Unknown:
Well, so I think that's reasonable and, you know, having, reproducible unsigned non cosigned binaries is, you know, definitely a a good first step. And so is just having reproducibility within something like a Docker container. Right? So, you know, you have if you have, like, a Docker container set up for doing bills and it's reproducible, like, that's a that's a good step because, both of these are are builds that people, peep other people can can still reproduce even if they don't exactly match what is being published. But, like, once you get there, you know, that's not the end. You have to keep going to to get past, some of the more obscure reproducibility issues and, you know, the harder problems like code signing.
[00:31:25] Unknown:
For sure. For sure. So just in terms of, my own kind of in this. I've been working on this for for at least at least a year now, and have finally, I I hope, reached the the point where the Star Aero binary will be able to be able to be produced as of the next next release. So that doesn't include the kerning as you may mentioned, and it doesn't include the actual install store files. So not quite all the way yet. But, certainly, Lee, it's been quite a journey to get just to the point point where, we are now. I was was wondering, you know, also, what your thoughts were on on on different kind of platforms and how how easier or or difficult it was, you know, in terms of the language that you use, and, you you know, all the kind of experiences there that you that you had that might, help as you are think thinking about to kind of go go about this? You know, what kind of languages, tool sets do you think should be looked at or maybe the some that maybe aren't very, ideal for this goal's goal?
[00:32:48] Unknown:
Well, so I think every every language is able to to have a reproducible build. Even so I I have, h w I is a Python project and, Python, you know, it's very high level, so it's it's a bit hard to get it to be reproducible. But, in the end, it is possible. And a lot of this is because reproducible bills is not just something that, Bitcoiners care about. Right? It's something that a a significant chunk of the the, free and open source software ecosystem cares about. So there are you know, GEEX is a project that is very focused on reproducible builds, so is Nix OS.
They they also do reproducible builds. But then, there's Debian. Like, Debian actually is really trying to get reproducible builds of all of their packages. Debian is massive. It's a very popular operating system, and it has tons of packages. And they have achieved reproducibility on a ton of them. So, it's actually really useful that other other people in open source software care about this because you can just go to their code, go to their code base, look at their build process, and borrow things. In in Bitcoin Core, we did this a lot. There's a lot of, like, you know, there's some project that does this one thing. Debian has this special patch to make it reproducible.
Well, we're just gonna go to the Debian repo, take that patch and and add it to our own build system and now we've made some extra thing reproducible. So so if you're trying to get a project to build reproducibly, it's really helpful to, you know, go to projects like Debian, find something related or maybe just like a dependency that you need and see how they're making it reproducible.
[00:34:57] Unknown:
Yeah. Cool. I think that that that makes a lot of lot of
[00:35:06] Unknown:
sense. Andrew, do you think you so you think we should hit a point, where all software that we use is reproducible, whether that's mobile or desktop?
[00:35:22] Unknown:
That would be ideal. It would be nice to get there. I don't think it will happen because not everyone not all the software developers are gonna be aware or care enough, I guess, to make their bills reproducible. But, even if, you know, even if the developers don't care, maybe the tools that they use, like the compilers and stuff, will care enough and and work towards making everything reproducible. But I I don't think we'll I don't think we will actually get there, though.
[00:35:56] Unknown:
So first of all, freaks, we're we're aware of the audio issues on Craig's side. He's trying to silently troubleshoot them. This is a live show. I'm not sure if he'll be able to before the end of the show, so it is what it is. To go back to your comment, Andrew, so, I mean, all of this is really big picture. The idea is to try and reduce as much trust in the supply process of our software as possible. And we've seen a lot of work kind of on, like, the centralized side, in terms of, you know, Apple requiring trusted signatures for all their software and trying to, you know, approve and disapprove what's in their App Store.
But from a pure, you know, free software movement perspective, that is kind of an issue in itself. Do you see a concern in a lot of Bitcoiners using, maybe these centralized app stores to download their their applications to download their software?
[00:37:12] Unknown:
I think that there isn't that much concern there, especially because, like, it is possible to reproduce, rebuild those, and and and it's not it's actually not that difficult to get the download from the store with you know, on a computer. I've done that before, and it's not that hard. But yeah. So I don't I don't think there's a huge amount of concern around doing around installing from centralized, app stores. But it it is it would be good to for people to be, using things like, other other app stores like F droid, which which does, I don't think they quite do reproducible builds, but it's like a repository of only open source apps, that kind of thing. So, and then, of course, if you can build your own the app yourself, that that would be that's much better than downloading from a from an app store.
[00:38:23] Unknown:
Right. I I guess there's there's varying levels of trust, and you're trying to just reduce them at different different levels. I mean, I guess, like, the big fear would be, or I I don't know if it's ever happened in practice. Craig is just dropping out and jumping back in to see if if that'll fix it. Is this idea that, you know, maybe like a Google or an Apple, replaces a package with the malicious package, and the user would have no idea because they're just downloading it through the App Store. Right?
[00:38:59] Unknown:
I mean, I think that's a concern, but I don't think it really matters because you're already running software, which Google and Apple can remotely update. So if even, like Right. If you're running their OS already Yeah. If you're running their OS. So if you even if you, you install the app from somewhere else, if you download an update that, an OS update that's malicious, I mean, you're screwed. So so it doesn't really matter, I guess, like, as long as you're in this ecosystem where they can just push out an automatic update to your device, and that that is an OS level thing, so it can do literally anything.
There's not much I don't think there's much reason to have a concern
[00:39:52] Unknown:
on, for App Store type things. Right. Really, the first step should be running an open source OS to begin with. Yeah. While we're while we're here on this, like, kinda tangential topic, do you wanna talk about why Bitcoin Core doesn't have automatic updates?
[00:40:10] Unknown:
Sure. Bitcoin Core doesn't want to force anyone to, use to enforce consensus rules that they aren't comfortable with, basically. You know, new new releases of Bitcoin Core have soft forks. You know, 0.21.1 had the Taproot software implemented in it, and we don't want to like, if you have automatic updates, it's kind of you would have users that upgrade without realizing that they're upgrading to a version that contains a consensus rule change, and so we don't want that. We want users to decide for themselves whether they want to support the new consensus rules.
[00:40:58] Unknown:
Right. So the up the updates are a security hole. Right? Someone can push an update to you and take your funds or change the consensus rules. Right.
[00:41:08] Unknown:
So that's the other part. It's, you know, a security problem and also kind of logistically, like, there's no Bitcoin core organization who's gonna run the servers. So who's gonna run the servers for automatic updates? That kind of thing also. Right. And who has the permission to push an update? We don't want to deal with that, and so we don't.
[00:41:28] Unknown:
Got it. So, I mean, in general, I mean, I think Bitcoin users should just none none of your Bitcoin software should have automatic updates on it, whether that's the actual Bitcoin software or whether that's the OS that you're running it on. We have Craig back. Before I get to Craig, so, Andrew, I mean, you had this thread where you were trying to reproduce a bunch of different projects, and you kept running into issues. You wanna go through, like, some of the issues that you noticed and and maybe, you know, mitigations that you hope happen.
[00:42:01] Unknown:
Yeah. So, the the most annoying issue was just the lack of documentation. That's a, you know, open an issue at the repo and tell them to put some docs up. And and and the reason I I really care about documentation is that it helps with getting new new people on board. Like, when I started contributing to Bitcoin Core, my first contributions weren't code. They were doing Gideon bills. I followed the docs. I read all the documentation and I did Gideon bills as a first my first foray into into working on Bitcoin Core. And so it's really helpful to have documentation on how to build your software and especially to build it for a release.
The the second problem is a is really a major issue even with Bitcoin Core, and it's that building, when you do a reproducible build or when you try to do a reproducible build, like, 6 months, a year after the release was made, there's a pretty good chance that you don't end up with the same binary. And that's because of the the build process depending on system system libraries or system software. So a lot of a lot of the build processes use Docker, and the way that they use Docker is just doing, like, you know, make a Docker container based on Ubuntu and then install this list of Ubuntu packages and then do the build. But if that if those Ubuntu packages have, you know, they changed their version number or, an older version got dropped from the package repository is no longer available, then the reproducible build might be different because, the version, some dependency installed to the system has changed.
This is a problem for I think it's a problem for pretty much every software I tested, including Bitcoin Core. Bitcoin Core doesn't use Docker, but Gideon works on basically the same principle. It starts virtual machines and install things from Ubuntu's package repositories. The way to fix this is to the only way to fix this is to ensure that the build environment uses exactly the same versions of every single dependency. This is difficult to do. It basically means you can't use package repositories, and so you have to, like, download everything or, you know, download the binaries for a specific version or build them from source.
And and so what we're doing in core to deal with that problem is using Geeks. Geeks is both Geeks and Nix do this, where the package the packages are reproducible and they are also specified by hash. So we say we want this version that has this hash, to build this version of Bitcoin Core. And when we do that, then if you try it again in, you know, a year or 2, it'll still use exactly that version instead of, instead of whatever the latest is. So that is how you would solve that problem, but, you know, that's the that's the next step after getting a reproducible build. You know, the the Docker and GIDEON methods, they work, you know, as a intermediate step.
So those are the 2, like, major issues. I guess the other major issue is the fact that someone just didn't build. That seems to be a documentation problem.
[00:45:57] Unknown:
That you just couldn't get them to build.
[00:45:59] Unknown:
Yeah. So sometimes docs get outdated, and so the command I run isn't the real command because the developer forgot to update the documentation and so the build just doesn't work. That happened a few times. When I was doing this, I I wasn't going to spend a whole lot of time troubleshooting, so I just kinda marked it down as build filled and moved on to the next one.
[00:46:24] Unknown:
So what do you think about MBK's idea that you have, basically, members of the community going out and trying to build these pieces of software and report back, in in return for, like, some kind of bounty. Do you think that's an achievable goal? Do you think that's the right way of going about it?
[00:46:48] Unknown:
Yeah. I think it's, it's good to encourage people to be doing reproducible builds. I think it's really important that multiple people are doing these, and like, you know, people outside of the developer groups of the specific software. And and I I like, like, you know, bringing up in public and and to encourage people to try it themselves. That's that's something that we should be trying to do and trying to get more people to do reproducible bills of the software that they use. It's, one thing is also to to always say, like, you know, do the build of a software that you use because that's something that people care more about than just build all the software. You know?
So if you get, users to do it, that that's good.
[00:47:42] Unknown:
So, I mean, I think, like, a lot of the issues here stem from just the fact that there's there's just not much eyeballs on these on these projects. Right? There's not many eyeballs, and there's just not much demand.
[00:47:54] Unknown:
Yeah. A a lot of issues come from, just kinda like things that are institutional knowledge that don't get written down for newcomers. And, you know, that that's kinda what release processes end up becoming. And, yeah, a lot of the problems are just from people who, you know, it works on their machine and someone hasn't come along with a different machine to test it out. Right. So, that that's where a lot of these issues, I think, are coming from.
[00:48:24] Unknown:
What's up, Craig? You still there with us?
[00:48:26] Unknown:
Yeah. How how do I sound? Oh, you sound way better. Yeah. Oh, great. Good. Good. Good. Yeah. So, really just interesting to hear, you know, the different kind of thoughts thoughts on on on this. The one, you know, thing that I can add is that, you know, I'm sort of writing in a Java kind of level, which gives me a virtual machine, which actually helps to abstract. It's kind of like my own little version of Docker, if you will, which which does actually make life a little bit easier. However, that is only true for the Java code that I write. Obviously, all of the non Java dependencies, you then have to have the same issues as before. So, that's been my sort of experience here in terms of being able to do this this this this work. Java has been a a great help, but I do recognize that there are certain parts of Sparrow that are not Java. And then, of course, those, you need to consider how you're gonna build build build build those. But it is, as I said, you know, sort of a journey. I think that you need to look at all the different parts and kind of just find a way to start at some place. Because even if you're just building just your own code, let alone all the other pieces that you pull in, you're still somewhat further down the path, and, you can build on that. So even though it seems like a insurmountable task, I think, to many devs at the start to try and get this right, I would just say begin at just the code that you write, bring in a a sort of intermediate build step, and then see if you can work from there.
[00:50:19] Unknown:
So, I mean, specifically with Sparrow, I saw you guys had a back and forth, you and Andrew. And I guess that particular issue was the version of Open Java that Andrew was using. Right?
[00:50:33] Unknown:
Yeah. So I have, too many versions of JDK installed. That's basically the problem. And, yeah. So I I was eventually able to build Sparrow, and, you know, this is one of the this is a relay this is one of the issues I talked about. It's, you know, it it it reproduces on my machine, but not on someone else's. Right? I I built it about 3 times, I think, 3 or 4 times. And each time, I got the same result, but they didn't match what Craig had published. And I think that's because, the way that Java packages work is, like, you package in the Java runtime too, or sometimes you do. And when you do that, it pulls the Java from your system. And if the Java version installed is not the exact same version as the developer has installed, then you get 2 different binaries.
And that seems to be what what I saw when I, examined it a bit closer. But but, you know, having having it reproducible across multiple attempts on the same system is also a really good first step for reproducibility.
[00:51:45] Unknown:
Yeah. And, that that's the that's act actually, one of the reasons that as Spiro Spiro did a recent upgrade from Java 14 to Java 6 16 was in fact to fix a bug in the sort of underlying system. And that's also a big part of this entire process is that often it's not the developers code, that it is more the build system that they depend on. In my case here, there was an issue in the Java 14 build system, that I use that prevented me from being able to do do do do do this. But after Andrew's tweet, last week, I spent some time on it this weekend, and and now at least, I've been able to build, on several different Windows and Linux systems and have got the same results across all all of them. So I'm hoping at this point that we have achieved that, but, I think it's gonna require some wider testing first before we know.
[00:52:53] Unknown:
Awesome. So, I mean, I have a plan in place, to get, core devs, Carl Dong and Nickler on to discuss GEEX and NYX, and how they can be helpful here. It might not be next Tuesday. It might not be the Tuesday after that, but some Tuesday and hopefully the near future, they will be on, and we will be discussing, those 2 projects along with the importance of reproducible builds. I think it it it's a good reoccurring it's a it's a topic that needs to be constantly addressed, I think. But before we, you know, wrap up here, I I know, Andrew, a lot of the work you've been doing, with Bitcoin Core has been in coin selection.
You know, I I think my users understand, my audience understands more so than other audiences that when you have a Bitcoin wallet, it might show, you know, you have 10,000,000 satoshis, or point 1 Bitcoin in your wallet, but, really, it's made up of a bunch of little UTXOs. You can think of them as like a bunch of bills, a bunch of cash bills in your wallet that are all different sizes. And these wallets, if if you use them in the simplest way, without actually choosing which bills you spend, the wallet is deciding which bills you spend. The so a lot of Andrew's work has been in how those UTXOs get chosen.
And I know that Craig has been working a lot about it on his side. I mean, he just added it's in test net right now, but he just added CoinJoin and, simulated CoinJoin transactions within, Sparrow Wallet. So I I was wondering, Andrew, if you wanted to go into it a little bit about how, you look at coin selection. I mean, it's it's a very difficult problem. I don't think there's necessarily an easy solution here.
[00:55:05] Unknown:
Yeah. Coin selection is kind of so there's a there's a computer science problem known as the knapsack problem, and it's like, how do you pick things out of a bag of stuff to to reach a certain value? And that's basically what coin selection does. It's, it is classified as a hard problem. And and, like, in computer science, hard means it's not polynomial time to find a solution. So so for coin selection, it ends up being like something exponential, I think, something like that. But anyways, there's a lot of different strategies, to to do coin selection, and each of them result in different, different, input sets and and different, like, wallet states.
Right? So so if you imagine, you know, the the most the easiest coin selection algorithm I could think of is first in first out. Right? I take, you know, my oldest input, and just keep grabbing the oldest ones until I reach the value that I need. That's a that algorithm, I think, you know, it does reasonably well, but it's not it's not the best and you could definitely do something smarter and better. And with coin selection, there's also a lot of like future problems to think about, you know. It's not just about can I find enough inputs to to reach the target that I want to send, but also if I spend these u UTXOs now, how does this affect me in the future?
And I would say a good example of this is if you think about the largest first selection algorithm, where I choose where the wallet chooses the largest UTXO to spend first, how this ends up working is that you break down all the really big UTXOs, and they get smaller and smaller and smaller until your wallet is just like a 1,000 Dust UTXOs, and that's not particularly helpful, because a 1,000 Dust UTXOs is basically valueless because you can't spend Dust. And so when it comes to coin selection, there's all these other problems that you have to think about like, you know, if I do this strategy, do I accidentally grind my wallet to dust? Or if I do this strategy, do I end up, you know, do I end up costing myself in the future?
And and then there's other things, like, if the fee rate is low now, do I wanna consider doing something that eats more Utexos now while the fee rates are low, or maybe I want to optimize for the lowest cost to the user? So, yeah, coin selection is a hard hard thing to to deal with.
[00:58:09] Unknown:
I mean,
[00:58:11] Unknown:
and I just think in addition to what Andrew said, you know, it's often not clear what goal the user is trying to achieve. You know? Are they trying to achieve a goal around being as efficient as they can in terms of fees? Do they wanna be as private as they can be? You know, these are also factors that need to be taken into account. Yeah.
[00:58:29] Unknown:
It's it's always hard, like, you know, what how do you define best? What is optimal? And and for each user, that could be something completely different.
[00:58:40] Unknown:
It's a it's a weird situation because it seems like in Bitcoin power user land, we are all just using coin control, and we're all just choosing which UTXOs we wanna spend for a given transaction. But then there's this massive disconnect where the majority of users are using wallets that are choosing for them, and they're using very naive algorithms to choose for them. Right?
[00:59:08] Unknown:
Yeah. This might be a hot take, but I don't think people should be using coin control because, because, people who use coin control are probably not thinking about the all of the other knock on side effects of the inputs that they choose. Although, granted, I'm sure that many wallet developers aren't thinking about those either, but also many do think about these, side effects. There are, there are ways to, you know, preserve privacy while using the coin selection algorithm by saying, you know, do selection but only on these UTXOs because I because I want these UTXOs to be grouped together and not with these other ones.
And so those are things that that are good for wallet developers to implement, in addition to, you know, trying to find a strategy that they think is good for their users.
[01:00:11] Unknown:
Craig, how do you approach this, concern?
[01:00:15] Unknown:
Yeah. Look, I think that that is, as Brad pointed out in the comments, a spicy take. I was about to say the same thing. You know, I think the reality is, is that, yeah, you you you can you can you can say, well, is the user likely to do better than the coin selection algorithm, which is being carefully considered? And there is a degree to which there is strength to that. But I I think that, you know, if if you just look at the at the the problems around trying to be private when you spend, if you look at the the fact that your change output really indicates a great deal about, you know, the sort of history of of, you know, you can as we all know, if you get paid in Bitcoin and you just take let's say you get one UTXO and you, you know, you go down to the store and you buy something, then the store owner can see how much you earn, you know, and that's obviously not ideal.
So, you know, if you are not using any form of coin control, whether you're sort of, putting UTXO's into groups or whether you're just selecting the, you know, the sort of UTXO itself, then you really are going to reveal far too much, I think. That's that's that's sort of I think the sort of default is you might reveal much more than you intend tend to. So, my my view is you can't really get away from from it. You're gonna have to have some kind of approach that doesn't just say here's a wallet, here's a whole lot of UTXO's, and, you know, the sort of algorithm will choose the best one. Because, ultimately, the algorithm doesn't know enough about how to keep you private.
[01:02:02] Unknown:
Yeah. There's it's always the algorithm doesn't know enough about how you want to spend your coins. That's, well, that's always a problem. We can't read people's minds yet.
[01:02:13] Unknown:
Yeah.
[01:02:14] Unknown:
But so do we do you guys think there's a place for I mean, well, I always tell users, and this is what I do myself, is to label all their UTXOs when they receive, so they know, you know, what what source what coins are from. Do you think there's, like, a place for that there's a there's a place for innovation here, or there's a there's a product market fit in terms of a wallet that can under can can understand those labels, kind of interpret those labels, and maybe be smarter in terms of knowing what's going on on chain. Like, I feel like a lot of these strategies don't really incorporate privacy, but if you're using your own node already, all that chain data is there. You already have the label data.
Is should we be hopeful for smarter algorithms that maybe can incorporate that kind of information, or is that far out of reach?
[01:03:20] Unknown:
I think that's a bit far out of reach because because that that that falls into, like, interpreting human text, and that's just not a fun time. The so one of the main ideas that we've had in Bitcoin Core is that if you want to separate groups of UTXO so that they aren't being spent together, then you should be using multiple wallet files. Right? You can still use Bitcoin Core but, you know, you have wallet a is, you know, income from my employer, wallet b is I bought these coins from an exchange, you know, things like that to to separate it. Because in the end, when you have these separations like that, the within when you have the separations within one wallet, you end up with basically just having multiple wallets except one place to view your full balance.
And, if you don't ever intend on having spending this those separated UTXOs together, like, then then what's the point of keeping them all in one place? That's kind of an I an idea that we have in Bitcoin Core. And it's, it's why, partially why I've also spent some time on getting multi wallet to not suck as much.
[01:04:48] Unknown:
I think, that is certainly an approach that I think makes a lot of sense and is one that, you know, you can use with just about any Bitcoin wallet as it stands. So long as you can fire up multiple wallets within the application, you should be good. The other thing I wanted to point out, I think a good approach to trying to deal with this is is actually if you can use CoinJoin. You can really, you know, it's it's obviously much more difficult to to figure out the source even if you're not spending necessarily, you know, funds that that have been through multiple rounds. At least if if you have broken up your UTXOs into equal sized amounts but much smaller amounts, then you're not as likely to give away as much information. You're you're, you know, you're you're not gonna sort of reveal how much you earn because hopefully, you've broken that that down into a number of more or less sized amounts.
[01:05:49] Unknown:
Right. I think, yeah, coin joins are great, and and people really should be using them more. Unfortunately, they're hard to coordinate.
[01:06:01] Unknown:
What about, so so Craig right now has this he's integrated Whirlpool into Sparrow. So he's into he's integrated coordinated 5 person coin join rounds into Sparrow. It's test net only right now. And then when you spend after the coin join round, it does a simulated coin join. It does like a it it has it has 2 inputs, or at the most basic sense, it has 2 inputs and 2 outputs regardless of the transaction afterwards, even though it's not necessarily a coin join. But then you could also do a 2 person coin join. So on chain, it looks it looks like a it looks like it could be a 2 person coin joint. You're not sure. You have no idea.
Do you think this is a a reasonable approach, Andrew, or is is that a
[01:07:01] Unknown:
is
[01:07:02] Unknown:
obviously, it increases your fee burden. Is is that too much of a trade off for the average user? Or I don't know. Simulated coin joints has always struck me as kind of odd,
[01:07:13] Unknown:
because it there's always struck me as, like, kinda like a false sense of security, because you're it's not really a coin joint. It doesn't add anonymity. It adds maybe some doubts.
[01:07:27] Unknown:
Like, if But it could be a coin join.
[01:07:30] Unknown:
It could be a coin join. Yeah. But if you are analyzing so for example, if, I forgot what the principle is called, but we do this in, secure cybersecurity analysis type things. But, basically, if I know that if I'm trying to figure out, you know, Matt's coins, and I know that you use Sparrow Wallet, and I know that this is a behavior that Sparrow does,
[01:08:00] Unknown:
then it's not entire then it's completely useless. Right? But the key aspect the key aspect is there's also a feature that would allow 2 Sparo users to do a coin join, just 2 people uncoordinated, just between the 2 of them.
[01:08:14] Unknown:
Right.
[01:08:15] Unknown:
So so that yeah. Okay. So we'd have to get usage of that up. But if you have usage of that up, then you really have no idea.
[01:08:24] Unknown:
Yes. Yeah. That that is reasonable. There there was something there was I remember there was some other wallet that did, like, simulated coin joins, where where I I remember it just didn't make any sense. But if there are actual coin joints, if Sparrow is going to do coin joints of this type, then that that is something I think is reasonable, to to do a simulated coin joint.
[01:08:52] Unknown:
I'm curious, Craig. I mean, we had you know, some of the audience might remember. I mean, we had a conversation maybe less than 2 months ago where we were talking about you integrating this kind of thing into Sparrow, and now we have it in testnet. What were some of, like, the issues that you ran into in terms of integration? What are these you know, did your perspectives change at all in terms of on chain privacy and how you implement this type of thing?
[01:09:24] Unknown:
Yeah. It's certainly you know, building Sparrow has, from the start been a journey for me and I've learned a lot along the way. So, yes, my views have changed for sure. But, in terms of the the sort of challenges of actually doing it, right, so obviously this Whirlpool is coming from the Samurai Wallet team, and the, the nice thing about that is that Sparrow is written in Java and Samura wallet is as well. So I was able to take the, Java client, which, has been very well written actually, and I was able to bring that into the Sparrow code codebase, and it can all, sort of run under the same VM. So that makes our life much easier. However, if you're trying to bring 2 wallet code bases together, you do have some overlap.
So there was, a challenge there to try and resolve solve that. Android applications run on Java 8, and, Spyro is running on Java 16, as I mentioned earlier. So, you know, we've got this huge difference in terms of the different versions that they run. And then, of course, we we we just have all of these, you know, for example, samurai uses a quite venerable Java package called Bitcoin j, which I'm sure Andrew is aware of. And Sparrow uses, basically, some of the same concepts, but not the same the same thing. However, they both have the same dependencies. So you have those kind of issues where you have to work through, that, you know, nobody really sees that sort of, but that's actually where a great deal of the time goes. It's just trying to figure out how to make these two things work together.
However, once that, had been overcome, it was really quite a pleasure to work with, that particular client. And it is the exact same client that you get with the some some some samurai wallet, which gives me a lot of confidence in going going forward, you know, that I'm not releasing code here that hasn't already, you know, had many years of of sort of iteration in it. But, yeah, you know, certainly was great to be able to build on the work of others. It's been a, you know, kind of building the Spire Spire haven't really, implemented a great deal. One of the things I have actually implemented is the branch and bound coin selection algorithm that is in Bitcoin Core, so I did a port of that. But, otherwise, it's been nice to have that sort of interaction and, as I say, good to be able to use the same code code base.
So, yeah, that's that's kind of how it went.
[01:12:22] Unknown:
I mean, I meant more so, I mean, you have you had to implement multiple subaccounts in the wallet. Right? So you had premix, postmix.
[01:12:36] Unknown:
Right. Right. Yeah. So sure. Sure. So so if you cost your buying back, we are there was actually introduction of database persistence into the wallet some months ago, and that was really just to prepare for this. Obviously, if you have multiple wallets and you have a single wallet file, you don't wanna be trying to write to that single wallet file every time one of those wallets has any kind of update. I don't think that's wise. Bitcoin Core also uses a database to say that it's it's it's so it would it sort of made sense to look at that exam sample and move towards that. So that was one of the the sort of pre requirements going into this, but then, being able to add multiple wallets into one wallet file was was, yes. So it's certainly something that you need to be able to do this.
And I hope that in the future, I'll be able to also use that for different subaccounts. So you'll be able to move your coin joins into different spending accounts as we were saying earlier.
[01:13:41] Unknown:
Yeah. So, I mean, I feel like with Sparrow, like, the goal has been kind of to give users, like, a power user type experience, the the ability to do, to to be more hands on with with their UTXO organization, and spending in, like, a very friendly GUI, which is kind of your way of tackling the challenges that we've discussed earlier. Right? Which is is this, you know, you're gonna put all your trust in a single algorithm in terms of coin selection. You're gonna use labeling. How are you gonna do all these things? How does it look on chain? And the way you do the visualization is pretty cool, in terms of how this transaction will look, before you send it.
I don't know. I I think a lot of people can learn from, basically, your approach. I don't know if it's for everybody. It definitely is kind of power usery. That's not a word. But,
[01:14:53] Unknown:
yeah. No. Look, I I really built the wallet that I wanted to use. I wasn't, you know, as many open source, maybe every open source project begins, I wasn't happy with what was out there. And, I just wanted to, you know, you know, in in sort of an apologetic way, just build the wallet that, I felt I wanted to use. And that's kind of been my guide throughout the entire entire thing. So I completely understand and realize that, you know, you know, it's not necessarily for everyone or even the wallet you might want to use in every occasion. For example, it's not a mobile wallet wallet wallet, so, you know, you might want something a little bit more automated in that case.
But in terms of, the kind of use cases that I wanted to try and support. The idea is really just to give the user as much information as you can, you know, to try and and reveal as much about the Bitcoin part part particle as you can. Because it's my belief that if you don't understand what's going on, Bitcoin is at this current time, and maybe the layers above it will solve these issues, but at this current time, working with layer 1 is you you're going to reveal too much about yourself or potentially make mistakes if you don't understand what's going on. I think that ultimately, you know, if you're taking sovereignty over your own funds, you need to put a bit of effort in to understand what you're doing.
That's that's honestly been my belief.
[01:16:36] Unknown:
I mean, Andrew, I'm like I'm curious about your perspective here because I feel like like, can you relate to this perspective, or or or or do you feel it's misguided?
[01:16:53] Unknown:
I don't know. Because so, like, when it comes to core, this is a project that I joined. It's not a project that I made on my own. So a lot of the things that I work on are because of decisions that were made by someone else. And and so, you know, sometimes it's not the software that I want to that I want to be making, but it's the it's what I have to work with.
[01:17:30] Unknown:
I don't I don't know. But I I I meant more from, I guess, from, like, your earlier comments, it I'm not trying to compare Core and Sparrow. I mean, I think it's good. I I I I I respect the decisions made in terms of core, in terms of trying to be as conservative as possible. I mean, it's the bedrock of of the whole network. I meant more from, like, your personal comments earlier in terms of how you approach coin selection. Right? Because it's it's it's a completely different viewpoint. Right? It's it's it's the the, like, Craig's Craig's Craig's perspective and his goal from the beginning have been to basically try and make power users out of non power users, while your perspective is kind of the opposite. Right? Your perspective is is they they should never be going into coin control. They should not be necessarily labeling explicitly UTXOs.
They should not really know exactly what's going on under the under the hood in terms of the terms of the UTXOs?
[01:18:41] Unknown:
Well, I mean, part of this is because it's these are decisions that that I didn't make. Right. These came from frankly, a lot of it came from Satoshi. Find a surprising amount of his code still lying around. And and, so so those decisions of, like, you know, do we expose do we expose these to the users, a lot of that ends up being, like, this in some previous version, we did it this way, and so it doesn't change. I've had that problem with even just changing our coin selection algorithm, and and I would say part of this is also that its core moves slowly and, doesn't want to we don't wanna break the current experience.
You know, what what users if if a user were to upgrade to the next version, is it a drastic change, that they are not expecting? So it's hard for us to completely shift to a different, like, model. Right.
[01:19:57] Unknown:
I have a question for you, Andrew. Do you get frustrated at the, you know, building a wallet GUI and releasing the consensus code for core are are about as diametrically opposed in terms of a release process as I could imagine. Do you find it's quite difficult to fit those 2 things into 1?
[01:20:22] Unknown:
Not really, because they're kind of separate. So so core, we've gotten to a point where where it's kind of modular. The wallet kinda lives in its own space. Consensus lives in its own space. The GUI lives in its own space. So there there isn't, like, conflicts between them, at least not that much. And when it comes to releases, you know, Core doesn't do feature based releases. Everything is time based. You know, every 6 or so months we do a release, and whatever got merged is in it, whatever didn't get merged is not in it. We don't wait around for features to get in, whether that's consensus or or wallet or otherwise. So I don't really find that to be
[01:21:15] Unknown:
frustrating at all. So if I was to approach that in a different way, is the the level of code review required for a consensus change is obviously huge. Do, does the same level of code review apply to a wallet GUI change, or or are they quite different?
[01:21:33] Unknown:
It's definitely less, but it's still quite a quite a lot of review, especially anything anything that touches people's coins, whether that's signing, coin selection, anything like that. Even like, storage for backwards compatibility reasons. All of this all of it requires, maybe not so much on the level of of consensus changes, which require also some conceptual review, but but still the code review is pretty rigorous. And part of this is because, it has a real world effect. You can observe in the UTXO set when coin selection algorithms change, and you can see how what you know, sometimes you can see a spike in UTXO, sometimes you see a dip, you can see changes in the distribution of values, that kind of thing.
And and so it's important for even wallet changes to get lots of review. There there's famously, one of, Merch's PRs, his his first PR decor caused a spike in the UTXO set, and it got reverted.
[01:23:00] Unknown:
Okay. What was that change? Was it, coin selection?
[01:23:03] Unknown:
It was coin selection. It was a simp it was ostensibly a simple change. I believe it was to drop unnecessary yeah. So sometimes a coin selection algorithm would choose more inputs than it really needed, and so it was drop unnecessary inputs, and reduce the change, which, you know, on the surface, that sounds fine. Right? What actually happened is that it caused change to become very small, which means that we got dust change or very close to dust change. When fee rates spiked, they became dust. And this also then caused the UTXO set to grow because less UTXO's were being cleaned up. And and some people's wallets ended up being becoming filled with dust inputs that they couldn't really use.
So
[01:23:53] Unknown:
so, yeah. That's interesting.
[01:23:55] Unknown:
Even things like coin selection have, not just impacts to the user, but impacts to the network.
[01:24:02] Unknown:
So that's that's like a dueling incentive there. Do you think do you think wallet developers should be should the priority be the user? Or or should you know, when when someone like Craig is is is dealing with his decisions in terms of of how coin selection happens, should his concern be, he has to balance the concern between the chain and the network versus the user?
[01:24:34] Unknown:
Yeah. And that's something that that, in core, we also struggle with. It's why for for the coin selection changes, I always get asked to run simulations, and I have to go and run simulations of, you know, how how many UTXOs do we end up on the network? How many end up, how big does the wallet get, that kind of stuff. So, yeah, it it is, it's a it's a very difficult balancing act.
[01:25:05] Unknown:
Yeah. I I think one of the other things that is, if that it makes it really hard to get the coin selection just to go back to that is that the fact that you don't actually know going into the algorithm all of the inputs that you require to get to the out output. So, usually what happens is you have to go around in a loop and try and and try different, kind of different ways of being able to solve solve things. So it's it's overall just just the way Bitcoin works makes that particular part of it difficult, and I think in many ways sort of an unsatisfactory from a scientific point of view because you you always feel that there's a better way to construct a particular transaction, but at some point you just have to give up because, particularly when your UTXO set is very large, it just ends up taking too long. So you end up just saying, oh, well, you know, this is gonna be enough.
But, you know, one of the reasons that I I just went ahead and ported the branch and bound algorithm directly from Bitcoin Core is really the amount of thought that needs to go into these these things. Thought and as Andrew says, testing, is very large. So, you know, and I and I I think you're absolutely right in that. Wallet developers do have to think about this and do have to take into account the health of the net network as well as their users. So it's one area in which wallets can can sort of sort of influence the health of the of of the network in the same way that Core has so many other factors that it needs to consider. But this is one that wallet developers have to consider as well.
[01:26:49] Unknown:
This is the second time you've mentioned Core's branch and bound, coin selection algorithm. Do one of you wanna describe how that works?
[01:26:58] Unknown:
I'll leave that to Andrew. He understands it much better than me. Yes.
[01:27:02] Unknown:
So branch and bound is an I algorithm that Murch came up with for his master's thesis. The idea is basically that you try every single combination of coin selection of inputs, Every single combination of inputs possible, and, and the goal is to exactly match within a small window of the to the target. So if I want 1 Bitcoin, this algorithm is intended to either fail or find 1 some input set that is between 1 and, like, 1 plus dust, basically. One of the one of the very important concepts in this is that we are willing to, throw away the amount that it would cost to make a change output.
Right? So if if a change output costs like a 100 satoshis, then we're willing to throw away up to a 100 satoshis in fees, to in order to not make a change output.
[01:28:17] Unknown:
So branch count be unspendable anyway?
[01:28:21] Unknown:
Yeah. So it's the the idea is that either we will make a change output and we have to spend those 100 satoshis in fees, or we cannot make a change output, but then, if we are as long as we're we throw away less than a 100 satoshis, it's always cheaper than if we had made a change output. Does that kinda make sense? Yeah. Yes. So that's one of the very important ideas that Merck came up with. And so what branch inbound does is it builds a tree of, like, all the possible inputs, and whether you can include an input or not, and all the various combinations. And whenever, like, if you follow one branch of that tree, you get to a point where you're past the target value, then that stops searching down that path. So that's the that's the bounding of branch and bound, and the branch is just searching all the different combinations.
And so this is fairly efficient, I think, because of the bounding, so we don't end up you don't end up searching all, 2 to the n possible combinations. And and this can with a very large input set, it probably will find something that fits in that exact match window. Awesome. And so branch and bound, we use this to to, avoid
[01:29:53] Unknown:
it's a good way to avoid making change, which is Right. So the main priority there is the chain, not the user.
[01:30:00] Unknown:
Well, there's 2 priorities with this. It's it's kinda both, actually. Like, it helps with the chain because it ends up being fairly consolidatory. And the other thing is it helps the user with their privacy because there's no change output.
[01:30:16] Unknown:
So it looks like they're spending to themselves maybe.
[01:30:20] Unknown:
Right. Well, it just ends up being that there isn't we just cut off one way that people can trace it because there's just gonna not be a change output. There won't be anything that someone could follow and say, this has probably changed, and try to link that to the rest of the wallet. So that that's that's where branch and bound helps with privacy.
[01:30:41] Unknown:
Fair enough. I mean, I I think I mean, a general heuristic that change surveillance company use uses as well is that this idea that if there's no change, you're probably consolidating to yourself.
[01:30:55] Unknown:
Right. And and in that case, this would throw that off.
[01:30:58] Unknown:
Right. Well, I'm not gonna listen to your advice, and I'm gonna keep using coin selection. But, I I appreciate the I'm still gonna use coin control, but I appreciate your perspective. So, I mean, we're we're kinda nearing the end of our time. We're over the end of our time here. I am respectful of your time. Before we get to final thoughts, we do have a question from Younglurk, asking if Craig plans to stay up to date with, Samura's current Whirlpool implementation. You wanna address that, Craig?
[01:31:35] Unknown:
Yeah. Sure. So my aim there is to, to keep on using the same clients. I've actually, have the some some of summarize teams help in just making sure that the client remains the same because I've had to, make some packaging related changes to their code, but at least we are staying on the same code code base. And I think that that's important. However, the way that it has been written is in a very modular sort of way. So it allows me to use the same code base, but to insert and add certain things that, allow me to add features that maybe the some samurai team haven't built in yet. So one of those features that we will be launching with is the ability to mix out from from a coin join after a certain number of rounds into a multisig wallet.
So you can currently do this, although it requires the command line. You can currently mix out into a single sig wallet, with the current some some some some samurai stack. And with Spiro Spiro, I will be able you'll you'll be able to mix out multi sig, which I think makes some sense for those who are looking to, you know, who who might be buying on a k by c exchange, sending it through a different, sort of through a number of different mixes, and then mixing out cold store storage. So that's, I think, one way in which, Sparrow will be able to sort of innovate on top of the Whirlpool client.
But in order to kind of keep things as aligned with a coordinator as I can, I'm gonna keep on using that same client and keep that client in sync with whatever the some some some some some some some some some some some some some some some some some some some some some some some some are doing.
[01:33:24] Unknown:
So that's the I mean, the dream is the dream is to have many wallets all using the same liquidity pool. So that's that's I mean, I'm speaking for you a little bit, but that's kind of the goal here. You share a liquidity pool. Hopefully, other wallets also decide to join the same liquidity pool. And I I guess you're also you're you're also including mixed spending tools that Samurais has, whether that's collaborative transactions like a 2 person uncoordinated coin join, or if that's the simulated coin joins, the stone walls.
[01:34:02] Unknown:
Yeah. So, I think that that's sort of almost a requirement, when you're coming out of post mix to at least have those those tools, because it just it helps everyone's an onset, to have have them them them them there. So for me, that was sort of a a basic requirement, that I wanted to have have in. And then the other one, as I just mentioned, was the ability to mix up to a different wallet, to a cold storage wallet. I think given what the Spire Wallet is trying to achieve here and its sort of primary use case as a desktop wallet, I thought that that was something which would be useful.
[01:34:37] Unknown:
Yeah. That's great.
[01:34:39] Unknown:
It's also just helpful. You know, I think the some Samurai team obviously have a concern with the amount of liquidity that they need to keep in the pool. So for them, they have no doubt considered the the the ability to mix out before, but obviously also considered what kind of impact that will have on the liquidity in those those pools. With, with what I'm doing, it kind of allows me to take a different tack without influencing theirs too much. And see, you know, my hope is that overall it will attract more people to come and coin join, and that the capability pool will actually increase even though some people are mixing out to cold storage, which obviously decreases.
So those are answers that we don't know yet, but I'm hopeful that, adding these features overall just makes the number of users more and ultimately increases the size of the pools. Awesome.
[01:35:40] Unknown:
Well, I appreciate both of your time. I think this was a great conversation. I know I'll have you back on, Craig, and I I hope that you'll join me again soon. And, Andrew, I hope to have you on again sometime soon to, to just you know, it's always great to have you. It's always a good conversation. Do you guys have any final thoughts? I guess we'll we'll end with some final thoughts. We'll start with Andrew.
[01:36:08] Unknown:
Final thoughts. Going back to the the reproducible build thing, if you are somewhat technical and you can read instructions, try doing try finding the instructions to do a reproducible build of software that you use, and try doing it and see if you get the same results. It's you know, it should just it should be more than just developers that do that. It should be, users too. The users of the software should be doing reproducible builds. So you should try doing that.
[01:36:42] Unknown:
I I mean, just encourage people in general. Right? They should be building from source.
[01:36:47] Unknown:
Yep.
[01:36:49] Unknown:
Craig?
[01:36:51] Unknown:
I I think I I kind of want to to add to that by just saying, you know, being able to build from source is important, but it's not obviously the only attack vector. And, you know, there's other ones as well, you know, for instance, there's the idea that you could have a dependency that could be updated, you know, sort of upstream for what you're doing, and that could introduce malicious code in. And building from source is not gonna help that. For example, some example, if indeed the developer has agreed to bring that pin pin pendency in. So, you know, teams that don't have a huge amount of resources need to look at all of the different attack vectors and you know, decide how many resources they're gonna put towards each each each each one.
But I I do think that being able to target intermediate steps in terms of your build process and have people be able to identify those and be able to build a the same, you know, kind of output to that point makes a lot of sense and helps kind of reach the end goal of a reproducible binary without having to try and do it in an all or not of nothing sort of
[01:38:11] Unknown:
step. I mean, I know those are final thoughts, but, Andrew, do you agree that an intermediate process, is a net benefit?
[01:38:20] Unknown:
Yes. In, getting some reproducibility is always good. Awesome.
[01:38:25] Unknown:
Andrew, I mean, I'm hoping to get, Carl Dong and Nickler on to talk about geeks and Nicks. So, I mean, it'd be awesome if you joined us for that conversation.
[01:38:36] Unknown:
Sure.
[01:38:37] Unknown:
And you've also before we fully wrap up, you've been doing Twitch yourself. You wanna show your yes, sir. Twitch account?
[01:38:48] Unknown:
If you wanna watch me work on this some stuff I work on for Core and other Bitcoin projects, you can come watch me every every Monday at 2 PM EST, on twitch.tv/hil 101.
[01:39:08] Unknown:
So Same name?
[01:39:10] Unknown:
Yep. Same same I use the same name everywhere. So if you want to, yeah, if you wanna come hang out and watch me complain about compilers not working, you can do that.
[01:39:24] Unknown:
Awesome. And that's 1800 UTC every Monday. That's awesome. Thank you to both of you guys for your time. Thank you for this conversation. It was a great conversation, and thank you to the freaks who joined us. I appreciate all your support. Thanks, guys. Cheers.
[01:40:26] Unknown:
My face above the water. Time. You are not around, slowly drifting, way drifting away. Wave after wave, wave after wave. Slowly drifting, drifting away. And it feels like I'm drowning, pulling against the stream, drifting, drifted away, wave after wave, wave after wave.
[01:43:37] Unknown:
Love you, freaks. Hope you enjoyed that rip. I know it was a bit technical, trying to find a balance here. If you appreciate the show, I appreciate your feedback, your support. That's what makes dispatch special to me. That's why I do it week after week. Yeah. I love you all. I'll see you, for rabbit hole recap on Thursday. Hopefully, I'll see you for dispatch next Tuesday. Gotta nip it in the bud, this biweekly meme of dispatch. So I will make sure that we have a solid topic and guest lineup next week. Yeah. Bitcoin binary.org is the project that NVK is working on to catalog all these reproducible builds, So pay attention to that.
I imagine it'll be updated. We'll see. And, I love you all. Stay humble and stack.
El Salvador's adoption of Bitcoin as legal currency
Bitcoin's price jump ahead of El Salvador's adoption
Challenges and concerns around El Salvador's Bitcoin adoption
Coin selection challenges and considerations
Different perspectives on coin selection and user goals
The importance of understanding coin selection and privacy
Challenges and lessons learned in integrating Whirlpool into Sparrow Wallet
The benefits and challenges of building a wallet GUI and releasing consensus code
The importance of reproducible builds and the need for users to build from source
Craig's plans to stay up to date with Samourai's Whirlpool implementation