I sat down with Andrew Poelstra on March 18th 2022 at the Bitcoin Takeover event held at Bitcoin Commons in Austin during SXSW to discuss his open source work.
Video: https://bitcointv.com/w/vmwWJVfFEL7vxY47EGtdLz
support the show: https://citadeldispatch.com/contribute
twitch: https://twitch.tv/citadeldispatch
bitcointv: https://bitcointv.com/video-channels/citadeldispatch/videos
podcast: https://www.podpage.com/citadeldispatch
telegram: https://t.me/citadeldispatch
stream sats to the show: https://www.fountain.fm/
join the chat: https://matrix.to/#/#citadel:bitcoin.kyoto
Understanding of the type of work that Andrew does, and his team is really critical to advancing the Bitcoin network. And it's at a layer that, that a lot of us don't get to look through into every day. So we really appreciate Andrew being here today. And thank you, Matt, for leading this talk.
[00:00:41] Unknown:
Yeah. Let's do that.
[00:00:47] Unknown:
Thank you, Parker. Yeah. So I'll just echo, what Parker said. We're very lucky to have Andrew joining us here. If we let him run, most of us will not understand what he's talking about. So we will try and bring it back, and keep it accessible. I guess a good place to start is how how long have you, been working in the open source space for?
[00:01:16] Unknown:
So in the open source space, beyond Bitcoin, I've been doing, I think, public open source work since I was 12, 11, or 12, so about 20 years. Originally, like, the first public open source tool that I worked on was something called PCB, which is what it sound like. It's a printed circuit board editor. It was part of the the GEDA, which is a GNU Electronic Design Apps, something like that. And I spent a summer working on just improving the UI, changing their units from, I think, 1 100th of a millimeter to nanometers so you could do more precise things when you're working in metric and things like that. And before that, I've been doing some, like, my own kind of hobby projects, but back then, I didn't have, like, an Internet connection at home, at least not all the time. I think when my parents weren't on the phone, then I could connect to the Internet. So it wasn't so easy wasn't so easy to be doing open source stuff. I got into Bitcoin, though. That's kind of a continuation of that. In late 2011, somebody on slashdot, maybe this was kind of the very end of slash dot, I think, was to kind of pump me into the Bitcoin space. Somebody was posting something wrong about Bitcoin, and I don't remember what it was. I think at the time, there was a meme that the way the Bitcoin worked, right, is that you would grind. You would find these these hashes, right, that was mining with, and then you would trade the hash values somehow. Right? You would, like, I don't know, put them in a secret envelope. And I don't know. I mean, it didn't make sense. Right? So I followed the slash daughter who who was making this claim to bitcoin.org to try to understand how all these idiots trading cashes thought that they were going to make some sort of financial system out of it.
And as it turned out, their, Bitcoin does not work by people trading hashes. And from bitcoin.org, I I wound up, on IRC, and I was able to meet I was very fortunate to show up on IRC at this time in late 2011, early 2012. At that time, Mt. Gox was in full swing. I think the price had just crashed from $30 down to, like, 15. So it was a steal. Right? I could get in at half price. And, no. No. Of course. It it continued down to, like, 2 or 3. And then it was going to 0, so I sold everything. No. Not quite. Not quite, but close. So I was very fortunate at that time. It was very early days. Sort of the only way that you could get into Bitcoin was something like that. There were sort of 2 big ways that I was familiar with.
Maybe 3. But, I mean, the the big ways were either you were a developer, like a cryptographer or a hobbyist or somebody who thought it was just a neat technical system, or you had heard of the Silk Road and it was a way to buy drugs, basically, is what you'd come in. And then there were I guess, the third way was the Ron Paul people, the kind of libertarian, like, hard money kind of set of people. But at that time, it was difficult to sell Bitcoin to a lot of the the hard money people, to the Mises Institute, for example, famously did did had a very negative view of Bitcoin. And this came from sort of an intuition that if something's on a computer, then it can't be hard money. Right? Like like, computer money is the opposite of hard money.
And a theme that we've seen throughout this day, right, is that for the most part, in everything except for Bitcoin, that intuition is true. As we move our ordinary money onto computers, we're finding we're exposing ourselves to censorship and, and things of that nature. But with Bitcoin, you don't, which is very surprising, and, like, it's not obvious that that should even be possible. Right? Because as silly as this idea was that Bitcoin was trading hash values, which is what people were saying back then, there have been a lot of good memes about Bitcoin, but that one was one of the oldest and and funnier ones.
It's difficult to think, like, how else could it work. Right? And now I think we we've as a as a community, we've started to develop some more intuition about how digital signatures work and the notion of signing transactions and transactions having inputs and outputs and so on, but it very quickly becomes very technical still. And this idea that you could have money that somehow is on a computer, and you spend the money by creating data and transmitting it over public networks that are, like, not only being wiretapped at every possible opportunity, but in the end, the data goes on to a public blockchain, which is then replicated 10th of 1000th times across the world and would be stored until long after everyone in this room is dead. It's surprising that you could transfer money that way, that somebody couldn't just download the blockchain and just copy all your coins off and and run laughing.
And so moving back to to the open source world, I I showed up on IRC. At the time, everybody on IRC, actually, still everybody on IRC, is kind of a crazy, like, gray beard developer kind of type. And the Bitcoin community was fairly small, and everybody was very welcoming. And, at that time in my life, I I was, again very fortunate in the timing. I was transitioning into graduate school. I was actually moving here to Austin, to start a math PhD program here at UT, and the first couple years of a math PhD program, I would say, are probably the easiest thing that anybody could possibly do. It was just like a free ride. It was great. They, they gave me $20,000 a year to just move to Austin and party all the time. And all I had to do was, like, write some tests every 3 or 4 months. And, and that's cool. And then, you know, eventually, they say, like, you know, you should get a supervisor. You should have a thesis topic and all this stuff. And at that point, I left. But they, at the time, it was just like it was great. I I had very little responsibility. I had nobody no deadlines, no anything. And at that time, I showed up on IRC, and there were people like Greg Maxwell there and Peter Wula and Matt Corallo, who are all still still around, by the way, although they're they're some of them are less public than they have been in the past.
And they said, hey. There's this thing called the eprint archive, which is a place where cryptographers post their papers that they're working on prior to going through the peer review and being published and stuff. And I said, hey, Andrew. Let me try reading these papers and try to understand this this Bitcoin stuff where we have this ECDSA signature scheme, and there's this other scheme called Schnorr that we think might be better for various reasons. Maybe we can investigate that kind of thing. And it's funny here to be be talking about this in 2022, and I guess, like, 7 or 8 later, we 7 or 8 years later, we finally did get Schnorr into Bitcoin. But we were talking about it back then. We were talking about Snarks, back then. There was a paper in late 2013 called Snarks for c, by, like, 8 different authors. Probably the most famous one in this movie, Eli Ben Sousen, who's the CEO of Starquare.
And it was a small community of people who were doing open source development, who were reading open access literature, cryptographic literature, and the idea to me that you could just download, like, the things that academics were doing, and you could see what was happening in this intellectual development of of the system of the whole field of cryptography was very exciting and eye opening. Right? Like, you don't need academic access to get access to the eprint archive. You don't need to be a cryptographer to read these papers.
There is some some sort of basic math literacy you need, but you can get that from the Internet as well. You don't need to go through a system. There's no no gates that you're being kept out of. There is if you want credentials perhaps, but that's not what you needed to do Bitcoin. You didn't need any credentials. You still don't need any credentials. It's very much an open access system where as long as you can be, you know, kinda kinda reasonably sociable and and get along with people, and not show up and start fights and and fool around on Twitter all day, then you can get stuff done. Right? People will recognize that that you know what you're talking about or don't, and they're very helpful. They're very friendly. They'll they'll help you to ramp up.
And so I showed up in the Bitcoin world. I had a bit of math experience, a bit of crypto experience. Like, I I was sort of primed to be working on this kind of stuff and a fair bit of open source experience at that time. I'd done a lot of public development and public projects. I was familiar with the process of, like, writing code and asking people to read it and and going through iterations and review. And people saying mean things to you and people asking me to rewrite things that you'd put weeks of, your blood and sweat into.
And so I was okay with that. And from there, there was sort of a natural a natural thing. I would I would suggest that Bitcoin, especially Bitcoin Core, is not the friendliest of open source projects, to join. It's very high friction because Bitcoin Core has consensus code in it. Right? Like, it's really and then consensus code for now, like, a multi $100,000,000,000 system is really kind of mind blowing. But at the time, it was a much smaller system, first of all, and we didn't understand consensus as well as we do now. There's an interesting, historical tidbit. If you look for the word soft fork on Bitcoin talk, you'll find that the first use of that term, I believe, dates to 2012 or so, around the same time that the idea of mining pools showed up, and other things that we take for granted, like, very old thing. But, actually, Bitcoin had been going for several years. There were big exchanges running. And at the time as a community, we didn't understand the notion of hard forks and soft forks and how consensus worked and and things like that.
It was a very, very, early Wild West day. Like, even even for Bitcoin, like, this was a Wild West. This was pretty crazy. And, secondly, as much as Bitcoin Core is in at the time, it was just called Bitcoin. It was a very controversial name change for for reasons I'm not going to go into. I gravitated more towards the crypto side of things and the cryptography behind Bitcoin is actually done in a separate software library called Libsec p 256k1. And I remember Greg Maxwell, who at the time was just like the god of Bitcoin. And and to this day, actually, if there's one person in the world who's responsible for Bitcoin existing and surviving, and thriving the way that it has, it will be Greg Maxwell.
I was talking to him on IRC because he's he's actually pretty easy to talk to on IRC, and he suggested that I work on an arm assembly implementation of some crypto algorithm. Right? Some, like, very obscure technical thing. And I was kinda shocked that Greg Maxwell would think that I could do something like that. Right? Like, it's just, like, I was kinda very flattered. And and even though I wasn't, I had a little bit of experience doing stuff like that. It was with arm assembly and crypto and stuff. I thought, wow. Like, I should go do that. And so I spent some time trying to do it. And in in the end, I didn't actually succeed at that. To this day, I have not touched the arm assembly code in the. But by doing that, I learned about how the library worked. I learned that who the different developers were. I learned how to interact with them.
And between that and my reading of stuff on the eprint archive, I was then able to become a cryptog, a contributor to the Libsakp library, which does all of the crypto. And, eventually, I became a maintainer, actually, for several years. I was one of the persons, who had merge access to to that library. And I remember when this first happened, I was very scared. Bitcoin at the time had a market cap, maybe $1,000,000,000 around this time, I think, maybe maybe 2 or $3,000,000,000, and the cryptography underlying it, everything in Bitcoin is cryptography somehow, but there's a core of it that's, like, is, like, hard cryptography. It's it's, like, pure cryptography. I I don't know what the right word is, but it's the signatures. Right? It's the stuff that deals with secret keys and with public keys where if those keys are leaked or lost or something, then all the money is gone forever.
And there are ways there are very subtle ways that you can lose key material. Whenever you have computers working with secret key material, that's a very dangerous and fragile situation. It's it's really incredible that cryptography works at all. Right? Because you have all these public networks. You have these public logs. You have everything. Everything can be seen. So how can you take a secret key? You put this into some sort of computer system. It takes that secret key and does something with it, and it outputs something that looks, you know, random, but somehow derived from that key in a way that you can verify, because you can verify the signatures and stuff. But somehow, it doesn't reveal anything about the key, and there is a lot of, like it really is dancing a fine line. The way that it works, right, is the computer needs to generate a second ephemeral key, and then it turns out that if you mix 2 uniformly random things together, the output will be uniformly random. Sorry. If you mix a uniformly random thing and anything else together, the result will be uniformly random. So, there's a sense in which you can erase data by mixing it with uniform randomness, and that's what happens as part of the signature process.
But if you mess up and you don't use quite uniform enough randomness, right, if you use, like, too many one bits or whatever, then it's not uniform, and you don't erase all the secret key data, and then your keys get leaked. And, in fact, this has happened on the real network. So I was working on the library that was used in Bitcoin Core or Bitcoin at the time, and this was the most popularly used library. On the verification side, it is, like, the by far the most popular. It's it's the library that's in the reference implementation, right, on the reference Bitcoin node. And this library is actually used inside of a lot of hardware wallets. It's used in all of the major Ethereum node implementation that's used in wallets for that. It's used in, like, all over the place, even in non cryptocurrency application. It's a very widely used library.
And if the code that I was writing or reviewing or merging was somehow wrong, then there was potentially 1,000,000,000 of dollars that could just go away. And at the time, I was, you know, in my early twenties. I was, like, screwing around, like, doing some sort of immigration scam, with the UT here so that I could fool around and, you know, play the piano and drink beer all the time. And I had this multibillion dollar system that somehow depended you know, I I think there were safeguards in retrospect. But, certainly, it felt at the time like it depended on me not doing anything too reckless.
And this is a very strange experience. It was a very strange phenomenon, really, like, at the time. And and now Bitcoin is much bigger, and there are many more stakeholders. And and in many ways, it's not the wild west, and there really are safeguards. Right? At this point, like, I actually probably couldn't screw up the system even if I tried. But at the time, maybe I could've. And the idea that I could do that despite not being credentialed in any real way, I was just a student who, you know, in the the professional world, certainly in software development and and in cryptography. Being a student is not a good thing. Right? People are very distrustful of students writing cryptography code, with good reason.
It was cool that I could just walk in and sort of do that. And so that was that was probably 2013, 2014, and I I continue to to ride that out pretty much to this day. In 2014, there was a company called Blockstream that started. And they're they're reasonably well known, certainly on the technical side of things. A lot of people have heard of Blockstream and outside of the the technical sphere or maybe not as well known, but they were at the time in late 2014 and and a project by a bunch of people basically on IRC. We had a couple, like, VCU, like, California people. I don't know where they came from, and they're not they're not there anymore, who showed up and kinda facilitate a thing that would buy us flights and and do all sorts of cool stuff. But what it felt like to me and what it was really was me and a bunch of people in IRC who were able somehow to form a company around all this experimental kind of crypto stuff that we were doing basically in our spare time for fun.
And for me, it meant that I could stay in Austin where I no longer had to be going. Like, I I had a student visa at the time with UT, and that was really why I was still in that PhD program. And I was able to drop that and switch to a work visa and stay in Austin, and that was cool. But it was also cool that we were making money, or at least raising money doing, cryptography, doing, like, experimental doing experimental, like, novel cryptocurrency research. Like, cryptocurrency research wasn't even really a thing at that time outside of outside of us and a couple other similar groups. And so I've continued basically in in that vein over the last, 6 or 8 years pretty much since Blockchain has been formed. At this point, I I I am the director of research at Blocks, and I run a team of researchers.
For a little while, we were kind of the only team, in town, but now there are many, which is super cool to see. Chainco Labs also funds a lot of Bitcoin development, and they they partially fund the Bitcoin OpTech newsletter and and also the outreach associated to that. There's Brink, which is a company, Mike Schmidt, who who lives here in Austin, although he's he's not here. He lives outside Austin. No. I shouldn't point in what direction, but, you know, he's around. And, and there's this block, of course, which was Square Crypto until recently. Right? There's there's all these different groups who are doing Bitcoin research as well as stakeholders, you know, exchanges and stuff.
Funding research, like paying for core developers to work and and contributing to this open source world. And this is a fascinating thing to see as aside from just the fascination in Bitcoin itself. From the open source perspective, there's been forever open source has been around forever. It's been around in some form or other probably since the seventies. I think it sort of came in as an idea. And before that, actually, everything was, like nobody even thought about trying to monetize source code. And then when people did open source, it was right there to say, well, you know, maybe we could do that without trapping people and trying to control access to information and, like, somehow forming a censorship based monetization model.
So open source has been around forever, and open source led to to many things that we know and love. Right? The Linux operating system, which is used in in half the phones that are out there as well as 80% of all the servers and stuff out there. And all over the Internet. Right? The the decoders that show you JPEGs and GIFs and the original web browsers and and actually all current web browsers are all open source. It's just everywhere, and historically, there was no way to monetize this. There was this this horrible phenomenon where people would invent groundbreaking critical things, as you have an xkcd about this. People who would write code that would then become, like, a foundational critical component of the infrastructure of the modern world, and they would be basically begging for donations on the Internet, Or they will be making, like, $40,000 a year because some random software company was willing to hire them in the in the, like, whatever part of Oklahoma that they lived in and stuff like that. And and they were doing, like, really critical stuff that's leading to trillion of dollars of wealth creation in other parts of the industry, but they personally don't see a dime of that.
And in the Bitcoin space, we are very fortunate in that there is a lot of money kind of by definition in the Bitcoin space, and more broadly geopolitically there's a lot of money in cryptography. There's a lot of money in computer security, and it's possible to get a very high paying job doing bitcoin development. And if you were here around in the the very early days of Bitcoin as well, it's also possible to make a nontrivial amount of money by just owning some Bitcoin and holding on to it. Although I should I should maybe clarify because there's a meme out there. One of the many, many ideas that people have about Bitcoin is that all the OG developers have just, like, 1,000,000,000 of dollars or whatever. And this is actually not true for a couple of reasons. And I'm obviously that's I have a very self serving motivation in saying this to everybody, but it is legitimately a a falsehood that, Bitcoin OGs all have tremendous piles of money. One reason is that they, like many people, after buying Bitcoin and seeing it go up 10 x, right, from 15¢ to a buck 50, just like fold it all and said like, all right, you know, I I can buy a car now.
Another reason is though at the time, it seemed like a really experimental kind of sketchy system that was going to fall over any day, and it really did for a very long time, like, longer than than anyone would expect. I would say personally that it was maybe, like, 2016 or 2017 before I thought that maybe the system could could maybe keep a phone, that there wasn't going to be some break and shot to where, like, the mining incentives would turn out not to work or, you know, who knows? Maybe the crypto would be broken in some way or maybe just, like, network systems. Maybe it couldn't handle partitioning or maybe there'd be a fork or or whatever. And and so the people who are closest to the system really building it had a much clearer idea of the fragility of the system and the inherent fragility of cryptography of of how narrow a dance all of this code was doing to work with secret key data to do meaningful things.
And so the people who were building the system and working on it the hardest were probably the most suspicious of it, beyond the people who would, like, look at it once and walk away, of course. And then finally, like, as a more general thing, it's good not to, to, to invest too heavily in something that your career already invests in. Right? We we learned that from Enron, but as a general thing. Right? That just your your personal what you're good at is a self, a form of investment. So by diverse the diversification mandates that you probably shouldn't have all your actual money is something that your career also depends on because that way, you know, you you while you have some protection against 1 or both of those sorry. You have some protection against one of those, not both going away.
And, and if instead you put all your chips where where, where your employment prospects are, you know, that's that's a very risky thing to do. So, so for that reason, I mean, a few people did get very rich, and I'm not gonna say who, but, you know, a few people did. But for the most part, developers in the Bitcoin space were not able to just, like, buy 10,000 coins back when they were worth 50¢ and and have held on to them today. But what we were able to do is form an industry, a thriving industry with a lot of economic activity happening in it where it's possible as an open source developer to be well paid and well respected. And that's a very novel thing in the history of open source and in the history of development, and it's a very precious thing. Like and and I again, obviously, this is very self serving. It's precious that I get paid so much, isn't it? But what I mean is, historically, open source I guess, I I haven't I realized I haven't defined open source, but it's the idea that that you can publish code and anybody can show up and contribute
[00:23:30] Unknown:
to it. Right? And that the code is public. You can vet it. You can verify it.
[00:23:31] Unknown:
If you think there's something suspicious about it, you can read it and and assure yourself that there's not, and there is. If you want to contribute to it, if it's broken, you can fix it. You can submit changes. People will take it, and, you know, everybody can share. And, historically, there hasn't been any model to to make that sustainable economically other than charity, basically. So, for example, the Linux operating system for a long time, basically, like, IBM was paying a couple developers to do it. And now there are, you know, probably a couple dozen companies like IBM paying Linux kernel developers.
But they're doing it because, you know, with 1 or 2 employees on, like, a, you know, 100000 person payroll. So So it's fine. It's not very much money, and their business does kinda depend on Linux continuing, but it's just sort of a charitable thing. And that works for Linux because Linux is huge and important, but it doesn't work for a lot of smaller projects that wind up being done by individuals in their spare time. So now it's personal charity. Right? It's people just doing a lot of these projects out of a labor of love, and it's very cool that for something like Bitcoin, which for a lot of us was a labor of love, we're able to be paid for that. And then, we're able to do it in a financially sustainable way. So so that's it's pretty cool.
[00:24:45] Unknown:
I mean, dude, you fucking killed it. You have 2 minutes left. It's 2
[00:24:54] Unknown:
minutes left. Cool. So I I think I'll seed my last 2 minutes, then if it's really 2 minutes. So thank you all for listening.
Understanding the type of work Andrew does and its importance to the Bitcoin network
Andrew's experience in the open source space and his introduction to Bitcoin
Andrew's involvement in Bitcoin Core and the challenges of working on consensus code
Andrew's role as the director of research at Blockstream and the growth of Bitcoin research
The significance of open source development in the Bitcoin space and the ability to be well paid and respected