Dedication
To the love of my life, my wife, Linnéa. Smart, faithful, real.
And to all awesome Bitcoiners everywhere.
Foreword
What is the difference between Bitcoin and the various monies used in multiplayer games? Each is defined by software, is created out of nothing, and can have prices comparable to traditional currencies. To a casual observer, they seem much the same.
But there is an important difference in how they’re designed that affects every aspect of their respective systems: whereas game currencies try to stop cheating by reactively banning users suspected of abuse, Bitcoin proactively prevents various types of cheating by making them impossible, impractical, or less profitable than honest behavior.
Grokking Bitcoin describes exactly how Bitcoin accomplishes that. As you learn about hash functions, digital signatures, proof of work, and many other technical concepts, you’ll see over and over that Bitcoin effectively prevents cheating without requiring the global banning of any user.
The book then demonstrates the many benefits of this key feature. As you learn about addresses, wallets, miners, and nodes, you’ll discover that if no one can cheat effectively, it’s safe to allow everyone to participate in every part of the protocol, leaving no one in a trusted position of authority over the system. And with no one holding a trusted position, there’s no need for identities in the first place. Grokking Bitcoin goes on to show how it’s possible for anyone to use the public Bitcoin blockchain with a degree of privacy.
A decentralized system that doesn’t use identities or depend on trust is quite different from the everyday systems with which most of us are familiar. Author Kalle Rosenbaum turns this liability into an asset by starting the explanation of each part of Bitcoin using a centralized, trusted, or identity-dependent system that any reader can intuitively understand. He then transforms that basic system step-by-step into the decentralized, trustless, and pseudonymous system implemented in Bitcoin. With each step filled with clear explanations and many excellent illustrations, even the most highly technical topic is made accessible to the interested layperson.
Bitcoin needs books like Grokking Bitcoin, but it also needs an active group of users who read those books and come to understand the technical principles on which Bitcoin is built. During these early days of what I hope will be the long history of Bitcoin, users are often asked to evaluate proposed changes to the system—changes that may affect the security and privacy of their bitcoins. Those who have read this book will understand how the system prevents cheating and will be able to help ensure that future changes preserve that essential feature and its many benefits.
—David A. Harding
Contributor to Bitcoin documentation
Preface
Bitcoin is changing the world.
The world has seen many major improvements to human life, among which are vaccines, electricity, radio, automobiles, and the internet. Some of these technologies start out as rich people’s games, but eventually they trickle down to the general public and bring huge benefits to Earth’s population. Bitcoin will soon be right there among them. This is what makes me super-excited about Bitcoin.
Luckily, I’m fortunate enough to live in a pretty functional society. When conducting financial transactions, I never have to worry that someone will come knocking on my door.  But that also makes it harder for me to grok why Bitcoin is important. Bitcoin is mostly theoretical to me, but when I hear reports from less fortunate people living under oppressive or incompetent regimes about how Bitcoin makes their lives better, it becomes very real. Bitcoin will give people an opportunity to opt out of the system that’s holding them hostage.
Satoshi Nakamoto, a pseudonym for a person or a group that wants to remain unknown, published a scientific paper on a cryptography-oriented e-mail list in October 2008. The title of the paper was, “Bitcoin: A Peer-to-Peer Electronic Cash System” (see web resource 1 in appendix C). Nakamoto’s paper described the vital parts of Bitcoin, the first digital money system where there’s no central authority to issue money or process transactions. In January 2009, Nakamoto published the first software program to implement the described system. Bitcoin didn’t get much attention then, apart from within a very limited set of cryptography experts. Gradually, as the system proved itself to be working, more people became interested. But the resistance against Bitcoin was and is still far bigger and louder than the adopters, as is the case with all groundbreaking new technology. As of 2019, hundreds of millions of people are aware of Bitcoin, and tens of millions of people are using Bitcoin.
When I started exploring Bitcoin in 2013, it took me a long time to acquire a decent understanding of the technology. That’s not because I’m remarkably stupid; it’s because Bitcoin is a complex system. It’s not just a fancy database—it’s a mishmash of economy, mathematics, technology, and anthropology.
I started a technical blog about Bitcoin in 2015, and I guess Manning liked my content, because the company sent me an email asking if I was interested in writing a book about “blockchain.” Since my passion is Bitcoin—not only the blockchain, an overly hyped word for Bitcoin’s database—I replied and thanked Manning and said I’d be interested in writing a book about Bitcoin. I’ve struggled a few years trying to find a place in the Bitcoin community, and this opportunity seemed like a great fit. The project started, and it turned out to be far harder and far more time-consuming than I expected.
This book began as a typical technical description of Bitcoin, but it was difficult to teach one topic without the full context. The cognitive load at the beginning of the book would be too overwhelming. It became clear that I needed to do something else. I discussed it with my wife, and we came up with the idea of conceptually building Bitcoin from the ground up. The book starts with a really simple spreadsheet system that anyone can understand, and that system evolves into Bitcoin. Each chapter points out some problems with the current system, and we fix them by adding the technology covered by that chapter.
The manuscript for Grokking Bitcoin will be released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license shortly after its print publication. Releasing a version under an open license was a requirement I had in writing this book. It is my way of giving back to the Bitcoin community that has given me so much over the years. It is my, and Manning’s, hope that those who can’t afford to buy this book will benefit from the open source version. There are, of course, less philanthropic reasons. We also hope that open sourcing this book will give it more visibility, as potential readers browse through it. Please buy the beautifully typeset version if you can afford it. Manning and I have spent an enormous amount of time and effort in producing this book, and we would greatly appreciate the income!
I hope you enjoy reading Grokking Bitcoin as much as I suffered writing it!
Acknowledgments
I have a lot of people to thank for making this book possible.
First, I’d like to thank my wife, Linnéa, for all our valuable discussions and your consistent encouragement. You are absolutely fantastic.
Christina Taylor at Manning, my dear development editor: thank you for keeping me busy. Work work work. Thank you also to Bert Bates for your very valuable input on the pedagogy.
I’d like to also thank everyone else at Manning who contributed with their professionalism, including Ozren Harlovic, Rebecca Rinehart, Candace Gillhoolley, Ana Romac, Michael Stephens, Erin Twohey, Christopher Kaufmann, Matko Hrvatin, and Greg Wild. A special thanks to the fantastic production team: Chuck Larson, illustrator; Rebecca Deuel-Gallegos, copy editor; Kevin Callahan, typesetter; Tiffany Taylor, proofreader; and Lori Weidert, production editor.
I’d also like to thank Jonathan Jogenfors and Pontus Lindblom for reviewing some chapters covering their respective fields of expertise.
A lot of people on various internet forums were very helpful during my research. The ones that stand out the most are David A. Harding, Pieter Wuille, and Mark “Murch” Erhardt. Thank you for all our Twitter conversations and great answers on Bitcoin Stack Exchange (see web resource 2 in appendix C).
I got help from Andreas M. Antonopoulos when negotiating the open source terms for this book. Your encouragement during that process was gold. Thank you! In addition, thanks for the “vegetarian restaurant” analogy used in Chapter 11. Thank you also for your amazing talks; you are a major source of inspiration for me.
I wrote this book using Asciidoctor, which is a text markup language. Thank you, Dan Allen, for your hard work on Asciidoctor; it’s fantastic.
The book went through three review rounds, with several reviewers in each round. Those reviews were extremely useful in vetting ideas and finding gaps in the learning path. Thank you to Jan Goyvaerts, Max Humber, Iryna Romanenko, Jean-François Morin, Al Krinker, Joel Kotarski, Markus Beckmann, Christopher Bailey, Viton Vitanis, Paolo Freuli, Tomo Helman, Marcello Seri, Maciej Drozdzowski, Cicero Zandona, Barnaby Norman, Frances Buontempo, Glenn Swonk, and Sergio Fernandez Gonzalez.
Thank you also to all the Manning Early Access Program (MEAP) readers who contributed their thoughts, corrections, and questions on the Manning Forum, and to Aruna Surya for providing feedback via email.
If I left someone out, I’m terribly sorry; thank you, too.
About this book
The primary goal of this book is for you to be able to decide for yourself whether you trust Bitcoin. On the way to that goal, you’ll learn a number of Bitcoin concepts—such as digital signatures, proof of work, and peer-to-peer networks—on a pretty deep level. Some secondary goals fall out naturally:
-
Install and use a Bitcoin wallet on your phone and understand what you’re doing.
-
Engage in technical Bitcoin discussions.
-
Make informed decisions about how to store your private keys depending on the number of bitcoins stored and the required level of security and convenience.
-
Run a full Bitcoin node to engage in financial transactions without trusting a third party.
-
See through extraordinary claims made by scammers, deceivers, and conmen who are piggybacking on Bitcoin’s success. Be careful out there!
Who should read this book
This book is intended for technically interested people who want to understand Bitcoin on a deep technical level. The book doesn’t require any programming skills, but a basic understanding of some technical concepts is beneficial—for example, databases, computer networks, computer programs, and web servers. A little math background can be useful too, but it’s certainly not required.
How this book is organized: A roadmap
This book consists of 11 chapters and 3 appendices:
-
Chapter 1 is an overview of Bitcoin. You’ll learn what Bitcoin is, why it matters, and roughly how it operates.
-
Chapter 2 discusses cryptographic hash functions and digital signatures. These are the fundamental building blocks needed for the rest of this book. I also lay the groundwork for a fictive money system, the cookie token spreadsheet, that we’ll build on in chapters 2–8.
-
Chapter 3 covers addresses. When you send bitcoins, you send them to the recipient’s Bitcoin address. What are Bitcoin addresses, why are they needed, and how are they created and used?
-
Chapter 4 goes through how a Bitcoin wallet keeps track of your secret keys and how multiple secret keys can be generated from a single huge random number called a seed. Backups are also discussed in detail.
-
Chapter 5 explores the anatomy of a Bitcoin transaction and how transactions are digitally signed and processed.
-
Chapter 6 discusses the blockchain: the database where transactions are stored. We walk through how the blockchain is structured and how it enables the use of so-called lightweight wallets.
-
Chapter 7 covers proof of work, which is used to select who gets to add new transactions to the blockchain. This process, called mining, is what keeps your bitcoins secure in the blockchain.
-
Chapter 8 explores the Bitcoin network. Bitcoin has no central point of control, and you’ll see how that’s possible with a peer-to-peer network. I also explain how to take active part in the Bitcoin network by running your own node.
-
Chapter 9 revisits transactions. We circle back to discover some bells and whistles that are important for various applications.
-
Chapter 10 introduces segregated witness. Bitcoin was upgraded with major improvements to transaction reliability, verification efficiency, and blockchain capacity in 2017, and this chapter gives you all the details.
-
Chapter 11 goes through soft forks and hard forks and how Bitcoin can be safely upgraded using a soft fork together with a careful deployment plan.
I suggest that you read chapters 2–8, where we’ll build the cookie token system from the ground up, sequentially. Each chapter adds a technology to the system to solve a specific problem, and by chapter 8, we’ll have built Bitcoin. Chapters 9, 10, and 11 can be read out of order, but I recommend reading chapter 11 carefully, because it covers the essence of Bitcoin. If you get chapter 11, you’re grokking Bitcoin.
I reuse some overview figures from Chapter 1 every now and then throughout the book to help you with orientation, both in chapter intros and embedded in the chapters. It’s easy to lose track of the big picture and the goal of the current topic; look for the periscope and section headers like “Where were we?”
Each chapter, except Chapter 1, contains exercises. They’re there for you to assess your skills. Each batch of exercises is divided into an easier section called “Warm up,” used for shorter fact checks; and a tougher section, “Dig in,” that requires more thinking. Some of the exercises in the “Dig in” sections are dreadfully difficult, so if you get stuck, please consult Appendix B for answers.
Code conventions
There’s not much code in this book. None, actually. But there are some Linux commands in Chapter 8 and Appendix A. A command is prefixed by a dollar sign and a space, as follows:
$ cd ~/.bitcoin
When a command is too long to fit on a single line, we break the line
with a backslash \
and indent the next line by four characters, as
follows:
$ ./bitcoin-cli getrawtransaction \ 30bca6feaf58b811c1c36a65c287f4bd393770c23a4cc63c0be00f28f62ef170 1
Backslashes can be used to write commands across multiple lines in most Linux command-line interpreters, so you can copy and paste such commands into your terminal. Lines of output from commands aren’t broken with backslashes; they’re instead wrapped as needed using a line break arrow as follows:
{"result":"000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f", "error":null,"id":"1"}
Throughout the book, data is written in a fixed-width
font: for
example, 7af24c99
. I usually don’t explicitly write out what encoding
is used (decimal numbers, hexadecimal strings, base64 strings, base58
strings, and so forth), because it’s often obvious from the context.
liveBook discussion forum
Purchase of Grokking Bitcoin includes free access to a private web forum run by Manning Publications where you can make comments about the book, ask technical questions, and receive help from the author and from other users. To access the forum, go to https://livebook.manning.com/#!/book/grokking-bitcoin/discussion. You can also learn more about Manning’s forums and the rules of conduct at https://livebook.manning.com/#!/discussion.
Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers, and between readers and the author can take place. It is not a commitment to any specific amount of participation on the part of the author, whose contribution to the forum remains voluntary (and unpaid). We suggest you try asking the author some challenging questions lest his interest stray! The forum and the archives of previous discussions will be accessible from the publisher’s website as long as the book is in print.
Other author resources
If you have specific questions about Bitcoin that you didn’t find the answer to in this book, I recommend Bitcoin Stack Exchange (Web resource 2), which is a platform for questions and answers where good answers are up-voted by readers.
I also recommend the Bitcoin Developer Reference (Web resource 3), for more comprehensive documentation of Bitcoin.
The Bitcoin Core source code (Web resource 4) is the most accurate source of information. It’s the reference implementation of the Bitcoin protocol, and reading that source code is sometimes the only way to find answers to questions.
If you want to search the contents of this book online, I recommend working with the source code available at Web resource 5. It will be released at the latest 90 days after the book is published.
About the author
Kalle Rosenbaum has worked as a software developer for 20 years. His passion for Bitcoin began in 2013 and has continued uninterrupted. Kalle started a Bitcoin consultancy company in 2015 and has worked in the Bitcoin industry since then. He also writes a technical blog that explains various technical Bitcoin topics, such as block-propagation improvements, sidechains, and replace-by-fee. The purpose of the blog is to teach himself and let others benefit, too.
1. Introduction to Bitcoin
-
Getting to know Bitcoin
-
Following a Bitcoin payment
-
Problems solved by Bitcoin
The goal of this book is to teach you enough about Bitcoin to make informed decisions about how you can use it to improve your private life or business. My hope is that you’ll learn enough to make up your own mind whether you trust Bitcoin or not (with any luck, the former). To get you off the ground, I’m going to assume you know roughly what the following terms mean:
-
Computer program
-
Database
-
Computer network
-
Web server
If you’re unsure of any of these terms, don’t worry. Either look them up or go ahead anyway—I think you’ll manage.
1.1. What is Bitcoin?
Bitcoin is a digital cash system. It allows for people to move bitcoins, the currency unit of Bitcoin, between each other without using a bank or any other trusted third party. It resembles traditional bank notes and coins, but it’s purely digital and used over the internet. The Bitcoin currency isn’t tied to any specific fiat currency like the US dollar or the Chinese renminbi; it has free-floating exchange rates against most fiat currencies. You can buy and sell bitcoins for fiat currencies online using one of several exchanges, such as kraken.com, bitstamp.net, or localbitcoins.com.
No government or company controls Bitcoin. Instead, thousands of computers around the globe—the Bitcoin network, shown in Figure 1—collectively keep the system working 24/7. You don’t need to register or sign up anywhere to use Bitcoin, you just need internet access and a computer program, like a mobile app, to use it.
Anyone can use or participate in the Bitcoin network without special permission from a bank or similar institution. Thanks to Bitcoin’s permissionless nature, a lot of Bitcoin-related technology has emerged over the years. We can roughly categorize participants in this Bitcoin ecosystem into several groups:
- End users
-
People using Bitcoin for their day-to-day needs, such as savings, shopping, speculation, or salaries
- Corporate users
-
Companies using Bitcoin to solve their business needs, such as paying wages internationally, or use cases similar to those of end users
- Merchants
-
For example, a restaurant or a bookstore accepting Bitcoin payments
- Bitcoin services
-
Companies providing Bitcoin-related services to customers, such as topping up mobile phones, anonymization services, remittance services, or tipping services
- Exchanges
-
Commercial services people can use to exchange their local currency to and from bitcoins
- Protocols on top
-
Systems that operate “on top” of Bitcoin to perform certain tasks, such as payment network protocols, specialized tokens, and decentralized exchanges
- Bitcoin developers
-
People working, often for free, with the open source computer programs that participants of the Bitcoin network use
The Bitcoin network’s job is to process Bitcoin payments, secure the ledger of who owns what from unauthorized modifications, and get new bitcoins into circulation at the predetermined rate. The network consists of thousands of computers around the world. We call these computers Bitcoin nodes, or just nodes. Any of the actors mentioned previously can also participate actively in the Bitcoin network by running their own Bitcoin node. You must run your own node if you don’t want to trust others to provide you with correct financial information.
1.2. The big picture
The Bitcoin network is a network of computers running Bitcoin software. This network verifies and confirms payments between Bitcoin users.
Suppose Alice wants to make a payment of 1 BTC to Bob. The payment starts with Alice creating a transaction and sending it to the Bitcoin network, as shown in Figure 2. I outline the process’s four steps here and explain each step further in the following subsections. Figure 2 will appear in the introduction of chapters 2 through 8, where I’ll point out which part of the figure we’ll cover in the chapter.
Now, let’s follow Alice’s payment from Alice to Bob:
-
Alice creates and signs a transaction that moves 1 bitcoin from her to Bob. She then sends the transaction to the Bitcoin network.
-
The computers in the network—the Bitcoin nodes—check that Alice actually has the money to spend and that the transaction is authentic. They then pass the transaction to their neighbors, called peers.
-
Each computer updates its own copy of the Bitcoin blockchain, or the ledger, with the new payment information.
-
The network notifies Bob that he has received 1 bitcoin.
Note how Alice does not really send 1 bitcoin to Bob, but asks the Bitcoin network to move 1 bitcoin from Alice to Bob in the Bitcoin blockchain.
The Bitcoin blockchain is a database that each computer in the Bitcoin network has a copy of. Think of the blockchain as a ledger of all transactions ever made.
We’ll go through these steps in more detail in the next four sections, one step per section.
1.2.1. Step 1: Transactions
Step 1 of the process (Figure 3) is when Alice asks the network to move 1 bitcoin to Bob. She does this by sending a Bitcoin transaction to the Bitcoin network. This transaction contains instructions on how to move the money and a digital signature that proves it’s really Alice requesting that the money be moved.
The Bitcoin transaction is a piece of data specifying
-
The amount to move (1 bitcoin)
-
The Bitcoin address to move the money to (Bob’s Bitcoin address,
15vwoaN74MBeF5nr2BH4DKqndEFjHA6MzT
) -
A digital signature (made with Alice’s private key)
The digital signature is created from the transaction and a huge secret number, called a private key, that only Alice has access to. The result is a digital signature that only the private key’s owner could have created.
Alice’s mobile wallet app is connected to one or more nodes in the Bitcoin network and sends the transaction to those nodes.
1.2.2. Step 2: The Bitcoin network
Alice has sent a transaction to one or more Bitcoin nodes. In step 2 of the process (Figure 4), each such node checks that the transaction is valid and passes it on to its peers. It does this by consulting its local copy of the blockchain and verifying that
-
The bitcoin that Alice spends exists.
-
Alice’s digital signature is valid.
If all checks pass, a node forwards the transaction to its peers in the Bitcoin network. This is known as relaying. Alice’s transaction will shortly have traveled the entire network while each node verifies it along the way. The blockchain hasn’t been updated yet; that’s the next step.
1.2.3. Step 3: The blockchain
In step 3, nodes update their local copies of the Bitcoin blockchain with Alice’s transaction. The blockchain contains historic information about all previous transactions; new transactions, such as Alice’s, are appended to it every now and then.
Updating the blockchain with Alice’s transaction isn’t as straightforward as it might seem. Alice’s transaction isn’t the only one going on in the Bitcoin network. Potentially thousands of transactions can be in flight at the same time. If all nodes updated their copy of the blockchain as they received transactions, the copies wouldn’t remain copies for long because transactions can come in different orders on different nodes, as Figure 5 shows.
To coordinate the ordering of transactions, one node takes the lead, saying “I want to add these two transactions to the blockchain in the order Y, X!” This message, known as a block, is sent out on the network by that leader (Figure 6), in the same way that Alice sent the transaction.
As nodes see this block, they update their copy of the blockchain according to the message and pass the block on to their peers. Alice’s transaction was one of the transactions in the block and is now part of the blockchain.
Why would a node want to take the lead? The node that takes the lead is rewarded with newly minted bitcoins and transaction fees paid by the transactions it includes in the block.
But wouldn’t every node constantly take the lead to collect the rewards? No, because to take the lead, a node must solve a hard problem. This requires the node to consume considerable time and electricity, which ensures that leaders don’t pop up often. The problem is so hard that most nodes in the network don’t even try. Nodes that do try are called miners because they mine new coins, similar to a gold miner digging for gold. We’ll discuss this process further in Chapter 7.
1.2.4. Step 4: Wallets
Bob and Alice are Bitcoin network users, and they both need a computer program to interact with the network. Such a program is called a Bitcoin wallet. Several types of Bitcoin wallets are available for different devices, such as mobile phones and desktop computers, and there are even specialized hardware wallet devices.
Before step 4 of the payment process, the nodes in the network update their local copy of the blockchain. Now, the network needs to notify Alice and Bob that the transaction went through, as Figure 7 shows.
Bob’s wallet is connected to some of the nodes in the Bitcoin network. When a transaction concerning Bob is added to the blockchain, the nodes that Bob’s wallet is connected to will notify Bob’s wallet. The wallet will then display a message to Bob that he received 1 bitcoin. Alice also uses a wallet. Her wallet will be notified of her own transaction.
Besides sending and receiving transactions, Bob’s and Alice’s wallets also manage their private keys for them. As described earlier, a private key is used to create digital signatures, as well as to generate a Bitcoin address. Alice created her digital signature with one of her private keys. When Bob later wants to spend the money he received at his Bitcoin address, which he generated from his private key, he needs to create a transaction and digitally sign it with that private key.
1.3. Problems with money today
Bitcoin wouldn’t be this widespread if it didn’t solve real problems for real people. Bitcoin solves several problems inherent to the traditional financial system. Let’s look at some commonly discussed problem areas.
1.3.1. Segregation
People with bank accounts and access to banking services such as online payments or loans are privileged. According to the World Bank, about 38% of the world’s population doesn’t have a bank account (see Web resource 6). The numbers are slowly improving, but many people are still stuck in a cash-only environment.
Without a bank account and basic banking services, such as online payments, people can’t expand their businesses outside their local communities. A merchant won’t be able to offer goods or services on the internet to increase its customer base. A person living in a rural area might have to travel half a day to pay a utility bill or top up their prepaid mobile phone.
This segregation between banked people and unbanked people is driven by several factors:
-
Banking services are too expensive for some people.
-
To use banking services, you need documentation, such as an ID card, that many people don’t have.
-
Banking services can be denied to people with certain political views or those conducting certain businesses. People might also be denied service due to their ethnicity, nationality, sexual preferences, or skin color.
1.3.2. Privacy issues
When it comes to electronic payments such as credit cards or bank transfers, traditional money poses several privacy problems. States can easily
-
Trace payments
-
Censor payments
-
Freeze funds
-
Seize funds
You might say, “I have nothing to hide, and the government needs these tools to fight crime.” The problem is, you don’t know what your government will look like in five years and how that government defines crime. New laws are just an election away. After the next election, your government could pass a law that allows it to freeze the funds of people with your political view. In some parts of the world, this is already happening.
We’ve seen lots of examples in which these powers are abused to disable someone’s ability to transact. For example, the nonprofit organization WikiLeaks was put under a blockade in 2010 in which all donations through traditional channels were blocked after pressure from the US government on the major payment networks, such as Visa and Mastercard (see Web resource 7). We’ve also seen how Cyprus seized 47.5% of all bank deposits exceeding 100,000 € as part of a financial rescue program in 2013 (Web resource 8).
Note that bank notes and coins usually aren’t affected. As long as there is cash, people can trade freely and privately. In some parts of the world—for example, Sweden—cash is being phased out, which means soon you won’t be able to buy chewing gum without someone recording your transaction.
1.3.3. Inflation
Inflation means the purchasing power of a currency decreases (Figure 8).
Most currencies are subject to inflation, some more than others. For example, the Zimbabwean dollar inflated nearly 1023% from 2007–2008, peaking at 80 billion percent per month during a few months in 2008. That’s an average daily inflation rate of nearly 100%. Prices roughly doubled every day.
Extreme cases of inflation like this are called hyperinflation, and are usually driven by a rapid increase in the money supply. Governments sometimes increase the money supply as a tool to extract value from the population and pay for expenses such as the national debt, warfare, or welfare. If this tool is overused, the risk of hyperinflation is apparent.
A rapid increase in the money supply will most likely lead to a depreciation of a country’s currency. This, in turn, pushes people to exchange their local currency for goods or alternative currencies that better hold value, which further drives down the value of the local currency. This can spiral to extremes, as in Zimbabwe. The result is devastating for people as they see their life savings diminish to virtually nothing. Table 1 shows examples of recent hyperinflations.
| Country | Year | Worst monthly inflation (%) |
|---|---|---|
Zimbabwe |
2007-2008 |
4.19*1016 |
Yugoslavia |
1992-1994 |
313*106 |
Peru |
1990 |
397 |
Ukraine |
1992-1994 |
285 |
Venezuela |
2012- |
120 |
Zimbabwe is one of the most extreme cases of inflation throughout history, but even today, some countries suffer from very high inflation. One is Venezuela, where its currency, the bolívar, experienced an 254% inflation rate during 2016 and suffered from about a 1,088% inflation rate in 2017. A staggering 1,370,000% inflation rate is forecast for 2018.
1.3.4. Borders
Moving value across national borders using national, or fiat, currency is hard, expensive, and sometimes even forbidden. If you want to send 1,000 Swedish crowns (SEK) from Sweden to a person in the Philippines, you can use a service like Western Union for the transfer. At the time I investigated this, 1,000 SEK was worth 5,374 Philippine pesos (PHP) or 109 US dollars. See Table 2.
| Send from | Receive to | Received by recipient | Fees | Fees % |
|---|---|---|---|---|
Bank |
Bank |
5,109 PHP |
265 PHP |
4.9% |
Bank |
Cash |
4,810 PHP |
564 PHP |
10.5% |
Credit card |
Cash |
4,498 PHP |
876 PHP |
16.3% |
If the recipient has a bank account that can receive an international money transfer, you can get away with a 4.9% fee. But a typical remittance recipient will be able to receive only cash, which doubles or triples the cost to 10.5% or 16.3%, depending on how quickly or conveniently they want it.
In contrast with international transfers, moving fiat currency within a nation state’s borders is usually convenient. For example, you can hand over cash directly to the recipient or transfer money using some mobile app made specifically for the currency. As long as you stay within one country and one currency, fiat currencies usually do a good job.
1.4. The Bitcoin approach
Bitcoin offers a fundamentally different model than traditional financial institutions. Let’s explore the major differences one by one.
1.4.1. Decentralized
Instead of a central organization such as the US Federal Reserve controlling the currency, control of Bitcoin is distributed among thousands of computers, or nodes. No single node or group of nodes has more privileges or obligations than any other. This equality between nodes makes Bitcoin decentralized, as opposed to centralized systems, such as banks or the Google search engine (Figure 9).
In a centralized system, the service is controlled by a single entity, such as a bank. This single entity can decide who gets to use the service and what the user is allowed to do. For example, an online video service can choose to provide a video only to people in a certain geographical location.
With a decentralized system such as Bitcoin, which has several thousands of nodes spread around the globe, it’s extremely hard to control who uses the system and how. No matter where or who they are, or to whom they’re sending money, the Bitcoin system will treat all users equally. The Bitcoin system has no central point that can be exploited to censor payments, deny users service, or seize funds.
As mentioned, Bitcoin is permissionless, which means you don’t need to ask anyone for permission to participate. Anyone with a computer and an internet connection can set up a Bitcoin node and take an active role in the Bitcoin network—no questions asked, no registration required.
Changing the rules of Bitcoin is nearly impossible without broad consensus. If a node doesn’t obey the rules, the rest of the nodes will ignore it. For example, one rule is that Bitcoin’s money supply is limited to 21 million bitcoins. This limit is nearly impossible to change because of decentralization; there’s no one you can threaten or bribe to change these rules.
1.4.2. Limited supply
Because Bitcoin’s money supply won’t exceed 21 million bitcoins, people can be sure that if they own 1 bitcoin, they will always own at least one 21-millionth of the total supply of bitcoins. This feature isn’t found in any fiat currency, where decisions on supply are made every so often by a company or state. Bitcoin is resistant to high inflation because you can’t increase the money supply at will.
Bitcoin’s money supply isn’t fixed today. It’s increasing, at a diminishing rate, according to a predetermined schedule and will eventually stop increasing around the year 2140. See Figure 10.
As of this writing, the money supply is about 17 million bitcoins, and the current yearly increase is at roughly 4%. This increase is halved every four years.
1.4.3. Borderless
Because Bitcoin is a system run by ordinary computers connected to the internet, it’s as global as the internet. This means anyone with an internet connection can send money to other people across the world, as Figure 11 illustrates.
There is no difference between sending a bitcoin to someone in the same room or sending it to someone on another continent. The experience is the same: money is sent directly to the recipient, who sees the payment nearly instantaneously. Within about 60 minutes, this recipient can be sure the money is theirs. Once settled, the transfer can’t be reversed without the recipient’s consent.
1.5. How is Bitcoin used?
So far, we’ve touched on a few common use cases for Bitcoin. This section will dig deeper into those use cases and a few others. It’s hard to predict what use cases we’ll see in the future, so let’s stick to what we know now.
1.5.1. Savings
One interesting Bitcoin feature is that you keep your money safe by storing a set of private keys: the secret pieces of information you’ll need when you want to spend your money. You choose how those private keys are stored. You can write them on paper, or you can store them electronically with a mobile app to have easy access to them. You can also memorize your private keys.  Keep them safe.
Savings is an attractive use case for Bitcoin. A simple way to save is to create a private key and write it down on a piece of paper that you store in a safe. This piece of paper is now your savings account, your savings wallet. You can then send bitcoins to your wallet. As long as your private key is kept safe, your money is safe. You can choose from a lot of different saving schemes to find the right balance between security and convenience. For example, you can keep your keys unencrypted in your mobile phone for easy access or store them encrypted on paper in a vault with armed guards.
1.5.2. Cross-border payments
As noted, moving money from one country to another is expensive (say, 15%), especially if you move money to a poor country, and the recipient doesn’t have a bank account. It’s becoming increasingly popular to use Bitcoin to circumvent this expensive and slow legacy system. It’s usually cheaper to exchange Swedish crowns for bitcoins in Sweden and transfer the bitcoins to your friend in the Philippines. Your friend will then exchange the bitcoins locally for Philippine pesos.
Some companies offer services so that you pay Swedish crowns to the company and the company pays out Philippine pesos to your friend (Figure 12). You won’t even know that Bitcoin is used under the hood. Such companies typically charge a few percent for the service, but it will still be cheaper than traditional remittance services.
Of course, if recipients can make good use of Bitcoin where they live, there’s no need for a middleman that takes a cut of the money. You can send bitcoins directly to your friend. This is what Bitcoin is all about. Exchanges and other such service companies are just bridges between the old legacy world and the new Bitcoin world.
1.5.3. Shopping
The most obvious use case for Bitcoin is shopping. Bitcoin’s borderlessness and security make it ideal for online payments for goods and services.
In traditional online payments, you send your debit card details to the merchant and hope the merchant will withdraw as much as you agreed on. You also hope the merchant handles your debit card details with great care. They probably store the details in a database. Think about that: for every debit card purchase you make, your card details will be stored in that merchant’s database. It’s likely that one of the databases will be hacked and your card details stolen. The more merchants store your details, the higher the risk.
With Bitcoin, you don’t have that problem because you don’t send any sensitive information to the merchant, or anyone else.  Speculation
The world is full of people wanting to get rich quick. Bitcoin can be alluring to them because of its price volatility, or tendency to change. Looking at the history of bitcoin’s price, as shown in Figure 13, it’s tempting to try to buy when it’s low and sell when it’s high.
In November 2013, the price climbed from about $100 USD to more than $1100 in a few weeks. This was clearly a so-called bubble, in which people were afraid of missing out on a great rise, so they bought in, driving the price further up, until it eventually started dropping again. The drop to 50% of its peak value was just as quick as its rise. The same pattern repeated in late 2017 but at a greater magnitude. This has happened many times already. Fluctuations like this are rarely driven by any specific news or technological advancement, but usually arise from speculation. Speculation can be fun, if you can afford to lose, but it’s more like a lottery than something to make a living from.
Sometimes a government or big corporation makes a negative statement about Bitcoin that creates fear in the market, but those events tend to have a limited effect on bitcoin’s value.
Bitcoin’s price volatility seems contradictory to the claims of it having a non-inflationary property; a 50% drop in market value appears pretty inflationary. Bitcoin is still relatively new, and lots of short-term speculation causes this volatility. But as Bitcoin grows and more people and institutions start using it to store their wealth, it will probably stabilize in the long run, and its deflationary property will emerge over time.
1.5.5. Noncurrency uses
Bitcoin is digital cash, but this form of cash can be used for things beyond money. This section covers two common uses, but there are others, including those not yet invented.
Ownership
Bitcoin lets you embed small pieces of data with payments. This data can be, for example, a chassis number of a car. When the car leaves the factory, the manufacturer can make a small Bitcoin payment to the new car owner, containing the chassis number. This payment will then represent the transfer of ownership for that car.
Bitcoin payments are public records, but they aren’t tied to people in any way. They’re tied to long strings of numbers called public keys, explained in detail in Chapter 2. The car manufacturer has made its public key available on its website, in newspapers, and in advertisements to tie the key to the manufacturer’s identity. Anyone can then verify that the manufacturer has transferred ownership of the car to the new owner. The new owner can show that she owns the car by proving that she owns the private key belonging to the public key to which the manufacturer has transferred ownership.
The new owner can sell the car to someone else and transfer ownership by sending the same bitcoins she got from the manufacturer to the new owner’s public key. The general public can follow the car’s ownership from the manufacturer through every owner’s public key up to the current owner.
Proof of existence
Using the same technique to store data in a Bitcoin payment to transfer ownership of a car, you can prove that a document existed prior to a certain point in time.
A digital document has a fingerprint: a cryptographic hash that anyone can calculate from that document. Creating a different document with the same fingerprint is practically impossible. This fingerprint can be attached to a Bitcoin payment. Where the money goes is irrelevant; the important thing is that the fingerprint is recorded in the Bitcoin blockchain. You “anchor” the document in the blockchain.
Bitcoin payments are public records, so anyone can verify that the document existed before the time of the payment by taking the document’s fingerprint and comparing it to the fingerprint stored in the blockchain.
1.5.6. How is Bitcoin valued?
As you read in the Section 1.5.4, a bitcoin’s price can fluctuate dramatically. But where does this price come from? Several Bitcoin exchanges exist, most of them internet-based. They resemble stock markets, where users wanting to sell bitcoins are matched with users wanting to buy bitcoins.
Different markets can have different market prices depending on the supply and demand in that market. For example, in countries such as Venezuela, where the government tries to hinder the Bitcoin market, the supply is low. But the demand is high because people want to escape their hyperinflating currency. These factors drive the Bitcoin price up in that market compared to, for example, the US and European markets, where people can trade more freely.
1.5.7. When not to use Bitcoin
Bitcoin is nice and all, but it’s not suitable for all financial activity. At least, not yet.
Tiny payments
A Bitcoin transaction should usually include a processing fee. This fee isn’t related to the amount sent but to how big the transaction is in bytes. This is because the Bitcoin network’s cost for processing a transaction depends mostly on how big (in bytes) the transaction is. High-value transactions aren’t bigger (in bytes) than low-value transactions, so the fee is about the same for both kinds of transactions. The fee required for a transaction also depends on supply and demand for available space in the blockchain. The blockchain can’t handle more than roughly 12 MB of transactions per hour, which means miners sometimes have to prioritize transactions. Paying a higher fee will probably give your transaction a higher priority.
If the fee is a significant share of the actual payment you want to make, it isn’t economically viable to pay with ordinary Bitcoin transactions (see Table 3).
| Amount to transfer | Fee | Fee % | Feasible |
|---|---|---|---|
2 BTC |
0.003 BTC |
0.15% |
Yes |
0.002 BTC |
0.001 BTC |
50% |
Probably not |
0.001 BTC |
0.005 BTC |
500% |
No |
But promising emerging technologies are being built on top of Bitcoin. One example is the Lightning Network, which allows for cheap, instantaneous micropayments of tiny fractions of a bitcoin. Using the Lightning Network, you could potentially pay just 100 satoshis (where 1 satoshi = 0.00000001 BTC) at a fee of as little as 1 satoshi.
Instant payments
Bitcoin payments take time to confirm. The recipient sees the payment immediately but shouldn’t trust the payment until the Bitcoin network confirms it, which typically happens within 20 minutes. Trusting an unconfirmed transaction can be risky; the sender can double spend the bitcoins by sending the same bitcoins in another transaction to another Bitcoin address—for example, the sender’s.
The confirmation time can add friction in brick-and-mortar shops because customers don’t want to wait 20 minutes before getting their coffee. This might not be a big issue in some online shops, where the shop can wait 20 minutes before sending the goods to the customer; but some online services, such as pay-per-view, could find the confirmation time problematic.
This limitation can also be fixed by systems built on top of Bitcoin—for example, the Lightning Network—especially when the payment amount is small.
Savings you can’t afford to lose
Bitcoin is probably the most secure money there is, but it’s still in its infancy.
-
Your private keys are stolen by some bad guy.
-
The government in your location tries to crack down on Bitcoin users by imprisonment or other means of force.
-
The price of Bitcoin swings down dramatically due to rumors or speculation.
-
Software bugs make Bitcoin insecure.
-
Weaknesses arise in the cryptography Bitcoin uses.
Although all these risks are possible, most of them are unlikely. This list is somewhat ordered with the most likely at the top. Always weigh the risks before putting money on the line, and select your security measures accordingly. This book will help you understand the risks and how to secure your money.
1.6. Other cryptocurrencies
This book will cover Bitcoin, but several other so-called cryptocurrencies exist, and new ones pop up all the time. Cryptocurrencies other than Bitcoin are often referred to as alt-coins, meaning alternative coins. I’ll list a few alt-coins along with their purpose and market capitalization, or market cap (Table 4). The market cap is the product of the money supply (number of coins) and the current market price per coin. Note that the market cap will most likely have changed a lot by the time you read this. I include this information only to give you a glimpse of Bitcoin’s position relative to other cryptocurrencies.
| Currency | Purpose | Market cap (billions of dollars) |
|---|---|---|
Global money; included for reference |
111 |
|
Running software on a decentralized abstract computer |
22.4 |
|
Privacy |
1.7 |
|
Privacy |
0.8 |
|
Naming system; complements the domain name system (DNS) |
0.008 |
I encourage you to look up these cryptocurrencies, because they all provide interesting new features beyond Bitcoin. Hundreds of other alt-coins exist. Some, such as those in the table, provide unique features that aren’t available in Bitcoin, and others provide little to nothing innovative. Some alt-coins may even be outright scams. Stay vigilant.
Anyone can create an alt-coin by taking existing cryptocurrency software and modifying it to their needs.
Let’s say Sheila wants to start an alt-coin, Wowcoin. She takes the Bitcoin software and changes the maximum money supply to 11,000,000, instead of Bitcoin’s 21,000,000 coins. When she starts Wowcoin, Sheila will be lonely because no one else is using her alt-coin. If she wants Wowcoin to have some real value, she must convince other people to begin using it. If she’s not providing anything innovative, she’s going to have a hard time getting other people on board, because they’re pretty happy with what Bitcoin already provides. Everybody else is using Bitcoin, so why would you use Wowcoin? Think of it as starting a new internet that you call Wownet. People on Wownet won’t be able to use services on the internet. Conversely, people on the internet won’t be able to use your service if you’re on Wownet. So why would anyone use Wownet? We call this the network effect (see Figure 14)—people tend to go where other people are.
Although some interesting alt-coins are out there, it’s hard to tell which of these will survive long-term. Also, picking one or a few alt-coins to cover in this book would be an arbitrary choice. Consequently, I focus solely on Bitcoin.
1.7. Summary
-
Bitcoin is global, borderless money that anyone with an internet connection can use.
-
Many different actors use Bitcoin, including savers, merchants, and traders for various purposes, such as payments, remittances, and savings.
-
A network of computers, the Bitcoin network, verifies and keeps records of all payments.
-
A transaction goes through the following steps: send transaction, verify transaction, add transaction to the blockchain, and notify the recipient and sender wallets.
-
Bitcoin solves problems with inflation, borders, segregation, and privacy by providing limited supply, decentralization, and borderlessness.
-
Several alternative cryptocurrencies exist apart from Bitcoin, such as Ethereum, Zcash, and Namecoin.
-
A (crypto)currency becomes more useful as more users use it. This is called the network effect.
2. Cryptographic hash functions and digital signatures
This chapter covers
-
Creating a simple money system: cookie tokens
-
Understanding cryptographic hash functions
-
Authenticating payments using digital signatures
-
Keeping your secrets secret
I’ll start this chapter by setting the stage for the rest of this book. We’ll look at a simple payment system that we can improve on using Bitcoin technologies. By the time we get to Chapter 8, this simple system will have evolved into what we call Bitcoin.
The second part of this chapter will teach you what you need to know about cryptographic hash functions. These are so important to Bitcoin that you really need to understand them before learning anything else. You’ll see how a cryptographic hash function can be used to verify that a file hasn’t changed since a previous point in time.
The rest of the chapter will solve the problem of the imposter: a bad guy claiming to be someone else to pay money from that someone’s account. We solve this problem by introducing digital signatures (Figure 15) into the simple system.
2.1. The cookie token spreadsheet
Suppose there’s a cafe in the office where you work. You and your coworkers use a spreadsheet to keep track of cookie tokens (Figure 16), which use the symbol CT. You can exchange cookie tokens for cookies in the cafe.
Lisa stores this spreadsheet on her computer. It’s shared read-only for everybody on the office network to open and watch, except Lisa. Lisa is very trustworthy. Everybody trusts her.  You and all the others can only view the spreadsheet by opening it in read-only mode.
Whenever Alice wants a cookie, she asks Lisa, who sits right next to the cafe, to transfer 10 CT from Alice to the cafe. Lisa knows who Alice is and can verify in the spreadsheet that she owns enough cookie tokens; she’ll search for “Alice” in the spreadsheet, sum all the amounts with Alice’s name in the To column, and subtract all the amounts with Alice’s name in the From column. Figure 17 shows the complete search result; three transfers involve Alice.
Lisa calculates that Alice has 70 CT, enough for Alice to pay 10 CT to the cafe. She appends a row at the end of the spreadsheet (Figure 18).
The cafe sees this new row in the spreadsheet and hands a cookie over to Alice.
When you run out of cookie tokens, you can buy tokens for dollars from someone who is willing to sell you some—possibly Anne or the cafe—at a price you both agree on. Lisa will then add a row to the spreadsheet accordingly.
Lisa has promised never to remove or change anything in the spreadsheet, just add to it. What happens in the spreadsheet, stays in the spreadsheet!
Lisa, who is performing valuable work to secure this money system, is rewarded with 7,200 newly minted cookie tokens per day (Figure 19). Every day, she adds a new row to the spreadsheet that creates 7,200 new cookie tokens with Lisa as the recipient.
This is how all the cookie tokens in the spreadsheet are created. The first row in the spreadsheet is a reward row—like the one in the spreadsheet just shown—that creates the very first 7,200 CT ever. The plan is that Lisa is rewarded with 7,200 CT per day during the first four years, and then the reward is halved to 3,600 CT/day for the next four years, and so on until the reward is 0 CT/day.
Don’t worry, for now, about what happens when the reward approaches 0—that’s far in the future. We’ll discuss that in Chapter 7. This reward halving makes the total money supply—the total number of cookie tokens in circulation—approach 21 million CT, but it will never exceed 21 million.
What Lisa does with the new cookie tokens she earns is up to her. She can buy cookies or sell the cookie tokens. She can also save them for later. The spreadsheet system works well, and everybody eats a healthy number of cookies.
Lisa basically performs the same work as miners in the Bitcoin network. She verifies payments and updates the ledger, the cookie token spreadsheet. Table 5 clarifies how the concepts in the spreadsheet correspond to concepts in Bitcoin.
| Cookie tokens | Bitcoin | Covered in |
|---|---|---|
1 cookie token |
1 bitcoin |
|
The spreadsheet |
The blockchain |
|
A row in the spreadsheet |
A transaction |
|
Lisa |
A miner |
This table will follow us throughout the book. It describes differences between the cookie token system and Bitcoin. I’ll delete rows from it as I introduce various Bitcoin stuff. For example, the row “The spreadsheet” will be deleted in Chapter 6, when we use a blockchain to store transactions. I’ll also add a few rows as I introduce new concepts for the cookie token system that differ from those in Bitcoin.
At the end of Chapter 8, this table will contain only the first row, mapping 1 cookie token to 1 bitcoin. This will mark the end of this cookie token example, and from that point, we’ll talk only about Bitcoin itself.
Table 6 is your starting point for learning how Bitcoin works, which we can call version 1.0 of the cookie token spreadsheet system.
| Version | Feature | How |
|---|---|---|
1.0 |
Simple payment system |
Relies on Lisa being trustworthy and knowing everyone’s face |
Finite money supply |
7,200 new CT rewarded to Lisa daily; halves every four years |
We’ll add a lot of fancy stuff to this system and release a new version in every chapter. For example, at the end of this chapter, we’ll release version 2.0, which uses digital signatures to solve the problem of imposters. Every chapter will take us closer to the end result: Bitcoin. But please be aware that this isn’t at all how Bitcoin evolved in reality—I’m just using this made-up system to help explain each important topic in isolation.
2.2. Cryptographic hashes
Cryptographic hashes are used everywhere in Bitcoin. Trying to learn Bitcoin without knowing what cryptographic hashes are is like trying to learn chemistry without knowing what an atom is.
You can think of a cryptographic hash as a fingerprint. A person will produce the same fingerprint of her left thumb every time it’s taken, but it’s extremely hard to find another person with the same left thumb fingerprint. The fingerprint doesn’t disclose any information about the person other than that particular fingerprint. You can’t know what math skills or eye color the person has by looking at this fingerprint.
Digital information also has fingerprints. This fingerprint is called a cryptographic hash. To create a cryptographic hash of a file, you send the file into a computer program called a cryptographic hash function. Suppose you want to create a cryptographic hash—a fingerprint—of your favorite cat picture. Figure 20 illustrates this process.
The output—the hash—is a 256-bit number; 256 bits equals 32 bytes because 1 byte consists of 8 bits. Thus, to store the number in a file, the file will be 32 bytes big, which is tiny compared to the size of the 1.21 MB cat picture. The particular cryptographic hash function used in this example is called SHA256 (Secure Hash Algorithm with 256-bit output) and is the most commonly used one in Bitcoin.
The word hash means something that’s chopped into small pieces or mixed up. That’s a good description of what a cryptographic hash function does. It takes the cat picture and performs a mathematical calculation on it. Out comes a big number—the cryptographic hash—that doesn’t look remotely like a cat. You can’t “reconstruct” the cat picture from just the hash—a cryptographic hash function is a one-way function. Figure 21 shows what happens when you change the cat picture a little and run it through the same cryptographic hash function.
This hash turns out completely different than the first hash. Let’s compare them:
Old hash: dee6a5d375827436ee4b47a930160457901dce84ff0fac58bf79ab0edb479561 New hash: d2ca4f53c825730186db9ea585075f96cd6df1bfd4fb7c687a23b912b2b39bf6
See how that tiny change to the cat picture made a huge difference in the hash value? The hash value is completely different, but the length of the hash is always the same regardless of input. The input “Hello” will also result in a 256-bit hash value.
2.2.1. Why are cryptographic hash functions useful?
Cryptographic hash functions can be used as an integrity check to detect changes in data.  This could happen, for example, due to disk errors or hackers. How can you make sure you detect corruption?
First, you calculate a cryptographic hash of the cat picture on your hard drive and write it down on a piece of paper (Figure 22).
Later, when you want to look at the picture, you can check if it’s changed since you wrote the hash on that paper. Calculate the cryptographic hash of the cat picture again, and compare it to the original hash on your paper (Figure 23).
If the new hash matches the one on paper, you can be sure the picture hasn’t changed. On the other hand, if the hashes don’t match, the cat picture has definitely changed.
Bitcoin uses cryptographic hash functions a lot to verify that data hasn’t changed. For example, every now and then—on average, every 10 minutes—a new hash of the entire payment history is created. If someone tries to change the data, anyone verifying the hash of the modified data will notice.
2.2.2. How does a cryptographic hash function work?
The real answer is complex, so I won’t go into exact detail. But to help you understand the operation of a cryptographic hash function, we’ll create a very simplistic one. Well, it isn’t really cryptographic, as I’ll explain later. Let’s just call it a hash function for now.
Suppose you want to hash a file containing the six bytes a1 02 12 6b
c6 7d
. You want the hash to be a 1-byte number (8 bits). You can
construct a hash function using addition modulo 256, which means
to wrap around to 0 when the result of an addition reaches 256
(Figure 24).
The result is the decimal number 99. What does 99 say about the original
input a1 02 12 6b c6 7d
? Not much—99 looks just as random as any other
single-byte number.
If you change the input, the hash will change, although a chance exists that the hash will remain 99. After all, this simple hash function has just 256 different possible outputs. With real cryptographic hash functions, like the one we used to hash the cat picture, this chance is unimaginably small. You’ll soon get a glimpse of this probability.
2.2.3. Properties of a cryptographic hash function
A cryptographic hash function takes any digital input data, called a pre-image, and produces a fixed-length output, called a hash. In the example with the cat picture on your hard drive, the pre-image is the cat picture of 1.21 MB, and the hash is a 256-bit number. The function will output the exact same hash each time the same pre-image is used. But it will, with extremely high probability, output a totally different hash when even the slightest variation of that pre-image is used. The hash is also commonly referred to as a digest.
Let’s look at what properties you can expect from a cryptographic hash function. I’ll illustrate using SHA256 because it’s the one Bitcoin uses most. Several cryptographic hash functions are available, but they all provide the same basic properties:
-
The same input will always produce the same hash.
-
Slightly different inputs will produce very different hashes.
-
The hash is always of the same fixed size. For SHA256, it’s 256 bits.
-
Brute-force trial and error is the only known way to find an input that gives a certain hash.
Figure 25 illustrates the first three properties. The fourth property of a cryptographic hash function is what makes it a cryptographic hash function, and this needs a bit more elaboration. There are some variations to the fourth property, all of which are desirable for cryptographic hash functions (Figure 26):
- Collision resistance
-
You have only the cryptographic hash function at hand. It’s hard to find two different inputs that result in the same hash.
- Pre-image resistance
-
You have the hash function and a hash. It’s hard to find a pre-image of that hash.
- Second-pre-image resistance
-
You have the hash function and a pre-image (and thus the hash of that pre-image). It’s hard to find another pre-image with the same hash.
2.2.4. Illustration of “hard”
The term hard in this context means astronomically hard. It’s silly to even try. We’ll look at second-pre-image resistance as an example of what hard means, but a similar example can be made for any of the three variants.
Suppose you want to find an input to SHA256 that results in the same hash as “Hello!”:
334d016f755cd6dc58c53a86e183882f8ec14f52fb05345887c8a5edd42c87b7
You can’t change the input “Hello!” just a little so the function “won’t
notice.” It will notice and will output a totally different hash. The
only way to find an input other than “Hello!” that gives the hash
334d016f…d42c87b7
is to try different inputs one by one and check
whether one produces the desired hash.
Let’s try, using Table 7.
| Input | Hash | Success? |
|---|---|---|
Hello1! |
82642dd9...2e366e64 |
|
Hello2! |
493cb8b9...83ba14f8 |
|
Hello3! |
90488e86...64530bae |
|
... |
... |
|
Hello9998! |
cf0bc6de...e6b0caa4 |
|
Hello9999! |
df82680f...ef9bc235 |
|
Hello10000! |
466a7662...ce77859c |
|
|
dee6a5d3...db479561 |
|
My entire music collection |
a5bcb2d9...9c143f7a |
|
As you can see, we aren’t very successful. Think about how much time it would take for a typical desktop computer to find such an input. It can calculate about 60 million hashes per second, and the expected number of tries needed to find a solution is 2255. The result is 2255 / (60 × 106) s ≈ 1068 s ≈ 3 × 1061 years, or about 30,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000 years.
I think we can stop trying, don’t you? I don’t think buying a faster computer will help, either. Even if we had 1 trillion computers and ran them concurrently, it would take about 3 × 1049 years.
Pre-image resistance, second-pre-image resistance, and collision resistance are extremely important in Bitcoin. Most of its security relies on these properties.
2.2.5. Some well-known hash functions
Table 8 shows several different cryptographic hash functions. Some aren’t considered cryptographically secure.
| Name | Bits | Secure so far? | Used in Bitcoin? |
|---|---|---|---|
SHA256 |
256 |
Yes |
Yes |
SHA512 |
512 |
Yes |
Yes, in some wallets |
RIPEMD160 |
160 |
Yes |
Yes |
SHA-1 |
160 |
No. A collision has been found. |
No |
MD5 |
128 |
No. Collisions can be trivially created. The algorithm is also vulnerable to pre-image attacks, but not trivially. |
No |
Generally, when a single collision has been found in a cryptographic hash function, most cryptographers will consider the function insecure.
2.2.6. Recap of cryptographic hashes
A cryptographic hash function is a computer program that takes any data as input and computes a big number—a cryptographic hash—based on that input.
It’s astronomically hard to find an input that will result in a specific output. This is why we call it a one-way function. You have to repeatedly guess different inputs.
We’ll discuss important topics throughout this book. When you’ve learned about a specific topic, like cryptographic hash functions, you can put a new tool into your toolbox for later use.
From now on, we’ll use these tool icons to represent cryptographic hash functions and cryptographic hashes, with some exceptions.
2.2.7. Exercises
Warm up
-
How many bits is the output of SHA256?
-
How many bytes is the output of SHA256?
-
What’s needed to calculate the cryptographic hash of the text “hash me”?
-
What are the decimal and the binary representations of the hexadecimal data
061a
? -
Can you, in practice, modify the text “cat” so the modified text gets the same cryptographic hash as “cat”?
Dig in
-
The simplistic hash function from Section 2.2.2, repeated for you as follows, isn’t a cryptographic hash function. Which two of the four properties of a cryptographic hash function is it lacking?
The four properties are also repeated as follows:
-
The same input will always produce the same hash.
-
Slightly different inputs will produce very different hashes.
-
The hash is always of the same fixed size. For SHA256, it’s 256 bits.
-
Brute-force trial and error is the only known way to find an input that gives a certain hash.
 Suppose someone wanted to change the cat picture on your hard drive without you noticing. What variant of the fourth property is important for stopping the attacker from succeeding?
2.3. Digital signatures
In this section, we explore how you can prove to someone that you approve a payment. To do that, we use digital signatures. A digital signature is a digital equivalent of a handwritten signature. The difference is that a handwritten signature is tied to a person, whereas a digital signature is tied to a random number called a private key.  Typical use of digital signatures
Suppose you want to send your favorite cat picture to your friend Fred via email, but you suspect the picture might be, maliciously or accidentally, corrupted during transfer. How would you and Fred make sure the picture Fred receives is exactly the same as the one you send?
You can include a digital signature of the cat picture in the email. Fred can then verify this digital signature to make sure the cat picture is authentic. You do this in three different phases, as Figure 27 shows.
Step 1 is preparation. You create a huge random number: the private key. You can use this to create digital signatures. You then create the public key, which is used to verify the signatures the private key creates. The public key is calculated from the private key. You hand the public key to Fred in person so Fred is sure it belongs to you.
Step 2 is signing. You write an email to Fred and attach the cat picture. You also use your private key and the cat picture to digitally sign the cat picture. The result is a digital signature that you include in your email message. You then send the email to Fred.
Step 3 is verifying. Fred receives your email, but he’s concerned the cat picture might be corrupt, so he wants to verify the signature. He uses the public key he got from you in step 1, the digital signature in the email, and the attached cat picture. If the signature or the cat picture has changed since you created the signature, the verification will fail.
2.3.2. Improving cookie token security
It’s time to return to our cookie token spreadsheet. The company is growing, and Lisa has a hard time recognizing everyone. She notices that some people aren’t honest. For example, Mallory says she is Anne, to trick Lisa into moving cookie tokens from Anne to the cafe, instead of from Mallory to the cafe. Lisa thinks of requiring everybody to digitally sign their cookie token transfers by writing a message and a digital signature in an email, as Figure 28 shows.
Suppose John is the new guy at the office. The company gave him some cookie tokens as a welcome gift when he started. Now, John wants to buy a cookie in the cafe for 10 CT. He needs to digitally sign a cookie token transfer. Figure 29 shows what he has to do.
Just as with the email to Fred in the previous section, there are three phases in this process (please compare with the steps in Figure 27 to see the similarities):
-
John prepares by generating a key pair. John keeps the private key secret and hands the public key over to Lisa. This is a one-time setup step.
-
John wants a cookie. He writes a message and signs it with his private key. He sends the message and the digital signature in an email to Lisa.
-
Lisa verifies the signature of the message using John’s public key and updates the spreadsheet.
2.3.3. Preparation: John generates a key pair
The signing and verification processes are based on a key pair. John needs a private key to sign payments, and Lisa will need John’s public key to verify John’s signatures. John needs to prepare for this by creating a key pair. He does this by first generating a private key and then calculating the public key from that private key, as Figure 30 shows.
John will use a random number generator to generate a huge, 256-bit random number. A random number generator is available on almost all operating systems. The random number is now John’s private key. The private key is then transformed into a public key using a public-key derivation function.
Public-key derivation is a one-way function, just like cryptographic hash functions; you can’t derive the private key from the public key. The security of digital signatures relies heavily on this feature. Also, running the private key through the public-key derivation function multiple times will always result in the same public key.
The public key is 33 bytes (66 hex digits) long. This is longer than the private key, which is 32 bytes (64 hex digits) long. The reason for the “extra” byte and how the public-key derivation function works is a hard topic, covered in Chapter 4. Luckily, you don’t have to be a cryptography expert to understand how signatures work from a user’s perspective.
Two ways to use the key pair
Keys are used to encrypt and decrypt data. Encryption is used to make messages unreadable to everybody but those who hold the proper decryption key.
We can think of the private and public keys as a pair because they have a strong relationship: the public key can be used to encrypt messages that only the private key can decrypt, and the private key can encrypt messages that only the public key can decrypt (Figure 31).
Following the left side of Figure 31, only John would be able to read the encrypted message because he’s the only one with access to his private key. Bitcoin doesn’t use this feature of public and private keys at all. It’s used when two parties want to communicate in private, as when you do your online banking. When you see the little padlock in the address bar of your web browser, then you know the process shown on the left side of the figure is being used to secure your communication.
Following the right side of the figure, Lisa can decrypt the message because she has the public key belonging to John’s private key. This feature is used for digital signatures. Using the private key to encrypt secret messages isn’t a good idea because the public key is, well, public. Anyone with the public key can decrypt the message. Digital signatures, on the other hand, don’t need any secret messages. We’ll explore digital signatures deeper soon. But first, some recap and orientation.
2.3.4. Recap of key pairs
Let’s summarize what you’ve learned about public and private keys. You create a key pair by first creating a private key. The private key is a huge, secret random number. The public key is then calculated from the private key.
You can use the private key to encrypt a message that can be decrypted only using the public key.
The encryption and decryption in this figure are the foundation for digital signatures. This process is not suitable for sending secret messages because the public key is usually widely known.
The reverse process is also common, in which the public key is used to encrypt and the private key is used to decrypt. This process is used to send secret messages. Bitcoin doesn’t use it.
2.3.5. Where were we?
Digital signatures were briefly mentioned in Chapter 1, where Alice signed her Bitcoin transaction of 1 BTC to Bob using her private key (Figure 32).
John has created a pair of keys and is about to digitally sign his payment to the cafe with his private key so Lisa can verify that it’s actually John making the payment. Lisa verifies this using John’s public key.
2.3.6. John signs his payment
Let’s have a close look at how the signing really happens (Figure 33).
The message John wants to sign is, “Lisa, please move 10CT to Cafe. /John”. The signing function will hash this message with SHA256, whose output is a 256-bit number. This hash value is then encrypted with John’s private key. The result is a digital signature that looks like this:
INxAs7oFDr80ywy4bt5uYPIv/09fJMW+04U3sJUfgV39 A2k8BKzoFRHBXm8AJeQwnroNb7qagg9QMj7Vp2wcl+c=
The signature is an encrypted hash of a message. If John had used another private key to sign with or a slightly different message, the signature would have looked completely different.
For example, using the input message “Lisa, please move 10CT to Mallory. /John” would generate this signature:
ILDtL+AVMmOrcrvCRwnsJUJUtzedNkSoLb7OLRoH2iaD G1f2WX1dAOTYkszR1z0TfTVIVwdAlD0W7B2hBTAzFkk=
This isn’t remotely similar to the previous signature. This is good for John, because he knows his signature can’t be used for messages other than his specific message.
John has now composed an email to Lisa. This email contains a message and a signature of that message. John finishes by sending the email to Lisa.
2.3.7. Lisa verifies the signature
Lisa looks at the email and sees it claims to be from John, so she looks up John in her table of public keys (Figure 34).
Lisa’s actions in this figure aim to determine that the cookie token transfer was signed by the private key it claims to be signed with. The message says it’s from John. She received John’s public key the other day and put that public key in her table of public keys. The things she has at hand are
-
The message “Lisa, please move 10CT to Cafe. /John”
-
The signature
INxAs7oFDr8…
-
John’s public key that she just looked up in her table
John encrypted the message’s hash with his private key. This encrypted hash is the signature. If Lisa decrypts the signature (B) with John’s public key (C), the result should be a hash that equals the hash of the message (A) in the email.
Lisa takes the signature (B) and decrypts it with the public key (C) she looked up in her table of public keys. The decryption outputs a big number. If this number is equal to the hash of the message (A), it proves John’s private key was used to sign the message. Lisa takes the message (A), exactly as written, and hashes that message just like John did when he created the signature. This message hash is then compared with the decrypted signature. The message hash and the decrypted signature match, which means the signature is valid.
Note that this process works only if John and Lisa use the exact same digital signature scheme. This must be agreed on beforehand, but it’s usually standardized. In Bitcoin, everyone knows exactly what digital signature scheme to use.
Lisa can now be sure no one is trying to fool her. She updates the spreadsheet with John’s transfer, as shown in Figure 35.
2.3.8. Private key security
John is in control of his cookie tokens because he owns the private key. No one but John can use John’s cookie tokens because he’s the only one with access to his private key. If his private key is stolen, he can lose any and all of his cookie tokens.
The morning after John’s transfer, he comes to the office, takes his laptop from his desk, and goes straight to the cafe to buy two morning cookies. He opens his laptop to write an email to Lisa:
Good morning Lisa! Please move 20 CT to Cafe. /John Signature: H1CdE34cRuJDsHo5VnpvKqllC5JrMJ1jWcUjL2VjPbsj X6pi/up07q/gWxStb1biGU2fjcKpT4DIxlNd2da9x0o=
He sends this email containing the message and a signature to Lisa. But the cafe doesn’t hand him any cookies. The guy behind the desk says he hasn’t seen an incoming payment of 20 CT yet. Lisa usually verifies and executes transfers quickly.
John opens the spreadsheet—he has read-only access, remember—and searches for “John.” Figure 36 shows what he sees.
John steps into Lisa’s office, asking for an explanation. She answers that she got a message signed with John’s private key, asking her to send money to a new coworker, Melissa. Lisa even shows him the message and signature. Of course, there is no Melissa at the office, even though several new employees have started at the company. Lisa doesn’t care about names anymore, only public keys and signatures. But she needs the name to look up the correct public key in the table.
The explanation to all this is that Mallory has
-
Managed to copy John’s private key. John’s laptop has been on his desk all night long. Anyone could have taken the hard drive out of the laptop to search for his private key.
-
Created a new key pair and sent the new public key to Lisa, with the following message:
Hi Lisa. My name is Melissa, and I’m new here. My public key is 02c5d2dd24ad71f89bfd99b9c2132f796fa746596a06f5a33c53c9d762e37d9008
-
Sent a fraudulent message, signed with the stolen private key, to Lisa as follows:
Hi Lisa, please move 90 CT to Melissa. Thanks, John Signature: IPSq8z0IyCVZNZNMIgrOz5CNRRtRO+A8Tc3j9og4pWbA H/zT22dQEhSaFSwOXNp0lOyE34d1+4e30R86qzEbJIw=
Lisa verified the transfer in step 3, concluded it was valid, and executed the transfer. John asks Lisa to revert the—according to him—fraudulent transfer. But Lisa refuses to do so. She thinks the transfer is perfectly valid. If John let someone see his private key, that’s his problem, not Lisa’s. That’s part of why she’s so trusted in the company—she keeps her promises.
John creates a new key pair and asks Lisa to add his new public key under the name John2. How can John secure his new private key and still have it readily available when he wants a cookie? John is pretty sure he won’t have more than 1,000 cookie tokens on that key.
The security of the spreadsheet has shifted from a system in which Lisa knows everyone’s face to one in which she knows everyone’s public key. In a sense, the security could be worse now, because it might be easier for Mallory to steal John’s private key than it is for her to trick Lisa into thinking Mallory is John. This depends on how John protects his private key. An important thing to note is that the security of John’s private key is totally up to him. No one will be able to restore John’s private key if he loses it. And Lisa sure isn’t going to reverse “fraudulent” transfers just because John is sloppy with security.
If John stores his private key in cleartext in a shared folder on the company’s intranet, anyone can easily copy it and use it to steal his cookie tokens. But if John stores the private key in an encrypted file, protected by a strong password, on his own laptop’s hard drive, getting a copy of his key is a lot harder. An attacker would have to
-
Get access to John’s hard drive
-
Know John’s password
If John never has more than 50 CT on his private key, he might not be that concerned with security. But the cafe, which manages about 10,000 CT daily, might be concerned. John and the cafe probably need different strategies for storing their private keys.
A trade-off exists between security and convenience. You can, for
example, keep your private key encrypted on an offline laptop in a bank
safe-deposit box. When you want to buy a cookie, you’ll need to go to
the bank, take the laptop out of your safe-deposit box, decrypt the
private key with your password, and use it to digitally sign a message
to Lisa that you save to a USB stick. Then, you’ll have to put the
laptop back into the safe-deposit box, bring the USB stick back to the
office, and send the email to Lisa. The private key never leaves
the laptop in the safe-deposit box. Very secure, and very inconvenient.
On the other hand, you can store your private key in cleartext on your mobile phone. You’ll have the key at your fingertips and can sign a message within seconds of when the urge for a cookie starts to nudge you. Very insecure, and very convenient.
Some of the different trade-offs, as illustrated in Figure 37, are as follows:
- Online vs. offline
-
Online means the private key is stored on a device with network access, like your mobile phone or general-purpose laptop. Offline means the private key is stored on a piece of paper or a computer without any network access. Online storage is risky because remote security exploits or malicious software on your computer, such as computer viruses, might send the private key to someone without you noticing. If the device is offline, no one can take the private key without physical access to the device.
- Cleartext vs. encrypted
-
If the private key is stored in cleartext in a file on your computer’s hard drive, anyone with access to your computer, either remotely over a computer network or physically, can copy the private key. This includes any viruses your computer might be victim to. You can avoid many of these attacks by encrypting your private key with a password that only you know. An attacker would then need access to both your hard drive and your secret password to get the private key.
- Whole key vs. split key
-
People usually store their entire private key on a single computer. This is convenient—you need only one computer to spend your cookie tokens. An attacker must get access to your hard drive to steal the private key. But if your private key is split into three parts (there are good and bad schemes for this—be careful), and you store the three parts separately on three different computers, then the attacker must get access to the hard drives of three computers. This is much harder because they must know what three computers to attack and also successfully attack them. Making a payment in this setup is a real hassle, but very secure.
You can use any combination of these methods to store your keys. But as a rule of thumb, the greater the security against attackers, the greater the risk of you accidentally losing access to your key. For example, if you store the private key encrypted on your hard drive, you risk losing your key due to both computer failure and forgetting your password. In this sense, the more securely you store your key, the less secure it is.
2.4. Recap
Lisa has solved the problem with people claiming to be someone else when they make a payment. She requires all payers to digitally sign the cookie token transfers. Every spreadsheet user needs a private key and a public key. Lisa keeps track of who owns which public key. From now on, a payment must be written in an email to Lisa, and the message must be digitally signed with the person’s private key. Lisa can then verify the signature to make sure she isn’t being fooled. The gist is that as long as John keeps his private key to himself, no one will be able to spend his money.
We need to add “Email to Lisa” to our concept table (Table 9).
| Cookie Tokens | Bitcoin | Covered in |
|---|---|---|
1 cookie token |
1 bitcoin |
|
The spreadsheet |
The blockchain |
|
Email to Lisa |
A transaction |
|
A row in the spreadsheet |
A transaction |
|
Lisa |
A miner |
The email to Lisa will be replaced by transactions in Chapter 5. Transactions will replace both the email to Lisa and the row in the spreadsheet. It’s time to release version 2.0 of the cookie token spreadsheet system (Table 10).
| Version | Feature | How |
|---|---|---|
2.0 |
Secure payments |
Digital signatures solve the problem with imposters. |
1.0 |
Simple payment system |
Relies on Lisa being trustworthy and knowing everyone’s face |
Finite money supply |
7,200 new CT rewarded to Lisa daily; halves every four years |
Everybody still trusts Lisa to not change the spreadsheet in any way except when executing signed cookie token transfers. If Lisa wanted to, she could steal anyone’s cookie tokens just by adding a transfer to the spreadsheet. But she wouldn’t do that—or would she?
You now have a lot of new tools to put in your toolbox for later use: key-pair generation, digital signing, the signature, and the verification.
2.5. Exercises
2.5.1. Warm up
-
Lisa is currently rewarded 7,200 CT per day for her work. Why won’t the supply increase infinitely over time? Why don’t we have 7,200 × 10,000 = 72 million CT after 10,000 days?
-
How can coworkers detect if Lisa rewards herself too much or too often?
-
How is the private key of a key pair created?
-
What key is used to digitally sign a message?
-
The signing process hashes the message to sign. Why?
-
What would Mallory need to steal cookie tokens from John?
2.5.2. Dig in
-
Suppose you have a private key and you’ve given your public key to a friend, Fred. Suggest how Fred can send you a secret message that only you can understand.
-
Suppose you (let’s pretend your name is Laura) and Fred still have the keys from the previous exercise. Now you want to send a message in a bottle to Fred saying,
Hi Fred! Can we meet at Tiffany’s at sunset tomorrow? /Laura
Explain how you would sign the message so Fred can be sure the message is actually from you. Explain what steps you and Fred take in the process.
2.6. Summary
-
Bitcoins are created as rewards to nodes securing the blockchain.
-
The reward halves every four years to limit the money supply.
-
You can use cryptographic hash functions to detect changes in a file or in a message.
-
You can’t make up a pre-image of a cryptographic hash. A pre-image is an input that has a certain known output.
-
Digital signatures are useful to prove a payment’s authenticity. Only the rightful owner of bitcoins may spend them.
-
Someone verifying a digital signature doesn’t have to know who made the signature. They just have to know the signature was made with the private key the signature claims to be signed with.
-
To receive bitcoins or cookie tokens, you need a public key. First, you create a private key for yourself in private. You then derive your public key from your private key.
-
Several strategies are available for storing private keys, ranging from unencrypted on your mobile phone to split and encrypted across several offline devices.
-
As a general rule of thumb, the more secure the private key is against theft, the easier it is to accidentally lose the key, and vice versa.
3. Addresses
This chapter covers
-
Basic privacy
-
Replacing names with public key hashes
-
Protecting against expensive typing errors
By the time you reach the end of this chapter, the cookie token spreadsheet will no longer have personal names—you’ll be replacing these names with hashes of public keys. This is useful from a privacy perspective. No one can easily see who’s paying whom, making it harder for others to extract information from the spreadsheet and see how many cookies any of your coworkers eat. Lisa also finds this useful because she doesn’t have to maintain a table of names and public keys.
When switching to public key hashes in the spreadsheet, coworkers will no longer use names in their emails to Lisa. They will instead use strings of hex code representing public key hashes. This means it will be easy to make typing errors. If you make a typing error, your money may end up digitally burned!
Some coworkers invent cookie token addresses (Bitcoin addresses) that protect them from losing money due to typing errors (Figure 38). Cookie token addresses are used between users to pay each other, pretty much like an email address, but they aren’t used in the spreadsheet.
3.1. Cookie-eating habits disclosed
You and many of your coworkers have health insurance with Acme Insurances. Acme has persuaded John to give it a copy of the spreadsheet. Acme figures it can adjust premiums or hold workers’ cookie-eating habits (Figure 39) against them in an eventual insurance dispute.
Another disturbing fact about the spreadsheet is that every coworker can easily look up other coworkers’ balances, as well as their cookie-eating habits.
The coworkers have asked Lisa to come up with a solution to these problems. Otherwise, they’ll stop using the spreadsheet.
3.2. Replacing names with public keys
Lisa has kept the table of names and public keys updated at all times since the coworkers started using digital signatures. She’s sick of doing this, so she comes up with an idea that will benefit both her and her coworkers: Lisa will replace all names in the spreadsheet with their respective public keys (Figure 40).
It’s now hard to see how many cookies Chloe has eaten without knowing
her public key. If Acme Insurances receives this new spreadsheet, it
won’t be able to see who the senders and recipients
are. It will see only the sender and recipient public keys of each
payment.
Lisa can now delete her cumbersome table of names and public keys. But when she does this, users should no longer use names when making payments. They must instead use the sender’s public key and the recipient’s public key (Figure 41).
The email to Lisa contains a few vital parts:
-
A message containing the
-
Amount
-
Sender public key
-
Recipient public key
-
-
A signature made with the sender’s private key
The essential difference is that the payment is now pseudonymous: names are replaced with the corresponding public keys. Otherwise, the payment looks the same as before.
3.2.1. New payment process
Suppose a new coworker just started at the company. Her name is Faiza. The company wants to send her 100 CT as a welcome gift. How can the company send 100 CT to Faiza?
First, the company needs the recipient’s—Faiza’s—public key. Faiza hasn’t used cookie tokens yet, so she needs to create a key pair and give the public key to the sender—the company—as Figure 42 shows.
Faiza creates a private and a public key, following the same process described in Section 2.3.2, but she doesn’t give her public key to Lisa. Now that Lisa doesn’t have the table of names and public keys, there’s no longer any point in giving her the public key. She doesn’t need it. Instead, Faiza gives the public key to the entity that wants to pay her cookie tokens—the company.
The company creates a message asking Lisa to move 100 CT from
037e944a…36de9496
to 029a726c…ad8f436d
. It then digitally signs
the message and sends it to Lisa. Lisa uses
-
The message
-
The sender’s public key
-
The signature
to verify that the message is signed with the private key belonging to the sender’s public key. She also verifies that the sender’s public key has enough funds in the spreadsheet. She does this the same way she did when the spreadsheet contained names—she searches for the sender’s public key and calculates the balance.
Lisa has never seen the recipient’s public key before, but she doesn’t care. She cares only that the sender has the money to spend and that the message is correctly signed. She’ll write into the spreadsheet’s recipient column whatever the message asks her to write.
Faiza sees the new row with her public key in the To column. It gives her a warm, fuzzy feeling. She can now spend her cookie tokens as she pleases. Faiza didn’t have to bother Lisa with her public key, saving Lisa a lot of work.
Let’s summarize what’s happened so far:
-
Names have been replaced with public keys in the spreadsheet.
-
Lisa has thrown away the table of names and public keys.
-
Payments are made using public keys instead of names to denote sender and recipient.
These changes have improved privacy and simplified Lisa’s work. At the end of this chapter, we’ll discuss more about how to further improve privacy.
The email to Lisa in this example probably reveals, to Lisa, who the sender is (the company, in this case) because of the From field of the email. For now, we can assume Lisa doesn’t reveal or use this personal information in any way. We use email in this example in place of Bitcoin’s peer-to-peer network. The Bitcoin network, discussed in detail in Chapter 8, doesn’t use any personal information.
Please take a moment to think about what Acme Insurances can now figure out from the spreadsheet. What information can it get if it figures out the name of the sender or recipient of one payment? It will be able to identify all payments that person has made.
3.3. Shortening the public key
Using public keys in the spreadsheet improved privacy, but such keys take up a lot of space compared to names. The name “John” takes 4 bytes in the spreadsheet, whereas a public key takes 33 bytes. Keeping the spreadsheet as small as possible is important because a smaller spreadsheet means faster downloading for coworkers wanting to check their balance; it also takes less space on Lisa’s hard drive.
3.3.1. Hashing the public key to 20 bytes
Among the coworkers, some developers think they can replace 33-byte public keys with something shorter while still preserving enough security. They suggest replacing each public key in the cookie token spreadsheet with a cryptographic hash of the public key. This shortens senders and recipients in the spreadsheet but also protects users’ money in the event of a flaw in the public key derivation function, as we’ll see later. The hashing isn’t made with a single cryptographic hash function but with two different cryptographic hash functions, as Figure 43 illustrates. We’ll discuss the reason for using two hash functions in the next section.
The public key is first hashed with SHA256, which you should be familiar with from the previous chapter. The output of this cryptographic hash function is then hashed with RIPEMD160, a cryptographic hash function that outputs a 160-bit (20-byte) number. We call this final hash the public key hash (PKH). All public keys in the spreadsheet are replaced with their respective PKHs.
The payment process now differs from when Faiza received her 100 CT from the company. Suppose John wants to buy a cookie (Figure 44).
First, the message to Lisa is changed a bit. John must use the cafe’s PKH—which was previously a public key—as the recipient. The sender is still a public key in the message because that public key is needed to verify the signature. Lisa doesn’t keep people’s public keys around anymore.
Second, because the spreadsheet now contains PKHs, Lisa must calculate the PKH from the sender’s public key to check the sender’s balance and be able to enter the payment into the spreadsheet.
3.3.2. Why SHA256 and RIPEMD160?
Using RIPEMD160 as the last cryptographic hash function is a deliberate choice to make the PKHs shorter. Compare the output from SHA256 with the output from RIPEMD160:
SHA256: 85ae273f0aa730eddf2285d3f3ab071eb29caba1e428db90e6dfbd71b8e1e918 RIPEMD160: 5f2613791b36f667fdb8e95608b55e3df4c5f9eb
It’s a well-balanced trade-off between security and size.
But why have two different cryptographic hash functions? We don’t really know why this scheme was chosen for Bitcoin because its inventor, Satoshi Nakamoto, has stopped corresponding with the Bitcoin community. We can only speculate. Instead, let’s discuss some of the scheme’s properties.
If either hash function isn’t pre-image-resistant, the other still is. This means if you can calculate an input to RIPEMD160 that gives a certain PKH output, you still need to pre-image attack SHA256 (with about 2255 guesses) to find the public key. Likewise, if you can calculate an input to SHA256 that gives a certain output, you first need to pre-image attack RIPEMD160 before you can use that pre-image to calculate the public key.
On the other hand, if it turns out that the output set of either cryptographic hash function is smaller than anticipated, then the security of the combined hash-function chain suffers. To make this clearer, pretend SHA256 has only 100 possible output values. You can steal money from anyone by trying different random private keys and calculating the corresponding PKHs. If a PKH matches your target, you’ve found a private key you can steal the money with. On average, you’d only have to test 50 different private keys to steal from one PKH. This property gives the worst of both worlds: if either of the two functions is weak, then the whole chain is weak. The probability that any function has this flaw is small. If any such flaw exists, the reduction in the output set likely isn’t significant enough to endanger security in practice. Remember, we’ve yet to find one single collision in any of these cryptographic hash functions.
Another thing to note is that different organizations developed the two cryptographic hash functions. RIPEMD160 was developed at a European university in open collaboration with a broad community of cryptographers. SHA256 was developed by the US National Security Agency (NSA). Both are considered secure, and both have been subject to scrutiny from a large number of people.
Now that we’ve strengthened the security of the cookie token spreadsheet, let’s think about privacy again. Has this improved privacy? Is it harder for Acme Insurances to figure out information about who’s paying whom now compared to when we used public keys in the spreadsheet?
The answer is no. There is practically a one-to-one correspondence between the public keys and the PKHs. Using PKHs doesn’t hide personal information any more than using plain public keys.
3.4. Avoiding expensive typing errors
When Lisa verifies a payment before executing it, she doesn’t care who the recipient is or if it’s even an existing recipient. She’ll put into the recipient column of the spreadsheet whatever the payer asks her to. She can’t even know if a recipient is valid or not, because she no longer knows everyone’s public keys.
This is convenient for Lisa, but it can cause people to lose money if they aren’t careful. Imagine once again that John wants to buy a cookie. This time, he’s not careful enough when writing the message, as Figure 45 shows.
He makes a typing error in the recipient PKH. The last character is d
when it should have been c
. What happens now?
John doesn’t notice the error and happily signs the message and sends
the email to Lisa. Lisa verifies the signature, which verifies fine,
and calculates the sender’s PKH. She doesn’t care about the
recipient. She inserts a new row in the spreadsheet paying from
5f261379…f4c5f9eb
to 87e3d169…8393b1cd
.
Then she considers herself done, moving on to other interesting tasks. The cafe owner, who is searching for his PKH in the spreadsheet, doesn’t see an incoming payment. John stands at the counter in the cafe yelling at the cafe owner that he did send money, so “Give me the freakin’ cookie!” The cafe owner refuses. John takes a close look at the spreadsheet and searches for his PKH. He finds the one he just made and realizes his spelling mistake.
John has sent money to a “public key hash” for which there is no known private key. No one will ever be able to spend those 10 CT—not the cafe, not John, nobody. John has just digitally burned 10 CT.
Unfortunately, this will probably happen again and again in the future if nothing is done to prevent it. The problem can happen at any point from when the cafe owner reads his own PKH to give to John to when John writes his message before signing it. You could argue that Lisa could also make this mistake when she updates the spreadsheet, but she’s so thorough that this would never happen. She’s just too good at what she’s doing. Lisa will never cause someone else’s funds to be burned.
3.4.1. Where were we?
This chapter deals with Bitcoin addresses. To remind you where this all fits into Bitcoin, remember the diagram from Chapter 1, shown again in Figure 46.
Toward the end of this chapter, we’ll end up with Bitcoin (cookie token) addresses. We’ve just replaced the names in the spreadsheet with PKHs. We’ll now get to Bitcoin addresses. A Bitcoin address is a converted PKH—that is, it’s a PKH written in a way more suitable for human users and safe against spelling errors. The PKH is sent to Lisa (or Bitcoin nodes), but the address is what users see and give to each other.
3.4.2. Base58check
Among the coworkers, the security-oriented people discuss the problem with typos and come up with the idea of cookie token addresses. A cookie token address is a PKH encoded to detect typing errors. The PKH can be converted back and forth between this encoding and plain byte format.
Suppose Faiza feels sorry for John and wants to give him 20 CT from her 100 CT to ease his pain. She doesn’t want to make the same mistake John did, so she asks him for his cookie token address. John creates this by encoding his PKH with a function called base58check (Figure 47).
The result is John’s cookie token address: 19g6oo8f…gCenRBPD
. John
hands this address to Faiza, who then makes a payment following the
process in Figure 48.
The payment process changes for the payer, but nothing changes for Lisa. Faiza will base58check decode John’s address into a PKH. This decoding ensures that no typing errors were made in the address.
As mentioned previously, a PKH can be converted to an address and back to a PKH. It is not a one-way function. It’s just different ways to represent the PKH, either as a series of bytes or as an address (Figure 49).
The email to Lisa is exactly the same as before. Only users employ the cookie token address. It isn’t part of Lisa’s validation process or the spreadsheet in any way.
Base58check encoding
Let’s see how this mysterious base58check encoding works (Figure 50). First, a version is added before the PKH. The people who came up with the idea of cookie token addresses wanted to make future upgrades to the address format easy. Right now, only one version of cookie token addresses is available. This version is a single 0 byte.
To detect typing errors, a checksum is added. This checksum is calculated from the versioned PKH. To create a checksum, base58check hashes the versioned PKH with double SHA256. This means it’s first hashed with SHA256, and the resulting hash is hashed again with SHA256. Take the first 4 bytes of the second hash, and let those 4 bytes be the checksum. This checksum is then appended to the versioned PKH. You’ll soon see how this checksum protects users from typing errors. Be patient!
You started with a PKH of 20 bytes (40 hex characters). But now that you’ve added a version and a checksum, you have 25 bytes (50 hex characters). To make up for this increase, you’ll encode the 25 bytes in a more compact way than hexadecimal encoding.
Using a more compact encoding
Hex encoding is an inefficient way to represent data bytes. It
requires 2 characters for each byte. We use only 16 characters, where
each character represent 4 bits, 0000
to 1111
.
Many encoding schemes exist that use more characters to represent data. The most widely known is base64, in which each character represents 6 bits of data; to do this, the scheme needs more characters than just letters and digits. Base64 uses the following alphabet:
ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/
The character A
represents the bits 000000
, B
represents
000001
, and the character /
represents 111111
. This is a nice,
easy, compact way to represent data with human-readable
characters. I’ve already used base64-encoded data several times in
this book to represent signatures.
But base64 doesn’t quite fit the bill for cookie token addresses. We
need an encoding that doesn’t just detect typing errors when they happen
but minimizes the risk of making them. Notice how some characters look
similar in some fonts, like lI
(lowercase L, capital I) and 0O
(zero
and capital O). We also need a format that users can easily copy and
paste, meaning special characters such as +
and /
shouldn’t be
allowed—they’ll prevent us from marking the whole address by
double-clicking it. Removing those six characters reduces the
possibility of typing errors. But now we have only 58 characters left,
so we need another type of encoding.
This new way to encode data is called base58 because the alphabet is the following 58 characters:
123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz
| If you feel put off by this low-level base58 mumbo jumbo, you can skip to Section <IP_ADDRESS> and just accept that base58 is a way to encode and decode data. For the rest of you, please continue. It’s fun. |
In base64, each character represents exactly 6 bits, which makes it straightforward to encode and decode data. But with base58, each character represents slightly less than 6 bits but more than 5 bits. We need to encode data differently.
Let’s get back to the example in which John creates his address. He’s just added a version and a checksum. Now it’s time to encode the 25 bytes into the final result: the address (Figure 51).
The overall strategy of base58 is to treat the data as a huge number
that you divide by 58 over and over until the quotient is 0; you keep
the remainders of every division. You look up each remainder in the
lookup table and append a 1
last for each leading 0 byte in the
input. The string is finally reversed, and the result is John’s
cookie token address. Note that all cookie token addresses, not just
John’s, will start with a 1
. This is because the version byte is 0,
which is encoded by the character 1
.
You can decode base58-encoded data such as John’s address back to the original input of the base58 encoding. I’ll leave this as an exercise for the interested reader.
 It’s a generic way to convert a decimal number to any other base. You can use the same algorithm to convert to base3, instead—divide by 3 instead of 58. Maybe you’d also like to change the lookup table to map 0 to 0, 1 to 1, and 2 to 2 to get the characters you’re used to. For example, write 17 in base 3:
Then, look up the remainders in the lookup table (same digits as the
ones you convert), and you’ll get 2 2 1
. Reverse that to get the final
result: 1 2 2
. Verify that it’s correct as follows:
Base58check decoding
John has just created his cookie token address by base58check encoding his PKH. He’s given the address to Faiza so she can send him 20 CT. Now, Faiza needs to write a message to Lisa. To do this, she needs John’s PKH. The great thing about base58check encoding is that the process can be reversed so you can get the PKH from the address while simultaneously checking for typing errors (Figure 52).
Faiza takes John’s cookie token address and base58 decodes it. Then, she removes the checksum and uses the remaining part, the versioned PKH, to calculate the checksum again. The newly calculated checksum and the just-removed checksum must match. Otherwise, a typing error has occurred, in which case Faiza won’t create the message. She’ll know the address was corrupted somewhere along the way and refrain from sending an email to Lisa. She’ll verify that she entered the address correctly and that John gave her the correct address to learn where it went wrong.
How safe is the checksum? Suppose a typing error occurred in an address. What’s the probability that the checksum won’t detect the error? The checksum is 4 bytes, which corresponds to 232 ≈ 4.3 billion values. The chance is about 1 in 4.3 billion that base58check fails to detect the typing error. It’s pretty safe.
3.5. Back to privacy
Although privacy improved when we replaced names with PKHs, the spreadsheet still reveals some information that Acme Insurances finds useful.
For example, Acme can probably figure out the cafe has the PKH
87e3d169…8393b1cc
because a lot of 10 CT payments have been made
to this PKH. From this, Acme will be able to see which PKHs are making
the most 10 CT payments to that PKH. Let’s say Acme talks to Faiza and
asks her for information about her recent payments. She’s made only one
payment so far, the one to John. Faiza, unaware of why Acme is asking
these questions, discloses that the transaction is for John.
A week later, John receives a letter from Acme politely informing him that he’s been promoted to a higher risk category, and his insurance premium has been adjusted accordingly.
Some privacy issues obviously remain. Luckily, users can create as many addresses they like. For example, the cafe could create a unique address for every incoming payment. And John could create a brand-new cookie token address the next time he accepts cookie tokens from Faiza.
Using unique addresses for each payment will make it harder for Acme to extract information from the cookie token spreadsheet, because they won’t be able to tell which payments belong to the same person.
3.6. Recap
This chapter started with replacing the names in the spreadsheet with users’ respective PKHs.
Then, we used base58check to create an address from a PKH. Let’s put the pieces together and look at the whole cookie token address-creation process from random number generator to the address.
Faiza makes sure no typing errors happen by base58check decoding the address before signing the message.
3.6.1. System changes
The concept table (Table 11) isn’t updated in this chapter. Cookie token addresses are exactly what Bitcoin uses, so we haven’t introduced any concepts that differ from Bitcoin.
| Cookie Tokens | Bitcoin | Covered in |
|---|---|---|
1 cookie token |
1 bitcoin |
|
The spreadsheet |
The blockchain |
|
Email to Lisa |
A transaction |
|
A row in the spreadsheet |
A transaction |
|
Lisa |
A miner |
Thanks to PKH and cookie token addresses, Lisa can ditch her table of public keys. You can add PKH and addresses to your toolbox for later use, and we release version 3.0 of the cookie token system (Table 12).
| Version | Feature | How |
|---|---|---|
3.0 |
Safe from expensive typing errors |
Cookie token addresses. |
Privacy improvements |
A PKH instead of a personal name is stored in the spreadsheet. |
|
2.0 |
Secure payments |
Digital signatures solve the problem with imposters. |
1.0 |
Simple payment system |
Relies on Lisa being trustworthy and knowing everyone’s face. |
Finite money supply |
7,200 new CT are rewarded to Lisa daily; the amount halves every four years. |
3.7. Exercises
3.7.1. Warm up
-
The PKH is shorter than the public key—only 160 bits. We made it shorter using RIPEMD160. Why do you want it to be shorter? There are two good reasons.
-
Base58check encoding is used to create a cookie token (Bitcoin) address from a PKH. Is it possible to reverse this process to create a PKH from an address?
-
When is base58check decoding used, and by whom?
-
Base58 encode the two hex bytes
0047
. Use the following diagram. You may skip this exercise if you didn’t read the section on base58 encoding. -
What in an address makes it mostly safe from typing errors?
3.7.2. Dig in
-
Imagine that John wants a cookie from the cafe. He has two addresses: @1 with a balance of 5 CT, and @2 with 8 CT. His total balance is 13 CT, so he should be able to afford 10 CT for a cookie. Give an example of how he could pay 10 CT to the cafe.
-
Is it possible to deduce what cookie token addresses were involved in a certain payment by looking at the following spreadsheet?
-
Is it possible to deduce what public keys were involved in a certain payment by looking at just the spreadsheet?
-
Suppose everybody always used unique addresses for each payment. What information from the spreadsheet could Acme use to roughly identify the cafe’s addresses?
-
Suppose there was a serious flaw in the public key derivation function, so anyone could calculate the private key from a public key. What prevents a bad guy from stealing your money in this scenario?
-
Suppose there was a serious flaw in RIPEMD160, so anyone could easily figure out a 256-bit pre-image of the PKH. This would mean it wasn’t pre-image resistant. What prevents a bad guy from stealing your money in this scenario?
3.8. Summary
-
Privacy is important for you, not just for criminals.
-
Using PKHs instead of personal names as recipients for payments is important for privacy and more secure.
-
Encoding a PKH as a Bitcoin address, or cookie token address, reduces the risk of sending money into the void, thanks to the checksum in the address.
-
Only users care about Bitcoin addresses. The Bitcoin network, or Lisa, deals with plain PKHs.
-
You can have as many Bitcoin addresses as you like. Using multiple addresses, preferably one per received payment, improves your privacy.
4. Wallets
This chapter covers
-
Automating payments
-
Creating and managing keys
-
Making simple, secure key backups
So far, we’ve done nothing to improve the user experience for the company’s coworkers using the cookie token spreadsheet. The situation has become worse for users because emails to Lisa now need more information than in the beginning. On top of this, users should take extra steps to use multiple addresses to preserve their privacy.
In this chapter, we’ll build a mobile app, called a wallet (Figure 53), that handles many of the common tasks users want to perform. This wallet will create new addresses, store private keys, simplify how addresses are transferred between users, and automate the payment process.
We’ll discuss different approaches to wallet backups. We’ll also look at a new way to generate keys, called hierarchical deterministic wallets (HD wallets), so backups become dead simple; you only need to back up a single random number, called a seed, once and for all. We’ll finish the chapter with an optional deep dive into the math behind public key derivation.
This chapter won’t change anything regarding Lisa’s work or the spreadsheet. We focus only on users here.
4.1. First wallet version
Among you and your coworkers, a group of software developers builds a mobile app called a wallet to simplify common tasks for themselves and other users. The group identifies the following tasks as the most common:
- Create new addresses
-
Users must create new cookie token addresses every now and then. They might want to use different addresses for different purposes or even different addresses for all payments for privacy and security reasons.
- Manage private keys
-
For each address created, the wallet needs to store and manage the corresponding private key. Keeping private keys safe from intruders is a delicate task.
- Transfer payment details from payee to payer
-
When John wants to buy a cookie, he needs to get the cafe’s address and the payment amount into his app. Writing it by hand is cumbersome and error-prone, so it would be nice if John could scan the details with his camera instead.
- Make a payment
-
The app should be able to send an email to Lisa with the digitally signed payment details.
- Keep track of funds
-
Users want to know how many cookies they can afford. The app should display the total number of cookie tokens a user has.
- Back up private keys
-
When private keys are created in the app, they only exist in the app. If the mobile phone is lost or broken, the private keys are gone. You know by now what happens when you lose your keys, don’t you? You need a backup facility for private keys.
The development team builds an initial version of the app and calls it the wallet. The term wallet isn’t perfect because the app doesn’t really contain money. It contains the keys needed to spend money. The actual money is stored in the spreadsheet. The app is more akin to a physical keyring; but the term wallet is widely used in the Bitcoin world for all things that store private keys, so we should get over it and move on. Let’s go through this wallet’s features.
Suppose, once again, that John wants to buy a cookie in the cafe (Figure 54). Both John and the cafe are using this new app.
The process goes through several steps:
-
The cafe asks its wallet to create a new address and request 10 CT to that address. This new address and the amount are displayed on the screen as a QR code. The QR code contains information on how much to pay, so John doesn’t have to type that in manually.
-
John points his phone’s camera at the QR code to scan the payment details. It scans the payment URI (uniform resource identifier, a general specification on how to identify stuff; a web URL is an example of a URI):
ct:19UzNFW4Fq8wm8mtWmoPZAzE3tcB9tZVtN?amount=10
This tells John’s phone to launch the cookie token wallet (ct:
) and
pay 10 (amount=10
) cookie tokens to the address
19UzNFW4Fq8wm8mtWmoPZAzE3tcB9tZVtN
.
-
John’s wallet displays the payment details to John, who checks that they’re reasonable and clicks OK.
-
John’s wallet creates an email to Lisa that looks the same as before. The wallet automatically selects an address to send from and signs the message with the correct private key. On Lisa’s side, nothing has changed. She verifies and executes the payment exactly as before.
Let’s take a closer look at what John’s wallet does in step 4 (Figure 55). The wallet does the same thing a user would do manually in the earlier examples.
Notice that the wallet manages three key pairs: two with funds and one with no funds. With this new wallet, users can have as many addresses as they want, which is good for privacy. The wallet will keep track of them for the user.
The cafe’s wallet, as well as John’s wallet, will check the spreadsheet every now and then to see if there are any new payments concerning any of the wallet’s keys, as a sender, a recipient, or both (Figure 56).
Even though John knows about the payment before Lisa confirms it in the spreadsheet, his wallet won’t update the balance until it’s confirmed. Why? Lisa might not approve the payment. Maybe the payment became corrupted during transfer, or the email ended up in Lisa’s spam folder, so she doesn’t see it.
If the wallet updates the balance without first seeing it in the spreadsheet, it could give false information to John. The wallet could, of course, be kind enough to inform John that a payment is pending confirmation.
4.2. Private key backups
The development team creates a feature to back up the wallet’s private keys. The idea is that the wallet creates a text file, the backup file, with all private keys in it and sends this file to an email address the user chooses.
Imagine that John wants to back up his private keys. The wallet collects all the keys it has ever created and writes them into a text file (Figure 57).
The text file is emailed to John’s email address. Can you see any problems with this? Yes, the biggest problem is that the keys have left the privacy of the wallet application and are being sent into the wild. Anyone with access to the email server or any other systems involved might be able to get the private keys without John noticing.
But another problem exists. As soon as John creates a new address after the backup is made, this new address isn’t backed up. John must make a new backup that includes the new key. For every new key, he must make a new backup. Doing backups for every address becomes tiresome for the user.
Let’s look at a few simple solutions to these two problems:
-
Automatically send a backup when an address is created. This increases the risk of theft because you send more backups.
-
Pre-create 100 addresses, and make a backup of them. Repeat when the first 100 addresses are used. This also increases the risk of theft, but not as much as solution 1.
-
Encrypt the backup with a password. This will secure the backed-up keys from theft.
A combination of solutions 2 and 3 seems like a good strategy; you seldom need to do a backup, and the backups are secured by a strong password.
The process is similar to the previous process, but this time John enters a password that’s used to encrypt the private keys (Figure 58). If John loses his phone, he needs the password and the backup file to restore his private keys.
If John loses his phone, he can easily install the wallet app on another phone. John sends the backup file to the app and enters his password; the keys are decrypted from the backup file and added to his wallet app.
4.2.1. A few words on password strength
A password’s strength is measured in entropy. The higher the entropy, the harder it is to guess the password. The word entropy, as used in information security, comes from thermodynamics and means disorder or uncertainty. Suppose you construct a password of 8 characters from among the following 64 characters:
ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/
Each character in the password would then represent 6 bits of entropy
because there are 64 = 26 possible characters. If you select the 8
characters randomly (no cherry-picking, please!), say E3NrkbA7
, the
eight-character password will have 6 × 8 = 48 bits of entropy. This is
equivalent in strength to 48 coin flips.
Suppose instead that you select random words from a dictionary of 211= 2,048 words. How many words do you need to use to beat the 48-bit entropy of your eight-character password? Four words wouldn’t be enough because 4 × 11 = 44 bits of entropy. But five words corresponds to 55 bits of entropy, which beats the password’s entropy.
A password’s real entropy also depends on what an attacker knows about
the password. For example, suppose an attacker, Mallory, steals John’s
encrypted backup file and tries to perform a brute-force attack
on it. A brute-force attack means the attacker makes repeated
password guesses, over and over, until they find the correct
password. If Mallory knows the password’s length is exactly 8 and the
characters are chosen from the 64 characters mentioned, the entropy is
48 bits. If she happens to know that the second character is 3
, the
entropy drops to 6 × 7 = 42 bits. On the other hand, if Mallory
doesn’t know how many characters the password has, it will be harder
for her, meaning the entropy will be higher.
This is true only if password selection is truly random. If John uses
cherry-picking to select the password j0Hn4321
, the entropy
decreases dramatically. Typical password brute-force attack programs
first try a lot of known words and names in different variations
before trying more “random-looking” passwords. John is a well-known
name, so an attacker will try a lot of different variations of that
name as well as many other names and words. For example:
butter122 … waLk129 … go0die muh4mm@d john John JOhn JOHn JOHN j0hn j0Hn jOhn jOHn jOHN … john1 … … john12 J0hn12 … j0Hn321 … j0Hn4321
Bingo! Suppose there are 1,000,000 common words and names, and each word can come in 100,000 variations, on average. That’s 100 billion different passwords to test, which corresponds to about 37 bits of entropy; 100 billion tries will take a high-end desktop computer a few days to perform. Let’s say, for simplicity, that it takes one day. If John uses a truly random password, the entropy for the attacker is around 48 bits. It would take around 2,000 days, or about 5.5 years, to crack the password.
4.2.2. Problems with password-encrypted backups
The process for password-encrypted backups works pretty well, but it also introduces new problems:
- More things to secure
-
John now needs to keep track of two things: a backup file and a password. In the first version, only a backup file was needed.
- Forgotten password
-
Passwords that are rarely used, as is the case with backup passwords, will eventually be forgotten. You can write them down on paper and store them in a safe place to mitigate this issue. You can also store them using password-manager software, such as LastPass or KeePass.
- Technology advancements
-
As time passes, new, more advanced hardware and software is built that makes password cracking faster. If your eight-character password was safe five years ago, it’s not good enough today. Passwords need more entropy as technology improves. You can re-encrypt your backup files every two years with a stronger password, but that’s a complicated process that few users will manage.
- Randomness is hard
-
Coming up with random passwords is really hard. When the app asks John for a password, he needs to produce one on the spot. He doesn’t have time to flip a coin 48 times to produce a good password. He will most likely make up something with far less entropy. One way to deal with this is to have the wallet give John a generated password. But this password is likely harder to remember than a self-invented password, which will increase the likelihood of a forgotten password.
It seems you haven’t yet come up with a good way of dealing with backups. Let’s not settle for this half-bad solution—there are better approaches.
4.3. Hierarchical deterministic wallets
One of the brighter developers at the company, a cryptographer, comes up with a new way to handle key creation to improve the backup situation and bring totally new features to wallets.
She realizes that if all private keys in a wallet were generated from a single random number called a random seed, the whole wallet could be backed up by writing down the seed on a piece of paper and storing it in a safe place (Figure 59).
She talks to some other cryptographers, and they decide on a strategy. They’re going to make an HD wallet. Basically, keys are organized as a tree, in which one key is the root of the tree, and this root can have any number of child keys. Each child key can in turn have a large number of children of its own, and so on.
Suppose Rita wants to organize her keys based on their purpose and generate five keys to use for shopping at the cafe and another three keys to use as a savings account. Figure 60 shows how her keys could be organized.