§9.4.

Cryptography

Modern cryptography is built upon four main ‘primitive’ capabilities:

  1. Random number generation

  2. One-way hashing

  3. Symmetric encryption/decryption

  4. Asymmetric encryption/decryption (public-key cryptography)

Familiarity with these four primitives is enough to understand most security protocols and techniques that use cryptography.

Warning
Don’t make the mistake of thinking you can write your own cryptographic code. Use libraries that have been written by cryptography professionals.
Warning
This is very important, so I’m going to repeat it. Don’t make the mistake of thinking you can write your own cryptographic code! Use thoroughly tested libraries developed by professional cryptographers. Cryptographers have meticulously coded the libraries for security against many attacks that are not obvious (for example, timing attacks).
Warning
One final time. Don’t write your own cryptography! It is important.

Random number generation

Computers are deterministic: they predicably follow instructions. Two computers running the same code at the same time with the same inputs will produce the same output.

In fact, before the introduction of special random number generation CPU instructions, such as RDRAND and RDSEED [1], it was theoretically impossible for a computer program to generate random numbers. [2]

Node.js and JavaScript have a Math.random() and other programming languages have similar functions. However, these use pseudorandom number generators.

An old example of pseudorandom number generators is the rand48() function in the C programming language standard library. The default implementation uses an update rule, such as the following:

r' = (25214903917 * r + 11) % 281474976710656

Starting with r = 1, the next ten ‘random’ numbers are as follows:

Iteration

r

r / 281474976710656

1

25214903928

0.0000895813340946

2

206026503483683

0.7319531771219197

3

245470556921330

0.872086605317186

4

105707381795861

0.3755480612563424

5

223576932655868

0.7943048269107607

6

102497929776471

0.3641457971656017

7

87262199322646

0.3100176091757803

8

266094224901481

0.9453565926573084

9

44061996164032

0.1565396564872117

10

147838658590923

0.5252284246314893

To an untrained eye, these numbers appear to be ‘random’: the third column looks like a list of ‘random’ numbers between 0 and 1. They’re good enough for games and for generating random visualizations.

Node.js (and Chrome) use a more complex function, XorShift128+, defined in the Node.js C++ source code as follows:

static inline void XorShift128(uint64_t* state0, uint64_t* state1) {
    uint64_t s1 = *state0;
    uint64_t s0 = *state1;
    *state0 = s0;
    s1 ^= s1 << 23;
    s1 ^= s1 >> 17;
    s1 ^= s0;
    s1 ^= s0 >> 26;
    *state1 = s1;
}

It isn’t necessary to understand what this code does. My point is simply that the function is far too simple to use for security purposes. Future numbers can be easily guessed based on older numbers. So, if a website uses Math.random() to generate login codes for SMS messages (i.e., in a multi-factor authentication system), an adversary will be able to predict the login codes sent to users, without needing access to their phones.

While computers cannot generate true random numbers, cryptographically secure pseudorandom numbers (CSPRNG) are considered a safe alternative. CSPRNGs start with a small amount of initial random data (e.g., mouse clicks, keyboard data, network packets). They then use extremely complex update rules to generate an endless stream of numbers that appear random. The rules are chosen so that there is no known technique to predict future values based on their outputs.

Node.js and modern web browsers expose CSPRNGs for situations where secure random numbers are required.

The following code generates 16 cryptographically secure random bytes in Node.js:

const crypto = require('crypto');
let bytes = crypto.randomBytes(16);
console.log(bytes);

A crypto object has also been available in modern web browsers since 2014. The following code generates 16 random numbers in a browser: [3]

let bytes = window.crypto.getRandomValues(new Uint8Array(16));
console.log(bytes);
Warning
Math.random() is NOT secure. Attackers can predict its output and it is not truly random.
Tip
If random numbers are essential to your application’s security, use require('crypto') in Node.js.
Reflection: Secure random numbers

A critical application of secure random numbers is the SMS codes sent to users’ phones during multi-factor authentication.

What other situations depend on secure random numbers to ensure the security of an application?

One-way hashing

A one-way hash function transforms any amount of data into a single fixed-length summary. For example, the following table shows the result of using the SHA-256 hash function on different data:

input

sha256(input)

"" (i.e., the empty string)

e3b0c44298fc1c14​9afbf4c8996fb924​27ae41e4649b934c​a495991b7852b855

"a"

ca978112ca1bbdca​fac231b39a23dc4d​a786eff8147c4e72​b9807785afee48bb

"b"

3e23e8160039594a​33894f6564e1b134​8bbd7a0088d42c4a​cb73eeaed59c009d

"ab"

fb8e20fc2e4c3f24​8c60c39bd652f3c1​347298bb977b8b4d​5903b85055620603

"hello"

2cf24dba5fb0a30e​26e83b2ac5b9e29e​1b161e5c1fa7425e​73043362938b9824

"hello, world"

09ca7e4eaa6e8ae9​c7d2611671291848​83644d07dfba7cbf​bc4c8a2e08360d5b

"a" repeated 50 times

160b4e433e384e05​e537dc59b467f7cb​2403f0214db15c5d​b58862a3f1156d2e

"a" repeated 100000 times

6d1cf22d7cc09b08​5dfc25ee1a1f3ae0​265804c607bc2074​ad253bcc82fd81ee

The full text of the Project Gutenberg edition of the book Gulliver’s Travels (1726) by Jonathan Swift

d00c4f379e789c14​6ce22749b8bcba65​59243e1eb9a457f7​d917a26b6ff7bf9b

The table above illustrates some desirable properties of a one-way hash:

  1. The output is always the same length, irrespective of the input

  2. The same input will always produce the same output

  3. Slight differences in the input result in entirely different outputs

A good one-way hash algorithm has additional properties:

  1. It is fast to calculate the hash output, given an input

  2. It is extremely difficult to find an input if you only know the output

  3. It is improbable and difficult to find two inputs that produce the same output (this is called a collision)

A one-way hash gets its name from the fact that it works in only one direction: it can go from input to output, but reversing from output to a matching input is nearly impossible.

Most modern one-way hash functions work internally by repeatedly applying simple functions that substitute and permute data. Each operation introduces a little bit of confusion and uncertainty. The result of repeated application of the simple operations is so complex that the only way to go from output to input is to randomly guess-and-check possible inputs.

Two common applications of one-way hash functions are as follows:

  1. A one-way hash provides evidence of a file’s integrity. Any change to the input file will result in a completely different hash.

  2. A one-way hash demonstrates that you know or have an input value, without revealing the actual input value.

Symmetric encryption/decryption

Symmetric encryption/decryption is a method for protecting data using a single key or password. The name symmetric comes from the idea that encryption and decryption use the same key. This approach may also be known as secret-key cryptography because the key is a password to keep secret.

The following table demonstrates the result of using the AES-256-CBC symmetric encryption algorithm on various inputs:

input

key

output = encrypt(input, key)

decrypt(output, key)

hello

aip

cb7d8qkuZDvY09bJUaY4Q

hello

hello

asdf

RzmNro/N4SrUZ9r1CCMNvw

hello

hello, world

aip

MTLh2I7fY15v6Ly66Tex+Q

hello, world

hello, world

asdf

g8iyyq1UopvWER0uZs0NXg

hello, world

Note that changing either the input or the password results in a completely different output. Without the password, the output is meaningless. The password will decrypt the output back into the original text.

If the password is known, then symmetric key encryption is fast during both encryption and decryption. However, if the password is unknown, then it is extremely difficult (i.e., impossible) to find the input from the output. [4]

Asymmetric encryption/decryption

Asymmetric encryption/decryption was invented in the 1970s to protect data using a pair of matching keys. This approach is also called public-key cryptography because one key is usually made public and the other is kept secret. In an asymmetric cryptosystem, the private key decrypts messages encrypted by the public key (and vice-versa).

RSA and elliptic curve cryptography are the two main approaches to public-key cryptography. Both depend on mathematical operations that are easy to compute in one direction, but harder to reverse.

RSA depends on factorization being difficult. For example, using pen-and-paper, most people would be able to multiply 89681 and 96079 to get 8616460799 within a minute or two. However, it might take years to reverse the operation to find the factors of 8616460799, without a computer. [5] Computers are faster than people, so RSA depends on massive numbers with over 600 digits (2048 bits).

Elliptic curve cryptography also uses an operation that is difficult to reverse. Elliptic curves are mathematical structures with a group operator. The mathematical symbol ‘+’ is often used to represent this operator, even though it is not ordinary addition. For example, in an elliptic curve, it is easy to combine a value with itself eight times: P = G + G + G + G + G + G + G + G. However, it is difficult to calculate that the group operator was applied eight times when given only G and P. In practice, systems such as Curve25519 apply the group operator many more times. For example, rather than using the one-digit number ‘eight’, elliptic curve systems use 77 digit numbers (256 bit) or greater.

The first step in using an asymmetric cryptosystem, is to generate a public and private key. For example, on my computer, I used the openssl command [6] to randomly and securely generate the following RSA private key and its matching public key.

-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEArehpGoTexi70UPL7w2AsOTWj+HJrewxVkM3DMKU4CDzVpV4d
CHRKciIA6rOYgrKnT/yH2z7UasHf9KBZENVNdFqMCBTCNM4Pt4NDKZT4DNSw+2qS
qxEtYb7gtOZfTJxv/MPhrZ1Fi672N8AfECqfZabf/HoVim4wtO1l+XMHdD/dpmQ7
wQjF/BpIGLYQbI+0sgZWydOArwP7tDgeZJG7nz+pgtgDhfkovLN9YYrgSr7IVydM
71waCuC71ASQ6tTeUmlznP7mMJChND7trdNPCSXuWt62JAPTSWqyjU3FW8pBNtOO
hYaUe8RK8j1DuJxz94m6MEobnyUGiH5SCCnTvQIDAQABAoIBACR+pjfLdFiQl/K4
2v6IGx+yUwObN1TuJLKri2+U7GpGIet/EYapqMnEuv6Fy9Z5mUTe0L/AsqDoqI/U
anxu1r85FTPI72xXZdLz988tFNTUeYN5POgrRaPCg7NSuOMB3Tpk/OILJAIJKGBQ
r/QbjbGuUEjScdzH/O6q9wBfFExflff//QusUi7SsIFTniA609Ah5BOuI7F/dTGl
QyN2Bu/oXF6MnH9hGSaLCUvKg4ZXyKyoP1J3zIM2dj7tKLl0mQV1ahg4B+mvcQgU
zRo6I/jaoGfhA2NLchH12T1SqJ95SDcCn0kc8C7zYESI3dG1ViM6K6sjrd4Ac2ua
V3+nev0CgYEA30TfYYEj0QUTrFFMx7Tv4hvsggyP0bxJmJY9Zqzt01dPFAmUmV0/
R4fHTGXxIc2XnPkYi6WqM7g4pUXNUBTE9do9L8YIy22TvSA1o21bZ0yWX7VvK721
QB/V1Jb7cWUpFKseuvKHshoMgiwKB7x7GbGOwbx1vvMpXint13o58XcCgYEAx2cM
U0QrNZJmdxJUsDu2AoS4h/bL7opuKwfk/72TOMBGRzP2nKjMe+WRMJ0ea29ygBv6
nWx7gnU5NsAv7QFfByA2jF38YALwewdwCVzLZ4qNVlkEVkF5Ah0Xy4fxFr008E4T
+fFddQacKCeWMvJDDAvcgErj/2vG6oEd2eCfEWsCgYBXVtTfiqodKRRCE2eqs+An
Hm9NjGZyUGql0xff44QBaaUYnIrR18VaUQYon7RNWeSWVmdAsaS8KLOYC48+ZXGL
Dz1iQ+DK22mw0TnKXYwlA7PLauk7PjH6DLoUOJ/SAxWn7SzPSvLEPCZqgZnG3vd0
3J2Qsg2Jjgu/tz1ATqL+DwKBgQCdlDryRonbETH2YT8Z8mYYwWfO0uNARJdhXCDF
VbxVeeVP+amnDeJi+v1tLI1Qm8chpHq+E2/bneWz9dcp9g5x5CwXa2K5QTloEG2i
iHmZ/q1JEpnRzHXjjLg0ON72eFmwmhNBT1Pq2mlndjlFU5xWlb0QiZ56SGLvCVBc
0R0DtwKBgQClTrEOpYAfBBKVIUnieBbYkfVcTIjE6Dhquxc8yHdXhl9xeWOFTSYl
a0syRFl7WZv1IXdARYtD/YCz3FOAGQPyixLDG2jKPUeeJDWuumf/7BtbqzqlHSfb
6JL4FEHViN60nEE8x5+tF81loJeb/0a2faAORgMJfs0kj01Spp1wHw==
-----END RSA PRIVATE KEY-----

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArehpGoTexi70UPL7w2As
OTWj+HJrewxVkM3DMKU4CDzVpV4dCHRKciIA6rOYgrKnT/yH2z7UasHf9KBZENVN
dFqMCBTCNM4Pt4NDKZT4DNSw+2qSqxEtYb7gtOZfTJxv/MPhrZ1Fi672N8AfECqf
Zabf/HoVim4wtO1l+XMHdD/dpmQ7wQjF/BpIGLYQbI+0sgZWydOArwP7tDgeZJG7
nz+pgtgDhfkovLN9YYrgSr7IVydM71waCuC71ASQ6tTeUmlznP7mMJChND7trdNP
CSXuWt62JAPTSWqyjU3FW8pBNtOOhYaUe8RK8j1DuJxz94m6MEobnyUGiH5SCCnT
vQIDAQAB
-----END PUBLIC KEY-----

The public key can be safely shared with anyone but the private key must be kept secret. [7]

In RSA, there is no difference between ‘encryption’ and ‘decryption’: the same algorithm accepts the private or public key. The following table illustrates the result of encrypting input data.

input

Operation

output

hello, world

apply(input, public_key)

OAkrSvD8qcM5oFqY…​

hello, world

apply(input, private_key)

nVB/z/rIqlv/kb47…​

OAkrSvD8qcM5oFqY…​

apply(input, public_key)

AHqRJWBGcmWaFDvf…​

OAkrSvD8qcM5oFqY…​

apply(input, private_key)

hello, world

nVB/z/rIqlv/kb47…​

apply(input, public_key)

hello, world

nVB/z/rIqlv/kb47…​

apply(input, private_key)

U7F29J2MwRH9kVOO…​

Notice in the table above that the public key and private key act as opposites of each other. Applying the public key and then the private key will get you back to the input. The input is also recovered if the private key is applied first, followed by the public key. However, applying the public key twice (or any other key) will result in unrelated output.

This public/private key structure makes it possible for two strangers to communicate with each other privately. If I want to send you a private message, I use your public key to encrypt the message. As you are the only person with the matching private key, only you can decrypt the message. You can then reply to me by using my public key to send a response back: only I will be able to decrypt the message with my matching private key.

Asymmetric encryption also makes it possible to create digital signatures. A digital signature is the reverse of an encrypted message: it is a public message from a person that only the sender can generate. If I encrypt a message with my private key, anyone with the public key can decrypt the message. Of course, this doesn’t offer any secrecy to the message. Yet, it tells the world that I sent the message because I am the only person with the private key who could have encrypted the message.

Applications of the primitives

Password salting and hashing

One of the most common security problems occurs when an attacker gains access to a back-end database and can see every user’s password.

For example, there may be an internal database table as follows:

Username

Email

Password

Mike

mike@example.com

asdf1234

Carol

carol@example.com

password

Greg

greg@example.net

chevrolet

Marcia

marcia@example.org

daveyjones

Peter

peter@example.com

chevrolet

One way to protect the password is to hash it. That way, attackers cannot see passwords even if they steal the database.

Username

Email

sha256(Password)

Mike

mike@example.com

312433c28349f63c​4f387953ff337046​e794bea0f9b9ebfc​b08e90046ded9c76

Carol

carol@example.com

5e884898da280471​51d0e56f8dc62927​73603d0d6aabbdd6​2a11ef721d1542d8

Greg

greg@example.net

9e48bad6e31c1282​58257bd997eacca6​66f0dae2049027b5​45ea48a0cf43ed71

Marcia

marcia@example.org

46a5d9d9f45ef11f​a3aada67a04467e8​591f220cbe1fea88​1b7c525e50451359

Peter

peter@example.com

9e48bad6e31c1282​58257bd997eacca6​66f0dae2049027b5​45ea48a0cf43ed71

The system verifies a password by hashing the input text. If the hashed input matches the saved hash of the password, then the input must correctly match the original password.

While this increases security, there is still a problem. We can see that popular passwords have identical hashes. For example, because Greg and Peter have the same password, their hashed passwords are also identical. If Peter accesses this database, he knows that he can use his password to also log in as Greg.

The way to address this problem is to add some random values to the password before hashing. This random data is called salt. A simple (but still slightly flawed) way to hash with salt is to add the salt to the end of the password when hashing:

Username

Email

Salt (randomly generated)

sha256(Password + Salt)

Mike

mike@example.com

ca978112ca

e6496e1fc73992ce​ac14c9fa40886635​d1713d149e5b3073​c4dd44a9c98af512

Carol

carol@example.com

3e23e81600

02f03ff9db9707dc​c1763f471e89b7ef​00298cae90e97761​75934f2fd42f34dc

Greg

greg@example.net

2e7d2c03a9

125677dfd55a0e2a​5c1b1ac3330451d0​10c78ecbef84607d​5779c53dd199f0fd

Marcia

marcia@example.org

18ac3e7343

53527b305b23d6b7​4a894b5b65b3dc86​a57e2ddde6bceb1c​98890a39644ea9e7

Peter

peter@example.com

3f79bb7b43

dec183b815f56678​e54fc7475f671923​d48d4c7b24ad2f2b​13aff0dedb99ba2e

Verification of a password then happens in three stages:

  1. Retrieve the user’s salt

  2. Hash the combined input text and salt

  3. Compare the hash to the stored salted hash

Note that every hash is different, even ones with identical passwords!

Of course, it is generally a bad idea to write your own cryptography code. A simple salting and hashing scheme will provide some security. To have state-of-the-art security, you should use libraries developed by professional cryptographers and have been ‘battle-tested’ by extensive use in major websites globally. The bcrypt library on NPM is an excellent choice and the most popular library. Other options include Argon2 (recommended by OWASP, but needs to be tuned) and PBKDF2 (recommended by NIST, but more difficult to configure).

Tip
If you are building a web application, create two separate accounts with the same password. If both accounts have the same password in the database (i.e., after hashing or salting), something is wrong! Every password should be unique after salting.

Tokens and identifiers

Some data has no apparent primary key. Examples are listed below:

  1. The shopping cart or session of a user who has not logged in (e.g., the shopping cart created on amazon.com when you visit as a guest)

  2. A newly created blank document in an online editor (e.g., a new document named “Untitled”, especially if the creation time might not be unique)

  3. A message in a social network (e.g., a tweet or a post on a social network)

These situations require a unique identifier or token to represent the data. If this token is predictable, then access needs to be carefully checked. However, if the token is unpredictable and long, then the token itself can be used to grant access to the underlying information.

For example, a Google Doc can be made visible to ‘anybody with the link’: https://docs.google.com/document/d/1UqutaK9WjvgBZ1Zo8qN8rZcaRzYQZTwSWcFL2pJMJ6w/edit?usp=sharing. This document URL contains a long random identifier (1UqutaK9WjvgBZ1Zo8qN8rZcaRzYQZTwSWcFL2pJMJ6w) that is impossible to guess by chance.

Another example is the concept of sessions available in most web development frameworks. [8] Sessions allow developers to associate data with users. When a new user arrives, the session management system generates a random token to identify the user. The server sends the token to the user’s browser as a cookie and uses it to match subsequent HTTP requests back to their sessions.

An important use of secure random numbers is to generate identifiers and tokens. Using insecure methods (such as a counter, the current time or Math.random()) can make the token predictable. Secure random numbers will preserve the security of systems that depend on identifiers or tokens being difficult to guess.

Warning
MongoDB automatically adds an _id when inserting in a collection. The value might look be something like ObjectId("5eb8ed79d16196d3e8ee2a3e"). While this looks like a random number, MongoDB uses a predictable algorithm based on the current time.

Hash-based message authentication code (HMAC)

A hash-based message authentication code (HMAC) is a form of digital signature, created from a secure hash and a secret key or password.

A common use of HMACs occurs in websites with many users. HMACs allow end-users’ web browsers to store critical data without concern about tampering.

For example, when Alice logs in, it might take a long time to retrieve account details from the database and verify her identity and access permissions. If this needs to happen often, the obvious solutions are not satisfactory:

  1. Verify the access permissions on every separate request (this is slow)

  2. Cache the access permissions of recent users (this may use lots of memory)

  3. Have Alice’s browser store a cache of the access permissions (Alice could tamper with the data to change her access permissions)

Instead, HMACs provide a way to prevent tampering of data. The server can digitally sign the access permissions and send the permissions and signature to Alice. Alice can present the digitally signed permissions on subsequent requests. The server can use the signature to verify that Alice has not tampered with the permissions.

A very basic HMAC can be created by hashing both data and a secret key known only by the server. Tampering by the client can be detected because changing the data will cause the hash to change. The client cannot regenerate a valid HMAC themselves because they don’t know the secret key.

In practice, more sophisticated algorithms securely combine the data and secret key to avoid security issues.

JSON web token (JWT)

JSON web tokens (JWT) are an industry-standard method to combine data with a digital signature in a URL-friendly format that works well with other web standards.

A JSON web token encodes three kinds of information into a single value:

  1. A header: the details of the encryption/signing algorithms (e.g., it might specify that it uses an HMAC based on SHA-256, or that it uses an RSA digital signature)

  2. A claim set: the data in the message (it might also contain information about the JWT, such as when it the token should expire) [9]

  3. A digital signature: an HMAC or signature calculated according to the algorithm specified in the header

Reflection: JWT in distributed systems

JWT can combine data with a digital signature. One application is to allow a web server to offload a cache of data to the client’s web browser.

In large scale applications, there may be multiple servers involved in handling requests. How could JWTs be useful in complex web applications where a server that performs authentication is separate from a server that handles shopping cart data?

Reflection: JWT invalidation

Cached data can become out of date. For example, Alice might originally have Manager access but be downgraded by an Administrator so that she only has Guest access.

However, the client’s web browser may store the JWT. How could the server invalidate a JWT that it cannot directly access?

SSL/TLS

In Chapter 3, I discussed how HTTP uses TCP sockets to send data over the network. TCP does not use encryption: any router handling TCP sockets can read and intercept messages.

HTTP requests for URLs that begin with the https:// scheme, use an additional layer of encryption on top of TCP. This layer was initially named Secure Sockets Layer (SSL). It has evolved in new standards, now called Transport Layer Security (TLS).

TLS is a complex standard, offering a range of operation modes. It serves several purposes:

  1. Privacy: only the sender and receiver understand the data

  2. Authentication: users can verify the server (so that nobody other than Google can claim to be www.google.com)

  3. Integrity: nobody can modify data sent between the sender and receiver

Symmetric encryption is usually very fast, but asymmetric encryption/decryption tends to be slower. For this reason, TLS combines both symmetric and asymmetric encryption. TLS uses symmetric encryption with a randomly chosen key to efficiently encrypt large amounts of data. The randomly chosen symmetric key is exchanged once at the start, using slower asymmetric encryption.

TLS also uses digital signatures to verify identity. Built into your web browser is the public key for a company called GlobalSign. This company is one of several companies that specialize in verifying identities. Google employees have provided GlobalSign with identification documents. In return, GlobalSign digitally signs Google’s public key. So, when you visit www.google.com, the web server provides its public key as well as the digital signature from GlobalSign to prove the authenticity of the public key.


1. Intel introduced RDRAND and RDSEED in 2012.
2. Security experts do not trust the RDRAND and RDSEED instructions. Intel has not provided enough information about the internal operation of the instructions to be considered reliable as a sole-source of randomness.
3. Be careful! You shouldn’t trust client browsers to generate random numbers. The user or their browser may be malicious.
4. Interestingly, symmetric encryption can perform one-way hashing. Using a message as both input and secret key (encrypt(input, input)), produces output that requires the original message for decryption.
5. This number was first published in 1874 by William Stanley Jevons. It took 15 years until Charles Busk published a factorization in 1889.
6. The openssl genrsa -out private.key 2048 command will generate a private key and openssl rsa -in private.key -pubout -out public.key will generate a matching public key.
7. The private key is sufficient to generate the public key at any time. However, it is extremely difficult (i.e., currently impossible) to use the public key to generate the private key again.
8. In Express, the express-session NPM package is the standard implementation of sessions.
9. The word claim is used in security applications to refer to information about a user. However, a JWT token may store any data.