Protect Vital Data End-to-End With Interior Encryption
Twitter exposed the plaintext passwords of 330 million users because those passwords were being written in to log files.
Stepping back, the design flaw is larger than simply injudicious logging: it is that in most systems, only the 'last hop' is actually encrypted.
In this article, we will explore the problem as well as solid solutions to it.
Example code is included.
-- by Arthur J Musgrove
In May of this year, Twitter warned
that the plaintext passwords of all 330 million users had been exposed. How did this happen
given that twitter was following all the best practices, including
storing passwords as hashes and using TLS encryption between clients
In most installations, only a small part of the traffic is encrypted,
and data is generally unencrypted at the first entry point into
the service deivery network and dealt with in unencrypted fashion
from there on. Below is a typical architecture.
As you can see, only the single hop from the client into the load balancer
is actually encrypted. Everything is plaintext from there on. Twitter
was good to come clean with the situation, but you can be sure
it is very common. Instead of workarounds of more
rigorous reviews, let's look at a solid solution.
Introduction to Asymmetric Key Cryptography
This section is a lightning fast introduction to asymmetric
cryptography or public/private key cryptography, in particular RSA. This is
very basic and only enough to understand this security technique.
If you understand asymemtric cryptography, you can safely skip this section.
Encryption is the science of coding messages to frustrate those that
may try to spy on your communications. Whenever you communicate with
https you are encrypting your communication with the server. This
is shown to the user with the little lock symbol on the address bar.
The encryption type that most people are familiar with, and is used
in so many aspects of security, is the shared secret method
of cryptography. In this mode, we have preagreed our keys. Remember
those secret decoder rings in boxes of cereal from when you
were a kid? Those are shared secret decoders. This type of
cryptography is called symmetric cryptography.
Shared secret encryption systems are the most commonly used because
they are secure and they are fast. The only real problem is that
both parties must keep the secret, well, secret. This is a problem
in the world of the Internet, because we want encryption
coming from the client machine, and that is by definition an insecure
Asymmetric cryptography uses 2 keys: one public and one private.
One key can encrypt data, and the other decrypts it. So you can give one
key to the world to encrypt data destined for you, with the confidence
that only you will be able do decrypt and read the data.
A full explanation of asymmetric cryptography and all of its various
uses, including confidentiality, integrity and nonrepudiation,
is well outside the scope of this article. Suffice it to say there are
many other uses than the simple case I am including here.
A Virtual Tunnel
While vital data is passed around the various parts of your architecture,
it is usually only needed by a specific component. For instance, the
username and password may pass through several components but it is
only a single component - the authentication server - that actually
needs the password. And it only needs it for a fleeting instant and
that plaintext password should not be persisted anywhere.
Wouldn't it be ideal if vital information could be sealed at the point
it is entered and can only be unsealed when needed by the component
that needs it, without it being accessible to any intermediate system?
Well, you can do exactly that.
The example here is using RSA encryption, but it can be equally done
with any asymmetric algorithm, including the newer elliptic curve
If you are unfamiliar with ECC, here is a quick explanation. RSA depends
on the difficulty of factoring large prime numbers, where ECC depends
on the difficulty of back-solving eliptical equations for the standard
equation: y2 = x3 + ax + b.
The basic process is simple.
Generate an RSA key pair
Client encrypts the subset of encrypted data and includes it within the payload (perhaps only a subset of JSON fields are encrypted while most are 'normal')
The encrypted vital data is passed around the system, and can only be decrypted by the ultimate component that needs the data.
To illustrate this design I've done a quick code example for you. On
the client side, this uses an excellent RSA library from a Stanford
University student which is available
Explore the example.
There is a README.txt file to explain the contents.
The traditional design - encrypted upfront then plaintext all through the
backend - has shown weakness and violates the defence-in-depth principle.
By implementing interior encryption you greatly improve the
security of the system and dramatically reduce attack surface.
All views expressed in this article are my own and do not represent the
opinions of any other entity whatsoever with which I have been, am now
or will ever be affiliated. No assurance of accuracy is given and
any use of any information provided is entirely at your own risk. The
author assumes no responsibility or liability for any errors or
omissions in the content of this article. No infomration provided is intended
to be a source of investment advise or credit analysis with respect to
any material presented or otherwise. Nothing contained in this article
is intended to defame or harm any person, business or other entity.
The author retains sole and exclusive
ownership of all material herein.