The Algorithm

The algorithm is divided into three stages: precalculation, encryption, and decryption. Precalculation is performed a single time for each person with a public/private key pair, and encryption/decryption is performed for each message.

Precalculation

1. Pick primes: Find two arbitrarily large prime numbers, p and q.

1. Determine the modulus: Multiply p and q to get n. This will be your modulus for all equations.

1. Calculate phi(n): Using Euler’s totient function, calculate . This number is a secret number, so don’t give it away. The strength of the algorithm depends on being hard to calculate when given a sufficiently large n = p * q

1. Determine an encryption exponent: Take any number, e, such that GCD(, e) = 1. This means that they are relatively prime, and share no common factors. This number is considered your public key (when combined with n), and you can give this number to whoever you like.

1. Compute the decryption exponent: Solve the Extended Euclidean Algorithm of GCD(, e) to find . This is your private key.

We have a private decryption key pair: {n, d}, and a public encryption key pair: {n, e}.

Encryption

is the encrypted message, and it can safely be sent publicly.

Decryption

is the decrypted message. The person with the private key for the message will be able to read it, and theoretically, nobody else.

Is the message preserved?

Applying the Corollary to Euler’s Theorem to , we get .

We also notice that

Why is it hard for an attacker to crack?

In a perfect world, the attacker needs to solve some equivalent of the Integer Factorization Problem to factor n, which is suspected to be outside of complexity class P using classical computation. If we have access to quantum computers, we have access to an algorithm, Shor’s Algorithm, to crack integer factorization efficiently, but probabilistically.

The essential problem that the attacker faces is as follows: they have the encryption exponent and the modulus. They know what n is, but they currently have no method of calculating what is without first factoring n. They need to find the decryption exponent, d = ,, but can’t find the exponent without being able to solve the Extended Euclidean Algorithm, where they need to know the value of . So long as n is hard to factor, RSA will remain difficult to break.

We don’t live in a perfect world, and there are plenty of examples of attacks that take advantage of weak implementations of RSA. See RFC 3447 for some best practices.

The Math

Extended Euclidean Algorithm

The Euclidean Algorithm finds the Greatest Common Divisor (GCD) of two integers, a and b. The Extended Euclidean Algorithm finds integers m and n in the following equation:

Euclidean Algorithm

To see how it does this, we will look at an example. Let a = 200, b = 37. Please note that 200 = (101 * 3). Since (200, 37) = 1, this is the equivalent of trying to find 37-1 (mod 200). For previous writings on the material, click here.

1. 
2. 
3. 
4. 

Line 3 is considered the “final” line of the algorithm because it is the last line where the remainder is nonzero. The remainder of that line, 1, is also GCD(200, 37).

Extended Euclidean Algorithm

We start by rewriting every equation so that the remainder is on the RHS.

1. 
2. 
3. 

From here, we start at line 3, and substitute in line 2. Don’t simplify any multiplications, just the additions.

Substitute line 1 into the resulting equation.

[Unparseable or potentially dangerous latex formula. Error 5 : 533x369]

We now have everything that we need for the inverse of 37. It turns out that is equivalent to saying:

so 173 is the multiplicative inverse of 37 (mod 200). Neat, huh?

Fermat’s Little Theorem + Euler’s Theorem

In a letter written in 1640, Fermat (of Pythagorean fame) offhandedly mentioned that he had noticed and proved the following relation:

Fermat’s Little Theorem

For a any integer, and p prime,

As he is occasionally noted for doing, he did not bother to write down the proof. Leibniz is said to have proved it, but didn’t get around to publishing it. He was an inventor of calculus, so we’ll cut him some slack.

Enter Leonhard Euler. Euler was 18 feet tall, shot laser beams from his eyes, and proved Mathematics every second he was awake. In 1736, he took 23 minutes off from a conquest of Mars to prove Fermat’s Little Theorem.

This wasn’t enough, though. He attempted to find a way to generalize it for any number, n, instead of just for primes, p, but couldn’t. The problem tortured him for 24 years, when in 1760* he was finally able to produce his proof.

His proof makes use of a function he defines, phi(x).

Euler’s Totient Function

For any positive integer, n, is equal to the number of positive integers where GCD(a, n) = 1, for a < n.

Most germane to this discussion, for any prime, p, . This makes perfect sense, of course, as a prime is indivisible, and therefore all numbers less than p don’t share factors with p, or else p could be divided!

The formula ends up being:

Euler’s Theorem

For any positive integers a, n:

* Wikipedia says 1736, my number theory book says 1760, tie goes to the book.

Corollary to Euler’s Theorem

Because , we can reduce the multiplication needed for any power of a > n.

Edit: added link to Wikipedia’s article on Integer Factorization, and a better explanation of why the attack is hard

and Loathing in Los Vegas

What will this article focus on?

This particular article looks at geometric sequences (mod n), and how we can use them instead of linear hashes. A geometric sequence is simply a sequence of powers of some number: 1, , , , … So instead of adding the same number together a bunch of times, we’re multiplying it together a bunch of times. And then you subtract one. More on that below!

First, the math

Euler’s Phi Function

When Euler was attempting to generalize Fermat’s Little Theorem, he defined a function using the Greek symbol (pronounced fee by most people I’ve encountered). It has a simple job: it takes in a natural number, n, and returns the number of positive integers less than n that are relatively prime to n. In this article, we are not concerned with ‘s calculation for anything but prime numbers.

It is easy to show that (p) = p-1 when p is prime: all numbers less than a prime are relatively prime to the prime in question, otherwise it wouldn’t be prime! Easy proof.

Euler’s phi function is of vital to the RSA encryption algorithm, and is the cornerstone of the generalization of Fermat’s Little Theorem, but it makes cameo appearances in many other areas of mathematics.

Examples:

(5) = 4, because gcd(5, 1) = gcd(5, 2) = gcd(5, 3), = gcd(5, 4) = 1.

(6) = 2, because gcd(6, 1) = gcd(6, 5) = 1, but gcd(6, 2) = 2, gcd(6, 3) = 3, and gcd(6, 4) = 2.

Order of a number (mod n)

The order of a number (mod n), where n is an integer, is the smallest positive value of x such that . If it is never equal to 1, it is considered infinite. 6 (mod 10) is an example that never has an answer. Note that this still has a solution under Euler’s generalization of Fermat’s Little Theorem. The laws of the universe won’t let you off that easy.

Example:

The order of 2 (mod 7) is 3, because [Unparseable or potentially dangerous latex formula. Error 1 ](prime) = prime-1, so (p) = p-1. If order(m) (mod p) is p-1, that means that m is a generator for all numbers (mod p) except p itself! Since this will not generate p, and 0 by extension, (since they are in the same congruence class), we must subtract our result by 1. So our generator is m, and our hash function is

It is not true that all numbers have a primitive root, but it WAS proved by Legendre that every prime has at least one generator (mod p). Interestingly, according to my college Number Theory textbook, Euler tried his hand at the proof, but was incorrect. To the uninitiated into the Cult of Euler, this would be akin to a team of Michael Jordan clones failing to score a single point in a basketball game against a team of middle school students.

We need to find one such that the first time this happens is for a power of p-1. Instead of testing every power, we can instead (because of this proof), just test powers where the power divides p-1. If we were looking mod 9, and we knew 3^8 == 1(mod p) (which it has to be because of Fermat’s Little Theorem), then , , , and are the only possible powers that can be equal to one. We will call this the generator test. We can check these particular values quickly through successive squaring. If any of the powers of 3 less than 8 are congruent to 1, then we have a failure, and it is not a generator.

If you do not have access to a good way to factor p-1, the following naive method will work well for small numbers. Please note that the preferable way is to factor p-1 and to find all of the divisors of p-1 that way.

// ***********************************************************************
// Precondition: p is a prime. If it is not, it will return 0 indicating
// failure
//
// This assumes that you are trying to do this for a small p, without being
// able to factorize p-1 quickly.
// ************************************************************************
unsigned int find_generator(int p)
{
  int phi_p(p-1);
  std::vector test_powers; 
 
  int i; 
 
  for(i=1; i
<0; --i)
      {
        if(powmod(test, test_powers[i], p) == 1)
	{
	      found = false;
	      break;
	}
      } 
 
      if(found)
      {
	  return test;
      }
   }
}

So what?

If we have an element a (mod n) who has , and is not equal to 1, we have a generator. The generator is for a set of integers of size (p-1), which is even.

Finding generators is nontrivial

A downside to this method is that there is no free lunch when it comes to finding generators. You have to find one, although fortunately for us, most numbers have generators that are less than 10, so you can find them by linearly searching. There are a few strategies of how we can pick primes that will allow us to (relatively) quickly find a generator (mod p). The one I use is:

One strategy is finding a prime, p, such that q = 2*p + 1 is also prime. The only two numbers that you have to check that violate our generator condition are 2 and p, in which case q is a generator. This helps reduce the complexity of the test. How do we know if our numbers are prime? Probabilistic primality testing, of course . It’s amazing how all of this stuff ties together.

A professor I had for a cryptology course said that the odds of the first generator NOT being less than 10 has been shown to be inordinately small, but I can’t for the life of me find any sort of reference to a figure that states that. As there is no trivial way to find a hash function, it is acceptable to search for the first generator (mod p) linearly, using our generator test, if you are looking for just any generator of p. Likewise, you can also find the largest such generator (mod p) by reverse searching.

This is so complicated. Why would I use this over a linear hash?

• The elements selected are not at a fixed interval, so data is usually less likely to cluster, which results in fewer collisions
• It does better at the avalanche test, which says that when a bit of the input changes, at least half of the bits of the output should change. The linear hash fails miserably at this, and geometric hashes (depending on your generator, of course), perform better than their linear counterparts.

Sometime in the future, (not in the next post, though), I will develop benchmarks to see what is better to deal with various different input scenarios. There’s no sense in developing the mathematics if we don’t actually put it all on the line and see if the “better” method works better in the real world. The real world has an amazing way of yelling “surprise!”, but we can limit that surprise through testing, testing, testing.

Number theory is the study of numbers, their properties, and what can be inferred from their properties. For programmers, it is most practical to focus on the theory of positive integers.

Who should use this guide?

• Those who did not know the answer to the above question
• Those who are interested in the math behind hash functions
• Those who found my last article interesting

What will this article focus on?

This article will focus on using the integers (mod n) as indices of a hash table, and the math behind different choices of hash functions. Our goal is to find a “good” hash function (see below). The mathematical explanation will be done irrespective of Group Theory, and I may write another article to look at a hash table as a group over addition or multiplication of the integers (mod n). For a quick refresher of the (mod n) concept, go here, or for another explanation, please look here.

Useful Tools

Greatest Common Divisor (GCD) of positive integers

Explanation:

Mathematically, the greatest common divisor of two numbers a and b is the product of all common divisors of a and b. For a simple explanation as to why, look here.

The Algorithm:

Naive:

unsigned int gcd(unsigned int a, unsigned int b)
{
    int remaind;
 
    if(!a) { return b; } // gcd(a, 0) = a
    if(!b) {return a; }
 
    if(a < b)
    {
        a ^= b;  // Swap a and b in place
        b ^= a;
        a ^= b;
    }
 
    while((remaind = a % b) > 0)
    {
        a = b;
        b = remaind;
    }
 
    return b;
}

Binary: (It’s always worth it to try to find the algorithms that take advantage of working with bits. If life gives you an integer as the sum of powers of two, make lemonade )

Wikipedia has a page that explains a binary algorithm that takes advantage of the binary format of the data. It reduces the problem by stripping out common multiples of two, and then applying the binary analogy of the GCD algorithm. For more details, follow the above link. I haven’t benchmarked it, but it relies heavily on bit operations, so it should run a little faster on modern popular architectures.

Least Common Multiple (LCM) of positive integers

Explanation:

The least common multiple is as it sounds: the smallest multiple that both a and b share. For example:
LCM(15, 20) = 60.

It appears that for each prime, the LCM of a and b includes the largest power from either a or b. In fact, this is true.

Relation between GCD and LCM

For integers a and b:

This is very powerful, and lets us efficiently calculate the LCM of a and b by dividing out the GCD of a * b. Why does this work? If a and b don’t have any prime factors in common, clearly the only way that we can have a multiple of a equal some multiple of b is by multiplying b by a. If a and b only have one prime factor in common (let’s call it d), if you multiply a by b, we get a*b as an answer. However, (a*b)/d is clearly a multiple of both a and b. We don’t need to multiply a by d, since a already HAS d as a factor. d is uncoincidentally the GCD of a and b, and clearly, GCD(a, b) * LCM(a, b) = a * b. An actual proof is left as an exercise to the reader.

What makes a good hash // hash table?

The short answer is that nobody knows. Hashes that work well for some kinds of inputs can produce intractable results for other kinds of input. For our purposes, we will say that a good hash function minimizes the odds of two different inputs ending up in the same congruence class (mod n). When two different inputs DO end up in the same index, this is called a collision, and is as undesirable in hash tables as it is while driving. Also bad is clustering, which is when collisions are much more likely to happen in certain indices than in other indices.

Ideally, we would like the hash function to be able to place elements at any index in the table. This makes it a generator, namely, it can generate any value in the table.

We will try to find a happy medium of all concerns through experimentation. I will define a few different hash functions in the upcoming articles, and then will show how to compare them. That will be where the “Science” part of Computer Science enters the picture

Linear Hashes

Linear hashes take in some number x, and place the object in the index ax + b (mod n). To make the mathematics easier, we will just use ax(mod n), as it should be obvious that adding b produces the set in the same order, but with a different starting point. In order for us to consider a as a hash function, a must be a generator (mod n). How do we know that it does that? Let’s look at a few different values of a (mod 16).

: {2, 4, 6, 8, 10, 12, 14, 0, 2} (mod 16) (doesn’t generate the integers (mod 16))

= 3: {3, 6, 9, 12, 15, 2, 5, 8, 11, 14, 1, 4, 7, 9, 12, 15, 3} (mod 16) (generates the integers (mod 16))

2*3 = 6: {6, 12, 2, 8, 14, 4, 10, 0, 6} (mod 16) (doesn’t generate the integers (mod 16)).

So what works? It works when gcd(a, n) = 1. This is known as being relatively prime or coprime, meaning they don’t share any common prime factors. and obviously don’t share any prime factors, so 3 is a generator using addition (mod 16).

Why does that work? The largest possible multiple of a that will give us 0 (mod n) is n, because a*n == a * 0 == 0 (mod n). We need to make the LCM of a and n equal to a * n, and since we know that LCM(a, n) = a * n / GCD(a, n), it follows that GCD(a, n) = 1.

Since most hash tables you make will have 2^n elements (this seems to be the standard, for addressing reasons), any odd number a will suffice to be a generator for linear hashes.

Theoretically, which hash value should I use?

Linear hashing is obviously a very simple hash function (the simplest one there is, I believe), and therefore, there is not a single hash fucntion that will work for every input set. In fact, this type of hash will have many input sets that will make it have very poor performance. However, if we have advanced knowledge of the kind of data that will be the input, we can stack the deck in our favor.

If your data is guaranteed to have no collisions (mapping unique integers less than the size of the hash table to some value), you can use any positive integer you want as your hash. I recommend 1 for ease of calculation

If your data is sorted ascending, use hash values close to 1. If you can find the mode of the data in advance, you can yourself by setting the hash value larger than the mode. If the mode is large with respect to the size of the hash table or with respect to the size of the data set, you can make the hash value larger than the average number of repetitions for each input.

If your data is sorted descending, you want to do as above, except make your hash value close to n-1. The reasoning can be derived from the above paragraph.

If your data is either purely random, or of several different varieties, your hash function is not always going to work no matter how hard you try. We should avoid hashes close to 1 and n-1, but other than that, we will need to benchmark to see if there is a better value.

Number theory is the study of numbers, their properties, and what can be inferred from their properties. For programmers, it is most practical to focus on the theory of positive integers.

Who should use this guide?

Those who did not know the answer to the above question.

How do we use modulus?

First, we should bridge the gap between a Programmer’s definition of modulus and a Mathematician’s.

Programmer: a % b is the remainder of a / b. Essentially, the programmer uses the following equation:

That is, the programmer says If we were finding 23 % 5, we would have:

% .

Mathematician: Mathematicians rearrange the above equation into the following:

. They write this as:

All this means is that you can move from a to r just by adding and subtracting b.
23 – 5 – 5 – 5 – 5 = 3, so

Efficient Exponentiation (mod n)

Let’s say that you need to find . The naive way of doing this is to perform the operation just as it’s written:

unsigned int powmod(unsigned int a, unsigned int x, unsigned int n)
{
    return pow(a, x) % n;
}

and it has a few disadvantages.

1. It has a very good chance of overflowing native data types
2. It has an algorithmic complexity of O(n) for the size of the exponent. For large integer types, this becomes O(nm), for integers of (on average) mwords

To show a more efficient way of doing it, we will use a method called “successive squaring”. I will explain it by using an example: Find :

We know that . We need to find :

. This is given.

.

This is where the leap of logic occurs. Since , it follows that

So % 5 = 3, and I was able to do it all in my head! For small numbers, this is usually the case. But it should be obvious that this is a lot easier than ordinary exponentiation, with on the order of O(logn) multiplications.

Code example

The best code example I have found is from Bruce Schneier’s “Applied Cryptography”. The C version using native unsigned integers is as follows:

unsigned int powmod(unsigned int base, unsigned int exp, unsigned int mod)
{
    unsigned int toret=1;
 
    while(exp > 0)
    {
        if(exp & 1)
            toret = (toret * base) % mod;
 
        exp >>= 1;
        base=(base*base) % mod;
    }
    return toret;
}

It’ll still overflow for the wrong values, but it is a quick and dirty example. If you have access to an infinite precision integer, it should be trivial to convert it.

Fermat’s Little Theorem

One of the many things that Fermat conjectured (and supposedly proved) is quite useful to the modern programmer. It says, for any prime number p, and for any integer a

.

Combined with the successive squaring method, this provides us a very powerful tool.

Probabilistic Primality Testing

For any number of applications, we need prime numbers. They are the crack-cocaine of modern mathematics. There are many simple ways to get prime numbers, such as the Sieve of Eratosthenes, but these methods fail when your application needs a 20-digit prime. There are newly developed (but complicated) tests that give a definite yes/no on a number in polynomial time, but they require Abstract Algebra, which is beyond the scope of this entry! For most developers, we don’t need to be 100% sure the numbers we are using are prime. We’re not using RSA in life-or-death (or multi-billion dollar banking) situations! All we want to do is tell whether or not an integer is most likely prime so that we can encrypt our Dawson’s Creek fan fiction and hide it from our father.

Fermat’s Little Theorem is always true if we know that the modulus is prime. The proof, however, doesn’t hold true in the opposite direction: if, for some number a, , we can’t say for sure that n is a prime. However, it is very frequently true, and often enough that we can form a probabilistic test, meaning that the numbers are probably prime. Mathematicians are noted for devastating understatement, so when we say “probably”, we mean “the chance is absurdly close to 100%”. According to pgp.net, PGP uses trial division for primes less than 8191, and the Fermat test for 2,3,5, and 7. I can’t find a reliable source covering the mathematics of why, but an unreliable source gives the chance that a composite is picked as less than 1 in . Yikes!

The Test

Ready?

For some number n, it is probably prime if:

If this makes you uncomfortable by using the first two primes, you can randomly pick two numbers (instead of 2 and 3). The test works just the same. For the ultra paranoid, try it three or four times.

The Carmichael Numbers

There are numbers that cause this test to fail for all test values. They are called Carmichael numbers, named after the first person to find an example. The first three are 561, 1105, and 1729. There are infinitely many Carmichael numbers, though they grow more scarce as the integers approach infinity. For fun, use the method of successive squaring to show that 561 is a Carmichael number.