and Loathing in Los Vegas

What will this article focus on?

This particular article looks at geometric sequences (mod n), and how we can use them instead of linear hashes. A geometric sequence is simply a sequence of powers of some number: 1, , , , … So instead of adding the same number together a bunch of times, we’re multiplying it together a bunch of times. And then you subtract one. More on that below!

First, the math

Euler’s Phi Function

When Euler was attempting to generalize Fermat’s Little Theorem, he defined a function using the Greek symbol (pronounced fee by most people I’ve encountered). It has a simple job: it takes in a natural number, n, and returns the number of positive integers less than n that are relatively prime to n. In this article, we are not concerned with ‘s calculation for anything but prime numbers.

It is easy to show that (p) = p-1 when p is prime: all numbers less than a prime are relatively prime to the prime in question, otherwise it wouldn’t be prime! Easy proof.

Euler’s phi function is of vital to the RSA encryption algorithm, and is the cornerstone of the generalization of Fermat’s Little Theorem, but it makes cameo appearances in many other areas of mathematics.

Examples:

(5) = 4, because gcd(5, 1) = gcd(5, 2) = gcd(5, 3), = gcd(5, 4) = 1.

(6) = 2, because gcd(6, 1) = gcd(6, 5) = 1, but gcd(6, 2) = 2, gcd(6, 3) = 3, and gcd(6, 4) = 2.

Order of a number (mod n)

The order of a number (mod n), where n is an integer, is the smallest positive value of x such that . If it is never equal to 1, it is considered infinite. 6 (mod 10) is an example that never has an answer. Note that this still has a solution under Euler’s generalization of Fermat’s Little Theorem. The laws of the universe won’t let you off that easy.

Example:

The order of 2 (mod 7) is 3, because [Unparseable or potentially dangerous latex formula. Error 1 ](prime) = prime-1, so (p) = p-1. If order(m) (mod p) is p-1, that means that m is a generator for all numbers (mod p) except p itself! Since this will not generate p, and 0 by extension, (since they are in the same congruence class), we must subtract our result by 1. So our generator is m, and our hash function is

It is not true that all numbers have a primitive root, but it WAS proved by Legendre that every prime has at least one generator (mod p). Interestingly, according to my college Number Theory textbook, Euler tried his hand at the proof, but was incorrect. To the uninitiated into the Cult of Euler, this would be akin to a team of Michael Jordan clones failing to score a single point in a basketball game against a team of middle school students.

We need to find one such that the first time this happens is for a power of p-1. Instead of testing every power, we can instead (because of this proof), just test powers where the power divides p-1. If we were looking mod 9, and we knew 3^8 == 1(mod p) (which it has to be because of Fermat’s Little Theorem), then , , , and are the only possible powers that can be equal to one. We will call this the generator test. We can check these particular values quickly through successive squaring. If any of the powers of 3 less than 8 are congruent to 1, then we have a failure, and it is not a generator.

If you do not have access to a good way to factor p-1, the following naive method will work well for small numbers. Please note that the preferable way is to factor p-1 and to find all of the divisors of p-1 that way.

// ***********************************************************************
// Precondition: p is a prime. If it is not, it will return 0 indicating
// failure
//
// This assumes that you are trying to do this for a small p, without being
// able to factorize p-1 quickly.
// ************************************************************************
unsigned int find_generator(int p)
{
  int phi_p(p-1);
  std::vector test_powers; 
 
  int i; 
 
  for(i=1; i
<0; --i)
      {
        if(powmod(test, test_powers[i], p) == 1)
	{
	      found = false;
	      break;
	}
      } 
 
      if(found)
      {
	  return test;
      }
   }
}

So what?

If we have an element a (mod n) who has , and is not equal to 1, we have a generator. The generator is for a set of integers of size (p-1), which is even.

Finding generators is nontrivial

A downside to this method is that there is no free lunch when it comes to finding generators. You have to find one, although fortunately for us, most numbers have generators that are less than 10, so you can find them by linearly searching. There are a few strategies of how we can pick primes that will allow us to (relatively) quickly find a generator (mod p). The one I use is:

One strategy is finding a prime, p, such that q = 2*p + 1 is also prime. The only two numbers that you have to check that violate our generator condition are 2 and p, in which case q is a generator. This helps reduce the complexity of the test. How do we know if our numbers are prime? Probabilistic primality testing, of course . It’s amazing how all of this stuff ties together.

A professor I had for a cryptology course said that the odds of the first generator NOT being less than 10 has been shown to be inordinately small, but I can’t for the life of me find any sort of reference to a figure that states that. As there is no trivial way to find a hash function, it is acceptable to search for the first generator (mod p) linearly, using our generator test, if you are looking for just any generator of p. Likewise, you can also find the largest such generator (mod p) by reverse searching.

This is so complicated. Why would I use this over a linear hash?

• The elements selected are not at a fixed interval, so data is usually less likely to cluster, which results in fewer collisions
• It does better at the avalanche test, which says that when a bit of the input changes, at least half of the bits of the output should change. The linear hash fails miserably at this, and geometric hashes (depending on your generator, of course), perform better than their linear counterparts.

Sometime in the future, (not in the next post, though), I will develop benchmarks to see what is better to deal with various different input scenarios. There’s no sense in developing the mathematics if we don’t actually put it all on the line and see if the “better” method works better in the real world. The real world has an amazing way of yelling “surprise!”, but we can limit that surprise through testing, testing, testing.

Number theory is the study of numbers, their properties, and what can be inferred from their properties. For programmers, it is most practical to focus on the theory of positive integers.

Who should use this guide?

• Those who did not know the answer to the above question
• Those who are interested in the math behind hash functions
• Those who found my last article interesting

What will this article focus on?

This article will focus on using the integers (mod n) as indices of a hash table, and the math behind different choices of hash functions. Our goal is to find a “good” hash function (see below). The mathematical explanation will be done irrespective of Group Theory, and I may write another article to look at a hash table as a group over addition or multiplication of the integers (mod n). For a quick refresher of the (mod n) concept, go here, or for another explanation, please look here.

Useful Tools

Greatest Common Divisor (GCD) of positive integers

Explanation:

Mathematically, the greatest common divisor of two numbers a and b is the product of all common divisors of a and b. For a simple explanation as to why, look here.

The Algorithm:

Naive:

unsigned int gcd(unsigned int a, unsigned int b)
{
    int remaind;
 
    if(!a) { return b; } // gcd(a, 0) = a
    if(!b) {return a; }
 
    if(a < b)
    {
        a ^= b;  // Swap a and b in place
        b ^= a;
        a ^= b;
    }
 
    while((remaind = a % b) > 0)
    {
        a = b;
        b = remaind;
    }
 
    return b;
}

Binary: (It’s always worth it to try to find the algorithms that take advantage of working with bits. If life gives you an integer as the sum of powers of two, make lemonade )

Wikipedia has a page that explains a binary algorithm that takes advantage of the binary format of the data. It reduces the problem by stripping out common multiples of two, and then applying the binary analogy of the GCD algorithm. For more details, follow the above link. I haven’t benchmarked it, but it relies heavily on bit operations, so it should run a little faster on modern popular architectures.

Least Common Multiple (LCM) of positive integers

Explanation:

The least common multiple is as it sounds: the smallest multiple that both a and b share. For example:
LCM(15, 20) = 60.

It appears that for each prime, the LCM of a and b includes the largest power from either a or b. In fact, this is true.

Relation between GCD and LCM

For integers a and b:

This is very powerful, and lets us efficiently calculate the LCM of a and b by dividing out the GCD of a * b. Why does this work? If a and b don’t have any prime factors in common, clearly the only way that we can have a multiple of a equal some multiple of b is by multiplying b by a. If a and b only have one prime factor in common (let’s call it d), if you multiply a by b, we get a*b as an answer. However, (a*b)/d is clearly a multiple of both a and b. We don’t need to multiply a by d, since a already HAS d as a factor. d is uncoincidentally the GCD of a and b, and clearly, GCD(a, b) * LCM(a, b) = a * b. An actual proof is left as an exercise to the reader.

What makes a good hash // hash table?

The short answer is that nobody knows. Hashes that work well for some kinds of inputs can produce intractable results for other kinds of input. For our purposes, we will say that a good hash function minimizes the odds of two different inputs ending up in the same congruence class (mod n). When two different inputs DO end up in the same index, this is called a collision, and is as undesirable in hash tables as it is while driving. Also bad is clustering, which is when collisions are much more likely to happen in certain indices than in other indices.

Ideally, we would like the hash function to be able to place elements at any index in the table. This makes it a generator, namely, it can generate any value in the table.

We will try to find a happy medium of all concerns through experimentation. I will define a few different hash functions in the upcoming articles, and then will show how to compare them. That will be where the “Science” part of Computer Science enters the picture

Linear Hashes

Linear hashes take in some number x, and place the object in the index ax + b (mod n). To make the mathematics easier, we will just use ax(mod n), as it should be obvious that adding b produces the set in the same order, but with a different starting point. In order for us to consider a as a hash function, a must be a generator (mod n). How do we know that it does that? Let’s look at a few different values of a (mod 16).

: {2, 4, 6, 8, 10, 12, 14, 0, 2} (mod 16) (doesn’t generate the integers (mod 16))

= 3: {3, 6, 9, 12, 15, 2, 5, 8, 11, 14, 1, 4, 7, 9, 12, 15, 3} (mod 16) (generates the integers (mod 16))

2*3 = 6: {6, 12, 2, 8, 14, 4, 10, 0, 6} (mod 16) (doesn’t generate the integers (mod 16)).

So what works? It works when gcd(a, n) = 1. This is known as being relatively prime or coprime, meaning they don’t share any common prime factors. and obviously don’t share any prime factors, so 3 is a generator using addition (mod 16).

Why does that work? The largest possible multiple of a that will give us 0 (mod n) is n, because a*n == a * 0 == 0 (mod n). We need to make the LCM of a and n equal to a * n, and since we know that LCM(a, n) = a * n / GCD(a, n), it follows that GCD(a, n) = 1.

Since most hash tables you make will have 2^n elements (this seems to be the standard, for addressing reasons), any odd number a will suffice to be a generator for linear hashes.

Theoretically, which hash value should I use?

Linear hashing is obviously a very simple hash function (the simplest one there is, I believe), and therefore, there is not a single hash fucntion that will work for every input set. In fact, this type of hash will have many input sets that will make it have very poor performance. However, if we have advanced knowledge of the kind of data that will be the input, we can stack the deck in our favor.

If your data is guaranteed to have no collisions (mapping unique integers less than the size of the hash table to some value), you can use any positive integer you want as your hash. I recommend 1 for ease of calculation

If your data is sorted ascending, use hash values close to 1. If you can find the mode of the data in advance, you can yourself by setting the hash value larger than the mode. If the mode is large with respect to the size of the hash table or with respect to the size of the data set, you can make the hash value larger than the average number of repetitions for each input.

If your data is sorted descending, you want to do as above, except make your hash value close to n-1. The reasoning can be derived from the above paragraph.

If your data is either purely random, or of several different varieties, your hash function is not always going to work no matter how hard you try. We should avoid hashes close to 1 and n-1, but other than that, we will need to benchmark to see if there is a better value.