Hadoop Assignment 1
The first map-reduce assignment is to modify my prime checking program
to not only determine if a number is prime, but also its prime
factors. The prime factors of a number are all the prime numbers that
evenly divides it. For example, 28 = 2*2*7, so its prime factors are
2 and 7. If a number is already prime, then its only prime factor is
itself. The key,value output of your reducer should indicate what the
prime factors are. For example, it could be something like:
28 false: 2,7
or
17 true: 17
As a hint, instead of BooleanWritable, you need to create a new Writable class
for your output values.
////////////////// ADDITIONAL REQUIREMENT ////////////////////
Instead of using int or long, use the java.math.BigInteger class.
A BigInteger can be created from an object, so this class does not need to
be a Writable. Instead of LongWritable, use Text as the Writeable class.
The BigInteger class is well documented.
***IMPORTANT: To determine if a number n is prime, you just need to
divide it by all primes up to sqrt(n). However, to find all the prime
*factors*, you need to check up to n/2. You must modify all relevant
parts of my program accordingly.
However, even if the number being checked (candidate) is greater than n*2, you should still
run the program, since it might still be able to determine if the number is NOT prime, and
output at least some of its factors.
You should modify the second, more advanced version of my program, which uses
an intermediate combiner.
You need to download and unzip the files I used from
http://primes.utm.edu/lists/small/millions/
You should download at least the first 20 million primes. If you download
more, be sure to adjust the maxprime value accordingly. Place all the
files into one directory.
However, you can also create a small file of primes to test your program first.
Be sure to include the following number in your tests and report its
prime factors:
8083734431887505
Please don't try to break any encryption algorithms using this program :)
Additional hints:
Unless you're running the stand-alone installation, System.out.print... won't work as you
expect - so to debug: throw new Error("something weird happened!");
/////////////////
NOTE: YOU NEED TO MODIFY MY PROGRAM TO ACCOMODATE MORE PRIMES