Himage Usage

Create hadoop.tar.gz

For now, assuming a single ‘hadoop’ directory (not three: ‘hadoop-common’, ‘hadoop-hdfs’ and ‘hadoop-mapreduce’).

cd hadoop
ant clean tar

Create hbase.tar.gz

cd hbase
mvn clean package

Preliminaries

>> $:.unshift("~/hbase-ec2/lib")
=> ["~/hbase-ec2/lib", ... ]
>> load 'hcluster.rb'
=> true
>> include Hadoop
=> Object

Uploading tar files from a local directory to S3:

>> builder = Himage.new :hbase => "/home/ekoontz/hbase/build/hbase-0.20-tm-3.tar.gz", 
                        :hadoop => "/home/ekoontz/hadoop/build/hadoop-0.20-tm-3.tar.gz", 
                        :tar_s3 => "ekoontz-tarballs",
                        :ami_s3 => "ekoontz-amis"
=> #<Hadoop::Himage:0x101f79700 ...>

The above command will upload the tars to the S3 bucket specified by :tar_s3.

Creating image

>> builder.create_image
Creating and registering image: hbase-0.20-tm-3
Starting a AMI with ID: ami-b00c34d9..

The above AMI (ami-b00c34d9) is a constant, publically available CentOS version that we use to build our image:

 ami-b00c34d9 => rightscale-us-east/RightImage_CentOS_5.4_x64_v5.4.6.2_Beta.manifest.xml

The return value of the create_image() call above is a string containing the name of the AMI that was just created:

=> "ami-977893fe"

You can now use your image in a new cluster as follows (supposing the AMI that was just created is called ami-977893fe, as in our example output above):

cluster = HCluster.new :ami => 'ami-977893fe'