Hadoop V2 - Software Setup and Environment Configuration

This in continuation with my previous blog on Hadoop V2 Pre-req Setup

Step 1
[As root -  Namenode - Download  Hadoop Binaries]
Download Hadoop Binaries
wget http://www-eu.apache.org/dist/hadoop/common/hadoop-2.7.5/hadoop-2.7.5.tar.gz

Step 2
[As root -  Namenode - Send Hadoop Binaries]
# for i in $(cat /tmp/all_hosts) ; do echo "scp /tmp/hadoop-2.7.5.tar.gz ${i}:/tmp &" >> /tmp/sendhdpv1.bash ; done
bash /tmp/sendhdpv1.bash


Step 3
[As root - Extract Hadoop]
#for i in $(cat /tmp/all_hosts) ;do ssh ${i} tar -xzf /tmp/hadoop-2.7.5.tar.gz -C /usr/local        --overwrite ; done

Step 4
[As root - Setup sudoers configuration]
#for i in $(cat /tmp/all_hosts) ; do ssh ${i}  echo '"hdfs        ALL=(ALL)       NOPASSWD: ALL" >> /etc/sudoers'; done


Step 5
[As root - Create Conf Directory]
#for i in $(cat /tmp/all_hosts) ;do ssh ${i} mkdir /etc/hadoop; done

[As root - Move the conf directory]
# for i in $(cat /tmp/all_hosts) ;do ssh ${i} mv /usr/local/hadoop-2.7.5/etc/hadoop /etc/hadoop/conf; done

[As root - Give Permissions]
# for i in $(cat hosts) ;do ssh ${i} chmod -R 755 /etc/hadoop ; done;


# for i in $(cat /tmp/all_hosts) ; do ssh ${i} 'mkdir -p /opt/HDPV2/logs  ; echo' ;  done;

# for i in $(cat /tmp/all_hosts) ; do ssh ${i} 'chmod 775  /opt/HDPV2/logs; echo' ;  done;

# for i in  $(cat /tmp/all_hosts) ; do ssh ${i} 'chown hdfs:hadoop  /opt/HDPV2/logs; echo' ;  done;

# for i in $(cat /tmp/all_hosts) ; do ssh ${i} 'mkdir -p /opt/HDPV2/pids ; echo' ;  done;

# for i in $(cat /tmp/all_hosts) ; do ssh ${i} 'chmod 775  /opt/HDPV2/pids; echo' ;  done;

# for i in  $(cat /tmp/all_hosts) ; do ssh ${i} 'chown hdfs:hadoop  /opt/HDPV2/pids; echo' ;  done;

# for i in $(cat /tmp/all_hosts) ; do ssh ${i} 'mkdir -p  /opt/HDPV2/1 /opt/HDPV2/2 /opt/HDPV2/tmp  ; echo' ;  done;

# for i in $(cat /tmp/all_hosts) ; do ssh ${i} 'chmod 775  /opt/HDPV2/1 /opt/HDPV2/2 /opt/HDPV2/tmp ; echo' ;  done;

# for i in  $(cat /tmp/all_hosts) ; do ssh ${i} 'chown hdfs:hadoop  /opt/HDPV2/1 /opt/HDPV2/2 /opt/HDPV2/tmp ; echo' ;  done;



[As root - Create Soft Link to Hadoop]
# for i in $(cat /tmp/all_hosts) ;do ssh ${i} ln -s /usr/local/hadoop-2.7.5 /usr/local/hadoop  ; done


[As root - Create Soft Link to Conf]
# for i in $(cat /tmp/all_hosts) ;do ssh ${i} ln -s /etc/hadoop/conf /usr/local/hadoop-2.7.5/etc/hadoop ; done