Hadoop V2 - Oozie Configuraton and Job Submissions

In this blog I discuss how to submit example job for Oozie


1. Shutdown oozie [As oozie]
    cd /usr/local/oozie
    bin/oozied.sh stop
   
2. Create Shared Lib [As oozie]
    cd /usr/local/oozie
    bin/ooziebin/oozie-setup.sh sharelib create -fs hdfs://192.168.2.101/users/oozie -locallib oozie-sharelib-5.0.0.tar.gz

3. Put Examples
    hdfs dfs -put examples /user/oozie

4. Update job.properties   
    Updae job.properties (/user/oozie/examples/apps/map-reduce/)  to  match your hadoop configuration and put it again to the same folder.
   

5. Update oozie-site.xml (/user/local/oozie/conf)
    See end of blog for full configuration
   
6. Resart oozie [As oozie user]
    cd /usr/local/oozie
    bin/oozied.sh start
   
7. Edit Capacity Scheduler to Match a queue name for oozie
    Here is my mapping u:oozie:oozie and a queue name oozie
   
8. Submit a workflow

bin/oozie job -oozie http://192.168.1.71:11000/oozie -config examples/apps/map-reduce/job.properties -run
job: 0000002-180507002619511-oozie-oozi-W

9. Verify Workflow Status
Oozie Web Console

 


Oozie CMD
[oozie@oem13cr2 oozie]$ bin/oozie job -oozie http://localhost:11000/oozie -info  0000002-180507002619511-oozie-oozi-W
Job ID : 0000002-180507002619511-oozie-oozi-W
------------------------------------------------------------------------------------------------------------------------------------
Workflow Name : map-reduce-wf
App Path      : hdfs://nn:8020/user/oozie/examples/apps/map-reduce/workflow.xml
Status        : RUNNING
Run           : 0
User          : oozie
Group         : -
Created       : 2018-05-07 04:39 GMT
Started       : 2018-05-07 04:39 GMT
Last Modified : 2018-05-07 04:39 GMT
Ended         : -
CoordAction ID: -

Actions
------------------------------------------------------------------------------------------------------------------------------------
ID                                                                            Status    Ext ID                 Ext Status Err Code
------------------------------------------------------------------------------------------------------------------------------------
0000002-180507002619511-oozie-oozi-W@:start:                                  OK        -                      OK         -
------------------------------------------------------------------------------------------------------------------------------------
0000002-180507002619511-oozie-oozi-W@mr-node                                  RUNNING   application_1525667692407_0002RUNNING    -
------------------------------------------------------------------------------------------------------------------------------------
10. Verify from Resource Manager UI

 
   
Appendix
oozie-site.xml

<property>
     <name>oozie.service.JPAService.jdbc.driver</name>
     <value>oracle.jdbc.driver.OracleDriver</value>
</property>

<property>
     <name>oozie.service.JPAService.jdbc.url</name>
     <value>jdbc:oracle:thin:@192.168.1.71:6633:EMPRD</value>
</property>

<property>
     <name>oozie.service.JPAService.jdbc.username</name>
     <value>oozie</value>
</property>

<property>
     <name>oozie.service.JPAService.jdbc.password</name>
     <value>oozie</value>
</property>
 <property>
        <name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
        <value>*=/etc/hadoop/conf</value>
  </property>
<property>
    <name>oozie.service.HadoopAccessorService.nameNode.whitelist</name>
    <value>nn:8020</value>
</property>

<property>
    <name>oozie.actions.default.name-node</name>
    <value>hdfs://nn:8020</value>
</property>

<property>
    <name>oozie.service.HadoopAccessorService.jobTracker.whitelist</name>
    <value>rm:8032</value>
</property>
<property>
    <name>oozie.actions.default.job-tracker</name>
    <value>rm:8032</value>
</property>