Symptoms -
Sqoop job fails when importing with avro type format.
Container logs show
org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoSuchMethodError: org.apache.avro.reflect.ReflectData.addLogicalTypeConversion(Lorg/apache/avro/Conversion;)V
Solution - Run the job by adding arguements to the comand line -
-Dmapreduce.job.user.classpath.first=true
sqoop import-all-tables -Dmapreduce.job.user.classpath.first=true --connect jdbc:oracle:thin:@192.168.1.71:6633:EMPRD \
--username SCOTT --password TIGER -m 1 -as-avrodatafile
This will make sure the right classes are accesible to sqop job during import and MR job.
No comments:
Write comments