Hive 쓰기 ES 오류 수정 실패: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

HIVE 통합 ES 장면:
ES 버전: 6.5.2
Hive 버전: 2.3.4
플러그인 패키지 다운로드 주소:https://repo.maven.apache.org/maven2/org/elasticsearch/elasticsearch-hadoop-hive/
통합 프로세스 네트워크의 많은 자료를 자신의 ES 버전과 일치하는 플러그인 패키지를 다운로드한 다음elasticsearch-hadoop-hive-6.5.2.jar는hive 설치 디렉터리lib로 복사합니다.왜냐하면 제 하이브는 이미 있어요.
commons-httpclient-3.0.1.jar 파일이므로 중복 복사를 할 필요가 없습니다.
hivesql를 실행하여 테이블 데이터를 elasticsearch에 쓰는 오류는 다음과 같습니다.

2020-05-30 14:08:12】org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
	at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:379)
	at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:257)
	at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
	at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:348)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
	at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:362)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

이 로그는 명확하지 않습니다. yarn 관리 페이지에서 구체적인 MR 작업을 찾으면 로그 신문에서 es의 클래스 오류를 찾을 수 없습니다. 로그는 다음과 같습니다.
 

Caused by: java.lang.ClassNotFoundException: org.elasticsearch.hadoop.hive.EsHiveInputFormat
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154)
	... 60 more

2020-05-30 14:56:01,845 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1590680711550_0090_m_000000_0: Error: java.lang.RuntimeException: Failed to load plan: hdfs://SINOTRUK/tmp/hive/hive/c7f92979-e7cc-4a39-beba-d8007ddcc817/hive_2020-05-30_14-55-44_617_4414312671758408146-2/-mr-10002/7db2e153-f315-4f63-9886-0774d2f85bf0/map.xml
	at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:481)
	at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:313)
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:394)
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:665)
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:658)
	at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:692)
	at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.(MapTask.java:175)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:444)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find class: org.elasticsearch.hadoop.hive.EsHiveInputFormat

mapreduce 임무를 수행하기 때문에, 분명히 (elasticsearch-hadoop-hive-6.5.2.jar)와(commons-httpclient-3.0.1.jar) 패키지를 특정한 노드에 놓으면 안 된다.그래서 (elasticsearch-hadoop-hive-6.5.2.jar)와(commons-httpclient-3.0.1.jar) 패키지를 모든hadoop 설치 디렉터리의lib에 복사해야 합니다.Hadoop과hive를 다시 시작하면 됩니다.
복사 (commons-httpclient-3.0.1.jar) 패키지가 없으면 다음 오류가 발생합니다.

2020-05-30 15:22:51,047 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1590680711550_0092_m_000000_1: Error: java.lang.ClassNotFoundException: org.apache.commons.httpclient.Credentials
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransportFactory.create(CommonsHttpTransportFactory.java:40)
	at org.elasticsearch.hadoop.rest.NetworkClient.selectNextNode(NetworkClient.java:102)
	at org.elasticsearch.hadoop.rest.NetworkClient.(NetworkClient.java:85)
	at org.elasticsearch.hadoop.rest.NetworkClient.(NetworkClient.java:61)
	at org.elasticsearch.hadoop.rest.RestClient.(RestClient.java:94)
	at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:615)
	at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.init(EsOutputFormat.java:173)
	at org.elasticsearch.hadoop.hive.EsHiveOutputFormat$EsHiveRecordWriter.write(EsHiveOutputFormat.java:58)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:762)
	at org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.process(VectorFileSinkOperator.java:101)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
	at org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:137)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
	at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
	at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.closeOp(VectorMapOperator.java:900)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697)
	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:465)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)

요약:hivesql가 제출되면 yarn은 MR 임무를 특정한 노드에 나누어 실행하기 때문에 이 노드의 Hadoop 아래에 (elasticsearch-hadoop-hive-6.5.2.jar)와(commons-httpclient-3.0.1.jar) 이 두 개의 가방이 이상 방출됩니다.
건의: (elasticsearch-hadoop-hive-6.5.2.jar)과 (commons-httpclient-3.0.1.jar) 이 두 패키지를 집단적인hadoop 설치 디렉터리의lib와hive 설치 디렉터리의lib로 복사합니다.