OSX의 Hadoop 환경에서 Apache Flink를 실행해보십시오.
Flink는 무엇입니까?
다운로드 & 배포
/usr/local2/
아래로 확장$ tar zxvf ~/Download/flink-0.9.0-bin-hadoop2.tgz -C /usr/local2/
Flink 시작
시작시 환경 변수에 HADOOP_CONF_DIR 설정
$ HADOOP_CONF_DIR=/usr/local2/hadoop-2.7.0/etc/hadoop /usr/local2/flink-0.9.0/bin/yarn-session.sh -n 1 -jm 1024 -tm 1024
즐거움으로 출력
20:15:47,636 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
20:15:47,739 INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032
20:15:47,759 INFO org.apache.flink.yarn.FlinkYarnClient - Using values:
20:15:47,760 INFO org.apache.flink.yarn.FlinkYarnClient - TaskManager count = 1
20:15:47,760 INFO org.apache.flink.yarn.FlinkYarnClient - JobManager memory = 1024
20:15:47,760 INFO org.apache.flink.yarn.FlinkYarnClient - TaskManager memory = 1024
20:15:48,040 INFO org.apache.flink.yarn.Utils - Copying from file:/usr/local2/flink-0.9.0/lib/flink-dist-0.9.0.jar to hdfs://localhost:9000/user/kodai_abe/.flink/application_1435403663845_0002/flink-dist-0.9.0.jar
20:15:48,966 INFO org.apache.flink.yarn.Utils - Copying from /usr/local2/flink-0.9.0/conf/flink-conf.yaml to hdfs://localhost:9000/user/kodai_abe/.flink/application_1435403663845_0002/flink-conf.yaml
20:15:48,980 INFO org.apache.flink.yarn.Utils - Copying from file:/usr/local2/flink-0.9.0/lib/flink-python-0.9.0.jar to hdfs://localhost:9000/user/kodai_abe/.flink/application_1435403663845_0002/flink-python-0.9.0.jar
20:15:48,990 INFO org.apache.flink.yarn.Utils - Copying from file:/usr/local2/flink-0.9.0/conf/logback.xml to hdfs://localhost:9000/user/kodai_abe/.flink/application_1435403663845_0002/logback.xml
20:15:49,000 INFO org.apache.flink.yarn.Utils - Copying from file:/usr/local2/flink-0.9.0/conf/log4j.properties to hdfs://localhost:9000/user/kodai_abe/.flink/application_1435403663845_0002/log4j.properties
20:15:49,018 INFO org.apache.flink.yarn.FlinkYarnClient - Submitting application master application_1435403663845_0002
20:15:49,036 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1435403663845_0002 to ResourceManager at /0.0.0.0:8032
20:15:49,036 INFO org.apache.flink.yarn.FlinkYarnClient - Waiting for the cluster to be allocated
20:15:49,038 INFO org.apache.flink.yarn.FlinkYarnClient - Deploying cluster, current state ACCEPTED
20:15:50,042 INFO org.apache.flink.yarn.FlinkYarnClient - Deploying cluster, current state ACCEPTED
20:15:51,047 INFO org.apache.flink.yarn.FlinkYarnClient - Deploying cluster, current state ACCEPTED
20:15:52,051 INFO org.apache.flink.yarn.FlinkYarnClient - Deploying cluster, current state ACCEPTED
20:15:53,055 INFO org.apache.flink.yarn.FlinkYarnClient - YARN application has been deployed successfully.
20:15:53,232 INFO org.apache.flink.yarn.FlinkYarnCluster - Start actor system.
20:15:53,236 INFO org.apache.flink.runtime.net.NetUtils - Determined /192.168.100.101 as the machine's own IP address
20:15:53,465 INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
20:15:53,495 INFO Remoting - Starting remoting
20:15:53,606 INFO Remoting - Remoting started; listening on addresses :[akka.tcp://[email protected]:62167]
20:15:53,610 INFO org.apache.flink.yarn.FlinkYarnCluster - Start application client.
Flink JobManager is now running on 192.168.100.101:62160
JobManager Web Interface: http://frkout-mbp15-l13.local:8088/proxy/application_1435403663845_0002/
Number of connected TaskManagers changed to 1. Slots available: 1
출력에 있는 Flink JobManager WebInterface의 URL에 액세스합니다.
JobManager Web Interface: http://frkout-mbp15-l13.local:8088/proxy/application_1435403663845_0002/
Flink 직업 던져보십시오
HDFS에 있는 파일에 Example에 있는 WordCount를 걸어 봅니다.
$ /usr/local2/flink-0.9.0/bin/flink run /usr/local2/flink-0.9.0/examples/flink-java-examples-0.9.0-WordCount.jar hdfs://localhost:9000/user/kodai_abe/hadoop-env.sh hdfs://localhost:9000/user/kodai_abe/flink_wordcount_out
JobManage WebInterface에서 실행 계획을 확인할 수 있습니다.
Reference
이 문제에 관하여(OSX의 Hadoop 환경에서 Apache Flink를 실행해보십시오.), 우리는 이곳에서 더 많은 자료를 발견하고 링크를 클릭하여 보았다 https://qiita.com/frkout/items/76c85347830736b38168텍스트를 자유롭게 공유하거나 복사할 수 있습니다.하지만 이 문서의 URL은 참조 URL로 남겨 두십시오.
우수한 개발자 콘텐츠 발견에 전념 (Collection and Share based on the CC Protocol.)