중앙 OS 7에 Digdag을 설치해 봤습니다.

작업 흐름 엔진으로 유명한 Apache Airflow와 digdag가 있지만, 이번에 도입해 볼 허들식 digdag가 있다고 생각합니다.

사전 준비

Digdag를 이동하려면 지정된 버전 8u72 이상의 JDK8(Java SE Development Kit8)을 설치해야 합니다.없으면 공식 페이지에서 설치 프로그램을 다운로드합니다.

[root@localhost ~]# rpm -ivh jdk-8u333-linux-x64.rpm
警告: jdk-8u333-linux-x64.rpm: ヘッダー V3 RSA/SHA256 Signature、鍵 ID ec551f03: NOKEY
準備しています...              ################################# [100%]
更新中 / インストール中...
   1:jdk1.8-2000:1.8.0_333-fcs        ################################# [100%]
Unpacking JAR files...
        tools.jar...
        plugin.jar...
        javaws.jar...
        deploy.jar...
        rt.jar...
        jsse.jar...
        charsets.jar...
        localedata.jar...
[root@localhost ~]#

JDK 버전 확인

[root@localhost ~]# java -version
openjdk version "1.8.0_322"
OpenJDK Runtime Environment (build 1.8.0_322-b06)
OpenJDK 64-Bit Server VM (build 25.322-b06, mixed mode)
[root@localhost ~]#

digdag 설치

[root@localhost ]# curl -o /usr/bin/digdag --create-dirs -L "https://dl.digdag.io/digdag-latest"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:06 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:06 --:--:--     0
100 84.5M  100 84.5M    0     0  6416k      0  0:00:13  0:00:13 --:--:-- 11.8M
[root@localhost ]#
[root@localhost bin]# chmod +x /usr/bin/digdag
[root@localhost bin]# which digdag
/usr/bin/digdag

샘플 워크플로우 만들기

[root@localhost ~]# mkdir digdag
[root@localhost ~]# cd digdag
[root@localhost mydag]# ls
mydag.dig
[root@localhost mydag]# digdag run mydag.dig
2022-05-06 17:32:37 +0900: Digdag v0.10.4
2022-05-06 17:32:38 +0900 [WARN] (main): Using a new session time 2022-05-06T00:00:00+00:00.
2022-05-06 17:32:38 +0900 [INFO] (main): Using session /root/digdag/mydag/.digdag/status/20220506T000000+0000.
2022-05-06 17:32:39 +0900 [INFO] (main): Starting a new session project id=1 workflow name=mydag session_time=2022-05-06T00:00:00+00:00
2022-05-06 17:32:40 +0900 [INFO] (0016@[0:default]+mydag+setup): echo>: start 2022-05-06T00:00:00+00:00
start 2022-05-06T00:00:00+00:00
2022-05-06 17:32:41 +0900 [INFO] (0016@[0:default]+mydag+disp_current_date): echo>: 2022-05-06 00:00:00 +00:00
2022-05-06 00:00:00 +00:00
2022-05-06 17:32:41 +0900 [INFO] (0016@[0:default]+mydag+repeat): for_each>: {order=[first, second, third], animal=[dog, cat]}
2022-05-06 17:32:42 +0900 [INFO] (0016@[0:default]+mydag+repeat^sub+for-0=order=0=first&1=animal=0=dog): echo>: first dog
first dog
2022-05-06 17:32:42 +0900 [INFO] (0017@[0:default]+mydag+repeat^sub+for-0=order=0=first&1=animal=1=cat): echo>: first cat
2022-05-06 17:32:42 +0900 [INFO] (0021@[0:default]+mydag+repeat^sub+for-0=order=2=third&1=animal=1=cat): echo>: third cat
third cat
first cat
2022-05-06 17:32:42 +0900 [INFO] (0018@[0:default]+mydag+repeat^sub+for-0=order=1=second&1=animal=0=dog): echo>: second dog
second dog
2022-05-06 17:32:42 +0900 [INFO] (0019@[0:default]+mydag+repeat^sub+for-0=order=1=second&1=animal=1=cat): echo>: second cat
second cat
2022-05-06 17:32:42 +0900 [INFO] (0020@[0:default]+mydag+repeat^sub+for-0=order=2=third&1=animal=0=dog): echo>: third dog
third dog
2022-05-06 17:32:44 +0900 [INFO] (0020@[0:default]+mydag+teardown): echo>: finish 2022-05-06T00:00:00+00:00
finish 2022-05-06T00:00:00+00:00
Success. Task state is saved at /root/digdag/mydag/.digdag/status/20220506T000000+0000 directory.
  * Use --session <daily | hourly | "yyyy-MM-dd[ HH:mm:ss]"> to not reuse the last session time.
  * Use --rerun, --start +NAME, or --goal +NAME argument to rerun skipped tasks.

GUI 액세스 시도

참고로 다음 명령을 통해 GUI에 액세스할 수 있지만 전혀 액세스할 수 없습니다...

[root@localhost mydag]# digdag server -o ./ -n 8081
2022-05-06 17:34:55 +0900: Digdag v0.10.4
2022-05-06 17:34:56 +0900 [INFO] (main): secret encryption engine: disabled
2022-05-06 17:34:57 +0900 [INFO] (main): XNIO version 3.3.8.Final
2022-05-06 17:34:57 +0900 [INFO] (main): XNIO NIO Implementation Version 3.3.8.Final
2022-05-06 17:34:57 +0900 [INFO] (main): Starting server on 127.0.0.1:8081
2022-05-06 17:34:57 +0900 [INFO] (main): Bound on 127.0.0.1:8081 (api)

방법이 없습니다. 오류를 시도한 후 아래 명령을 사용하여 접근할 수 있습니다.
아마도 VMware에 구축된 CentOS일 것입니다. localhost는 안 됩니다.
따라서 -b 옵션을 통해 IP 주소를 지정했습니다.

[root@localhost mydag]# digdag server -o ./ -n 8081 -b 192.168.250.58
2022-05-06 17:36:44 +0900: Digdag v0.10.4
2022-05-06 17:36:46 +0900 [INFO] (main): secret encryption engine: disabled
2022-05-06 17:36:46 +0900 [INFO] (main): XNIO version 3.3.8.Final
2022-05-06 17:36:46 +0900 [INFO] (main): XNIO NIO Implementation Version 3.3.8.Final
2022-05-06 17:36:46 +0900 [INFO] (main): Starting server on 192.168.250.58:8081
2022-05-06 17:36:46 +0900 [INFO] (main): Bound on 192.168.250.58:8081 (api)

순조롭게 방문하였다.