Impala Heartbeat Timeout

안테나의 ImpalaJob 로그에서 오류가 자주 발생합니다.
ERROR:No backends configured
Couldnot execute command:xxx
그리고 계속 오류를 보고하는 것이 아닙니다. statstore의 로그를 보면 다음과 같습니다.

I022316:18:34.338698 10924 state-store.cc:194] Creating new topic:''impala-membership' on behalf of subscriber: 'xxxhostname:22000
I022316:18:34.338739 10924 state-store.cc:200] Registering: xxxhostname:22000
I022316:18:34.339864 10904 state-store.cc:355] Unable to update subscriber atxxxhostname:23000,  received errorCouldn't open transport for xxxhostname:23000(connect() failed: Connectionrefused)
I022316:18:34.840463 10904 state-store.cc:355] Unable to update subscriber atxxxhostname:23000,  received errorCouldn't open transport for xxxhostname:23000(connect() failed: Connectionrefused)
I022316:18:35.341156 10904 state-store.cc:355] Unable to update subscriber atxxxhostname:23000,  received errorCouldn't open transport for xxxhostname:23000(connect() failed: Connectionrefused)
I022316:18:36.843724 10904 state-store.cc:365] Subscriber: xxxhostname:22000 haseither failed or disconnected.
I022316:18:47.536650 10911 state-store.cc:355] Unable to update subscriber atxxxhostname:23000,  received errorCouldn't open transport for xxxhostname:23000(connect() failed: Connectionrefused)
I022316:18:54.343158 10924 state-store.cc:200] Registering: xxxhostname:22000
W022316:18:54.343209 10924 state-store.cc:215] Duplicate registration of subscriber:xxxhostname:22000, possible duplicate subscriber IDs or recovering subscriber

다시 보면 xxxxhostname 역시 등록에 실패했습니다.

I022421:32:38.173282 33088 state-store-subscriber.cc:169] Trying to register...
I022421:32:38.173743 33088 state-store-subscriber.cc:172] Reconnected tostate-store. Exiting recovery mode
I022421:32:48.173959 33088 state-store-subscriber.cc:166] xxxhostname:22000:Connection with state-store lost, entering recovery mode
I022421:32:48.174018 33088 state-store-subscriber.cc:169] Trying to register...
W022421:32:48.174432 33088 state-store-subscriber.cc:181] Failed to re-register withstate-store: Duplicate registration of subscriber: xxxhostname:22000

하트비트 타임아웃으로 추정되지만 문서에 관련 파라미터에 대한 상세한 설명을 찾지 못하고 코드를 뒤져서 관련 파라미터를 열거합니다. 마지막으로statestore_ 수정subscriber_timeout_seconds=60s 리셋이 적용됩니다.

statestore_subscriber_timeout_seconds, 10, "The amount of time (in seconds) that may elapse before the connection with the statestore is considered lost.";
statestore_subscriber_cnxn_attempts, 10, "The number of times to retry an RPC connection to the statestore. A setting of 0 means retry indefinitely";
statestore_subscriber_cnxn_retry_interval_ms, 3000, "The interval, in ms, to wait between attempts to make an RPC connection to the statestore.";
statestore_max_missed_heartbeats, 5, "Maximum number of consecutive heartbeats an impalad can miss before being declared failed by the statestore.";
statestore_suspect_heartbeats, 2, "(Advanced) Number of consecutive heartbeats an impalad can miss before being suspected of failure by the statestore";
statestore_num_heartbeat_threads, 10, "(Advanced) Number of threads used to  send heartbeats in parallel to all registered subscribers.";
statestore_heartbeat_frequency_ms, 500, "(Advanced) Frequency (in ms) with which the statestore sends heartbeats to subscribers.";