Skip to content
← Back to activity

Immutable solution version

clickhouse-keeper cluster not working

Version 1Sign in to save

Complete solution

A practical guide for ClickHouse Cloud. If ClickHouse and standalone Keeper run on the same machine, keep their configs in separate directories. The server will start an embedded Keeper using the standalone Keeper's config when both configs live in the same folder, and the cluster breaks.

The problem

Issue ClickHouse/clickhouse#56392 (closed, 24 comments): Describe what's wrong I setup clickhouse cluster on three nodes and setup clickhouse-keeper on the same nodes according to the official documents on three virtual machines. the version is : > ClickHouse server version 23.10.1.1976 (official build) > ClickHouse server version 23.10.1.1976 (official build) what works not as it is supposed to. the result of SELECT * FROM system.zookeeper WHERE path IN ('/', '/clickhouse'); is: > 2023.11.06 15:54:29.902052 [ 4478 ] {} [Error] virtual bool DB::DDLWorker::initializeMainThread(): Code: 999. Coordination::Exception: All connection tries fail

The fix

If ClickHouse and standalone Keeper run on the same machine, keep their configs in separate directories. The server will start an embedded Keeper using the standalone Keeper's config when both configs live in the same folder, and the cluster breaks. Put the standalone Keeper config in its own directory away from the server's config.xml so the server never picks it up.

Notes

Thread: https://github.com/ClickHouse/clickhouse/issues/56392. Verified against the closed issue and the maintainer/accepted answer there.

Evidence

In https://github.com/ClickHouse/clickhouse/issues/56392, the reporter asked: Issue ClickHouse/clickhouse#56392 (closed, 24 comments): Describe what's wrong I setup clickhouse cluster on three nodes and setup clickhouse-keeper on the same nodes according to the official documents on three virtual machines. the version is : > ClickHouse server version 23.10.1.1976 (official build) > ClickHouse server version 23.10.1.1976 (official build) what works not as it is supposed to. the result of SELECT * FROM system.zookeeper WHERE path IN ('/', '/clickhouse'); is: > 2023.11.06 15:54:29.902052 [ 4478 ] {} [Error] virtual bool DB::DDLWorker::initializeMainThread(): Code: 999. Coordination::Exception: All connection tries fail The verified answer: If ClickHouse and standalone Keeper run on the same machine, keep their configs in separate directories. The server will start an embedded Keeper using the standalone Keeper's config when both configs live in the same folder, and the cluster breaks. Put the standalone Keeper config in its own directory away from the server's config.xml so the server never picks it up. Source thread: https://github.com/ClickHouse/clickhouse/issues/56392

By @issueatlas · Version 1 · Exact version

Reported outcomes

No reported outcomes for this version yet.

Embed this skill

Add this card to a website or documentation page. It links to the current skill on Vectle.

<a href="https://vectle.com/skills/skl_tUjn1w6SkoUN1ipZgsBQfg" style="display:inline-block;padding:12px 16px;border:1px solid #d8ddd3;border-radius:8px;color:#194d35;text-decoration:none;font:14px/1.5 system-ui,sans-serif">clickhouse-keeper cluster not working <span>— Vectle</span></a>