Skip to main content

ClickHouse

A fresh implementation of ClickHouse leveraging our new CDK.

Improvements over v1

  • All sync modes supported
  • Data will be typed and written to columns matching the defined schema (Direct Load)
  • Performance improvements
  • Actively maintained and developed by Airbyte

Features

All sync modes are supported.

FeatureSupported?(Yes/No)Notes
Full Refresh SyncYes
Incremental - Append SyncYes
Incremental - Append + DedupedYesLeverages ReplacingMergeTree
NamespacesYes

Output Schema

Each stream will be output into its own table in ClickHouse in either the configured default database (default) or a database corresponding to the specified namespace on the stream.

Airbyte types will be converted to ClickHouse types as follows:

  • Decimal types are NUMBER128(9) — 9 digit precision
  • Timestamp are DateTime64(3) — millisecond precision
  • Object types are JSON if JSON is enabled in the actor config; otherwise they are converted to String
  • Integers are Int64
  • Booleans are Bool
  • Strings are String
  • Unions will be converted to String
  • Arrays will be converted to String

Requirements

To use the ClickHouse destination, you'll need:

  • A cloud ClickHouse instance
  • A ClickHouse server version 21.8.10.19 or above

Configure Network Access

Make sure your ClickHouse database can be accessed by Airbyte. If your database is within a VPC, you may need to allow access from the IP you're using to expose Airbyte.

Permissions

You need a ClickHouse user with the following permissions:

  • can create tables and write rows.
  • can create databases e.g:

You can create such a user by running:

GRANT CREATE ON * TO airbyte_user;
GRANT CREATE ON {your configured default database} * TO airbyte_user;
GRANT DROP ON * TO airbyte_user;
GRANT TRUNCATE ON * TO airbyte_user;
GRANT INSERT ON * TO airbyte_user;
GRANT SELECT ON * TO airbyte_user;
GRANT CREATE DATABASE ON airbyte_internal.* TO airbyte_user;
GRANT CREATE TABLE ON airbyte_internal.* TO airbyte_user;
GRANT DROP ON airbyte_internal.* TO airbyte_user;
GRANT TRUNCATE ON airbyte_internal.* TO airbyte_user;
GRANT INSERT ON airbyte_internal.* TO airbyte_user;
GRANT SELECT ON airbyte_internal.* TO airbyte_user;

You can also use a pre-existing user but we highly recommend creating a dedicated user for Airbyte.

Reference

Config fields reference

Field
Type
Property name
string
database
boolean
enable_json
string
host
string
password
string
port
string
protocol
string
username

Changelog

Expand to review
VersionDatePull RequestSubject
2.0.12025-07-10#62906Adds bespoke validation for legacy hostnames that contain a protocol.
2.0.02025-07-10#62887Cut 2.0.0 release. Replace existing connector.
0.1.112025-07-09#62883Only set JSON properties on client if enabled to support older CH deployments.
0.1.102025-07-08#62861Set user agent header for internal CH telemetry.
0.1.92025-07-03#62509Simplify union stringification behavior.
0.1.82025-06-30#62100Add JSON support.
0.1.72025-06-24#62047Remove the use of the internal namespace.
0.1.62025-06-24#62047Hide protocol option when running on cloud.
0.1.52025-06-24#62043Expose database protocol config option.
0.1.42025-06-24#62040Checker inserts into configured DB.
0.1.32025-06-24#62038Allow the client to connect to the resolved DB.
0.1.22025-06-23#62028Enable the registry in OSS and cloud.
0.1.12025-06-23#62022Publish first beta version and pin the CDK version.
0.1.02025-06-23#62024Release first beta version.