Inquiry about Iceberg Zero-ETL Integration Support and Configuration in InfluxDB 3.0 OSS Environment

InfluxDB 3
1

Hello,
This is Namuk Kim from LS_THIRAUTECH.

We are currently building and testing a data lake environment with the following stack:

  • InfluxDB 3.0 (OSS, containerized)
  • MinIO (S3-compatible object storage)
  • Apache Iceberg (JDBC Catalog: PostgreSQL)
  • Trino (SQL analytics engine)

InfluxDB 3.0 is integrated with MinIO, and we are inserting data using the influxdb3 CLI as follows:

text

influxdb3 write --database IFC --precision ns 'tb_eqp_param_mset,eqp_id=EQP_01 body="{\"val01\":12.34}" 1720000000000000000'
  • DATABASE: IFC
  • TABLE NAME: tb_eqp_param_mset

According to the official documentation and blog posts, InfluxDB 3.x supports Zero-ETL integration with Iceberg (i.e., InfluxDB data is automatically exposed as Iceberg tables and can be queried directly from Trino and similar engines without additional ETL). Based on this, we have tested the following:

  1. Even after inserting data into InfluxDB, no Iceberg metadata (i.e., /metadata directory) is created in the MinIO (s3://warehouse) path, and the Iceberg table is not automatically generated.
  2. Even if we manually create the Iceberg table in Trino, no data appears in the table.

sql

CREATE TABLE iceberg.IFC.tb_eqp_param_mset (
  eqp_id VARCHAR,
  time TIMESTAMP(6),
  body VARCHAR
)
WITH (
  partitioning = ARRAY['hour(time)'],
  location = 's3://warehouse/tb_eqp_param_mset'
);

Questions

  • Are there any additional configurations or procedures required for Iceberg Zero-ETL integration to work properly with InfluxDB 3.0?
  • In the above OSS environment, what are the necessary conditions or settings (e.g., Iceberg catalog, object storage options, permissions, etc.) to enable automatic linkage from InfluxDB to Iceberg tables?
  • Is this feature unsupported in the InfluxDB 3.0 OSS version and only available in Cloud Dedicated/Clustered or other enterprise environments?

I would appreciate any guidance on whether Iceberg Zero-ETL integration is possible in the OSS environment, or if it is only supported in commercial environments, as well as any specific configuration steps or requirements for enabling automatic integration.

Thank you very much.

2

Hi Namuk Kim,

Thanks for these important questions. Current integration between InfluxDB 3 and Apache Iceberg is in the works including processing engine plugin for the same etc. More information will be shared soon once it comes out publicly. Please let us know if any other questions etc.