What type of enhancement is this?

User experience

What does the enhancement do?

GreptimeDB, a hybrid time series and analytical database, can manage real-time data ingestion and analytics independently. However, for many established infrastructures, integrating with existing big data systems such as Flink, Kafka, BigQuery, Hadoop, and Spark is essential. This issue focuses on the integration process within the big data ecosystem.

To clarify further, we can categorize these integrations into two parts: GreptimeDB as a data sink and GreptimeDB as a data source.

GreptimeDB as data sink

This direction aims to ingest data from big data softwares to faciliate fast, low cost and efficient analytics.

• Elasticsearch exporter: Export logs in Elasticsearch to GreptimeDB in an efficient way.
• Flink sink: Set up sinks to write streaming data to GreptimeDB.
• Kafka connector: Use Kafka Connect or custom sink connectors to send data to GreptimeDB.

GreptimeDB as data source

• Kafka: Use change data capture (CDC) connectors to stream GreptimeDB updates to Kafka.

Implementation challenges

No response