Confluent Platform for Flink¶
The official product documentation after 12/2024 release is here.
The main features are:
- Fully compatible with open-source Flink.
- Deploy on Kubernetes using Helm
- Define environment, which groups applications
- Deploy application with user interface and task manager cluster
- Exposes custom kubernetes operator for specific CRD
The figure below presents the Confluent Flink components deployed on Kubernetes:
- CFK supports the management of custom CRD, based on the Flink for Kubernetes Operator
- CMF (Confluent Manager for Apache Flink) adds security control, and a REST API server for the cli or a HTTP client
- FKO is the open source Flink for Kubernetes Operator
- Flink cluster are created from command and CRDs and run Flink applications within an environment
Be sure to have confluent cli.
See dedicated session for Kubernetes deployment
Important source of information for deployment¶
- Deployment overview and for Apache Flink.
- CP Flink supports K8s HA only.
- Flink fine-grained resource management documentation.
Metadata management service for RBAC¶
- Metadata Service Overview
- Single broker Kafka+MDS Deployment
- Git repo with working CP on K8s deployment using RBAC via SASL/Plain and LDAP
- Configure CP Flink to use MDS for Auth
- Additional Flink RBAC Docs
- How to secure a Flink job with RBAC
- Best Practices for K8s + RBAC