Hi Team,
I am trying to run Jikkou within docker image - streamthoughts/jikkou-api-server:main in mac and it is running fine. Now I would like to automate kafka topic creation by passing a config yml file -
My kafka cluster is running locally using localhost:9092.
I am using below yml file to run the topic to be created in kafka cluster -
apiVersion: io.streamthoughts.jikkou/v1beta1
kind: ResourceList
resources:
# Kafka connection configuration
- apiVersion: jikkou.io/v1beta1
kind: KafkaConfiguration
metadata:
name: kafka-cluster
spec:
bootstrap.servers: "localhost:9092"
security.protocol: PLAINTEXT
# Define a Kafka topic
- apiVersion: kafka.jikkou.io/v1beta1
kind: KafkaTopic
metadata:
name: test-topic-20012025
spec:
partitions: 3
replicas: 2
config:
cleanup.policy: "delete"
retention.ms: "604800000" # 7 days
max.message.bytes: "1048576"
My docker file -
# Use the base image for Jikkou API Server
FROM streamthoughts/jikkou-api-server:main
COPY kafka-topics.yml /config/kafka-topics.yml
# Default command to apply Kafka topic configurations using Jikkou
CMD ["jikkou", "apply", "--files", "/config/kafka-topics.yml"]
Now I am able to run Jikkou on my machine but could not able to connect to the kafka cluster.
below is my admin config -
AdminClientConfig values:
auto.include.jmx.reporter = true
bootstrap.controllers = []
bootstrap.servers = [localhost:9092]
client.dns.lookup = use_all_dns_ips
client.id =
connections.max.idle.ms = 300000
default.api.timeout.ms = 60000
enable.metrics.push = true
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 2147483647
retry.backoff.max.ms = 1000
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.connect.timeout.ms = null
sasl.login.read.timeout.ms = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.login.retry.backoff.max.ms = 10000
sasl.login.retry.backoff.ms = 100
sasl.mechanism = GSSAPI
sasl.oauthbearer.clock.skew.seconds = 30
sasl.oauthbearer.expected.audience = null
sasl.oauthbearer.expected.issuer = null
sasl.oauthbearer.jwks.endpoint.refresh.ms = 3600000
sasl.oauthbearer.jwks.endpoint.retry.backoff.max.ms = 10000
sasl.oauthbearer.jwks.endpoint.retry.backoff.ms = 100
sasl.oauthbearer.jwks.endpoint.url = null
sasl.oauthbearer.scope.claim.name = scope
sasl.oauthbearer.sub.claim.name = sub
sasl.oauthbearer.token.endpoint.url = null
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
socket.connection.setup.timeout.max.ms = 30000
socket.connection.setup.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
ssl.endpoint.identification.algorithm = https
ssl.engine.factory.class = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.certificate.chain = null
ssl.keystore.key = null
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLSv1.3
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.certificates = null
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
10:27:02.791 [scheduled-executor-thread-1] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version: 3.7.0
10:27:02.792 [scheduled-executor-thread-1] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId: 2ae524ed625438c5
10:27:02.792 [scheduled-executor-thread-1] INFO o.a.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1737455222791
10:27:02.794 [scheduled-executor-thread-1] INFO i.s.j.k.internals.KafkaBrokersReady - Checking for Kafka to be ready. Expected broker(s): 1
10:27:32.799 [kafka-admin-client-thread | adminclient-52] INFO o.a.k.c.a.i.AdminMetadataManager - [AdminClient clientId=adminclient-52] Metadata update failed
org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: fetchMetadata
10:28:02.800 [kafka-admin-client-thread | adminclient-52] INFO o.a.k.c.a.i.AdminMetadataManager - [AdminClient clientId=adminclient-52] Metadata update failed
I am able to create the same topic using cli command -
Please help me here what I am doing wrong - I am guessing my configuration is wrong or provide me a sample config and docker file.
Hi Team,
I am trying to run Jikkou within docker image - streamthoughts/jikkou-api-server:main in mac and it is running fine. Now I would like to automate kafka topic creation by passing a config yml file -
My kafka cluster is running locally using
localhost:9092.I am using below yml file to run the topic to be created in kafka cluster -
My docker file -
Now I am able to run Jikkou on my machine but could not able to connect to the kafka cluster.
below is my admin config -
I am able to create the same topic using cli command -
Please help me here what I am doing wrong - I am guessing my configuration is wrong or provide me a sample config and docker file.