Command-Line Interface

kafka-python ships simple command-line interfaces for consumer, producer, and admin clients. They can be invoked either as the kafka-python console script or as module entry points:

kafka-python consumer -b localhost:9092 -t my-topic
kafka-python producer -b localhost:9092 -t my-topic
kafka-python admin    -b localhost:9092 cluster describe

# equivalent module invocations
python -m kafka.consumer -b localhost:9092 -t my-topic
python -m kafka.producer -b localhost:9092 -t my-topic
python -m kafka.admin    -b localhost:9092 cluster describe

The kafka-python admin command, in particular, is a convenient alternative to the apache kafka bin/ scripts when a compatible JVM is not available.

• kafka-python consumer
  • Examples
  • Reference
• kafka-python producer
  • Examples
  • Reference
• kafka-python admin
  • Global options
  • acls
  • cluster
  • configs
  • topics
  • partitions
  • groups
  • transactions
  • users

Common Options

All three commands share a common set of connection, logging, and configuration options. They are documented in full on the individual command pages; the summary below highlights the most commonly used flags.

Connection

-b/--bootstrap-servers HOST:PORT

One or more bootstrap servers used to discover the rest of the cluster. May be supplied multiple times.

-S/--security-protocol

One of PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL. Defaults to PLAINTEXT.

-M/--sasl-mechanism

One of PLAIN, GSSAPI, OAUTHBEARER, SCRAM-SHA-256, SCRAM-SHA-512. Defaults to PLAIN.

-U/--sasl-user / -P/--sasl-password

Credentials for SASL PLAIN and SCRAM-* mechanisms.

Logging

-l/--log-level

Python logging level (DEBUG, INFO, …). Defaults to CRITICAL so the CLI is quiet by default.

-L/--enable-logger / -D/--disable-logger

Selectively turn on or off a single logger by name. Both flags may be supplied multiple times.

--log-format

Provide customized log message format string (passed to logging.Formatter).

--log-date-format

Provide customized log date format string (passed to logging.Formatter).

--log-file

Specify file for logging output. If not provided, log output is written to stderr.

--log-config

Path to logging configuration file for full control over handlers, formatters, etc. Supports .json and .ini formats; .yaml requires installing PyYAML separately. If provided, the file is loaded as a logging.config.dictConfig for .json/.yaml files, or logging.config.fileConfig for all others. Any values provided for --log-format, --log-date-format and/or --log-file will be ignored (--enable-logger and --disable-logger still apply as logger level adjustments). See logging.config for details.

Extended Configuration

-C/--extra-config key=value

Pass arbitrary keyword arguments through to the underlying client constructor (KafkaConsumer, KafkaProducer, or KafkaAdminClient). Values that parse as int, True, False, or None are converted; everything else is passed through as a string. May be supplied multiple times.

kafka-python consumer -b localhost:9092 -t foo \
    -C auto_offset_reset=earliest \
    -C consumer_timeout_ms=1000