kafka.coordinator.assignors package

Submodules

kafka.coordinator.assignors.abstract module

class kafka.coordinator.assignors.abstract.AbstractPartitionAssignor

Bases: object

Abstract assignor implementation which does some common grunt work (in particular collecting partition counts which are always needed in assignors).

assign(cluster, members)

Perform group assignment given cluster metadata and member subscriptions

Parameters:

• cluster (ClusterMetadata) – metadata for use in assignment
• (dict of {member_id (members) – MemberMetadata}): decoded metadata for each member in the group.

Returns:

{member_id: MemberAssignment}

Return type:

dict

metadata(topics)

Generate ProtocolMetadata to be submitted via JoinGroupRequest.

Parameters:topics (set) – a member’s subscribed topics Returns:MemberMetadata struct

name

.name should be a string identifying the assignor

on_assignment(assignment)

Callback that runs on each assignment.

This method can be used to update internal state, if any, of the partition assignor.

Parameters:assignment (MemberAssignment) – the member’s assignment

kafka.coordinator.assignors.roundrobin module

class kafka.coordinator.assignors.roundrobin.RoundRobinPartitionAssignor

Bases: kafka.coordinator.assignors.abstract.AbstractPartitionAssignor

The roundrobin assignor lays out all the available partitions and all the available consumers. It then proceeds to do a roundrobin assignment from partition to consumer. If the subscriptions of all consumer instances are identical, then the partitions will be uniformly distributed. (i.e., the partition ownership counts will be within a delta of exactly one across all consumers.)

For example, suppose there are two consumers C0 and C1, two topics t0 and t1, and each topic has 3 partitions, resulting in partitions t0p0, t0p1, t0p2, t1p0, t1p1, and t1p2.

The assignment will be:

C0: [t0p0, t0p2, t1p1] C1: [t0p1, t1p0, t1p2]

classmethod assign(cluster, member_metadata)

classmethod metadata(topics)

name = 'roundrobin'

classmethod on_assignment(assignment)

version = 0

Module contents