The Author Online Book Forums are Moving

The Author Online Book Forums will soon redirect to Manning's liveBook and liveVideo. All book forum content will migrate to liveBook's discussion forum and all video forum content will migrate to liveVideo. Log in to liveBook or liveVideo with your Manning credentials to join the discussion!

Thank you for your engagement in the AoF over the years! We look forward to offering you a more enhanced forum experience.

Paolo Patierno (13) [Avatar] Offline

in section 2.6.6 Rebalancing, could it be worth to mention that new consumers get partitions from the current ones in the consumer group until the number of consumers is equal to the number of partitions; after hitting this limit, a new consumer will be idle with no partitions assigned.

Bill Bejeck (47) [Avatar] Offline

Sumant Tambe (17) [Avatar] Offline
I don't think the suggestion made here is accurate. The number of partitions have pretty much nothing to do with the number of consumers. Except that having more consumers than the number of partitions is useless. A single consumer can consume 0 to all partitions. Rebalancing is complete when all partitions have been assigned a (any) consumer. It may be so that all partitions are consumed by a single consumer if there's only one consumer in the group.
Sumant Tambe (17) [Avatar] Offline
After reading the initial suggestion more carefully, I realized that we're saying the same thing, However, its confusing to read about the number of consumers equal to number of partitions. It could be misleading.
Bill Bejeck (47) [Avatar] Offline
Hi Sumant,

Thanks for taking the time to comment.

Sorry that it's a bit confusing. The reason I mentioned a consumer per partition is that in practice, I've always used a consumer in a separate thread per partition. I'll take another look at 2.6.6 and see if I can make things a little more clear.