kafka consumer patterns with groups -


i trying head around possible consumer patterns kafka 0.8.1.1. let's ignore replication because dont believe affects these patterns. ran command line console consumer tests.

can please confirm understanding correct?

1) 1 topic, many partitions m, many consumers n, m=>n, no groups defined.

in case every consumer every message on topic.

2) 1 topic, many partitions m, many consumers n m less n , no groups defined.

same behavior. every consumer every message on topic.

3) 1 topic, many partitions m, many consumers n, m less n , 1 consumer group defined consumers

i see "no broker partitions consumed consumer thread" on 1 of consumer consoles. is because there more consumers partitions? ( in case m=3, n=4 )

4) 1 topic, many partitions m, many consumers n, m=n , 1 consumer group defined consumers

from using kafka monitor, see each partition assigned 1 consumer now. however, seems there no parallelism in data consumption. see happening 1 consumer gets messages time t0 t1 partition p1. consumer gets messages t1 t2 partition p2 , on.

why there no parallel consumption happening ? looks me producer's data goes p1 t0 t1 , t1 t2 p2. thought if dont specify partitioning key, producer's data partitioned randomly. it's randomness seems "delayed". why ?

i think 2 links understand happening :

should choose multiple group ids or single 1 consumers?

why of consumers in consumer group never receive message?

if don't specifies partition key, won't random stick time 1 partition period , round robin partition next period. default period 10 minutes i think can change topic.metadata.refresh.interval.ms


Comments

Popular posts from this blog

database - VFP Grid + SQL server 2008 - grid not showing correctly -

jquery - Set jPicker field to empty value -

.htaccess - htaccess convert request to clean url and add slash at the end of the url -