meta之前辈kafka
?
6.?Producer
?
6.1?Automatic producer load balancing
Kafka supports client-side load balancing for message producers or use of a dedicated load balancer to balance TCP connections.??
The advantage of using a level-4 load balancer is that each producer only needs a single TCP connection, and no connection to zookeeper is needed.??The disadvantage is that the balancing is done at the TCP connection level, and hence it may not be well balanced (if some producers produce many more messages then others, evenly dividing up the connections per broker may not result in evenly dividing up the messages per broker).
Client-side zookeeper-based load balancing solves some of these problems. It allows the producer to dynamically discover new brokers, and balance load on a per-request basis.?It allows the producer to partition data according to some key instead of randomly.
The working of the zookeeper-based load balancing is described below. Zookeeper watchers are registered on the following events—
-a new broker comes up
-a broker goes down
-a new topic is registered
-a broker gets registered for an existing topic
Internally, the producer maintains an elastic pool of connections to the brokers, one per broker.?This pool is kept updated to establish/maintain connections to all the live brokers, through the zookeeper watcher callbacks. When a producer request for a particular topic comes in, a broker partition is picked by the partitioner (see section on semantic partitioning). The available producer connection is used from the pool to send the data to the selected broker partition.
producer通过zk，管理与broker的连接。当一个请求，根据partition rule 计算分区，从连接池选择对应的connection，发送数据。
?
6.2?Asynchronous send
Asynchronous non-blocking operations are fundamental to scaling messaging systems.
This allows buffering of produce requests in a in-memory queue and batch sends that are triggered by a time interval or a pre-configured batch size.?
?
6.3?Semantic partitioning
The producer has the capability to be able to semantically map messages to the available kafka nodes and partitions.?
This allows partitioning the stream of messages with some semantic partition function based on some key in the message to spread them over broker machines.?
?
?
相关资料：
Meta：https://github.com/killme2008/Metamorphosis?【推荐】
raid5：http://baike.baidu.com/view/969385.htm
磁盘种类：http://www.china001.com/show_hdr.php?xname=PPDDMV0&dname=66IP341&xpos=172
?
