Kafka(1)Install on Redhat and Try Java Client
Kafka(1)Install on Redhat and Try Java Client
1. Install and Try Kafka
>wget http://people.apache.org/~nehanarkhede/kafka-0.7.0-incubating/kafka-0.7.0-incubating-src.tar.gz
>tar zxvf kafka-0.7.0-incubating-src.tar.gz
>mv kafka-0.7.0-incubating-src /opt/tools/kafka-0.7.0
>cd /opt/tools/kafka-0.7.0
>./sbt update
>./sbt package
2. Start the Server and try messages
Start the zookeeper first
>bin/zookeeper-server-start.sh config/zookeeper.properties
>bin/kafka-server-start.sh config/server.properties
We can send the message from producer and consumer
Producer, we type in this window
>bin/kafka-console-producer.sh --zookeeper localhost:2181 --topic test
Consumer, we got message which we typed.
>bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
3. Try to Run JAVA Client
First problem, how to find the jar package for kafka. This is a problem. There is not easy way like maven or pom to do that.
Kafka is not build via maven, there is a build tool named sbt. Simple Build Tool. I have no time to learn this right now. So I just use
this command to download and build all the related jar packages.
>./sbt release-zip
There is example using java, it is under this directory /opt/tools/kafka-0.7.0/examples/bin. How do I know where the jar packages live in classpath? Check this in the file java-producer-consumer-demo.sh and java-simple-consumer-demo.sh.
I download the jar package from there directories:
$base_dir/project/boot/scala-2.8.0/lib/*.jar
$base_dir/core/lib_managed/scala_2.8.0/compile/*.jar
$base_dir/core/lib/*.jar
$base_dir/core/target/scala_2.8.0/*.jar
$base_dir/examples/target/scala_2.8.0/*.jar
Update these related jar package to my local maven repository.
The 2 demos can run well on redhat server, localhost.
The POM.xml will be as follow:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.10</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>kafka</groupId>
<artifactId>kafka</artifactId>
<version>0.7.0</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.8.0.RC7</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
</dependency>
<dependency>
<groupId>org.I0Itec</groupId>
<artifactId>zkclient</artifactId>
<version>20110412</version>
</dependency>
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.3.3</version>
</dependency>
references:
http://www.54chen.com/java-ee/linkedin-kafka-usage.html
http://incubator.apache.org/kafka/
http://incubator.apache.org/kafka/quickstart.html
