Apache Kafka installation

Apache Kafka installation

Apache Kafka is the most popular messaging system
It has been developed specifically for

• High throughput
• Scalability
• Fault tolerance

Apache Kafka heavily dependent to Zookeeper for state management and Synchronisation

Download Kafka binary from http://kafka.apache.org/downloads.html
Un-tar in a directory

"bin" directory has all the startup scripts to run Kafka and zookeeper
"config" directory contains all the server side (configurations) settings with which Kafka and zookeeper run

Start Zookeeper

aux-137@thashitharan:~/kafka/kafka_2.12-1.1.0$ bin/zookeeper-server-start.sh config/zookeeper.properties

Start Kafka

aux-137@thashitharan:~/kafka/kafka_2.12-1.1.0$ bin/kafka-server-start.sh config/server.properties

Create a topic

aux-137@thashitharan:~/kafka/kafka_2.12-1.1.0$ bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic mytesttopic

aux-137@thashitharan:~/kafka/kafka_2.12-1.1.0$ bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic mytesttopic12

List created topic

aux-137@thashitharan:~/kafka/kafka_2.12-1.1.0$ bin/kafka-topics.sh --list --zookeeper localhost:2181

Starting producer

aux-137@thashitharan:~/kafka/kafka_2.12-1.1.0$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic mytesttopic

Starting consumer (subscribed to 'mytesttopic' topic)

aux-137@thashitharan:~/kafka/kafka_2.12-1.1.0$ bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic mytesttopic --from-beginning

Start sending the message from the producer -> to consumer