Spring Boot Kafka Consumer Producer Example Tutorial

In this tutorial, we will create a Spring Boot Kafka consumer producer example. We will cover the basics of Kafka, how to create a Kafka cluster, and how to produce and consume messages using Spring Boot.

Introduction to Kafka

Apache Kafka is a distributed streaming platform that is used for building real-time data pipelines and streaming apps. It is highly scalable, fault-tolerant, and provides low-latency data processing.

Prerequisites

Before we begin, make sure you have the following installed on your system:

• Java 8 or higher
• Maven or Gradle
• Kafka 2.6 or higher

Step 1: Create a Kafka Cluster

To create a Kafka cluster, you can use the following command:

bin/kafka-server-start.sh config/server.properties

This will start a Kafka broker on your local machine.

Step 2: Create a Spring Boot Project

To create a Spring Boot project, you can use the Spring Initializr web tool or your favorite IDE. Add the following dependencies to your pom.xml file:

<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
  </dependency>
</dependencies>

Or, if you are using Gradle, add the following to your build.gradle file:

dependencies {
  implementation 'org.springframework.boot:spring-boot-starter'
  implementation 'org.springframework.kafka:spring-kafka'
}

Step 3: Create a Kafka Producer

To create a Kafka producer, you can use the following code:

import org.springframework.beans.factory.annotation.Value;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;

@Component
public class KafkaProducer {

  @Value("${kafka.topic}")
  private String topic;

  private final KafkaTemplate<String, String> kafkaTemplate;

  public KafkaProducer(KafkaTemplate<String, String> kafkaTemplate) {
    this.kafkaTemplate = kafkaTemplate;
  }

  public void sendMessage(String message) {
    kafkaTemplate.send(topic, message);
  }
}

This code creates a Kafka producer that sends messages to a specified topic.

Step 4: Create a Kafka Consumer

To create a Kafka consumer, you can use the following code:

import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;

@Component
public class KafkaConsumer {

  @KafkaListener(topics = "${kafka.topic}")
  public void receiveMessage(String message) {
    System.out.println("Received message: " + message);
  }
}

This code creates a Kafka consumer that listens for messages on a specified topic.

Step 5: Configure Kafka Properties

To configure Kafka properties, you can use the following code:

kafka.bootstrap-servers=localhost:9092
kafka.topic=my-topic

This code configures the Kafka bootstrap servers and topic.

Common Mistakes

Here are some common mistakes to avoid when creating a Spring Boot Kafka consumer producer example:

• Not configuring the Kafka bootstrap servers correctly
• Not specifying the correct topic
• Not handling errors and exceptions properly

Conclusion

In this tutorial, we created a Spring Boot Kafka consumer producer example. We covered the basics of Kafka, how to create a Kafka cluster, and how to produce and consume messages using Spring Boot. We also discussed common mistakes to avoid and how to configure Kafka properties.