Centralized Logging with ELK Stack (Elasticsearch, Logstash, Kibana)

In a microservices architecture, logging becomes more complex as logs are scattered across multiple services. This makes debugging and monitoring challenging.

Centralized Logging with ELK Stack solves this problem by aggregating logs from all services into a central platform, enabling real-time visualization and search.

In this guide, you’ll learn how to integrate Spring Boot microservices with the ELK Stack — Elasticsearch, Logstash, and Kibana — for powerful centralized logging.

📦 Stack Overview

• Elasticsearch: Stores structured log data
• Logstash: Parses and transports log data
• Kibana: Visualizes log data with dashboards

🧰 Step 1: ELK Stack Setup Using Docker

Create a docker-compose.yml:

version: '3.7'
services:

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.10.0
    container_name: elasticsearch
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
    ports:
      - 9200:9200

  logstash:
    image: docker.elastic.co/logstash/logstash:8.10.0
    container_name: logstash
    volumes:
      - ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf
    ports:
      - 5000:5000

  kibana:
    image: docker.elastic.co/kibana/kibana:8.10.0
    container_name: kibana
    ports:
      - 5601:5601
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200

⚙️ Step 2: Logstash Configuration (logstash.conf)

input {
  tcp {
    port => 5000
    codec => json_lines
  }
}

output {
  elasticsearch {
    hosts => ["http://elasticsearch:9200"]
    index => "springboot-logs-%{+YYYY.MM.dd}"
  }
  stdout { codec => rubydebug }
}

📁 Step 3: Spring Boot Logback Configuration

You’ll use Logback + Logstash encoder in your Spring Boot apps.

🔧 Add dependencies to pom.xml:

  net.logstash.logback
  logstash-logback-encoder
  7.4

📄 logback-spring.xml Configuration:

  

  
    localhost:5000
    
      
        
          timestamp
        
        
          
            {
              "level": "%level",
              "logger": "%logger",
              "message": "%message",
              "thread": "%thread"
            }
          
        
      
    
  

  
    
  

🧑‍💻 Example Microservice Logging

package com.kscodes.springboot.microservice.orderservice;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.*;

@RestController
@RequestMapping("/orders")
public class OrderController {

    private static final Logger logger = LoggerFactory.getLogger(OrderController.class);

    @GetMapping("/{id}")
    public String getOrder(@PathVariable String id) {
        logger.info("Fetching order with ID: {}", id);
        return "Order details for " + id;
    }
}

🧪 Step 4: Run Everything

1. Run docker-compose up
2. Run your Spring Boot microservice
3. Visit Kibana UI at http://localhost:5601
4. Create an index pattern:
   springboot-logs-*

Now you can search and visualize logs.

🧠 Best Practices

• 🧾 Use structured JSON logs
• 🛡️ Mask sensitive data before logging
• ⚠️ Set log levels wisely (ERROR, INFO, DEBUG)
• 📁 Separate environments by index (e.g., logs-dev-*, logs-prod-*)

📈 Benefits of Centralized Logging with ELK Stack

• 🔍 Full-text search of logs with Kibana
• 🧠 Easy debugging and traceability across microservices
• 🧩 Seamless integration with Spring Boot via Logback
• 📊 Real-time dashboards and alerts

🏁 Conclusion

Implementing Centralized Logging helps unify and analyze logs across all your microservices. With structured logging, searchable indexes, and visual dashboards, the ELK stack provides visibility and observability into your applications.