Skip to main content

Logger Service

The Logger Service provides centralized logging framework for the Rippler platform. It works with Spring Boot's logging infrastructure to ingest logs from partner services, providing a unified view of system-wide logging for debugging, monitoring, and auditing purposes.

Architecture Diagram​

The following diagram illustrates the internal architecture of the Logger Service, showing how logs are collected from various microservices, processed, stored, and made available for querying and monitoring.

Update the Architecture Diagram

To view and edit the architecture diagram:

  1. Open /docs/architecture/services/rippler-logger-service.drawio in diagrams.net or VS Code with the Draw.io extension
  2. The diagram shows the complete logging architecture including log ingestion from services, processing pipeline, storage, and query capabilities
  3. After making changes, export to HTML and copy to /website/static/architecture/services/rippler-logger-service.drawio.html

Overview​

Technology: Spring Boot, Java 17
Integration: Spring Logger Framework

The Logger Service acts as a centralized logging hub for all Rippler microservices, collecting, processing, and storing logs in a structured format for easy querying and analysis.

Features​

  • ✅ Centralized log ingestion from all partner services
  • ✅ Integration with Spring Boot logging framework
  • ✅ Structured log storage and indexing
  • ✅ Real-time log streaming
  • ✅ Log level filtering and categorization
  • ✅ Query API for log retrieval and analysis
  • ✅ Log retention and archival policies
  • ✅ Performance monitoring and metrics

Architecture​

graph TD
    A[Microservices] -->|Send Logs| B[Logger Service]
    B -->|Process| C[Log Processor]
    C -->|Store| D[Log Storage]
    E[Admin/Developer] -->|Query| B
    B -->|Return| E

Log Categories​

The Logger Service handles various types of logs from Rippler services:

  • Application Logs - Service-specific application events
  • Request/Response Logs - API call tracking
  • Error Logs - Exception and error tracking
  • Performance Logs - Timing and performance metrics
  • Security Logs - Authentication and authorization events
  • Integration Logs - External service integration events

Log Levels​

The service supports standard log levels:

LevelDescriptionUse Case
TRACEVery detailed diagnostic informationDeep debugging
DEBUGDetailed debugging informationDevelopment and troubleshooting
INFOInformational messagesNormal operation tracking
WARNWarning messagesPotential issues
ERRORError eventsApplication errors
FATALCritical errorsSystem failures

Integration with Services​

All Rippler microservices can integrate with the Logger Service by configuring their Spring Boot logging to forward logs to the centralized service.

Example Configuration​

logging:
  level:
    root: INFO
    com.rippler: DEBUG
  pattern:
    console: "%d{yyyy-MM-dd HH:mm:ss} - %msg%n"
    file: "%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n"

API Endpoints​

Query Logs​

Get Logs by Service​

GET /api/v1/logs?service={service-name}&level={log-level}&from={timestamp}&to={timestamp}

Query logs from a specific service within a time range.

Get Recent Logs​

GET /api/v1/logs/recent?limit={count}

Retrieve the most recent logs across all services.

Search Logs​

GET /api/v1/logs/search?query={search-term}

Search logs using full-text search capabilities.

Environment Variables​

# Service Configuration
SERVER_PORT=18012
SPRING_APPLICATION_NAME=logger-service

# Log Storage
LOG_STORAGE_TYPE=database
LOG_RETENTION_DAYS=90

# Eureka Service Registry
EUREKA_CLIENT_SERVICE_URL_DEFAULTZONE=http://eureka:8761/eureka/

Log Storage​

The Logger Service stores logs in a structured format with indexing for fast queries:

  • Short-term storage: Recent logs in database for quick access
  • Long-term storage: Older logs archived to cost-effective storage
  • Indexing: Full-text search indexing for efficient queries
  • Retention: Configurable retention policies based on log level and age

Monitoring & Alerts​

The Logger Service can be configured to:

  • Monitor for specific error patterns
  • Send alerts when error rates exceed thresholds
  • Track service health based on log patterns
  • Generate reports on system-wide logging trends

Health Check​

GET /actuator/health

Response:

{
  "status": "UP",
  "components": {
    "db": { "status": "UP" },
    "diskSpace": { "status": "UP" }
  }
}

Development​

Running Locally​

# With Docker Compose
docker-compose up logger-service

# Standalone
cd services/logger-service
mvn spring-boot:run

Testing​

# Test log ingestion
curl -X POST http://localhost:18012/api/v1/logs \
  -H "Content-Type: application/json" \
  -d '{
    "service": "test-service",
    "level": "INFO",
    "message": "Test log message",
    "timestamp": "2024-01-15T10:30:00"
  }'

# Query logs
curl "http://localhost:18012/api/v1/logs?service=test-service&limit=10"

Best Practices​

What to Log​

✅ Do Log:

  • Request/response for important operations
  • Business logic decisions
  • Integration points with external services
  • Performance metrics
  • Security events

❌ Don't Log:

  • Passwords or secrets
  • Complete JWT tokens
  • Credit card numbers
  • Personal Identifiable Information (PII)
  • Sensitive business data

Log Message Format​

Use structured logging with consistent formats:

log.info("User {} performed action {} on resource {}", 
    userId, action, resourceId);

Performance Considerations​

  • Use appropriate log levels to avoid excessive logging
  • Avoid logging in tight loops
  • Use async logging for high-throughput scenarios
  • Implement sampling for very high-volume logs

Troubleshooting​

Logs Not Appearing​

  1. Verify service is configured to send logs to Logger Service
  2. Check network connectivity between services
  3. Review Logger Service health status
  4. Check log level configuration

High Storage Usage​

  1. Review retention policies
  2. Implement log archival
  3. Adjust log levels to reduce volume
  4. Enable log compression

Slow Query Performance​

  1. Ensure indexes are properly configured
  2. Optimize query patterns
  3. Consider adding caching layer
  4. Review database performance

See Also​