Filebeat Prometheus exporter

⚠️ Disclaimer: This sidecar is developed and tested with specific Filebeat versions (9.x.x). Compatibility across all Filebeat versions is not guaranteed, and the metrics format or API endpoints may change in future Filebeat releases. Please test thoroughly with your specific Filebeat version before deploying to production.

⚠️ Disclaimer 2: This was done with the help of Kiro and Cursor. I am not a exprienced go developer, I am using this oportunity to learn a bit about it. Use this at your own risk ;).

A lightweight Go application that acts as a metrics transformation proxy for Filebeat. It fetches JSON metrics from Filebeat's /stats endpoint and converts them to Prometheus exposition format, making Filebeat metrics compatible with Prometheus monitoring infrastructure.

Quick Start

All metrics with an atempt own its meaning can be found here.

Using Docker

# Build the image
docker build -t filebeat-prometheus-exporter .

# Run with default configuration
docker run -p 9090:9090 filebeat-prometheus-exporter

# Run with custom Filebeat URL
docker run -p 9090:9090 -e FILEBEAT_URL=http://your-filebeat:5066 filebeat-prometheus-exporter

Using Docker Compose

# Copy the example configuration
cp examples/docker-compose.yml .
cp examples/filebeat.yml .
cp examples/prometheus.yml .

# Start the stack
docker-compose up -d

Binary Installation

# Build from source
go build -o sidecar ./cmd/sidecar

# Run with default settings
./sidecar

# Run with custom configuration
./sidecar -port 8080 -filebeat-url http://filebeat:5066 -log-level debug

Configuration

The application can be configured using environment variables, command-line flags, or a combination of both. Command-line flags take precedence over environment variables.

Configuration Options

Flag	Environment Variable	Default	Description
-port	SIDECAR_PORT	9090	Port to listen on
-filebeat-url	FILEBEAT_URL	http://localhost:5066	Filebeat stats endpoint URL
-timeout	REQUEST_TIMEOUT	10s	Request timeout duration
-prefix	METRIC_PREFIX	filebeat_	Metric name prefix
-log-level	LOG_LEVEL	info	Log level (debug, info, warn, error)

Configuration Examples

Environment Variables

export SIDECAR_PORT=9090
export FILEBEAT_URL=http://filebeat:5066
export REQUEST_TIMEOUT=30s
export METRIC_PREFIX=filebeat_
export LOG_LEVEL=info
./sidecar

Command Line Flags

./sidecar -port 9090 -filebeat-url http://filebeat:5066 -timeout 30s -prefix filebeat_ -log-level info

Using Configuration File

# Load environment variables from file
source examples/config.env
./sidecar

API Endpoints

GET /metrics

Returns Filebeat metrics in Prometheus exposition format.

Example Response:

# HELP filebeat_beat_cpu_system_ticks CPU system ticks
# TYPE filebeat_beat_cpu_system_ticks counter
filebeat_beat_cpu_system_ticks 1234

# HELP filebeat_beat_cpu_user_ticks CPU user ticks  
# TYPE filebeat_beat_cpu_user_ticks counter
filebeat_beat_cpu_user_ticks 5678

# HELP filebeat_libbeat_output_events_acked Output events acknowledged
# TYPE filebeat_libbeat_output_events_acked counter
filebeat_libbeat_output_events_acked 1070

GET /health

Returns health status of the sidecar and its connection to Filebeat.

Example Response:

{
  "status": "healthy",
  "filebeat_connection": "ok",
  "timestamp": "2024-01-15T10:30:00Z"
}

Deployment

Docker

Build Image

docker build -t filebeat-prometheus-exporter:latest .

Run Container

docker run -d \
  --name filebeat-sidecar \
  -p 9090:9090 \
  -e FILEBEAT_URL=http://filebeat:5066 \
  filebeat-prometheus-exporter:latest

Kubernetes

Deploy using the provided manifests:

# Deploy the application
kubectl apply -f examples/kubernetes/deployment.yaml
kubectl apply -f examples/kubernetes/service.yaml

# Optional: Deploy ServiceMonitor for Prometheus Operator
kubectl apply -f examples/kubernetes/servicemonitor.yaml

Docker Compose

Use the provided Docker Compose configuration:

# Start the complete stack (Filebeat + Sidecar + Prometheus)
docker-compose -f examples/docker-compose.yml up -d

Monitoring Setup

Prometheus Configuration

Add the following job to your prometheus.yml:

scrape_configs:
  - job_name: 'filebeat-sidecar'
    static_configs:
      - targets: ['filebeat-sidecar:9090']
    scrape_interval: 30s
    metrics_path: /metrics

Grafana Dashboard

The sidecar exposes standard Filebeat metrics that can be visualized in Grafana:

• CPU Usage: filebeat_beat_cpu_*
• Memory Usage: filebeat_beat_memstats_*
• Event Processing: filebeat_libbeat_output_events_*
• Harvester Stats: filebeat_filebeat_harvester_*

Troubleshooting

Common Issues

Connection Refused

Error: Failed to fetch metrics from Filebeat: connection refused

Solution: Ensure Filebeat is running and the HTTP endpoint is enabled:

# In filebeat.yml
http.enabled: true
http.host: "0.0.0.0"
http.port: 5066

Invalid JSON Response

Error: Failed to parse Filebeat response: invalid JSON

Solution: Check Filebeat version compatibility and ensure the /stats endpoint is accessible.

Port Already in Use

Error: Failed to start server: port 9090 already in use

Solution: Use a different port:

./sidecar -port 9091
# or
export SIDECAR_PORT=9091

Debug Mode

Enable debug logging for troubleshooting:

./sidecar -log-level debug

This will show detailed information about:

• Configuration loading
• HTTP requests to Filebeat
• Metrics transformation process
• Server request handling

Health Check

Check if the sidecar is healthy:

curl http://localhost:9090/health

Metrics Validation

Verify metrics are being generated correctly:

curl http://localhost:9090/metrics

Development

Prerequisites

• Go 1.22.2 or later
• Docker (optional, for containerized development)

Building

# Build binary
go build -o sidecar ./cmd/sidecar

# Build with race detection (for development)
go build -race -o sidecar ./cmd/sidecar

# Cross-compile for Linux
GOOS=linux GOARCH=amd64 go build -o sidecar-linux ./cmd/sidecar

Testing

# Run all tests
go test ./...

# Run tests with coverage
go test -cover ./...

# Run tests with race detection
go test -race ./...

Project Structure

.
├── cmd/sidecar/           # Main application entry point
├── internal/
│   ├── config/           # Configuration management
│   ├── filebeat/         # Filebeat client implementation
│   ├── handlers/         # HTTP request handlers
│   ├── interfaces/       # Interface definitions
│   ├── logging/          # Structured logging
│   ├── server/           # HTTP server implementation
│   └── transformer/      # Metrics transformation logic
├── examples/             # Example configurations and deployments
├── Dockerfile           # Container build configuration
└── README.md           # This file

Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add some amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

For issues and questions:

1. Check the Troubleshooting section
2. Search existing GitHub Issues
3. Create a new issue with detailed information about your problem

Changelog

v1.0.0

• Initial release
• Basic metrics transformation functionality
• Docker and Kubernetes deployment support
• Comprehensive configuration options
• Health check endpoint