No description
Find a file
iQ 9ca8ad43cb
All checks were successful
Build Go Binary / Build ${{ matrix.goarch }} (amd64, , amd64) (push) Successful in 4m16s
Build Go Binary / Build ${{ matrix.goarch }} (arm64, , arm64) (push) Successful in 4m13s
Build Go Binary / Build ${{ matrix.goarch }} (ppc64le, , ppc64le) (push) Successful in 4m7s
Build Go Binary / Build ${{ matrix.goarch }} (arm, 7, arm-v7) (push) Successful in 4m15s
Build Go Binary / Package release (push) Has been skipped
Merge branch 'main' of gitea:jwalzer/btrfs_exporter
2025-04-02 08:06:47 +01:00
.gitea/workflows troubleshooting CI 2025-03-16 20:27:36 +00:00
testsuite Implement comprehensive testing for BTRFS exporter, including mock filesystem setup and enhanced metric collection. Added tests for real-world data and improved error handling in collection functions. Updated expected metric counts and ensured non-blocking operations during metric collection. 2025-04-01 15:53:36 +01:00
.gitignore Enhance BTRFS exporter with error tracking and build improvements 2025-03-16 01:38:56 +00:00
btrfs_exporter_test Enhance BTRFS exporter with error tracking and build improvements 2025-03-16 01:38:56 +00:00
btrfs_metrics.adoc Enhance BTRFS metrics: Added device information and error statistics. Updated device metrics to include detailed error types and synthesized "all" metrics. Adjusted labels for device metrics in code and documentation. 2025-03-22 22:55:47 +00:00
Dockerfile Enhance Dockerfile and Makefile for multi-architecture support 2025-03-16 14:26:01 +00:00
exporter_test.go Refactor BTRFS exporter tests to improve structure and coverage. Introduced a helper function for running test cases, enhanced metric collection validation, and added device information metrics. Updated the main exporter to support command line flags for BTRFS path and improved error handling in metric collection. 2025-04-01 20:24:11 +01:00
go.mod Refactor logging and update dependencies 2025-03-16 00:26:45 +00:00
go.sum Refactor logging and update dependencies 2025-03-16 00:26:45 +00:00
main.go Merge branch 'main' of gitea:jwalzer/btrfs_exporter 2025-04-02 08:06:47 +01:00
main_test.go Implement comprehensive testing for BTRFS exporter, including mock filesystem setup and enhanced metric collection. Added tests for real-world data and improved error handling in collection functions. Updated expected metric counts and ensured non-blocking operations during metric collection. 2025-04-01 15:53:36 +01:00
Makefile Refactor Makefile and README for enhanced multi-architecture support 2025-03-16 14:55:56 +00:00
README.md Enhance BTRFS metrics: Added device information and error statistics. Updated device metrics to include detailed error types and synthesized "all" metrics. Adjusted labels for device metrics in code and documentation. 2025-03-22 22:55:47 +00:00
TechSpec.md Enhance TechSpec with detailed metrics, container build process, security features, error tracking, and testing strategy. Added sections for Go runtime metrics, multi-architecture support, and comprehensive logging capabilities. 2025-03-26 13:44:32 +00:00
test_error_expiry.sh Enhance BTRFS exporter with error tracking and build improvements 2025-03-16 01:38:56 +00:00
test_nonexistent.sh Enhance BTRFS exporter with error tracking and build improvements 2025-03-16 01:38:56 +00:00
transfer_btrfs.sh Implement comprehensive testing for BTRFS exporter, including mock filesystem setup and enhanced metric collection. Added tests for real-world data and improved error handling in collection functions. Updated expected metric counts and ensured non-blocking operations during metric collection. 2025-04-01 15:53:36 +01:00

BTRFS Exporter for Prometheus

A Prometheus exporter for BTRFS filesystem metrics.

Overview

This exporter collects metrics from the /sys/fs/btrfs directory on Linux systems running BTRFS. It exposes allocation statistics, device information, scrub status, and other BTRFS-specific metrics.

Installation

From source

Once you've set up your private git repository, you can install using:

go install your-private-git-host/btrfs_exporter@latest

Build from source

git clone your-private-git-repo-url/btrfs_exporter.git
# or for local development:
# git clone /path/to/local/repo
cd btrfs_exporter
go build

Using Docker

You can run this exporter using Docker. The image requires access to the host's /sys filesystem.

Build the Docker image

For the host's native architecture:

make container

Build for multiple architectures

The exporter supports building container images for multiple architectures:

  • linux/amd64 (x86_64)
  • linux/arm64 (ARM 64-bit)
  • linux/ppc64le (PowerPC 64-bit)
  • linux/arm/v7 (ARM 32-bit)

To build for all supported architectures:

make container-multiarch

This uses Docker's platform flag to pull the correct architecture-specific base images and builds binaries using the toolchain in those containers. Each architecture gets its own image tagged with the architecture suffix, for example:

  • btrfs_exporter:v0.1.1-amd64
  • btrfs_exporter:v0.1.1-arm64
  • btrfs_exporter:v0.1.1-ppc64le
  • btrfs_exporter:v0.1.1-arm-v7

Note: Requires Docker with QEMU support for emulating different architectures.

Building for a specific architecture

To build for a specific architecture:

# For PowerPC 64-bit
make ARCH=linux/ppc64le container-arch

# For ARM 32-bit
make ARCH=linux/arm/v7 container-arch

The resulting image will be tagged with the architecture suffix, e.g., btrfs_exporter:v0.1.1-ppc64le.

Registry and Distribution

To push architecture-specific images to the registry:

make container-push

This will tag and push each architecture-specific image to registry.intern.in6-addr.net.

To create a multi-architecture manifest that combines all architectures into a single reference:

make container-manifest

This creates a manifest list that allows Docker to automatically pull the correct architecture when users run the image.

Verifying architecture

To verify that a container image contains a binary built for the correct architecture:

make verify-arch CONTAINER=btrfs_exporter:v0.1.1-ppc64le

This will show both the container's architecture and the Go binary's architecture, which should match.

Run the Docker container

docker run -d --name btrfs_exporter \
  -p 9573:9573 \
  -v /sys:/sys:ro \
  --restart=unless-stopped \
  btrfs_exporter:$(git describe --tags --always)

If using an architecture-specific build:

docker run -d --name btrfs_exporter \
  -p 9573:9573 \
  -v /sys:/sys:ro \
  --restart=unless-stopped \
  btrfs_exporter:$(git describe --tags --always)-amd64

Using pre-built images

If you have access to the container registry with pre-built images, you can pull and run them directly:

docker pull registry.intern.in6-addr.net/btrfs_exporter:latest
docker run -d --name btrfs_exporter \
  -p 9573:9573 \
  -v /sys:/sys:ro \
  --restart=unless-stopped \
  registry.intern.in6-addr.net/btrfs_exporter:latest

Environment variables

The container supports the following environment variables:

  • BTRFS_EXPORTER_LOG_LEVEL: Set the log level (default: "info")

Usage

./btrfs_exporter [flags]

Flags

  -web.listen-address string
    	Address to listen on for web interface and telemetry. (default ":9573")
  -web.telemetry-path string
    	Path under which to expose metrics. (default "/metrics")
  -version
    	Print version information.
  -log.level string
        Log level: debug, info, warn, error (default "info")
  -test
        One-shot test mode: collect metrics once and exit

Test Mode

The -test flag enables one-shot test mode where the exporter:

  1. Collects metrics once
  2. Outputs them directly to stdout in Prometheus text format
  3. Exits immediately

This mode is useful for testing the exporter's functionality without starting a web server:

./btrfs_exporter -test

Continuous Integration

This project uses Gitea CI with act-runners to automatically build binaries for multiple architectures.

CI Features

  • Automated builds triggered on pushes to main/master branches and tags
  • Tests run before builds to ensure code quality
  • Cross-compilation for multiple architectures:
    • linux/amd64 (x86_64)
    • linux/arm64 (ARM 64-bit)
    • linux/ppc64le (PowerPC 64-bit)
    • linux/arm/v7 (ARM 32-bit)
  • Build artifacts stored for each successful build
  • Release binaries created automatically when tags are pushed

Working with CI

  • To trigger a build: push to main/master branch or create a pull request
  • To create a release: push a tag with the version (e.g., v1.0.0)
  • View build logs and artifacts in your Gitea instance

Metrics

The exporter provides the following metrics:

Basic Metrics

  • btrfs_allocation_bytes - BTRFS allocation statistics in bytes
  • btrfs_reserved_bytes - BTRFS reserved space in bytes
  • btrfs_device_bytes - BTRFS device size and allocation information in bytes
  • btrfs_scrub_status - BTRFS scrub status (1 for running, 0 for not running)
  • btrfs_scrub_progress_ratio - BTRFS scrub progress ratio (0.0-1.0)
  • btrfs_filesystem_info - BTRFS filesystem information
  • btrfs_feature_enabled - BTRFS enabled features (1 for enabled, 0 for disabled)
  • btrfs_error_count - BTRFS error counts
  • btrfs_space_bytes - BTRFS space information in bytes
  • btrfs_device_error_count - BTRFS device error counters
  • btrfs_qgroup_bytes - BTRFS quota group information in bytes
  • btrfs_qgroup_limit_bytes - BTRFS quota group limit in bytes
  • btrfs_compression_ratio - BTRFS compression ratio
  • btrfs_io_stats - BTRFS disk IO statistics
  • btrfs_checksum_info - BTRFS checksum algorithm information
  • btrfs_config_info - BTRFS configuration information

Advanced Metrics

  • btrfs_allocation_advanced_bytes - Detailed BTRFS allocation statistics including:

    • disk_total and disk_used - Physical disk space allocation
    • bytes_pinned - Bytes currently pinned in memory
    • bytes_readonly - Bytes marked as read-only
    • bytes_zone_unusable - Bytes unusable in zoned devices
    • flags - Allocation flags
    • total_bytes_pinned - Total bytes pinned across the system
  • btrfs_device_info - BTRFS device information:

    • fsid - Filesystem ID of the device (as a label)
    • Provides detailed identification information for each device
  • btrfs_device_status - BTRFS device status information:

    • missing - Whether the device is missing (1 for missing)
    • writeable - Whether the device is writeable (1 for writeable)
    • in_fs_metadata - Whether the device is in filesystem metadata (1 for yes)
    • scrub_speed_max - Maximum scrub speed for the device
    • replace_target - Whether the device is a target for replacement operations (1 for yes)
  • btrfs_filesystem_generation - BTRFS filesystem generation number

  • btrfs_global_feature_supported - BTRFS globally supported features (1 for supported)

  • btrfs_supported_checksum - BTRFS supported checksum algorithms (1 for supported)

  • btrfs_supported_sectorsize - BTRFS supported sector sizes (1 for supported)

RAID Metrics

  • btrfs_raid_config - BTRFS RAID configuration information:

    • Includes profile type (e.g., raid1, raid1c3) and allocation type (data, metadata, system)
    • Helps monitor complex multi-device RAID setups
    • Shows which RAID features are available and which are in use
  • btrfs_space_bytes includes additional RAID-specific metrics:

    • Each RAID profile gets its own metrics (e.g., data_raid1_total, metadata_raid1c3_used)
    • Shows allocation data for each RAID level separately

Exporter Information

  • btrfs_exporter_info - Information about the exporter itself:
    • git_commit - Full git commit hash
    • git_commit_short - Abbreviated git commit hash
    • git_repo_state - State of the git repository ("clean" or "unclean")
    • git_version - Version derived from the git tag or branch name
    • build_timestamp - Timestamp when the exporter was built

Error Metrics

  • btrfs_exporter_error_since - Timestamp when an error condition was first detected:
    • error_level - Severity level of the error ("error" or "warn")
    • message - Detailed error message describing the issue
    • Value is the Unix timestamp when the error condition was first detected
    • This metric is automatically removed when the error condition is resolved

The error metrics system tracks various issues that might occur during operation:

  • Missing BTRFS filesystems
  • Inability to access the /sys/fs/btrfs directory
  • Problems reading specific metrics or files
  • Configuration issues

These error metrics help identify when problems started and automatically clear when resolved.

Device Metrics

Device metrics provide information about the individual devices that make up a BTRFS filesystem:

  • btrfs_device_bytes: Device size and allocation information
    • Labels: fsid, device_id, type (total, used)
  • btrfs_device_error_count: Device error counters
    • Labels: fsid, device_id, type (read, write, flush, corruption, generation)
    • Includes all errors from both /sys/fs/btrfs/<uuid>/devices/<id>/ and /sys/fs/btrfs/<uuid>/devinfo/<id>/error_stats
    • The error_stats file contains all error types in a single file with format error_type value
    • Always exports metrics even when error counts are zero
    • Includes an aggregate "all" metric for each error type with device_id="all" that sums errors across all devices
  • btrfs_device_status: Device status information
    • Labels: fsid, device_id, type (missing, writeable, in_fs_metadata, scrub_speed_max, replace_target)
  • btrfs_device_info: Additional device information
    • Labels: fsid, device_id, info_type, info_value - where info_type can be "fsid" and info_value is the device's filesystem UUID
  • btrfs_io_stats: Device I/O statistics
    • Labels: fsid, device_id, type

These metrics are collected from:

/sys/fs/btrfs/<uuid>/devices/           # Symbolic links to block devices
/sys/fs/btrfs/<uuid>/devinfo/           # Device information
/sys/fs/btrfs/<uuid>/devinfo/<id>/error_stats  # Detailed error statistics

Requirements

  • Go 1.22 or higher
  • Linux system with BTRFS filesystems mounted
  • Read access to /sys/fs/btrfs directory

Prometheus Configuration

scrape_configs:
  - job_name: 'btrfs'
    static_configs:
      - targets: ['localhost:9573']

License

This project is licensed under the MIT License - see the LICENSE file for details.