Skip to main content

17: Netdata - Real-time Performance Monitoring

Netdata is a distributed, real-time performance monitoring solution built with C and Node.js. It provides sub-second granularity metrics collection, zero-configuration operation, and comprehensive system monitoring including CPU, memory, disk, network, processes, and Docker containers. For API documentation, advanced configuration, and development guides, see the official Netdata documentation.

Prerequisites

  • ✅ Docker installed (Chapter 3)
  • ✅ Docker Compose (Chapter 3)
  • ✅ Optional: Traefik installed (Chapter 4) for HTTPS with Let's Encrypt
  • ✅ Optional: Domain configured (Chapter 4.5), e.g., monitor.example.com
  • ✅ Optional: Apprise installed (Chapter 5) for alert notifications

Installation via Infinity Tools

Menu Installation

📱 APPLICATIONS → Netdata → Install

CLI Installation

sudo bash /opt/InfinityTools/Solutions/setup-netdata.sh --install

Deployment Modes

Traefik Mode (Default)

Uses Traefik for SSL termination and domain routing:

  • Automatic Let's Encrypt certificate provisioning
  • Domain-based access: https://monitor.example.com
  • Security headers configured
  • Requires: Traefik running, DNS A record configured

Standalone Mode

Direct access with HTTP or HTTPS (self-signed):

  • HTTP: http://SERVER_IP:19999
  • HTTPS: https://SERVER_IP:19999 (self-signed cert via nginx proxy)
  • Default port: 19999 (configurable)
  • No domain required

Architecture

Container

  • netdata - Main application (netdata/netdata:latest)
  • netdata-ssl-proxy - Nginx SSL proxy (standalone HTTPS mode only)

Data Persistence

  • Config: /opt/speedbits/netdata-client/netdata/ (configuration files)
  • Lib: /opt/speedbits/netdata-client/netdata/lib/ (database, metrics)
  • Cache: /opt/speedbits/netdata-client/netdata/cache/ (temporary data)
  • SSL: /opt/speedbits/netdata-client/ssl/ (standalone mode certificates)

Host Access

Netdata requires access to host system for monitoring:

  • /proc - Process and system information (read-only)
  • /sys - System information (read-only)
  • /var/run/docker.sock - Docker API (read-only, for container monitoring)
  • /etc/passwd, /etc/group - User information (read-only)
  • /etc/os-release - OS information (read-only)

Networks

  • Traefik network: Joins Traefik's proxy network (Traefik mode)
  • netdata-internal: Isolated bridge network (standalone mode)
  • borgmatic-db: Network for Apprise integration (if enabled)

Installation Process

Configuration Steps

  1. SSL Mode Selection: Choose Traefik (default) or Standalone
  2. If Traefik: Provide domain name
  3. If Standalone: Specify port (default: 19999) and SSL mode
  4. Streaming: Optional parent-child streaming configuration
  5. Apprise Integration: Optional alert notification setup

What Gets Created

  • Directory: /opt/speedbits/netdata-client
  • Container: netdata
  • Docker Compose: /opt/speedbits/netdata-client/docker-compose.yml
  • Config Files: Health alerts, streaming config, Apprise integration

Access Methods

Traefik Mode

https://monitor.example.com

Direct web access after DNS propagation and SSL certificate generation (30-60 seconds).

Standalone Mode

HTTP:

http://SERVER_IP:19999

HTTPS:

https://SERVER_IP:19999

Accept self-signed certificate warning (Advanced → Proceed).

Security Configuration

Access Security

  • ✅ Traefik mode uses Let's Encrypt SSL (production-ready)
  • ✅ Standalone HTTPS uses self-signed certificates (acceptable for internal use)
  • ✅ Security headers configured (X-Frame-Options, CSP, etc.)
  • ⚠️ NO default authentication - Dashboard is publicly accessible

Container Security

  • Runs with SYS_PTRACE and SYS_ADMIN capabilities (required for monitoring)
  • Security option: apparmor:unconfined
  • Read-only mounts for host filesystem access
  • Docker socket mounted read-only

Authentication

⚠️ CRITICAL: Netdata has NO username/password protection by default!

  • Traefik mode: Add Basic Auth via websiteprotection.sh
  • Standalone mode: Keep on private network or use SSH tunnel
  • VPN: Access via WireGuard VPN for secure remote access
  • SSH Tunnel: ssh -L 19999:localhost:19999 user@server

Metrics Collection

System Metrics

  • CPU - Per-core usage, load averages, interrupts
  • Memory - RAM, swap, buffers, cache
  • Disk - I/O, space usage, per-filesystem metrics
  • Network - Per-interface traffic, connections, errors
  • Processes - Per-process CPU, memory, I/O
  • System Load - Load averages, context switches

Docker Metrics

  • Auto-discovery of all containers
  • Per-container CPU, memory, disk, network
  • Container health status
  • Real-time metrics (1-second granularity)

Data Retention

  • Configurable retention period
  • Default: 1 hour of 1-second data
  • Can extend for longer historical data
  • Low storage footprint (~50MB RAM)

Alert Configuration

Default Alerts

Pre-configured alerts in health.d/:

  • CPU Usage: Warning > 80%, Critical > 95%
  • RAM Usage: Warning > 80%, Critical > 95%
  • Disk Space: Warning > 80%, Critical > 90%

Apprise Integration

If Apprise is enabled:

  • Alerts sent to Apprise via HTTP POST
  • Apprise forwards to configured channels
  • Supports all Apprise notification providers
  • Config file: health_alarm_notify.conf

Custom Alerts

Create custom alerts in health.d/:

# Example: Custom alert
alarm: custom_metric
    on: system.cpu
  lookup: average -3m unaligned of user,system
   units: %
   every: 1m
    warn: $this > 75
    crit: $this > 90
   delay: down 5m multiplier 1.5 max 1h
    info: Custom CPU alert
      to: sysadmin

Parent-Child Streaming

Configuration

Stream metrics to a Netdata Director (parent server):

  • Configured in stream.conf
  • Requires director hostname/IP and API key
  • Enables centralized monitoring of multiple servers
  • Local dashboard remains available

Use Cases

  • Multi-server monitoring
  • Centralized dashboards
  • Unified alerting
  • Historical data aggregation

Environment Variables

Netdata Container

  • NETDATA_CLAIM_TOKEN - Optional Netdata Cloud claim token
  • NETDATA_CLAIM_ROOMS - Optional Netdata Cloud rooms
  • NETDATA_CLAIM_URL - Netdata Cloud URL (default: https://app.netdata.cloud)
  • DOCKER_HOST - Docker socket path (default: /var/run/docker.sock)

Configuration Files

Main Configuration

  • netdata.conf - Main Netdata configuration
  • stream.conf - Parent-child streaming configuration
  • health_alarm_notify.conf - Alert notification configuration
  • health.d/*.conf - Individual alert definitions

Customization

# Edit main config
nano /opt/speedbits/netdata-client/netdata/netdata.conf

# Edit alerts
nano /opt/speedbits/netdata-client/netdata/health.d/cpu_usage.conf

# Edit streaming
nano /opt/speedbits/netdata-client/netdata/stream.conf

Troubleshooting

Container Not Starting

docker logs netdata
docker ps -a | grep netdata

Missing Metrics

  • Verify host mounts: docker exec netdata ls /host/proc
  • Check Docker socket: docker exec netdata ls /var/run/docker.sock
  • Verify container capabilities

Docker Containers Not Showing

  • Verify Docker socket access
  • Check Docker daemon is running
  • Restart Netdata container

Alerts Not Working

  • Verify Apprise is running and accessible
  • Check health_alarm_notify.conf configuration
  • Test alert thresholds
  • Check Netdata logs for errors

Production Considerations

  • Access Method: Use Traefik mode for production (trusted SSL)
  • Authentication: Add Basic Auth protection (critical!)
  • Network Security: Restrict access via firewall or VPN
  • Resource Usage: ~50MB RAM, minimal CPU impact
  • Data Retention: Configure appropriate retention period
  • Alerting: Configure multiple notification channels for redundancy
  • Streaming: Use director for multi-server environments

Integration with Infinity Tools

Netdata complements Infinity Tools by providing:

  • Real-time monitoring of all Infinity Tools applications
  • Docker container monitoring for infrastructure
  • System resource monitoring
  • Alert integration with Apprise
  • All Infinity Tools application containers
  • System resources (CPU, RAM, disk)
  • Network traffic
  • Docker daemon health

API & Automation

REST API

  • Netdata provides REST API for metrics access
  • API documentation available in web interface
  • Useful for automation and integration
  • Export metrics to external systems

Exporting Data

  • Export graphs as images
  • Export metrics as JSON/CSV
  • Integrate with external monitoring systems

Next Steps

Netdata is now operational. Use it to:

  • Monitor all Infinity Tools applications
  • Track Docker container health
  • Monitor system resources in real-time
  • Set up alert notifications
  • Analyze performance trends

For advanced features, API usage, custom collectors, and development guides, refer to the official Netdata documentation.

No comments to display

Back to top