Loggregator Architecture

This topic describes the Loggregator architecture and components. It also describes the Cloud Foundry components that send BOSH-reported VM metrics to Loggregator.

Overview

Loggregator gathers and streams logs and metrics from user apps in a Cloud Foundry deployment. It also gathers and streams metrics from Cloud Foundry components and health metrics from BOSH-deployed VMs. Loggregator allows you to view these logs and metrics through the Loggregator CLI plugins or through a third-party service. For more information, see the Loggregator repository on GitHub.

Loggregator architecture includes components for collecting, storing, and forwarding logs and metrics.

Loggregator Architecture and Components

This section includes the following Loggregator architecture diagrams:

Loggregator Architecture

The following diagram shows the architecture of a Loggregator deployment.

A Loggregator Agent appears in squares that depict 'Host VM' and 'Cloud Controller VMs'. Represented by arrows, the Loggregator Agent receives logs from a Forwarder Agent, which forwards logs received from a Syslog Agent. Loggregator Agents send the logs to Dopplers over gRPC communication. Dopplers then forward the logs to Traffic Controllers. For more descriptions of all of the components in Loggregator, see the 'Loggregator Components' section below.

View a larger version of this image.

The following are Loggregator components, as shown in the diagram above:

  • Loggregator Agent: Loggregator Agents run on both Cloud Foundry component VMs and Diego Cell VMs. They receive logs and metrics from the apps and Cloud Foundry components located on those VMs. Loggregator Agents then forward the logs and metrics to Dopplers.

  • Doppler: Dopplers receive logs and metrics from Loggregator Agents, store them in temporary buffers, and forward them to Traffic Controllers.

  • Traffic Controller: Traffic Controllers poll Doppler servers for logs and metrics, then translate these messages from the Doppler servers as necessary for external and legacy APIs. The Loggregator Firehose is located on the Traffic Controller.

  • Reverse Log Proxy: Reverse Log Proxies (RLPs) collect logs and metrics from Dopplers and forward them to Log Cache and Traffic Controllers. Operators can scale up the number of RLPs based on overall log volume.

  • Syslog Agents: Syslog Agents run on Cloud Foundry component VMs and host VMs to collect and forward logs and metrics to configured syslog drains. This includes syslog drains for individual apps as well as aggregate drains for all apps in your foundation.

  • Syslog Binding Cache: Syslog Agents can overwhelm CAPI when querying for existing bindings. This component acts a a proxy for the CAPI Binding query.

  • Firehose: The Firehose is a WebSocket endpoint that streams all the event data from a Cloud Foundry deployment. The data stream includes HTTP events, app logs, container metrics from apps, and metrics from Cloud Foundry platform components. The Firehose cf CLI plugin allows you to view the output of the Firehose. For more information about the Firehose plugin, see Installing the Loggregator Firehose Plugin for cf CLI.

  • Log Cache: The Log Cache allows you to view logs and metrics from the Firehose over a specified period of time. The Log Cache includes API endpoints and a CLI plugin to query and filter logs and metrics. To download the Log Cache CLI plugin, see Cloud Foundry Plugins. The Log Cache API endpoints are available by default. For more information about using the Log Cache API, see Log Cache on GitHub.

  • Nozzles: Nozzles are programs that consume data from the Loggregator Firehose. Nozzles can be configured to select, buffer, and transform data, and to forward it to other apps and services. For more information about creating and deploying nozzles, see Deploying a Nozzle to the Loggregator Firehose.

System Metrics Agents Architecture

The following diagram shows the architecture of a Loggregator deployment that uses System Metrics Agents to collect VM and system-level metrics.

System Metrics Agent appear in squares that depict 'Host VMs'. Represented by arrows, the System Metrics Agents send VM system-level metrics to a System Metrics Scraper, which forwards these metrics to Loggregator Agents over mTLS. For more descriptions of all of the components in Loggregator, see the 'Loggregator Components' section below.

View a larger version of this image.

The following describes the components of a Loggregator deployment that uses System Metrics Agents, as shown in the diagram above:

  • Loggregator Agent: Loggregator Agents run on both Cloud Foundry component VMs and Diego Cell VMs. They receive logs and metrics from the apps and Cloud Foundry components located on those VMs. Loggregator Agents then forward the logs and metrics to Dopplers.

  • System Metrics Agent: A standalone agent to provide VM system metrics using a Prometheus-scrapable endpoint. System Metrics Agents also convert Loggregator metrics from the Loggregator Agent into Prometheus-formatted metrics and host them on a Prometheus-scrapable endpoint. For more information, see Using Metrics Agent in the Metrics Discovery Release repository on GitHub.

  • System Metrics Scraper: The System Metrics Scaper forwards metrics from System Metrics Agents to Loggregator Agents over mTLS.

Related BOSH Components

This section describes the Cloud Foundry components that forward BOSH-reported VM metrics to Loggregator. BOSH-reported VM metrics measure the health of BOSH-deployed VMs on which apps and Cloud Foundry components are deployed. Loggregator streams BOSH-reported VM metrics through the Firehose.

The following are the Cloud Foundry components that send BOSH-reported VM metrics to Loggregator:

  • BOSH Agent: BOSH Agents are located on component VMs and Diego Cell VMs. They collect metrics, such as Diego Cell capacity remaining, from the VM and forward them to the BOSH Health Monitor.

  • BOSH Health Monitor: The BOSH Health Monitor receives metrics from the BOSH Agents. It then forwards the metrics to a third-party service or to the BOSH System Metrics Forwarder.

  • BOSH System Metrics Plugin: This plugin reads health events, such as VM heartbeats and alerts from the BOSH Health Monitor, and streams them to the BOSH System Metrics Server.

  • BOSH System Metrics Server: The BOSH System Metrics Server streams metrics and heartbeat events to the BOSH System Metrics Forwarder over gRPC.

  • BOSH System Metrics Forwarder: The BOSH System Metrics Forwarder is located on the Loggregator Traffic Controller. It forwards heartbeat events from the BOSH System Metrics Server as envelopes to Loggregator through a Loggregator Agent.

View the source for this page in GitHub