Choose an instrumentation approach

This page contains recommendations for instrumenting your app on different Google Cloud platforms, such as Google Kubernetes Engine (GKE) and Cloud Run. If your app isn't already instrumented, then use these recommendations for guidance as to how to instrument your app to send telemetry data to Google Cloud. The recommendations on this page aren't the only solutions, and other approaches can work. For additional guidance, contact Google Cloud Support.

There are recommendations for the following:

For information about instrumentation examples, see the documents listed in the Code samples section.

GKE

For general information about GKE, see GKE overview.

Type Recommendation
Metrics

We recommend that you use Google Cloud Managed Service for Prometheus.

For instrumentation, do one of the following:
Traces

Do the following:

  1. Deploy the Google-Built OpenTelemetry Collector on Google Kubernetes Engine.

    The collector receives trace data from the SDK's in-process OTLP exporter, processes that data, and then sends the processed data to your Google Cloud project by using the Telemetry (OTLP) API.

  2. Use the OpenTelemetry SDK and the OTLP exporter for your language.
Logs

Configure your app to output JSON-structured logs to stdout and stderr. For a list of frameworks, see Recommended logging frameworks.

GKE collects logs written to stdout and stderr automatically. For more information, see About GKE logs.

Compute Engine

For general information about Compute Engine, see Virtual machine instances.

Type Recommendation
Metrics and Traces

Do the following:

  1. Use the Ops Agent to collect metrics and traces.

    For an example, see Collect OpenTelemetry Protocol (OTLP) metrics and traces. This guide describes how to configure the Ops Agent to receive metric and trace data from the SDK's in-process OTLP exporters, transforms that data, and then send the data to your Google Cloud project. Metric data is sent by using the Cloud Monitoring API and trace data is sent by using the Cloud Trace API.

  2. Use the OpenTelemetry SDK and the OTLP exporter for your language.

Alternatively, if you only want to configure collection for Prometheus-format metrics, then use the Ops Agent Prometheus Receiver to collect metrics instrumented using Prometheus client libraries or the OpenTelemetry SDK.
Logs

Do the following:

  1. Configure your app to output JSON-structured logs to a file. For a list of frameworks, see Recommended logging frameworks.
  2. Install the Ops Agent and configure a receiver. For an example, see Logging receivers.

Cloud Run

For general information about Cloud Run, see What is Cloud Run.

Type Recommendation
Metrics and Traces

Do the following:

  1. Use the OpenTelemetry SDK and the OTLP exporter for your language.

  2. Deploy an OpenTelemetry sidecar to collect metrics and traces.

    For examples, see the following documents:

  3. For your Cloud Run service, use instance-based billing. With instance-based billing, the CPU is allocated for the entire instance lifecycle, which is necessary because OpenTelemetry instrumentation does background processing. For more information, see Billing settings for Cloud Run services.

Alternatively, if you only want to configure collection for Prometheus-format metrics, then use the Prometheus sidecar for Cloud Run to collect metrics instrumented using Prometheus client libraries or the OpenTelemetry SDK.
Logs

Configure your app to output JSON-structured logs to stdout and stderr. For a list of frameworks, see Recommended logging frameworks.

Cloud Run collects logs written to stdout and stderr automatically. For more information, see Write container logs.

Cloud Run functions

For general information about Cloud Run functions, see Cloud Run functions overview.

Type Recommendation
Metrics Direct writing of metrics isn't supported in Cloud Run functions. You can generate log-based metrics.
Traces Use the OpenTelemetry SDK and the Cloud Trace exporter for your language.
Logs

Configure your app to output JSON-structured logs to stdout and stderr. For a list of frameworks, see Recommended logging frameworks.

Cloud Run functions collects logs written to stdout and stderr automatically. For more information, see Cloud Run functions: Monitoring and logging overview.

App Engine

For general information about App Engine, see An overview of App Engine.

Type Recommendation
Metrics Use the OpenTelemetry SDK and the Cloud Monitoring exporter for your language.
Traces Use the OpenTelemetry SDK and the Cloud Trace exporter for your language.
Logs

Configure your app to output JSON-structured logs to stdout and stderr. For a list of frameworks, see Recommended logging frameworks.

App Engine collects logs written to stdout and stderr automatically. For more information, see Write and view logs.

Recommended logging frameworks

To collect logs, we recommend that you use a framework which can be configured to output serialized JSON objects to stdout, stderr, or to a file. The logging agent, whether integrated or installed, scrapes the file and writes structured logs for Cloud Logging. For writing log data, we recommend the following:

Code samples

When you instrument an application with OpenTelemetry libraries, you decide how your telemetry data is sent to your Google Cloud project. There are two fundamental approaches:

  • You instrument your application code with the OpenTelemetry SDK and use the in-process OTLP exporter that is included in the SDK to send data to an OpenTelemetry collector. The collector receives data from the in-process exporter and then sends that data to your Google Cloud project. The changes you make to your application code are vendor neutral. The collector contains the logic to send the telemetry to your Google Cloud project.

  • You instrument your application code with the OpenTelemetry SDK and use an in-process exporter that directly sends data to your Google Cloud project. Some changes that your make to your application are vendor specific. However, you don't need to configure or deploy a collector.

We recommend that you use an OpenTelemetry collector to export your telemetry data when your environment supports use of collector. For some environments, you must use an in-process exporter that directly sends data to your Google Cloud project.

Samples for collector-based export

For language-specific instrumentation samples, see the following documents:

If you are interested in how the collectors are defined and deployed, see Sample overview.

Samples for direct-export using in-process exporters

References to OpenTelemetry

This section provides links to the OpenTelemetry SDK, and the exporters for OTLP, Cloud Trace, and Cloud Monitoring.

General references:

Go

Java

JavaScript

Python