Monitoring Che Server

You can configure Che to expose JVM metrics such as JVM memory and class loading for Che Server.

Enabling and exposing Che Server metrics

Che exposes the JVM metrics on port 8087 of the che-host Service. You can configure this behaviour.

Procedure

Configure the CheCluster Custom Resource. See Using the CLI to configure the CheCluster Custom Resource.
```
spec:
  components:
    metrics:
      enable: <boolean> (1)
```
1 true to enable, false to disable.

When metrics are enabled, the Che Operator automatically creates the necessary Prometheus resources (ServiceMonitor, Role, and RoleBinding) and adds the openshift.io/cluster-monitoring=true label to the Che namespace to enable metrics collection by the in-cluster Prometheus instance.

Collecting Che Server metrics with Prometheus

The Che Operator automatically manages the Prometheus resources required to collect Che Server metrics when metrics are enabled.

Prerequisites

Your organization’s instance of Che is installed and running in Red Hat OpenShift.
Che metrics are enabled. See Enabling and exposing Che server JVM metrics.

Procedure

The Che Operator automatically creates and manages the following resources when spec.components.metrics.enable is set to true:

A ServiceMonitor object named che-host to scrape metrics from the Che Server
A Role and RoleBinding to grant the prometheus-k8s service account permission to scrape metrics
The openshift.io/cluster-monitoring=true label on the Che namespace to enable discovery by the in-cluster Prometheus instance

No manual configuration is required.

Verification

In the Administrator view of the OpenShift web console, go to Observe → Metrics.
Run a PromQL query to confirm that the metrics are available. For example, enter process_uptime_seconds{job="che-host"} and click Run queries.

To troubleshoot missing metrics, view the Prometheus container logs for possible RBAC-related errors:

Get the name of the Prometheus pod:

$ oc get pods -l app.kubernetes.io/name=prometheus -n openshift-monitoring -o=jsonpath='{.items[*].metadata.name}'

Print the last 20 lines of the Prometheus container logs from the Prometheus pod from the previous step:
```
$ oc logs --tail=20 <prometheus_pod_name> -c prometheus -n openshift-monitoring
```

Additional resources

Viewing Che Server from an OpenShift web console dashboard

After configuring the in-cluster Prometheus instance to collect Che Server JVM metrics, you can view the metrics on a custom dashboard in the Administrator perspective of the OpenShift web console.

Prerequisites

Your organization’s instance of Che is installed and running in Red Hat OpenShift.
An active oc session with administrative permissions to the destination OpenShift cluster. See Getting started with the CLI.
The in-cluster Prometheus instance is collecting metrics. See Collecting Che Server metrics with Prometheus.

Procedure

Create a ConfigMap for the dashboard definition in the openshift-config-managed namespace and apply the necessary label.

$ oc create configmap grafana-dashboard-devspaces-server \
  --from-literal=devspaces-server-dashboard.json="$(curl https://raw.githubusercontent.com/eclipse-che/che-server/main/docs/grafana/openshift-console-dashboard.json)" \
  -n openshift-config-managed

```
$ oc label configmap grafana-dashboard-devspaces-server console.openshift.io/dashboard=true -n openshift-config-managed
```
The dashboard definition is based on Grafana 6.x dashboards. Not all Grafana 6.x dashboard features are supported in the OpenShift web console.

Verification steps

In the Administrator view of the OpenShift web console, go to Observe → Dashboards.
Go to Dashboard → Che Server JVM and verify that the dashboard panels contain data.

Figure 1. Quick Facts

Figure 2. JVM Memory

Figure 3. JVM Misc

Figure 4. JVM Memory Pools (heap)

Figure 5. JVM Memory Pools (Non-Heap)

Figure 6. Garbage Collection

Figure 7. Class loading

Figure 8. Buffer Pools