Loki

Tutorial to collect your namespace’s pod logs with a Grafana Alloy Deployment, reading them through the Kubernetes API, and ship them to Grafana Loki.

This guide assumes a Spring Boot service, but the collection model is application-agnostic: any container that writes to stdout/stderr is picked up automatically. Adapt it to your needs.

Requirements

• A Grafana instance (see the Grafana Helm chart)
• A deployed service (Kubernetes Deployment) that logs to stdout/stderr
• kubectl and helm installed and configured for your Kubernetes cluster
• (optional) a namespace dedicated to your cross-namespace tooling (example: mycorp-monitoring). It can host both Loki and Grafana.

Loki

Install Loki with its Helm chart. For a small, single-tenant customer setup, run Loki in monolithic mode (the SingleBinary deployment, equivalent to -target=all): all Loki components run in one process, which keeps operations simple and scales fine to a few hundred GB/day. See the deployment modes guide for when to graduate to the scalable or distributed topologies.

The OSS Loki Helm chart moved to the grafana-community/helm-charts repository in March 2026. If you have an existing grafana/loki release, review the community repo migration guide before upgrading.

Configure

Create a loki-values.yaml. The two key choices are the deployment mode and the storage backend. Below is a minimal sketch; consult the chart values.yaml for the exhaustive and current schema before applying.

deploymentMode: SingleBinary
loki:
  # Single-tenant setup: disable multi-tenancy so pushes/queries need no tenant header.
  auth_enabled: false
  commonConfig:
    replication_factor: 1
  schemaConfig:
    configs:
      - from: 2024-04-01
        store: tsdb
        object_store: filesystem
        schema: v13
        index:
          prefix: index_
          period: 24h
singleBinary:
  replicas: 1
# Disable the scalable-mode component groups in monolithic mode.
backend:
  replicas: 0
read:
  replicas: 0
write:
  replicas: 0

For production-grade durability, point object_store at your S3-compatible bucket (Ceph RGW on h8lio) rather than filesystem. The chart docs cover the exact storage block.

Installation

# OSS chart, community repository (March 2026 onward)
helm repo add grafana-community https://grafana-community.github.io/helm-charts
helm repo update
helm search repo grafana-community/loki
helm install loki grafana-community/loki -f loki-values.yaml --namespace [NAMESPACE]

(optional) Expose

Deploy Traefik routes to reach your Loki instance from outside the cluster. Most setups keep Loki internal (queried by an in-cluster Grafana) and skip this.

• routes.yaml

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: loki
spec:
  entryPoints:
    - http
  routes:
    - kind: Rule
      match: Host(`loki.mydomain.com`)
      middlewares:
        - name: https-redirect
          namespace: traefik
      services:
        - name: loki
          port: 3100
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: loki-tls
spec:
  entryPoints:
    - https
  routes:
    - kind: Rule
      match: Host(`loki.mydomain.com`)
      services:
        - name: loki
          port: 3100
  tls:
    certResolver: default

Collect logs with Grafana Alloy

Promtail reached End-Of-Life on 2026-03-02. It receives no further development. Grafana Alloy is its successor and the recommended collector going forward. If you are migrating an existing Promtail config, the alloy convert --source-format=promtail command translates it for you.

On a managed h8lio cluster you operate within Kubernetes namespaces, not whole nodes, so log collection stays namespace-scoped: run a single Grafana Alloy Deployment that reads your pods’ logs through the Kubernetes API with the loki.source.kubernetes component and pushes them to Loki. A node-level DaemonSet is deliberately not used here: tailing node log files requires cluster-wide privileges (and scheduling a pod on every node of shared infrastructure) that a namespace tenant does not have on a managed cluster.

The recommended layout mirrors Prometheus: host Loki, Alloy, and Grafana in a dedicated monitoring namespace (an h8lio cluster such as acme-monitoring) and collect logs from your organization’s other namespaces (acme-prod, acme-staging, …). Alloy reaches each namespace through the API with a namespace-scoped Role granted in that namespace, so no cluster-wide access is ever needed. The simplest single-namespace case (collect only your own namespace) is shown first; the organization-wide variant follows.

RBAC: a namespace-scoped Role

Alloy needs read access to pods and their logs in your namespace only. A Role (not a ClusterRole) is sufficient, and it grants no node access.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: alloy-logs
  namespace: [NAMESPACE]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: alloy-logs
  namespace: [NAMESPACE]
rules:
  - apiGroups: [""]
    resources: ["pods", "pods/log"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: alloy-logs
  namespace: [NAMESPACE]
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: alloy-logs
subjects:
  - kind: ServiceAccount
    name: alloy-logs
    namespace: [NAMESPACE]

Organization-wide collection. To collect from your other namespaces, keep the ServiceAccount in the monitoring namespace (acme-monitoring) and apply the same Role plus a RoleBinding in each namespace you collect from (acme-prod, acme-staging, …). Each RoleBinding references the single ServiceAccount in acme-monitoring as its subject (just like the Prometheus scraping RBAC). A namespace tenant cannot use a ClusterRole, so cross-namespace access is granted one explicit RoleBinding at a time.

Alloy configuration

This pipeline discovers the pods in your namespace, tails their logs via the API, and forwards them to Loki. The field names below are the current discovery.kubernetes, loki.source.kubernetes, and loki.write component arguments; check those pages for the exhaustive schema.

discovery.kubernetes "pods" {
  role = "pod"
  namespaces {
    own_namespace = true
  }
}

loki.source.kubernetes "pods" {
  targets    = discovery.kubernetes.pods.targets
  forward_to = [loki.write.default.receiver]
}

loki.write "default" {
  endpoint {
    url = "http://loki:3100/loki/api/v1/push"
  }
}

For the organization-wide layout, replace own_namespace = true with an explicit list of the namespaces you granted the Role in (Alloy only lists pods where its ServiceAccount is allowed):

discovery.kubernetes "pods" {
  role = "pod"
  namespaces {
    names = ["acme-prod", "acme-staging", "acme-monitoring"]
  }
}

Deploy Alloy

Install Alloy with its Helm chart, set the controller type to Deployment (not DaemonSet), and bind it to the ServiceAccount above. Pass the configuration through the chart’s alloy.configMap values (or mount it as a ConfigMap). A plain Kubernetes Deployment that runs the grafana/alloy image with the config mounted works just as well.

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
helm install alloy-logs grafana/alloy \
  -f alloy-values.yaml --namespace [NAMESPACE]

In alloy-values.yaml, set controller.type: deployment, controller.replicas: 1, and serviceAccount.create: false with serviceAccount.name: alloy-logs. See the chart docs for the current values schema rather than copying keys that may drift.

Once running, every pod in your namespace that writes to stdout/stderr appears in Loki automatically. There is nothing to add to your application’s Deployment.

Alternative: a sidecar for file-based logs

If an application genuinely cannot log to stdout/stderr (for example a legacy app that only writes rotating log files), add a per-pod Grafana Alloy sidecar that tails a shared emptyDir volume with the loki.source.file component, forwarding to the same loki.write endpoint. The sidecar must be Alloy, not Promtail, which is EOL. Prefer fixing the application to log to the console; the sidecar is a last resort.

Application logging best practice

Log to stdout/stderr (a console appender), ideally as structured JSON, so the Kubernetes API source picks logs up with no extra wiring and Loki/Grafana can parse fields directly. Do not write to a shared log file for a collector to tail; that pattern is obsolete.

For a Spring Boot service, the simplest option is to keep the default console appender and let Logback emit JSON. With logstash-logback-encoder on the classpath, a minimal logback-spring.xml looks like:

<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
  </appender>
  <root level="INFO">
    <appender-ref ref="STDOUT"/>
  </root>
</configuration>

If you do not need JSON, the Spring Boot default console pattern already writes to stdout and is collected as-is. No logging.file.name and no emptyDir volume are required.

Grafana

h8lio provisions a shared Grafana at https://monitoring.h8l.io with read-only dashboards scoped to your organization, but it does not expose your Loki logs. To explore your logs and build your own dashboards, run your own Grafana in the monitoring namespace (the same place that hosts Prometheus for your metrics) and add Loki as a datasource.

DataSource

Configure a Loki datasource on http://loki:3100 (if Grafana runs in the same namespace as Loki). To wire an external Grafana instance, use the public route you exposed above. This step is unchanged from earlier Loki setups.

Dashboard

Here is a link to a Loki Dashboard you can import into your Grafana instance. You can also import a dashboard from the Grafana Marketplace.

Explore the logs

Once the Alloy Deployment is running in your namespace and your services are producing logs, open your Grafana instance, go to Explore, select the Loki datasource, and query with a label selector such as {namespace="byzaneo-one", pod=~"myservice.*"}. Logs appear within a few seconds of being emitted.

You can build LogQL-based alerts on critical log levels across services and be notified through your usual Grafana alerting channels.