Data | Domains
Open Sourcing Airflow Local Development 
Kris Geusebroek 05 Apr, 2019
Share 
Self-hosting Langfuse with Kubernets
A local GPU cluster is great for developing sensitive LLM applications. If you are working on sensitive applications, you don’t want logging and monitoring sent over the internet to a server managed by a third party.
Langfuse is a great open-source framework for managing traces, evals, prompt management and metrics to debug and improve your LLM application. This blog post describes how we set up our self-hosted Langfuse server for a local GPU cluster.
Our stack
Before we dive in, let’s highlight the tools we use to get a local Langfuse server deployed:
- Helm: Charts for the Kubernetes config
- Kustomize: To modify the Charts
- Secret generator: To create and manage secrets
- Traefik: To route incoming traffic
- Pihole: DNS server for DNS-level filtering
Configuration
Langfuse provides a Helm Chart in their repo and a minimal installation example. This section shows how to modify these to work with the stack mentioned above.
Kustomize
We start by referencing the Helm Chart in our kustomization.yml:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: langfuse
helmCharts:
- name: langfuse
releaseName: langfuse
version: "1.2.5"
repo: https://langfuse.github.io/langfuse-k8s
namespace: langfuse
Namespace
To create a namespace, under the resources/ folder, a file called namespace.yml is created:
apiVersion: v1
kind: Namespace
metadata:
name: langfuse
This file is referenced by appending the following to the kustomization.yml:
...
resources:
- resources/namespace.yml
Secrets
Next, we set mandatory secrets for langfuse and the services langfuse depends on (postgresql, redis, clickhouse and minio) in the kustomization.yml:
...
helmCharts:
- name: langfuse
...
valuesInline:
langfuse:
...
salt:
secretKeyRef:
name: langfuse-salt
key: salt
nextauth:
secret:
secretKeyRef:
name: langfuse-nextauth
key: encryption-key
postgresql:
auth:
existingSecret: langfuse-postgresql
secretKeys:
userPasswordKey: postgres-password
adminPasswordKey: postgres-password
redis:
auth:
existingSecret: langfuse-redis
existingSecretPasswordKey: password
clickhouse:
auth:
existingSecret: langfuse-clickhouse
existingSecretKey: password
s3:
deploy: true
bucket: langfuse
endpoint: http://langfuse-s3:9000
forcePathStyle: true
auth:
existingSecret: langfuse-minio
rootUserSecretKey: user
rootPasswordSecretKey: password
...
We use kubernetes-secret-generator to manage secrets in resources/secrets.yml:
apiVersion: v1
kind: Secret
metadata:
name: langfuse-postgresql
annotations:
secret-generator.v1.mittwald.de/autogenerate: postgres-password
secret-generator.v1.mittwald.de/encoding: hex
secret-generator.v1.mittwald.de/length: "32"
---
...
This is repeated for all secrets referenced in the kustomization.yml, with the addition of a user for minio:
...
---
apiVersion: v1
kind: Secret
metadata:
name: langfuse-minio
annotations:
secret-generator.v1.mittwald.de/autogenerate: password
secret-generator.v1.mittwald.de/encoding: base64
secret-generator.v1.mittwald.de/length: "32"
data:
user: bWluaW8=
The secrets file is referenced under resources in the kustomization.yml, similarly to the namespaces:
resources:
...
- resources/secrets.yml
Configure Ingress – Traefik
To allow local traffic to reach our self-hosted Langfuse server, the ingress configuration is passed to langfuse in the kustomization.yml:
langfuse:
ingress:
enabled: true
classname: traefik
hosts:
- host: langfuse.my-server.internal
paths:
- path: /
pathType: Prefix
...
Traefik configures the ingress to go from langfuse.my-server.internal to langfuse.langfuse.svc.cluster.local.
Complete Example
To recap, this results in the following files:
langfuse/
├── kustomization.yml
└── resources
├── namespace.yml
└── secrets.yml
kustomization.yml:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: langfuse
helmCharts:
- name: langfuse
releaseName: langfuse
version: "1.2.5"
repo: https://langfuse.github.io/langfuse-k8s
namespace: langfuse
valuesInline:
langfuse:
ingress:
enabled: true
classname: traefik
hosts:
- host: langfuse.my-server.internal
paths:
- path: /
pathType: Prefix
salt:
secretKeyRef:
key: salt
name: langfuse-salt
nextauth:
secret:
secretKeyRef:
key: key
name: langfuse-nextauth
postgresql:
port: 5432
auth:
existingSecret: langfuse-postgresql
secretKeys:
userPasswordKey: postgresql-password
adminPasswordKey: postgresql-password
redis:
auth:
existingSecret: langfuse-redis
existingSecretPasswordKey: redis-password
clickhouse:
auth:
existingSecret: langfuse-clickhouse
existingSecretKey: clickhouse-password
s3:
deploy: true
bucket: langfuse
endpoint: http://langfuse-s3:9000
forcePathStyle: true
auth:
existingSecret: langfuse-minio
rootUserSecretKey: user
rootPasswordSecretKey: minio-password
resources:
- resources/namespace.yml
- resources/secrets.yml
resources/namespace.yml:
apiVersion: v1
kind: Namespace
metadata:
name: langfuse
resources/secrets.yml:
apiVersion: v1
kind: Secret
metadata:
name: langfuse-salt
annotations:
secret-generator.v1.mittwald.de/autogenerate: salt
secret-generator.v1.mittwald.de/encoding: base64
secret-generator.v1.mittwald.de/length: "32"
---
apiVersion: v1
kind: Secret
metadata:
name: langfuse-nextauth
annotations:
secret-generator.v1.mittwald.de/autogenerate: key
secret-generator.v1.mittwald.de/encoding: base64
secret-generator.v1.mittwald.de/length: "32"
---
apiVersion: v1
kind: Secret
metadata:
name: langfuse-postgresql
annotations:
secret-generator.v1.mittwald.de/autogenerate: postgresql-password
secret-generator.v1.mittwald.de/encoding: hex
secret-generator.v1.mittwald.de/length: "32"
---
apiVersion: v1
kind: Secret
metadata:
name: langfuse-redis
annotations:
secret-generator.v1.mittwald.de/autogenerate: redis-password
secret-generator.v1.mittwald.de/encoding: base64
secret-generator.v1.mittwald.de/length: "32"
---
apiVersion: v1
kind: Secret
metadata:
name: langfuse-clickhouse
annotations:
secret-generator.v1.mittwald.de/autogenerate: clickhouse-password
secret-generator.v1.mittwald.de/encoding: base64
secret-generator.v1.mittwald.de/length: "32"
---
apiVersion: v1
kind: Secret
metadata:
name: langfuse-minio
annotations:
secret-generator.v1.mittwald.de/autogenerate: minio-password
secret-generator.v1.mittwald.de/encoding: base64
secret-generator.v1.mittwald.de/length: "32"
data:
user: bWluaW8=
Access and Test
Thanks to Traefik, Langfuse can be accessed on the internal network of the Kubernetes cluster on: http://langfuse.my-server.internal
App
When connecting to Langfuse, e.g. if you want to log traces from your agentic app, you set the env variable to the same URL:
LANGFUSE_HOST="http://langfuse.my-server.internal"
UI
In Langfuse’s UI, you can create an Organization, a Project, and API keys. To access the UI from your local device, we make use of an existing DNS server (Pihole) and VPN (Tailscale) setup. All that is needed to access Langfuse is to add a DNS entry.
DNS entry
The DNS entry is added to the kustomize.yml of Pihole:
helmCharts:
- name: pihole
releaseName: pihole
version: "v2.24.0"
repo: https://mojo2600.github.io/pihole-kubernetes/
valuesInline:
dnsmasq:
customDnsEntries:
...
- address=/langfuse.my-server.internal/100.74.16.73
...
Now, Langfuse can be accessed from a local device: http://langfuse.my-server.internal. After creating an Organization and Project, you can create the API keys (LANGFUSE_SECRET_KEY, LANGFUSE_PUBLIC_KEY) in the UI.
Test via Python
To test the deployment, run a temporary container:
kubectl run langfuse-test --image=python:3.12-slim-bookworm -n langfuse --rm -it -- /bin/bash
Inside that container, install the Python package langfuse:
pip install langfuse
Export the environment variables:
export LANGFUSE_HOST=http://langfuse.my-server.internal
export LANGFUSE_SECRET_KEY=sk-lf-...
export LANGFUSE_PUBLIC_KEY=pk-lf-...
Run the following Python code:
from langfuse import observe
@observe()
def fn():
pass
@observe()
def main():
fn()
main()
⚠️ Note that from v3 it is simply from langfuse instead of from langfuse.decorators in v2.
If everything went successfully, the traces should now appear in the UI: 
Extensions
To professionalize this setup, the Langfuse server can be extended by:
- Using an existing keyvault or sealed-secrets to keep track of secrets.
- Setting up encryption.
- Optimize setup for reliability and uptime.
- Setting up SSO.
References
Jetze is a well-rounded Machine Learning Engineer, who is as comfortable solving Data Science use cases as he is productionizing them in the cloud. His expertise includes: MLOps, GenAI, and Cloud Engineering. As a researcher, he has published papers on: Computer Vision and Natural Language Processing and Machine Learning in general.
