In previous posts, we covered how IBC works and why it's secure, and how to run reliable Cosmos nodes on Kubernetes. Now it's time to put those pieces together — because a well-run node is only half the story. Without a relayer, those nodes can't actually communicate across chains.
This post walks through two approaches: a production-grade Kubernetes setup with health checks, resource management, and persistent storage, followed by a simpler local setup for testing and custom chain exploration.
Option 1: Kubernetes Setup (Production-Ready)
The Core Idea
We use a StatefulSet to give the relayer a permanent home in the cluster. This ensures the bridge retains its synchronization point and transaction history across pod restarts — critical for a process that needs to pick up exactly where it left off.
For secrets, we use the External Secrets Operator (ESO) so that private keys never end up hardcoded in the repository.
Prerequisites
Docker
Minikube
Helm & External Secrets Operator
1. The Startup Script (configs/entrypoint.sh)
This script runs when the container boots. It handles folder creation, injects secrets from environment variables, fetches chain metadata, and starts the relayer if a path is configured.
sh
#!/bin/sh
set -e
mkdir -p /home/relayer/.relayer/config
envsubst < /tmpl/config.template.yaml > /home/relayer/.relayer/config/config.yaml
rly keys restore cosmoshub default "$COSMOS_MNEMONIC" || true
rly keys restore osmosis default "$OSMOSIS_MNEMONIC" || true
rly paths fetch
if [ -n "$PATH_NAME" ]; then
exec rly start "$PATH_NAME" --enable-metrics-server --metrics-listen-addr 0.0.0.0:5183
else
while true; do sleep 3600; done
fi
#!/bin/sh
set -e
mkdir -p /home/relayer/.relayer/config
envsubst < /tmpl/config.template.yaml > /home/relayer/.relayer/config/config.yaml
rly keys restore cosmoshub default "$COSMOS_MNEMONIC" || true
rly keys restore osmosis default "$OSMOSIS_MNEMONIC" || true
rly paths fetch
if [ -n "$PATH_NAME" ]; then
exec rly start "$PATH_NAME" --enable-metrics-server --metrics-listen-addr 0.0.0.0:5183
else
while true; do sleep 3600; done
fi
#!/bin/sh
set -e
mkdir -p /home/relayer/.relayer/config
envsubst < /tmpl/config.template.yaml > /home/relayer/.relayer/config/config.yaml
rly keys restore cosmoshub default "$COSMOS_MNEMONIC" || true
rly keys restore osmosis default "$OSMOSIS_MNEMONIC" || true
rly paths fetch
if [ -n "$PATH_NAME" ]; then
exec rly start "$PATH_NAME" --enable-metrics-server --metrics-listen-addr 0.0.0.0:5183
else
while true; do sleep 3600; done
fi
2. Config Template (configs/config.template.yaml)
The config template is populated at runtime using envsubst, keeping sensitive RPC addresses and chain settings out of version control.
yaml
global:
timeout: 20s
chains:
cosmoshub:
type: cosmos
value:
chain-id: cosmoshub-4
rpc-addr: ${COSMOS_RPC}
account-prefix: cosmos
gas-prices: 0.01uatom
osmosis:
type: cosmos
value:
chain-id: osmosis-1
rpc-addr: ${OSMOSIS_RPC}
account-prefix: osmo
gas-prices
global:
timeout: 20s
chains:
cosmoshub:
type: cosmos
value:
chain-id: cosmoshub-4
rpc-addr: ${COSMOS_RPC}
account-prefix: cosmos
gas-prices: 0.01uatom
osmosis:
type: cosmos
value:
chain-id: osmosis-1
rpc-addr: ${OSMOSIS_RPC}
account-prefix: osmo
gas-prices
global:
timeout: 20s
chains:
cosmoshub:
type: cosmos
value:
chain-id: cosmoshub-4
rpc-addr: ${COSMOS_RPC}
account-prefix: cosmos
gas-prices: 0.01uatom
osmosis:
type: cosmos
value:
chain-id: osmosis-1
rpc-addr: ${OSMOSIS_RPC}
account-prefix: osmo
gas-prices
3. Helm Templates
ConfigMap (templates/configmap.yaml)
This manifest reads the files from your configs/ folder and mounts them into the cluster:
yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "cosmos-relayer.fullname" . }}-files
data:
{{ (.Files.Glob "configs/*"
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "cosmos-relayer.fullname" . }}-files
data:
{{ (.Files.Glob "configs/*"
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "cosmos-relayer.fullname" . }}-files
data:
{{ (.Files.Glob "configs/*"
External Secret (templates/external-secret.yaml)
The ExternalSecret pulls mnemonics and RPC endpoints from Vault and creates a native Kubernetes secret that the pod can consume:
yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: {{ include "cosmos-relayer.fullname" . }}-eso
spec:
refreshInterval: 1h
secretStoreRef:
name: vault-backend
kind: SecretStore
target:
name: relayer-secrets-target
creationPolicy: Owner
data:
- secretKey: COSMOS_MNEMONIC
remoteRef:
key: {{ .Values.keyName }}
property: COSMOS_MNEMONIC
- secretKey: OSMOSIS_MNEMONIC
remoteRef:
key: {{ .Values.keyName }}
property: OSMOSIS_MNEMONIC
- secretKey: COSMOS_RPC
remoteRef:
key: {{ .Values.keyName }}
property: COSMOS_RPC
- secretKey: OSMOSIS_RPC
remoteRef:
key: {{ .Values.keyName }}
property
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: {{ include "cosmos-relayer.fullname" . }}-eso
spec:
refreshInterval: 1h
secretStoreRef:
name: vault-backend
kind: SecretStore
target:
name: relayer-secrets-target
creationPolicy: Owner
data:
- secretKey: COSMOS_MNEMONIC
remoteRef:
key: {{ .Values.keyName }}
property: COSMOS_MNEMONIC
- secretKey: OSMOSIS_MNEMONIC
remoteRef:
key: {{ .Values.keyName }}
property: OSMOSIS_MNEMONIC
- secretKey: COSMOS_RPC
remoteRef:
key: {{ .Values.keyName }}
property: COSMOS_RPC
- secretKey: OSMOSIS_RPC
remoteRef:
key: {{ .Values.keyName }}
property
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: {{ include "cosmos-relayer.fullname" . }}-eso
spec:
refreshInterval: 1h
secretStoreRef:
name: vault-backend
kind: SecretStore
target:
name: relayer-secrets-target
creationPolicy: Owner
data:
- secretKey: COSMOS_MNEMONIC
remoteRef:
key: {{ .Values.keyName }}
property: COSMOS_MNEMONIC
- secretKey: OSMOSIS_MNEMONIC
remoteRef:
key: {{ .Values.keyName }}
property: OSMOSIS_MNEMONIC
- secretKey: COSMOS_RPC
remoteRef:
key: {{ .Values.keyName }}
property: COSMOS_RPC
- secretKey: OSMOSIS_RPC
remoteRef:
key: {{ .Values.keyName }}
property
StatefulSet (templates/statefulset.yaml)
The StatefulSet handles scheduling, security context, probes, and persistent volume claims. The podAntiAffinity rule is optional but recommended — it spreads replicas across nodes to improve availability:
yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ include "cosmos-relayer.fullname" . }}
spec:
serviceName: {{ include "cosmos-relayer.fullname" . }}
replicas: {{ .Values.replicaCount }}
template:
spec:
{{- if .Values.affinity.enabled }}
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
topologyKey: kubernetes.io/hostname
labelSelector:
matchLabels:
app: {{ include "cosmos-relayer.name" . }}
{{- end }}
securityContext:
fsGroup: 1000
runAsUser: 1000
runAsNonRoot: true
containers:
- name: rly
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
envFrom:
- secretRef:
name: relayer-secrets-target
resources:
{{- toYaml .Values.resources | nindent 12 }}
readinessProbe:
tcpSocket:
port: {{ .Values.metrics.port }}
initialDelaySeconds: {{ .Values.probes.readiness.initialDelaySeconds }}
livenessProbe:
tcpSocket:
port: {{ .Values.metrics.port }}
initialDelaySeconds: {{ .Values.probes.liveness.initialDelaySeconds }}
command: ["/bin/sh"]
args: ["/scripts/entrypoint.sh"]
volumeMounts:
- name: go-relayer-storage
mountPath: /home/relayer/.relayer
- name: scripts
mountPath: /scripts/entrypoint.sh
subPath: entrypoint.sh
- name: scripts
mountPath: /tmpl/config.template.yaml
subPath: config.template.yaml
volumes:
- name: scripts
configMap:
name: {{ include "cosmos-relayer.fullname" . }}-files
defaultMode: 0755
volumeClaimTemplates:
- metadata:
name: go-relayer-storage
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ include "cosmos-relayer.fullname" . }}
spec:
serviceName: {{ include "cosmos-relayer.fullname" . }}
replicas: {{ .Values.replicaCount }}
template:
spec:
{{- if .Values.affinity.enabled }}
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
topologyKey: kubernetes.io/hostname
labelSelector:
matchLabels:
app: {{ include "cosmos-relayer.name" . }}
{{- end }}
securityContext:
fsGroup: 1000
runAsUser: 1000
runAsNonRoot: true
containers:
- name: rly
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
envFrom:
- secretRef:
name: relayer-secrets-target
resources:
{{- toYaml .Values.resources | nindent 12 }}
readinessProbe:
tcpSocket:
port: {{ .Values.metrics.port }}
initialDelaySeconds: {{ .Values.probes.readiness.initialDelaySeconds }}
livenessProbe:
tcpSocket:
port: {{ .Values.metrics.port }}
initialDelaySeconds: {{ .Values.probes.liveness.initialDelaySeconds }}
command: ["/bin/sh"]
args: ["/scripts/entrypoint.sh"]
volumeMounts:
- name: go-relayer-storage
mountPath: /home/relayer/.relayer
- name: scripts
mountPath: /scripts/entrypoint.sh
subPath: entrypoint.sh
- name: scripts
mountPath: /tmpl/config.template.yaml
subPath: config.template.yaml
volumes:
- name: scripts
configMap:
name: {{ include "cosmos-relayer.fullname" . }}-files
defaultMode: 0755
volumeClaimTemplates:
- metadata:
name: go-relayer-storage
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ include "cosmos-relayer.fullname" . }}
spec:
serviceName: {{ include "cosmos-relayer.fullname" . }}
replicas: {{ .Values.replicaCount }}
template:
spec:
{{- if .Values.affinity.enabled }}
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
topologyKey: kubernetes.io/hostname
labelSelector:
matchLabels:
app: {{ include "cosmos-relayer.name" . }}
{{- end }}
securityContext:
fsGroup: 1000
runAsUser: 1000
runAsNonRoot: true
containers:
- name: rly
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
envFrom:
- secretRef:
name: relayer-secrets-target
resources:
{{- toYaml .Values.resources | nindent 12 }}
readinessProbe:
tcpSocket:
port: {{ .Values.metrics.port }}
initialDelaySeconds: {{ .Values.probes.readiness.initialDelaySeconds }}
livenessProbe:
tcpSocket:
port: {{ .Values.metrics.port }}
initialDelaySeconds: {{ .Values.probes.liveness.initialDelaySeconds }}
command: ["/bin/sh"]
args: ["/scripts/entrypoint.sh"]
volumeMounts:
- name: go-relayer-storage
mountPath: /home/relayer/.relayer
- name: scripts
mountPath: /scripts/entrypoint.sh
subPath: entrypoint.sh
- name: scripts
mountPath: /tmpl/config.template.yaml
subPath: config.template.yaml
volumes:
- name: scripts
configMap:
name: {{ include "cosmos-relayer.fullname" . }}-files
defaultMode: 0755
volumeClaimTemplates:
- metadata:
name: go-relayer-storage
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage
Monitoring & PDB (templates/monitoring.yaml)
The Service exposes the metrics port. A ServiceMonitor wires it into Prometheus, and the PodDisruptionBudget prevents the pod from being evicted during cluster maintenance:
yaml
apiVersion: v1
kind: Service
metadata:
name: {{ include "cosmos-relayer.fullname" . }}
spec:
selector:
app: {{ include "cosmos-relayer.name" . }}
ports:
- name: metrics
port: {{ .Values.metrics.port }}
---
{{- if .Values.metrics.enabled }}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ include "cosmos-relayer.fullname" . }}
labels:
release: {{ .Values.metrics.prometheusReleaseLabel }}
spec:
selector:
matchLabels:
app: {{ include "cosmos-relayer.name" . }}
endpoints:
- port: metrics
path: {{ .Values.metrics.path }}
{{- end }}
---
{{- if .Values.pdb.enabled }}
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: {{ include "cosmos-relayer.fullname" . }}
spec:
minAvailable: {{ .Values.pdb.minAvailable }}
selector:
matchLabels:
app: {{ include "cosmos-relayer.name"
apiVersion: v1
kind: Service
metadata:
name: {{ include "cosmos-relayer.fullname" . }}
spec:
selector:
app: {{ include "cosmos-relayer.name" . }}
ports:
- name: metrics
port: {{ .Values.metrics.port }}
---
{{- if .Values.metrics.enabled }}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ include "cosmos-relayer.fullname" . }}
labels:
release: {{ .Values.metrics.prometheusReleaseLabel }}
spec:
selector:
matchLabels:
app: {{ include "cosmos-relayer.name" . }}
endpoints:
- port: metrics
path: {{ .Values.metrics.path }}
{{- end }}
---
{{- if .Values.pdb.enabled }}
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: {{ include "cosmos-relayer.fullname" . }}
spec:
minAvailable: {{ .Values.pdb.minAvailable }}
selector:
matchLabels:
app: {{ include "cosmos-relayer.name"
apiVersion: v1
kind: Service
metadata:
name: {{ include "cosmos-relayer.fullname" . }}
spec:
selector:
app: {{ include "cosmos-relayer.name" . }}
ports:
- name: metrics
port: {{ .Values.metrics.port }}
---
{{- if .Values.metrics.enabled }}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ include "cosmos-relayer.fullname" . }}
labels:
release: {{ .Values.metrics.prometheusReleaseLabel }}
spec:
selector:
matchLabels:
app: {{ include "cosmos-relayer.name" . }}
endpoints:
- port: metrics
path: {{ .Values.metrics.path }}
{{- end }}
---
{{- if .Values.pdb.enabled }}
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: {{ include "cosmos-relayer.fullname" . }}
spec:
minAvailable: {{ .Values.pdb.minAvailable }}
selector:
matchLabels:
app: {{ include "cosmos-relayer.name"
4. Values Files
You'll need two values files: a base values.yaml with defaults and structure, and a values-relayers.yaml with your actual deployment overrides.
values-relayers.yaml
yaml
image:
repository: <your-repo>/relayer
tag: v2.6.0
pullPolicy: IfNotPresent
replicaCount: 1
keyName: relayer-k8s
persistence:
size: 2Gi
metrics:
enabled: true
port: 5183
path: /relayer/metrics
interval: 30s
prometheusReleaseLabel
image:
repository: <your-repo>/relayer
tag: v2.6.0
pullPolicy: IfNotPresent
replicaCount: 1
keyName: relayer-k8s
persistence:
size: 2Gi
metrics:
enabled: true
port: 5183
path: /relayer/metrics
interval: 30s
prometheusReleaseLabel
image:
repository: <your-repo>/relayer
tag: v2.6.0
pullPolicy: IfNotPresent
replicaCount: 1
keyName: relayer-k8s
persistence:
size: 2Gi
metrics:
enabled: true
port: 5183
path: /relayer/metrics
interval: 30s
prometheusReleaseLabel
values.yaml
yaml
image:
repository: <your-repo>/relayer
tag: v2.6.0
pullPolicy: IfNotPresent
replicaCount: 1
keyName: relayer-k8s
rpc:
osmo: https://rpc.testnet.osmosis.zone:443
pulsar: https://pulsar.rpc.secretnodes.com:443
mnemonics:
osmo: ""
pulsar: ""
persistence:
size: 2Gi
storageClassName: ""
relayer:
pathName: ""
affinity:
enabled: true
resources:
requests:
cpu: "100m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
probes:
readiness:
enabled: true
initialDelaySeconds: 15
periodSeconds: 20
timeoutSeconds: 5
failureThreshold: 3
liveness:
enabled: true
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
pdb:
enabled: true
minAvailable: 1
metrics:
enabled: true
port: 5183
path: /relayer/metrics
interval: 30s
prometheusReleaseLabel
image:
repository: <your-repo>/relayer
tag: v2.6.0
pullPolicy: IfNotPresent
replicaCount: 1
keyName: relayer-k8s
rpc:
osmo: https://rpc.testnet.osmosis.zone:443
pulsar: https://pulsar.rpc.secretnodes.com:443
mnemonics:
osmo: ""
pulsar: ""
persistence:
size: 2Gi
storageClassName: ""
relayer:
pathName: ""
affinity:
enabled: true
resources:
requests:
cpu: "100m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
probes:
readiness:
enabled: true
initialDelaySeconds: 15
periodSeconds: 20
timeoutSeconds: 5
failureThreshold: 3
liveness:
enabled: true
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
pdb:
enabled: true
minAvailable: 1
metrics:
enabled: true
port: 5183
path: /relayer/metrics
interval: 30s
prometheusReleaseLabel
image:
repository: <your-repo>/relayer
tag: v2.6.0
pullPolicy: IfNotPresent
replicaCount: 1
keyName: relayer-k8s
rpc:
osmo: https://rpc.testnet.osmosis.zone:443
pulsar: https://pulsar.rpc.secretnodes.com:443
mnemonics:
osmo: ""
pulsar: ""
persistence:
size: 2Gi
storageClassName: ""
relayer:
pathName: ""
affinity:
enabled: true
resources:
requests:
cpu: "100m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
probes:
readiness:
enabled: true
initialDelaySeconds: 15
periodSeconds: 20
timeoutSeconds: 5
failureThreshold: 3
liveness:
enabled: true
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
pdb:
enabled: true
minAvailable: 1
metrics:
enabled: true
port: 5183
path: /relayer/metrics
interval: 30s
prometheusReleaseLabel
5. Deployment and Verification
With everything in place, deploy with Helm and verify the pod comes up healthy:
bash
kubectl create namespace relayer
helm upgrade --install cosmos-relayer ./cosmos-relayer-helm \
-n relayer \
-f values-relayers.yaml
kubectl -n relayer get pods
kubectl -n relayer logs -f
kubectl create namespace relayer
helm upgrade --install cosmos-relayer ./cosmos-relayer-helm \
-n relayer \
-f values-relayers.yaml
kubectl -n relayer get pods
kubectl -n relayer logs -f
kubectl create namespace relayer
helm upgrade --install cosmos-relayer ./cosmos-relayer-helm \
-n relayer \
-f values-relayers.yaml
kubectl -n relayer get pods
kubectl -n relayer logs -f
6. Funding Wallets and Linking Paths
Before the relayer can submit transactions, the accounts need funds on both chains.
Get wallet addresses: kubectl -n relayer exec -it cosmos-relayer-0 -- rly keys list cosmoshub
Link the path manually (if needed): kubectl -n relayer exec -it cosmos-relayer-0 -- rly tx link hubosmo
Start relaying: Update relayer.pathName: "hubosmo" in your values.yaml and redeploy.
Option 2: Local Testing (Manual Setup)
If you're testing against a custom chain or just want to iterate quickly without a full cluster, local setup is much faster.
bash
git clone https://github.com/cosmos/relayer.git && cd relayer && make install
rly config init
rly chains add cosmoshub osmosis
rly keys restore cosmoshub default
rly keys restore osmosis default
rly start
git clone https://github.com/cosmos/relayer.git && cd relayer && make install
rly config init
rly chains add cosmoshub osmosis
rly keys restore cosmoshub default
rly keys restore osmosis default
rly start
git clone https://github.com/cosmos/relayer.git && cd relayer && make install
rly config init
rly chains add cosmoshub osmosis
rly keys restore cosmoshub default
rly keys restore osmosis default
rly start
Note on custom chains: If you're connecting two locally-running Cosmos chains rather than mainnet, the same flow applies — you'll just need to manually define the chain configs instead of pulling from the chain registry. This is particularly useful when building or testing new IBC-enabled modules before deploying to a live network.
Summary
A production Cosmos relayer isn't just rly start — it needs persistent storage to survive pod restarts, externally managed secrets to stay secure, and health probes so Kubernetes knows when something's wrong. The StatefulSet + ESO pattern covers all three cleanly.
For local development and custom chain work, the manual setup gets you going in minutes. Start there to validate your configuration, then promote to Kubernetes when you're ready to run it reliably.