Skip to main content

Kubernetes Executor Setup

This guide configures the yardrail-worker to run CWL container tools as Kubernetes Pods with S3-backed CSI volumes instead of local Docker containers. For an overview of how this works, see Kubernetes Executor Architecture.

1. Install the S3 CSI driver​

The Kubernetes executor needs a CSI driver that can mount S3 buckets as inline volumes on Pods. Choose the path that matches your S3 provider.

Option A: AWS S3 with Mountpoint CSI (EKS)​

The Mountpoint for Amazon S3 CSI driver is the recommended driver for AWS S3.

Create an IAM policy​

The CSI driver needs read/write access to the workflow artifacts bucket. Create a policy:

yardrail-s3-csi-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::yardrail-workflows",
"arn:aws:s3:::yardrail-workflows/*"
]
}
]
}
aws iam create-policy \
--policy-name YardrailS3CSIPolicy \
--policy-document file://yardrail-s3-csi-policy.json

Create an IAM role for the CSI driver (IRSA)​

eksctl create iamserviceaccount \
--name s3-csi-driver-sa \
--namespace kube-system \
--cluster your-cluster-name \
--role-name YardrailS3CSIRole \
--attach-policy-arn arn:aws:iam::YOUR_ACCOUNT:policy/YardrailS3CSIPolicy \
--approve

Install the driver​

# Via EKS add-on (recommended for EKS)
aws eks create-addon \
--cluster-name your-cluster-name \
--addon-name aws-mountpoint-s3-csi-driver \
--service-account-role-arn arn:aws:iam::YOUR_ACCOUNT:role/YardrailS3CSIRole

# Or via Helm
helm repo add aws-mountpoint-s3-csi-driver https://awslabs.github.io/mountpoint-s3-csi-driver
helm repo update
helm install aws-mountpoint-s3-csi-driver \
aws-mountpoint-s3-csi-driver/aws-mountpoint-s3-csi-driver \
--namespace kube-system \
--set node.serviceAccount.name=s3-csi-driver-sa

Verify the driver​

kubectl get daemonset -n kube-system aws-mountpoint-s3-csi-driver

The DaemonSet should show pods running on every node in the cluster.

Option B: MinIO or self-hosted S3​

For non-AWS S3 providers, you need a CSI driver that supports S3-compatible endpoints. Options include:

  • csi-s3 — supports custom S3 endpoints, used with MinIO and other S3-compatible stores.

Install the driver following its documentation. The key requirement is that the driver supports CSI inline volumes with configurable volumeAttributes for bucket name, prefix, and endpoint.

Note the driver name — you will need it for worker.csiDriverName in the next step.

2. Create the workflow artifacts bucket​

The Kubernetes executor stores staged inputs and workflow outputs in a dedicated S3 bucket, separate from the frontend's avatar bucket.

# AWS S3
aws s3 mb s3://yardrail-workflows

# MinIO
mc mb myminio/yardrail-workflows

3. Configure the Helm values​

Add the Kubernetes executor configuration to your values.yaml:

values.yaml
worker:
executor: kubernetes

# CSI driver name — must match your installed driver
# AWS Mountpoint: s3.csi.aws.amazon.com
# csi-s3: ch.ctrox.csi.s3-driver (check your driver's docs)
csiDriverName: "s3.csi.aws.amazon.com"

# Namespace for workflow Pods (empty = same as the worker)
workflowNamespace: ""

# Extra attributes passed to CSI volumeAttributes on every workflow Pod
# Use this for driver-specific config like endpoint, region, or auth
# csiVolumeAttrs: '{"region": "us-east-1"}'

# Kubernetes Secrets for pulling private workflow container images
workflowImagePullSecrets: []

# S3 bucket for workflow artifacts
s3:
bucket: yardrail-workflows

serviceAccount:
create: true
automount: true

CSI volume attributes​

The executor always injects bucketName and prefix into each volume's attributes. You can pass additional driver-specific attributes via CSI_VOLUME_ATTRS (JSON) or the Helm value above. Common examples:

DriverAttributePurpose
Mountpoint S3regionAWS region for the bucket
csi-s3endpointS3-compatible endpoint URL
csi-s3accessKeyID, secretAccessKeyInline credentials (prefer Secrets in production)

4. Deploy​

helm upgrade yardrail yardrail/yardrail \
--namespace yardrail \
--values values.yaml

5. Verify RBAC​

The Helm chart creates a Role and RoleBinding automatically when worker.executor is kubernetes:

kubectl -n yardrail get role,rolebinding | grep yardrail-worker

Expected output:

role.rbac.authorization.k8s.io/yardrail-worker
rolebinding.rbac.authorization.k8s.io/yardrail-worker

The Role grants create, get, list, watch, delete on pods and create, get on pods/attach.

Cross-namespace workflow Pods​

If worker.workflowNamespace is set to a namespace different from the worker's, the chart-created Role only covers the worker's namespace. You must manually create a Role and RoleBinding in the target namespace:

workflow-namespace-rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: yardrail-worker
namespace: yardrail-workflows # your workflowNamespace
rules:
- apiGroups: [""]
resources: [pods]
verbs: [create, get, list, watch, delete]
- apiGroups: [""]
resources: [pods/attach]
verbs: [create, get]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: yardrail-worker
namespace: yardrail-workflows
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: yardrail-worker
subjects:
- kind: ServiceAccount
name: yardrail-worker
namespace: yardrail # the worker's namespace
kubectl apply -f workflow-namespace-rbac.yaml

6. Check the worker​

kubectl -n yardrail logs deployment/yardrail-worker | head -5

Confirm the log line shows executor=kubernetes:

{"level":"INFO","msg":"yardrail-worker starting","executor":"kubernetes","s3_endpoint":"s3.amazonaws.com",...}

Next steps​