Security & RBAC¶
Kubernaut follows a least-privilege model: each service runs under its own ServiceAccount with only the permissions it needs. This page is the consolidated reference for all RBAC resources.
Helm vs Operator RBAC
The Helm chart and the Kubernaut Operator create the same logical set of ClusterRoles, but the Operator prefixes each name with the CR's namespace (e.g., kubernaut-system-gateway-role) to prevent collisions when multiple Kubernaut CRs exist. The Operator creates 14 baseline ClusterRoles (including the API Frontend ClusterRole in v1.5+), plus 2 additional ones (alertmanager-view, gateway-signal-source) when spec.monitoring.enabled: true, 6 per-persona tool ClusterRoles for SAR authorization (v1.5+), and an optional workflowexecution-awx ClusterRole when Ansible integration is enabled.
The Operator also supports spec.kubernautAgent.additionalClusterRoleBindings — a list of pre-existing ClusterRole names to bind to the Kubernaut Agent ServiceAccount (max 64). Use with caution: any writable cluster-scoped privileges referenced here are granted to the agent, creating a privilege escalation path. Restrict who may edit the Kubernaut CR via cluster RBAC. See the Operator threat model for details.
Signal Ingestion¶
The Gateway authenticates every signal ingestion request using Kubernetes TokenReview + SubjectAccessReview (SAR):
sequenceDiagram
participant AM as AlertManager
participant GW as Gateway
participant K8s as Kubernetes API
AM->>GW: POST /api/v1/signals/prometheus<br/>Authorization: Bearer <token>
GW->>K8s: TokenReview (validate token)
K8s-->>GW: Authenticated as SA "alertmanager-..."
GW->>K8s: SubjectAccessReview<br/>(can SA create services/gateway-service?)
K8s-->>GW: Allowed
GW->>GW: Process signal, create RemediationRequest
Gateway RBAC¶
The Gateway's own ClusterRole (gateway-role) includes:
| apiGroup | Resources | Verbs | Purpose |
|---|---|---|---|
kubernaut.ai |
remediationrequests, remediationrequests/status |
create, get, list, watch, update, patch | Create and manage RRs from incoming signals |
| (core) | namespaces |
get, list, watch | Scope label checks (kubernaut.ai/managed) |
| (core) | nodes, pods, services, persistentvolumes |
get, list, watch | Owner chain resolution for fingerprinting |
apps |
deployments, replicasets, statefulsets, daemonsets |
get, list, watch | Owner chain resolution |
batch |
jobs, cronjobs |
get, list, watch | Owner chain resolution |
authentication.k8s.io |
tokenreviews |
create | Validate bearer tokens from signal sources |
authorization.k8s.io |
subjectaccessreviews |
create | Check signal source RBAC via SAR |
coordination.k8s.io |
leases |
get, create, update, delete | Leader election |
Signal Source RBAC¶
External signal sources (AlertManager, custom webhooks) must satisfy two requirements:
- A valid bearer token -- The source must send its ServiceAccount token in the
Authorizationheader. The Gateway validates it via TokenReview. -
SAR authorization -- The ServiceAccount must have
createpermission onservices/gateway-service. The chart provides thegateway-signal-sourceClusterRole for this:
The Helm value gateway.auth.signalSources creates a ClusterRoleBinding for each entry:
gateway:
auth:
signalSources:
- name: alertmanager
serviceAccount: alertmanager-kube-prometheus-stack-alertmanager
namespace: monitoring
If you deploy a Kubernetes Event Exporter separately (not included in the chart since v1.1), its ServiceAccount must also be registered as a signal source in the same signalSources list.
Without the bearer token, the Gateway returns 401 Unauthorized. Without the ClusterRoleBinding, the Gateway returns 403 Forbidden.
See Installation for the complete AlertManager configuration example.
CRD Controllers¶
Each CRD controller runs under its own ServiceAccount with a dedicated ClusterRole scoped to the CRDs it manages. All controllers also get a namespace-scoped Role for reading ConfigMaps and Secrets in the release namespace (Rego policies, credentials).
Four services (Kubernaut Agent, WorkflowExecution, RemediationOrchestrator, EffectivenessMonitor) include read access to security.istio.io and networking.istio.io resources for service mesh awareness during investigation and remediation.
| Controller | ServiceAccount | CRDs Managed | Additional Access | Notes |
|---|---|---|---|---|
| RemediationOrchestrator | remediationorchestrator-controller |
All 7 child CRDs (full CRUD) | Pods, nodes, events, namespaces, services, deployments, statefulsets, daemonsets, jobs, cronjobs (read) | Broadest permissions -- creates and watches all child CRDs |
| SignalProcessing | signalprocessing-controller |
SignalProcessing, RemediationRequest | Pods, services, namespaces, nodes, events, deployments, replicasets, statefulsets, daemonsets, HPAs, PDBs, network policies (read); leases (full) | Owner chain resolution and enrichment |
| AIAnalysis | aianalysis-controller |
AIAnalysis | Events (create) | Also bound to kubernaut-agent-client for Kubernaut Agent access and data-storage-client for DataStorage access |
| WorkflowExecution | workflowexecution-controller |
WorkflowExecution | Tekton PipelineRuns (full), TaskRuns (read), Jobs (full), events (create); leases (full) | Creates Jobs and PipelineRuns in the execution namespace. When the ansible engine is enabled, also requires get on the AWX API token Secret -- see Ansible Engine Setup. |
| EffectivenessMonitor | effectivenessmonitor-controller |
EffectivenessAssessment, RemediationRequest (read) | Pods, nodes, services, PVCs, events, deployments, replicasets, statefulsets, daemonsets, HPAs, PDBs, jobs, cronjobs (read) | Post-remediation health checks |
| Notification | notification-controller |
NotificationRequest | Events (create) | Minimal scope |
| AuthWebhook | authwebhook |
All Kubernaut CRDs (read), status subresources (update, patch) | -- | Admission webhook validation, defaulting, and catalog registration. Intercepts CREATE and UPDATE operations on RemediationWorkflow CRDs. Uses retry-on-conflict for ActionType status updates. |
Broad Read Access via view ClusterRole¶
The RemediationOrchestrator and EffectivenessMonitor are additionally bound to the Kubernetes built-in view ClusterRole via remediationorchestrator-view and effectivenessmonitor-view ClusterRoleBindings. This provides broad read access to CRD types not individually enumerated in their dedicated ClusterRoles -- for example, cert-manager Certificate resources and Istio networking resources -- which is required for pre- and post-remediation hash capture (DD-EM-002).
If the view ClusterRole lacks read permission for a particular resource type (e.g., a third-party CRD), the Remediation Orchestrator emits a HashCaptureDegraded Kubernetes event on the RemediationRequest when CapturePreRemediationHash returns a degraded reason. The EffectivenessAssessment then proceeds in degraded mode — the EA skips the hash comparison component and relies on the remaining health-check signals (alert state, metric thresholds, pod readiness) to determine effectiveness.
Workflow Execution¶
Remediation workflows (Jobs, Tekton PipelineRuns, Ansible playbooks) execute in the kubernaut-workflows namespace. By default, workflow executions run with the execution namespace default ServiceAccount. Many deployments bind that default to kubernaut-workflow-runner, which carries the broadest ClusterRole in the system because workflows need to act on the cluster to remediate issues.
| apiGroup | Resources | Verbs | Purpose |
|---|---|---|---|
apps |
deployments, statefulsets, daemonsets |
get, list, patch, update | Scale, restart, or patch workloads |
apps |
replicasets |
get, list, watch | Read replica state |
| (core) | pods, pods/eviction |
get, list, create, delete | Evict pods, read pod state |
| (core) | configmaps, secrets |
get, list, create, update, patch, delete | Read/write configuration |
| (core) | nodes |
get, list | Read node state for drain/cordon |
| (core) | namespaces, services, persistentvolumeclaims |
get, list | Read cluster state |
autoscaling |
horizontalpodautoscalers |
get, list, patch | Scale HPAs |
policy |
poddisruptionbudgets |
get, list, patch | Adjust PDBs during remediation |
networking.k8s.io |
networkpolicies |
get, list, create, update, patch, delete | Manage network policies |
argoproj.io |
applications |
get, list | Read ArgoCD application state |
cert-manager.io |
certificates, clusterissuers |
get, list | Read certificate state |
policy.linkerd.io |
authorizationpolicies, servers, meshtlsauthentications |
get, list, delete | Manage Linkerd policies (legacy) |
security.istio.io |
authorizationpolicies, peerauthentications, requestauthentications |
get, list, delete | Manage Istio security policies |
networking.istio.io |
virtualservices, destinationrules, gateways, serviceentries |
get, list, create, update, patch, delete | Manage Istio networking resources |
kubernaut.ai |
workflowexecutions |
get | Ansible playbooks read WFE ownerReferences for RR correlation |
storage.k8s.io |
storageclasses |
get, list | Discover default StorageClass for PVC migration |
| (core) | endpoints |
get, list | Check service endpoint health |
batch |
jobs |
get, list, create, delete | pg_dump/pg_restore Job lifecycle (disk-pressure-emptydir scenario) |
The last four rules were added for production Ansible playbooks (DD-WE-007).
Additionally, a namespace-scoped workflowexecution-dep-reader Role grants get, list, watch on Secrets and ConfigMaps in the execution namespace for dependency validation before workflow launch.
Per-Workflow ServiceAccount (v1.2)¶
Starting with v1.2, workflows can declare a dedicated ServiceAccount via the spec.execution.serviceAccountName field on the RemediationWorkflow CRD. This value is propagated into WorkflowExecution.spec.serviceAccountName.
- Job/Tekton execution: the service account name is set directly on the created Job or PipelineRun.
- Ansible execution: the WE controller requests a short-lived token via the Kubernetes TokenRequest API and injects it into AWX credentials.
This enables least-privilege RBAC per workflow — each SA needs only the permissions required by its specific remediation.
The WE controller's ClusterRole includes the following permissions for TokenRequest:
| apiGroup | Resources | Verbs | Purpose |
|---|---|---|---|
| (core) | serviceaccounts |
get | Look up the per-workflow SA |
| (core) | serviceaccounts/token |
create | Create short-lived tokens via TokenRequest |
TTL validation (Ansible path): The controller requests tokens with a 3600s (1 hour) TTL by default. When an execution timeout is configured, the controller validates that the requested token TTL covers the execution window. If the TTL is insufficient, it sets TokenTTLInsufficient on the WorkflowExecution and emits a warning event (TokenTTLShortened).
Cluster-level TTL constraints
The Kubernetes API server flag --service-account-max-token-expiration (or OpenShift ServiceAccountTokenMaxExpiration) can silently cap token TTLs below the requested duration. If the cap is lower than the workflow execution timeout, Ansible jobs may receive 401 Unauthorized errors mid-execution. Ensure the cluster-level maximum is at least 3600s (or greater than your longest workflow execution timeout). Check for TokenTTLShortened warning events on WorkflowExecution CRDs if Ansible jobs fail unexpectedly with authentication errors.
Fallback behavior: If serviceAccountName is not set:
- Job/Tekton run with the execution namespace default ServiceAccount.
- Ansible falls back to controller in-cluster credentials for AWX credential injection.
For the Ansible executor (#501), TokenRequest tokens replace the controller's own in-cluster SA token for AWX credential injection, ensuring each playbook runs with the minimum permissions declared by the workflow author.
OCP Monitoring RBAC¶
When effectivenessmonitor.external.ocpMonitoringRbac is true, the chart creates additional RBAC resources for EM to access OCP's monitoring stack through kube-rbac-proxy:
| Resource | Kind | Purpose |
|---|---|---|
effectivenessmonitor-monitoring-view |
ClusterRoleBinding | Binds EM ServiceAccount to the built-in cluster-monitoring-view ClusterRole for Prometheus API access |
kubernaut-alertmanager-view |
ClusterRole | Grants get on monitoring.coreos.com/alertmanagers/api for AlertManager API access through kube-rbac-proxy |
effectivenessmonitor-alertmanager-view |
ClusterRoleBinding | Binds EM ServiceAccount to kubernaut-alertmanager-view |
The AlertManager ClusterRole and ClusterRoleBinding are only created when both ocpMonitoringRbac and alertManagerEnabled are true.
OCP's kube-rbac-proxy requires resource-level RBAC (monitoring.coreos.com/alertmanagers/api) rather than nonResourceURLs for AlertManager API access. Standard nonResourceURLs rules are silently ignored by kube-rbac-proxy, causing EM AlertManager queries to fail with 403 Forbidden.
Ansible Credential Injection¶
When the Ansible/AWX execution engine is enabled, the WorkflowExecution controller injects the cluster's Kubernetes API credentials into AWX Job Templates so that kubernetes.core Ansible modules can authenticate against the target cluster. The v2 custom credential type (kubernaut-k8s-bearer-token-v2) uses kubeconfig-file injection rather than environment variables, because in-cluster ServiceAccount config inside AAP execution environments takes precedence over K8S_AUTH_* env vars. If resolution picks a built-in or kind-matched type, the injector may differ.
The credential type resolution follows a 6-step process:
- Look for the built-in AWX type ("OpenShift or Kubernetes API Bearer Token")
- Look up a credential type by kind (
FindCredentialTypeByKind("kubernetes", true)) - Fall back to
kubernaut-k8s-bearer-token(custom type from earlier versions) - Fall back to
kubernaut-k8s-bearer-token-v2(kubeconfig-based type) - If none exist, create the v2 type with a Jinja2 kubeconfig template that AWX renders at job launch
- Create an ephemeral credential populated with the controller's in-cluster SA token, API server host, and CA certificate
The v2 kubeconfig template conditionally includes certificate-authority-data when the cluster CA is available, or sets insecure-skip-tls-verify: true otherwise. AWX injects the rendered kubeconfig as a temp file and sets K8S_AUTH_KUBECONFIG to point to it, ensuring kubernetes.core modules use the injected credentials instead of in-cluster config.
Ephemeral credentials are cleaned up after the AWX job completes. See BR-WE-017 for the full shared SA model. In v1.2, workflows can declare a dedicated ServiceAccount via spec.execution.serviceAccountName — see Per-Workflow ServiceAccount above.
Internal Service Communication¶
DataStorage Authentication¶
DataStorage uses the same TokenReview + SAR pattern as the Gateway. The data-storage-auth-middleware ClusterRole grants DataStorage permission to validate client tokens:
| apiGroup | Resources | Verbs |
|---|---|---|
authentication.k8s.io |
tokenreviews |
create |
authorization.k8s.io |
subjectaccessreviews |
create |
Clients must have create permission on services/data-storage-service (via the data-storage-client ClusterRole). The chart binds every Kubernaut service to this role:
Gateway, SignalProcessing, RemediationOrchestrator, AIAnalysis, WorkflowExecution, EffectivenessMonitor, Notification, AuthWebhook, Kubernaut Agent, and DataStorage itself.
Kubernaut Agent Access¶
The AIAnalysis controller communicates with Kubernaut Agent via the kubernaut-agent-client ClusterRole, which grants create and get on services/kubernaut-agent.
Kubernaut Agent itself has a broad read-only ClusterRole (kubernaut-agent-investigator) for its kubectl-based investigation:
| apiGroup | Resources | Verbs | Purpose |
|---|---|---|---|
| (core) | pods, pods/log, events, services, endpoints, configmaps, secrets, nodes, namespaces, replicationcontrollers, PVCs, resourcequotas | get, list, watch | Cluster state investigation |
apps |
deployments, replicasets, statefulsets, daemonsets | get, list, watch | Workload investigation |
batch |
jobs, cronjobs | get, list, watch | Job investigation |
events.k8s.io |
events | get, list, watch | Event investigation |
autoscaling |
horizontalpodautoscalers | get, list, watch | HPA investigation |
policy |
poddisruptionbudgets | get, list, watch | PDB investigation |
networking.k8s.io |
networkpolicies | get, list, watch | Network policy investigation |
cert-manager.io |
certificates, clusterissuers, certificaterequests | get, list, watch | Certificate investigation |
argoproj.io |
applications | get, list, watch | ArgoCD investigation |
policy.linkerd.io |
servers, authorizationpolicies, meshtlsauthentications | get, list, watch | Linkerd mesh investigation (legacy) |
security.istio.io |
authorizationpolicies, peerauthentications, requestauthentications | get, list, watch | Istio security policy investigation |
networking.istio.io |
virtualservices, destinationrules, gateways, serviceentries | get, list, watch | Istio networking investigation |
monitoring.coreos.com |
prometheusrules, servicemonitors, podmonitors, probes | get, list, watch | Monitoring investigation |
config.openshift.io |
nodes, clusteroperators, clusterversions, infrastructures | get, list, watch | OCP platform investigation (Operator only) |
This read-only access allows the LLM to investigate root causes using live cluster data without making changes.
Lease RBAC for Interactive Sessions (v1.5+)¶
Interactive MCP sessions use Kubernetes Leases for distributed locking. The Kubernaut Agent ServiceAccount requires Lease permissions in addition to its existing investigator role:
| apiGroup | Resources | Verbs | Purpose |
|---|---|---|---|
coordination.k8s.io |
leases |
get, create, update, delete, list | Session locking + orphan reclamation |
The list verb (new in v1.5) is required for ReconcileOrphanedLeases — on startup, KA scans for Leases whose holder identity no longer exists and reclaims them.
Automatic provisioning
Both the Helm chart and the Kubernaut Operator provision this permission automatically. Only custom RBAC configurations need manual updating.
API Frontend RBAC (v1.5+)¶
The API Frontend ServiceAccount requires:
| apiGroup | Resources | Verbs | Purpose |
|---|---|---|---|
kubernaut.ai |
investigationsessions, investigationsessions/status |
get, list, watch, create, update, patch, delete | Session CRD management |
kubernaut.ai |
remediationrequests |
get, list, watch, create, update, patch | RR lifecycle for MCP/A2A sessions |
kubernaut.ai |
remediationapprovalrequests, remediationapprovalrequests/status |
get, list, create, update, patch | Approval request access |
authorization.k8s.io |
subjectaccessreviews |
create | SAR-based tool authorization |
| (core) | events |
get, list, create, patch | Cluster context triage tools |
| (core) | pods, replicationcontrollers |
get, list | Cluster context triage tools |
apps |
deployments, replicasets, statefulsets, daemonsets |
get, list | Cluster context triage tools |
batch |
jobs, cronjobs |
get | Cluster context triage tools |
Infrastructure and Hooks¶
PostgreSQL and Valkey¶
Both run with dedicated ServiceAccounts that have automountServiceAccountToken: false, preventing unnecessary API token mounting.
Helm Hooks¶
The shared hook ServiceAccount (kubernaut-hook-sa) and its ClusterRole are used by TLS certificate generation jobs, the database migration job, and the CRD upgrade job:
| apiGroup | Resources | Verbs | Purpose |
|---|---|---|---|
| (core) | secrets, configmaps |
get, create, update, patch, delete | TLS cert/CA storage, migration state |
admissionregistration.k8s.io |
mutatingwebhookconfigurations, validatingwebhookconfigurations |
get, patch | Patch caBundle (hook mode only, see #334) |
apiextensions.k8s.io |
customresourcedefinitions |
get, list, create, update, patch | CRD pre-upgrade server-side apply (#521) |
| (core) | pods |
get, list | Post-install verification |
apps |
deployments |
get | Post-install verification |
batch |
jobs |
get, list | Migration job monitoring |
Hook jobs only run during helm install, helm upgrade, and helm delete. They do not have long-lived pods.
Prompt Injection Defense — Shadow Agent (v1.4)¶
Kubernaut v1.4 introduces a shadow agent that evaluates every LLM tool output for prompt injection attacks. The shadow agent runs as a fail-closed defense layer inside Kubernaut Agent.
Architecture¶
The shadow agent operates at two levels:
-
Per-step evaluation — After each tool call, the shadow agent scans the tool output using random boundary markers, head+tail truncation, and data exfiltration detection patterns. This catches injection attempts embedded in Kubernetes resource data (ConfigMaps, Secrets, annotations).
-
Full-context grounding review — At the RCA-to-workflow boundary, a second evaluation layer reviews the entire investigation conversation through the shadow LLM. This detects distributed injection ("boiling frog" attacks) where individually benign tool outputs combine into a malicious instruction. Runs in parallel with workflow discovery for zero added latency.
Enforcement modes¶
The shadow agent supports two modes controlled by the Kubernaut Agent configuration:
| Mode | Behavior |
|---|---|
| Monitor | Log alignment verdicts and emit audit events, but do not block investigations |
| Enforce | Cancel the investigation via circuit breaker when suspicious content is detected |
In enforce mode, a positive detection triggers context.WithCancelCause(ErrCircuitBreaker), immediately terminating the primary investigation. The kubernaut_alignment_circuit_breaker_total Prometheus counter tracks activations.
Alignment verdicts¶
Shadow agent results are propagated through the system:
alignment_verdictfield on the Kubernaut AgentIncidentResponse(OpenAPI) andAIAnalysisStatus(CRD) carrying the verdict (result,circuit_breaker_activated,summary,findings)aiagent.alignment.verdictaudit event — emitted byInvestigatorWrapper.emitVerdictEvent()after every investigation. Payload includesresult(aligned/suspicious),circuit_breaker_activated,summary,flagged/totalcounts,findings[], and optionalgrounding_review(v1.5.1)- NotificationRequest
ReviewContextincludesalignmentVerdictandcircuitBreakerActivatedfields for routing rule support - Manual review notifications render shadow agent findings prominently when
alignment_check_failedSubReason escalates toNotificationPriorityCritical
Observability¶
| Metric | Type | Description |
|---|---|---|
kubernaut_alignment_grounding_total |
Counter | Grounding review invocations |
kubernaut_alignment_grounding_duration_seconds |
Histogram | Grounding review latency |
kubernaut_alignment_circuit_breaker_total |
Counter | Circuit breaker activations |
Per-step audit events include shadow LLM request/response payloads and token counts for cost tracking (aiagent.alignment.step).
False positive handling¶
v1.4 includes targeted fixes to reduce false positives:
- Well-known Kubernetes/OpenShift annotation namespaces, container commands, probe commands, event messages, RBAC verbs, and registry URLs are whitelisted (CLEAN classification)
- The shadow agent evaluates raw tool output (post-sanitizer, pre-summarizer) to avoid false positives from LLM-generated analysis content
NetworkPolicies (v1.4)¶
Kubernaut v1.4 deploys 12 NetworkPolicy templates covering all services with a default-deny ingress posture. Each policy restricts ingress to only the expected callers.
Default behavior¶
The default depends on your deployment method:
| Deployment | Default | How to change |
|---|---|---|
| Helm | Enabled for all services | Set networkPolicies.<service>.enabled: false per service |
| Operator | Disabled (spec.networkPolicies.enabled: false) |
Set spec.networkPolicies.enabled: true in the Kubernaut CR |
Operator users: NetworkPolicies are off by default
If you deploy via the Kubernaut Operator and expect network segmentation, you must explicitly enable it. The Operator uses a single toggle (spec.networkPolicies.enabled) rather than per-service toggles.
Verify your cluster's CNI plugin supports NetworkPolicy enforcement (Calico, Cilium, etc.) — clusters with CNI plugins that do not enforce NetworkPolicies silently ignore them.
Configuration¶
Per-service toggle via Helm values:
Custom CIDR ranges can be configured for services that need external access (e.g., Gateway ingress from AlertManager).
Tool Authorization¶
SAR-based tool authorization (v1.5)¶
v1.5 replaces the API Frontend's file-based rbac_roles.yaml with Kubernetes-native SubjectAccessReview (SAR) authorization (PR #1222). Every MCP tools/call invocation triggers a SAR check before execution. tools/list is unfiltered (ADR-020) — authorization is enforced only at call time.
How it works¶
- Client authenticates via OIDC — the API Frontend extracts the user identity from the JWT
- On each tool invocation, AF issues a SAR:
can <user> use tools/<tool-name> in apiGroup kubernaut.ai? - The Kubernetes API server evaluates the user's ClusterRoleBindings
- If denied (or if the SAR API is unreachable), the tool call is rejected — fail-closed
All 3 AF RBAC code paths (MCP bridge checkRBAC(), A2A agent newRBACGuard(), and main wiring) delegate to the SARChecker.
Per-persona ClusterRoles¶
The Helm chart ships 6 per-persona ClusterRoles via a data-driven template (apifrontend.config.rbac.personas):
| ClusterRole | Persona | MCP Tools | Tool List |
|---|---|---|---|
kubernaut-tool-sre |
Full investigation + remediation | 21 | kubernaut_list_remediations, kubernaut_get_remediation, kubernaut_approve, kubernaut_cancel_remediation, kubernaut_watch, kubernaut_await_session, kubernaut_investigate, kubernaut_discover_workflows, kubernaut_select_workflow, kubernaut_present_decision, kubernaut_list_workflows, kubernaut_list_alerts, kubernaut_get_remediation_history, kubernaut_get_effectiveness, kubernaut_get_audit_trail, kubernaut_message, kubernaut_complete, kubernaut_complete_no_action, kubernaut_cancel, kubernaut_status, kubernaut_reconnect |
kubernaut-tool-ai-orchestrator |
Automated agent orchestration (no approval) | 15 | kubernaut_list_remediations, kubernaut_get_remediation, kubernaut_watch, kubernaut_await_session, kubernaut_investigate, kubernaut_discover_workflows, kubernaut_select_workflow, kubernaut_present_decision, kubernaut_message, kubernaut_complete, kubernaut_complete_no_action, kubernaut_cancel, kubernaut_status, kubernaut_reconnect |
kubernaut-tool-cicd |
CI/CD pipeline integration | 4 | kubernaut_list_remediations, kubernaut_get_remediation, kubernaut_watch, kubernaut_await_session |
kubernaut-tool-observability |
Read-only observability | 6 | kubernaut_list_remediations, kubernaut_get_remediation, kubernaut_watch, kubernaut_await_session, kubernaut_get_effectiveness, kubernaut_list_workflows |
kubernaut-tool-l3-audit |
Compliance and auditing | 6 | kubernaut_list_remediations, kubernaut_get_remediation, kubernaut_list_workflows, kubernaut_get_remediation_history, kubernaut_get_effectiveness, kubernaut_get_audit_trail |
kubernaut-tool-remediation-approver |
Human approval workflows | 7 | kubernaut_approve, kubernaut_list_approval_requests, kubernaut_get_approval_request, kubernaut_list_remediations, kubernaut_get_remediation, kubernaut_watch, kubernaut_await_session |
Internal tools
The AF also uses 5 internal tools (kubectl_get, kubectl_list, kubectl_list_events, kubernaut_check_existing_remediation, kubernaut_remediate) that run under the AF pod's own ServiceAccount. These are not exposed via MCP/A2A, are not SAR-gated, and are not included in persona tool counts.
Custom ClusterRoles¶
For organizations that need tool subsets not covered by the 6 built-in personas — for example, separation of duties where SREs can investigate but not approve — the operator supports user-managed ClusterRoles via the clusterRoleName field on ToolRoleBinding.
How it works:
- You create a ClusterRole with
verb: useonresource: toolsinapiGroup: kubernaut.ai, listing the desired tool names asresourceNames - In the Kubernaut CR, add a
roleBindingsentry withclusterRoleName(instead ofrole) pointing to your ClusterRole - The operator creates only the ClusterRoleBinding — it does not create or modify your ClusterRole
Validation: role and clusterRoleName are mutually exclusive. Entries with both or neither set are rejected by CR validation.
Custom ClusterRole: restricted investigator (investigate + read, no approval)
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubernaut-restricted-investigator
rules:
- apiGroups: ["kubernaut.ai"]
resources: ["tools"]
verbs: ["use"]
resourceNames:
- kubernaut_list_remediations
- kubernaut_get_remediation
- kubernaut_watch
- kubernaut_investigate
- kubernaut_message
- kubernaut_complete
- kubernaut_cancel
- kubernaut_status
- kubernaut_reconnect
Custom ClusterRole: readonly audit (history + effectiveness only)
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kubernaut-readonly-audit
rules:
- apiGroups: ["kubernaut.ai"]
resources: ["tools"]
verbs: ["use"]
resourceNames:
- kubernaut_list_remediations
- kubernaut_get_remediation
- kubernaut_get_remediation_history
- kubernaut_get_effectiveness
- kubernaut_get_audit_trail
Mixed CR configuration (built-in + custom)
Available tool resourceNames¶
The following tool names can be used in resourceNames when authoring custom ClusterRoles. The SAR check matches against these exact strings.
MCP bridge tools (23) — available to external MCP clients and the Kubernaut Console:
resourceName |
Description |
|---|---|
kubernaut_list_remediations |
List active and recent remediations |
kubernaut_get_remediation |
Get details of a specific remediation |
kubernaut_cancel_remediation |
Cancel an active remediation |
kubernaut_watch |
Stream live status updates for a remediation |
kubernaut_list_approval_requests |
List approval requests |
kubernaut_get_approval_request |
Get full details of an approval request |
kubernaut_approve |
Approve or reject a pending approval request |
kubernaut_await_session |
Wait for an investigation session to become ready |
kubernaut_investigate |
Start or join an investigation |
kubernaut_message |
Send a message to an active investigation session |
kubernaut_complete |
Complete an investigation session |
kubernaut_complete_no_action |
Complete with no remediation (dismiss or escalate) |
kubernaut_cancel |
Cancel an active investigation session |
kubernaut_status |
Get the current status of an investigation session |
kubernaut_reconnect |
Reconnect to a disconnected session |
kubernaut_discover_workflows |
Discover available workflows for a remediation |
kubernaut_select_workflow |
Select a workflow for execution |
kubernaut_present_decision |
Present investigation results for a decision |
kubernaut_list_workflows |
List available workflows from the catalog |
kubernaut_list_alerts |
Query firing Prometheus alerts (conditional) |
kubernaut_get_remediation_history |
Query historical remediations |
kubernaut_get_effectiveness |
Get effectiveness scores for workflows |
kubernaut_get_audit_trail |
Retrieve the audit trail for a remediation |
A2A agent internal tools (5) — used inside the A2A agent loop, SAR-gated on the A2A path:
resourceName |
Description |
|---|---|
kubectl_get |
Get any namespaced K8s resource (Secret .data redacted) |
kubectl_list |
List namespaced K8s resources (Secret .data redacted) |
kubectl_list_events |
List K8s events with reason/object filters |
kubernaut_check_existing_remediation |
Check for duplicate RemediationRequest |
kubernaut_remediate |
Create a new RemediationRequest CRD |
Binding example¶
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: sre-team-kubernaut-tools
subjects:
- kind: Group
name: sre-team
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: kubernaut-tool-sre
apiGroup: rbac.authorization.k8s.io
SAR caching¶
Authorization results are cached with a SHA256-keyed TTL cache (default 30s) to avoid per-call SAR overhead:
Migration from rbac_roles.yaml¶
Breaking change
The rbac_roles.yaml ConfigMap and RBACConfig.GroupMapping are removed in v1.5. Customers who customized file-based RBAC must create equivalent ClusterRoleBindings.
- Identify which tools each role had access to
- Map those to the closest per-persona ClusterRole (or create a custom one with
verb: useonresource: toolsinapiGroup: kubernaut.ai) - Create ClusterRoleBindings for your OIDC groups
- The
rbac_roles.yamlConfigMap is no longer read
Unified ServiceAccount model (v1.5, ADR-022)¶
All AF Kubernetes API calls use the AF pod's own ServiceAccount — there is no per-user impersonation or OIDC-direct token forwarding. This simplifies the security model: the AF SA has a fixed, auditable set of permissions, and application-level authorization is enforced entirely through SAR-based tool gating (see Per-persona ClusterRoles).
User attribution is preserved in the application audit trail (tool.executed events include UserID, actor_ip, target_namespace, target_kind) rather than in Kubernetes audit logs.
Accepted risks (documented in ADR-022):
| ID | Risk | Mitigation |
|---|---|---|
| SEC-01 | AF SA has broad kubernaut CRD access | Namespace-scoped where possible; SAR gates each tool call |
| SEC-02 | K8s audit logs attribute actions to AF SA, not the end user | Application audit trail carries user identity; correlate via actor_ip |
| SEC-04 | AF SA privilege aggregation | ClusterRole is explicit and minimal; reviewed per release |
| SEC-09 | Persona misconfiguration could over-grant | Helm values are the source of truth; kubernaut-tool-* roles are data-driven |
Approval Consent Guard (DD-AF-006, v1.5.1)¶
kubernaut_approve is structurally absent from the A2A agent's buildToolList() as of v1.5.1. It remains registered on the MCP bridge for the Kubernaut Console's Approve/Reject buttons and direct MCP client access.
Rationale: An LLM with access to kubernaut_approve could autonomously approve RARs via prompt injection, bypassing the human consent gate that RARs exist to enforce.
Defense-in-depth layers:
- Tool absent from
buildToolList()-- structurally impossible for the A2A agent to call - Explicit prompt instruction telling the agent the tool is unavailable
- SAR RBAC on the MCP bridge path
- Audit trail attributing every approval to the human user
Similarly, kubernaut_complete_no_action is MCP-only (DD-AF-007) -- it is not available to the A2A agent.
Trusted Intermediary Delegation (#1287)¶
When a human operator approves or rejects a RemediationApprovalRequest (RAR) via the MCP bridge, the API Frontend acts as a trusted intermediary:
- The operator's OIDC token is validated and a
TokenReviewresolves the human identity (username + groups). - The AF ServiceAccount patches the RAR's
status.decisionon behalf of the human user. - The
decidedViafield on the RAR status records the trusted intermediary's SA identity (e.g.system:serviceaccount:kubernaut-system:kubernaut-apifrontend). The auth webhook sets this field; it is blank when the RAR is decided directly (e.g., by the operator CRD controller).
This allows the AF to record who made the decision (the human) while executing the Kubernetes API call with its own SA token — consistent with the unified SA model above.
TokenReview Identity Enrichment¶
When a client connects via OIDC, the AF performs a Kubernetes TokenReview to resolve the full user identity (username, UID, groups, extras). This enriched identity is:
- Used in every SAR check for tool authorization
- Injected into the Rego policy context as
input.identity(see Rego Reference) - Recorded in application audit trail events
OIDC CA Trust (#1245)¶
For environments using a private OIDC provider (non-public CA), the AF supports a custom CA certificate via the oidcCaFile configuration field:
apifrontend:
config:
auth:
issuerURL: https://idp.internal.corp.com
oidcCaFile: /etc/certs/oidc-ca.pem
Mount the CA certificate as a Kubernetes Secret or ConfigMap volume. The AF uses this CA when fetching the OIDC discovery document and JWKS keys.
Auth Mode Auto-Detection (#1309)¶
The AF auto-detects its authentication mode from the configuration:
| Condition | Mode | Behavior |
|---|---|---|
| JWT providers configured | OIDC/JWKS | Validates bearer tokens against configured issuers and JWKS endpoints |
| No JWT providers + TokenReview wired | K8s TokenReview | Validates tokens via the Kubernetes API server's TokenReview endpoint |
This removes the need for an explicit authMode toggle — the AF infers the correct strategy from which providers are present in the configuration.
Keycloak OIDC Configuration¶
When using Keycloak as the OIDC provider (common with kagenti deployments), additional configuration is needed to ensure the API Frontend can authorize tool calls via SAR.
Groups Client Scope¶
The AF uses Kubernetes SubjectAccessReview (SAR) to authorize tool access based on OIDC group claims in the user's JWT. Keycloak must include group membership in issued tokens.
- In the Keycloak admin console, navigate to the target realm (e.g.,
kagenti) - Go to Client scopes > Create client scope: name
groups, protocolopenid-connect - Under the new scope, go to Mappers > Create mapper:
| Setting | Value |
|---|---|
| Name | groups |
| Mapper type | Group Membership |
| Token claim name | groups |
| Full group path | off |
| Add to ID token | on |
| Add to access token | on |
| Add to userinfo | on |
- Go to Clients >
kagenti> Client scopes > Add client scope > addgroupsas a Default scope
The group names in the token must match the groups array in spec.apiFrontend.rbac.roleBindings (Operator CR) or apifrontend.config.rbac.roleBindings (Helm values).
Audience Mapper¶
For kagenti to successfully authenticate with the API Frontend via SPIFFE identity, add an oidc-audience-mapper protocol mapper to the kagenti client:
| Setting | Value |
|---|---|
included.custom.audience |
spiffe://<trust-domain>/ns/kubernaut-system/sa/apifrontend |
access.token.claim |
true |
id.token.claim |
false |
Assign this scope as a default scope to the kagenti client. The trust domain must match your SPIRE deployment.
Stale Token Warning¶
Users must re-authenticate after scope changes
Users must log out and log back in after being added to a group (or after the groups scope is first created) to receive a fresh token with the groups claim. Stale tokens issued before the scope was added will have an empty or missing groups array. SAR checks will fail with 403 Forbidden for all tool calls, even if the user's group has a valid roleBinding.
To verify a token contains the expected groups:
If groups is null or empty, the user must re-authenticate.
Next Steps¶
- Installation -- Configure AlertManager and other signal sources
- Configuration Reference -- Helm values for all services
- Disconnected Installation -- Air-gapped installation with Keycloak integration
- Troubleshooting -- Diagnose RBAC-related issues