Skip to content

Troubleshooting

It couldn’t load a kubeconfig. The screen names the path it tried and why it failed.

  1. Check that kubectl cluster-info works with the same $KUBECONFIG.
  2. In Docker, mount the kubeconfig at /home/nonroot/.kube/config. If it points at 127.0.0.1, use --network host on Linux. On macOS and Windows, change the address to host.docker.internal and, for kind and minikube, whose certificates don’t name that host, add tls-server-name: localhost to the cluster’s entry; without it the connection fails with an x509 error.
  3. Paste a kubeconfig on the setup screen, or start a local cluster; KubeGlass continues by itself once one works.

Every request runs as you, so KubeGlass shows what your RBAC allows. Check with kubectl as the same user:

Terminal window
kubectl auth can-i list pods -n <namespace>
kubectl auth can-i create pods/exec -n <namespace>

If you can’t list namespaces at all, add the ones you use under Settings › Clusters › Allowed namespaces.

A change that fails with read_only was refused because KubeGlass runs with --read-only or because you marked the cluster read-only; the message says which.

In a cluster deployment, the person needs RBAC of their own. If nothing works for anyone, check that KubeGlass’s ServiceAccount may impersonate (the chart’s ClusterRole grants impersonate on users, groups and ServiceAccounts) and that your proxy sends X-Forwarded-User.

A cluster says “Not signed in” or doesn’t answer

Section titled “A cluster says “Not signed in” or doesn’t answer”

Clusters › All clusters and the Overview check every context in your kubeconfig. Contexts that sign in through a credential plugin that can start a login, such as kubelogin for AKS or kubectl oidc-login, are not checked until you open them, so KubeGlass never opens a login page you didn’t ask for. Switch to the cluster to sign in. Contexts that use aws, aws-iam-authenticator, gke-gcloud-auth-plugin, gcloud, doctl or oci are checked right away, because those fail quietly when your login has expired.

For a cluster that doesn’t answer, open its row on All clusters: it tests the server name, the connection, the credentials and the permissions in turn and marks the step that fails.

KubeGlass runs credential plugins without a terminal. If a plugin needs to ask you something, run a kubectl command against that context first to log in.

KubeGlass keeps checking that its server and the current cluster answer. One failed check isn’t enough: a failure counts when the check 2 seconds later fails too.

  • A page that has loaded keeps what it shows, under a banner that says what failed (“Can’t reach” and the cluster’s name, “Lost the connection to KubeGlass”, “You’re offline”, a server error or a timeout), how long ago, and “Data may be out of date. Retrying…”. Retry now checks again at once. The page is live again about 3 seconds after the answers come back.
  • A page with nothing loaded yet shows that in its place: “Lost the connection to KubeGlass” when the KubeGlass server stopped answering, “You’re offline” when the browser has no network, and for a cluster that doesn’t answer, the steps of reaching it with the one that fails.
  • When the cluster’s API server stops answering, the Overview and the map show an error instead of an empty list that would read as nothing wrong; their API routes answer 502 api_unreachable (see Error codes).

Check that KubeGlass runs (curl -s http://localhost:8090/healthz, or the pod’s status in a cluster), then the cluster with kubectl cluster-info. If only live updates stop, see Live updates stop or the terminal disconnects.

Adding and removing clusters, changing settings, the audit log and Drift policies are limited to admins. Add the user or one of their groups to KUBEGLASS_ADMIN_USERS or KUBEGLASS_ADMIN_GROUPS.

Browser sessions, which remember each person’s cluster, are kept in memory. After a restart an open tab names the cluster it shows, and its new session starts there when a switch to it would be allowed and the person’s credentials can be used for it; otherwise it starts on the default cluster and the tab moves there with a warning. Behind a sign-in proxy people stay signed in, because KubeGlass reads the proxy’s headers on every request. In oidc and token modes sign-ins are kept in the data directory: people stay signed in if it is on a PersistentVolumeClaim and the session key stays the same (session.keySecretRef in the chart), and have to sign in again otherwise.

Live updates stop or the terminal disconnects

Section titled “Live updates stop or the terminal disconnects”

WebSockets carry live updates, logs and terminals. A proxy or load balancer in front of KubeGlass must pass them through and allow long-lived connections, for example proxy_read_timeout 3600s; in nginx. KubeGlass runs as one replica (see High availability), so keep replicaCount at 1. A terminal also closes after 30 minutes without input or output.

  1. Check the pod is running and you may exec: kubectl auth can-i create pods/exec -n <namespace>.
  2. Pick the right container in pods with several.
  3. An image without a shell (distroless, scratch) has nothing to open: the tab says so and offers a debug container, which needs patch on pods/ephemeralcontainers.
  4. Node shells need KUBEGLASS_ENABLE_NODE_SHELL=true on the server.
  5. A shell on the computer KubeGlass runs on needs kubeglass --local-shell, local auth mode, KubeGlass outside a cluster and no --read-only. When one of those is missing, the log says Local shell not turned on and why, and Settings › Terminal › Shell on this computer says Off.
  6. In read-only mode, or on a cluster you marked read-only, no shells open.

Live CPU and memory need metrics-server: kubectl top nodes must work. PromQL needs Prometheus. KubeGlass looks for it in the cluster; if it doesn’t find yours, set KUBEGLASS_PROMETHEUS_URL. Prometheus is used only for people with cluster-wide read access.

KubeGlass keeps in memory the lists of the types that pages and search use, per user who opens them; each stops after 10 minutes without use. In very large clusters, give the pod more memory. Performance has the memory we measured, and the cluster sizes we haven’t tested.

Look up the code from the error in Error codes. When you open an issue, include the KubeGlass version (kubeglass --version) and the requestId from the error; the server log has the details for that ID. When you start kubeglass at a terminal, the log is in ~/.kubeglass/data/kubeglass.log; kubeglass --verbose prints it instead.