diff options
Diffstat (limited to 'logs/filters.txt')
| -rw-r--r-- | logs/filters.txt | 207 |
1 files changed, 207 insertions, 0 deletions
diff --git a/logs/filters.txt b/logs/filters.txt new file mode 100644 index 0000000..daf2bab --- /dev/null +++ b/logs/filters.txt @@ -0,0 +1,207 @@ +RBAC DENY: user # what it was? +Pod ended with: too old resource version +containerd: unable to save +an empty namespace may not be set when a resource name +timeout expired waiting for volumes to attach +Fast watcher, slow processing + +# Temporary (current evaluation) +# --------- +apiserver.*http: Handler timeout # apiserver was not restarted. Why? +etcdserver: request timed out +Error syncing job: Timeout: request did not complete within allowed duration +Timeout: request did not complete within allowed duration +the server cannot complete the requested operation at this time, try again later +unable to create pods: the server cannot complete the requested operation at this time +Server.processUnaryRPC failed to write status # Whats what???? Why it was triggered? +route ip\+net: no such network interface # And what? +dockerd-current:.*Error getting v2 registry # OpenShift docker registry died by that moment +forbidden: no providers available to validate pod request # This when I tried to old supplementalGroup with coma. Unclear why this was wrong, but ok. +unable to validate against any security context constrain # This was supplementalGroup missing in namespace definition +ImagePullBackOff: "Back-off pulling image # This was due to registry diying, making it inaccessible +ImagePullBackOff: Back-off pulling image +PullImage.*getsockopt: connection refused +Error syncing pod.*getsockopt: connection refused +secret.*not registered # Unclear but was related to development of davmail +Couldn't get secret.*not registered +Chown failed on +Chmod failed on + + + +# Too many parallel mounts cause malfunction (transient scope) https://github.com/kubernetes/kubernetes/issues/79194 +# ----------------------- +Mount failed: exit status 1 +mount failed: exit status 1 +Failed to start transient scope unit +Mounting command: systemd-run # potentially we can use different mounter? +Mounting arguments: +origin-node: Output: umount.*target is busy # this is probably a very different problem +origin-node: Unmounting arguments: +Error: UnmountVolume.TearDown failed for volume + +# docker cgroup-driver related stuff (needs inverstigation) https://github.com/kubernetes/kubernetes/issues/76531 +# -------------------- +Error while processing event +/sys/fs/cgroup/memory +RecentStats: unable to find data for container +Failed to get RecentStats +Failed to read "/sys/fs/cgroup +Failed to update stats for container "/user.slice +Failed to create summary reader for +Failed to detect containers: readdirent: no such file or directory +unable to find data for container /system +origin-node: container /kubepods.slice + +# Known errors which we tolerate +# ------------------------------ +has become an orphan, killing it # left-over resources, etc. +is already stopped +but volume paths are still present on disk +Same superblock, different security settings +required revision has been compacted # etcd +Could not get instant cpu stats: different number of cpus + +# performance (may be worth looking into) +# ----------- +du and find on following dirs took + +# Set of problems with sandboxes, cleaning resources (CronJobs). It was there before and probably not critical +# ------------------------------ +Failed to stop sandbox +killPodWithSyncResult failed +Cannot find network namespace for the terminated container +rpc error: code = 4 desc = context deadline exceeded +rpc error: code = 2 desc = no such process +cannot delete builds.build.openshift.io +Unmount skipped because path does not exist +stderr: du: cannot access +CNI request failed with status 400 # https://github.com/kubernetes/kubernetes/issues/72044 (not a problem) +network: failed to Statfs +Error while adding to cni lo network +ns/net: No such file or directory # this is still the same +ns/net": no such file or directory +garbagecollector.go.*not found + +# containers stopped/dying simmultaneously +# ------------------- +not found in pod's containers # Unclear, containers died in the pod (before cgroups killed one container, but many more dissapperead for some reason shortly) +failed to exit within.*trying direct SIGKILL # Unclear, for some reason many containers (pods) were stopped simulatenously.... +failed to exit within.*using the force +No ref for container # Related. Then, many of them seems exited before or what happened? +Can't make a ref to pod +Error: No such container +Error deleting network when building cni runtime +rpc error: code = 2 desc = container not running # Related. The same +rpc error: code = 2 desc = Error: No such container +rpc error: code = 2 desc = containerd: process not found for container +returned error: No such container +containerd: container not found +json returned error: No such container +devmapper: Unknown device # also emptyDir volumes seems missing +Both sandbox container and checkpoint for id.*could not be found +NetworkPlugin cni failed to teardown pod +Failed to retrieve checkpoint for sandbox +cannot exec a container that has run and stopped +cannot join network of a non running container +Container.*is not running + + +# Filtering some known and repeated stuff +# -------------------- +adei # adei cron jobs mounting/unmounting adei-* volumes + +# Normal errors, due to current development efforts +# ------------- +restarting failed container +error processing PVC +Could not construct volume information: no volume plugin matched +Readiness probe for +Liveness probe for +Failing rollout for +is dead, but RestartPolicy says that we should restart it +the object has been modified; please apply your changes to the latest version and try again +container start failed: ErrImagePull +starting container process caused +returned error: No such image +Error setting up exec command in container +endpoints "gfs" not found # I forgot to create it while creating new ns +Unable to authenticate the request due to an error +volume-subpaths: no such file or directory + + +# Normal errors, like communication failures, etc. +# ------------- +Error proxying data from client to backend +Error proxying data from backend to client + +# Suppress multi-line output what we can't interpret after filtering +# -------------------------- +In some cases useful info about processes that use # suppress multi-line output +the device is found by +with error: exit status +origin-node: - exit status +origin-node: ' +the following error information was pulled from the glusterfs log to help diagnose this issue + +# Non errors from openshift +# ------------------------- +origin-master-controllers: I0923 # informational +origin-master-controllers: I0925 +origin-master-controllers: I0926 # DS: May worth investigating, scheduling failures +origin-node: I0923 +origin-node: I0924 +origin-node: I0925 +origin-node: I0926 +tomic-openshift-master-api: I0926 +atomic-openshift-master-api:.*Trace +origin-master-controllers: Trace +Starting watch for +updating pod condition for +Using node IP +Delete endpoint +is released and reclaim policy +found previous inflight deployment +already in flight, abandoning +entered phase "Released" +Checking for prebound volumes with node affinity +Discovered runtime cgroups name +is not a mountpoint, deleting +List /apis +LoadBalancerRR: Setting endpoints for +dockerd-current:.*cleaned up +dockerd-current:.*Attempting next endpoint for pull after error # This is a problem, but this is secondary messages... +dockerd-current:.*Failed to cancel upload: unauthorized: authentication required + +# System informational Messages +# ----------------------------- +kernel: XFS +systemd: Starting +systemd: Created +systemd: Stopping +systemd: Removed +systemd: Started +systemd: Stopped +systemd-logind: New session +systemd-logind: Removed session +etcd: store.index +etcd: segmented wal file +etcd: finished scheduled compaction +etcd: compacted rev +etcd: purged +etcd: start +etcd: saved +etcd: compacted +etcd: avoid queries with large range +device-mapper: ioctl: remove_all left.*device +dnsmasq.*setting upstream servers from DBus +dnsmasq.*using nameserver +kernel: nf_conntrack: falling back to vmalloc +IPv6: ADDRCONF +NetworkManager.*veth +ntpd.*veth +kernel: device veth # infamouse veth devices +veth.*link connected +Listen normally on +Creating netns +rotating log files |