What Happened?

Observations

Without carefully matched file permissions minikube delete quietly leaves most data on the host on linux when the driver is docker.

When running as a non root user on linux with the driver set to docker minikube start is successful and most data is stored under the folder /var/lib/docker/volumes/

When minikube delete is called however there appears to be a permissions issue that can result in the silent failure to delete most data. When this happens all this data is left intact (full data of containerized host - so all images/volumes etc).

I believe the issue occurs when there is a mismatch between the user that originally called start (and thus created the /var/lib/docker/volumes/minikube folder) and the user that calls delete

Running with sudo does delete the remaining data but this has the side effect that purge then looks in the wrong home folder (root) rather than the current user for config data to clean

Note that in the successful case the user is additionally reminded that (the relatively small) "Kicbase images have not been deleted".

Expectations

Expect either for data to be deleted or for information to be displayed informing the user that most data could not be deleted. To me the biggest issue is this silent failure, this can lead to a system's hard drive filling up and the system becoming unresponsive.

Expect not to have to call delete twice (once with and once without sudo) to delete and purge all data.

This was seen on a CENTOS 7 system.

Some anonymised logs - first we list the data, then try minikube delete. Notice that some 30GB of data remains under /var/lib/docker/volumes/ and yet no problems are reported. Then I re-run using sudo. This time 30GB is removed. Note that a different home folder is then purged.

[myuseraccount@myserver myfolder]$ sudo du -h / | sort -h -r | head -n 20 53G / 37G /var 34G /var/lib/docker 34G /var/lib 29G /var/lib/docker/volumes/minikube/_data/lib 29G /var/lib/docker/volumes/minikube/_data 29G /var/lib/docker/volumes/minikube 29G /var/lib/docker/volumes 28G /var/lib/docker/volumes/minikube/_data/lib/docker/overlay2 28G /var/lib/docker/volumes/minikube/_data/lib/docker 10G /data/jenkins 10G /data 9.9G /data/jenkins/workspace 4.7G /var/lib/docker/overlay2 2.8G /usr 2.0G /home ...

[myuseraccount@myserver myfolder]$ ./minikube delete --all --purge 🔥 Successfully deleted all profiles 💀 Successfully purged minikube directory located at - [/home/myuseraccount@MYDOMAIN/.minikube]

[myuseraccount@myserver myfolder]$ sudo du -h / | sort -h -r | head -n 20 53G / 37G /var 34G /var/lib/docker 34G /var/lib 29G /var/lib/docker/volumes/minikube/_data/lib 29G /var/lib/docker/volumes/minikube/_data 29G /var/lib/docker/volumes/minikube 29G /var/lib/docker/volumes 28G /var/lib/docker/volumes/minikube/_data/lib/docker/overlay2 28G /var/lib/docker/volumes/minikube/_data/lib/docker 10G /data/jenkins 10G /data 9.9G /data/jenkins/workspace 4.7G /var/lib/docker/overlay2 2.8G /usr 2.0G /home ...

[myuseraccount@myserver myfolder]$ sudo ./minikube delete --all --purge 🔥 Successfully deleted all profiles 💀 Successfully purged minikube directory located at - [/root/.minikube] 📌 Kicbase images have not been deleted. To delete images run: ▪ docker rmi gcr.io/k8s-minikube/kicbase:v0.0.34

[myuseraccount@myserver myfolder]$ sudo du -h / | sort -h -r | head -n 20 23G / 10G /data/jenkins 10G /data 9.9G /data/jenkins/workspace 6.8G /var 3.9G /var/lib 3.7G /var/lib/docker 3.6G /var/lib/docker/overlay2 2.8G /usr 2.0G /home ...

Attach the log file

nologfile.txt

Operating System

No response

Driver

No response