July 25, 2018 by jcberthon Containers, volumes and file permissions Permission Denied for Container's Volume There is a subject which seems to be completely abstruse to many users of containers on Linux, it is about sharing data between a host and a container or between containers. # docker run -it alpine sh / # whoami root / # mount /dev/sda1 /tmp mount: permission denied (are you root?) Or / # swapoff -a swapoff: /dev/sda2: Operation not permitted. You can create custom Seccomp profiles for your container, disabling for example calls to chmod. Let's retrieve the default docker Seccomp profile. You can access the /etc and /usr directories within SELinux context, but you cannot obtain write everywhere, so z and Z will occasionally give you unable to label issues when spinning up docker containers with volume mounts from those locations. However, if you have SELinux protected files elsewhere, e.g. In a users home directory, you'd be. The result should be that when docker creates a directory in there for the volume mount (as root), it will be owned and writable by the group 100. If your docker container runs with that GID, then it will be able to create files in there. To reference C: Users admin volume, you would use //c/Users/admin/volume. Configure Oracle VirtualBox to have access to that folder. This is the important solution that is not well documented. Oracle VirtualBox, the virtual machine behind Docker Toolbox, does not have access to your folders by default.
Recently I was leveraging Azure App Services to deploy my Docker packaged .NET Core app. My setup includes VS 2017 v15.2, Docker CE v17.03.1-ce-win12 (stable) and Windows 10 Enterprise (with Creators update).
My app ran fine locally without Docker but as soon as I tried deploying to a Linux container VS gave me a weird error:
I figured there was something funky going on with my Docker settings. Navigating to Docker Client -> Settings -> Shared Drives none of my drives were shared (also weird since I am pretty sure I had set them up earlier). Maybe the Windows 10 Creators update had something to do with that? Anyways…
Re-sharing my local drive with Docker, I uncovered another error:
I tried a number of times to share, including resetting cached credentials, using local credentials etc. No dice :(
Of course looking at the documentation sent me down some bunny trails around making sure inbound firewall rules were setup correctly between the Host and VM processes.
After a chunk of time researching the issue and trying a bunch of things, I have a solution that worked for me - one that might save you time. The solution actually has nothing to do with the error displayed!
- Make sure your target drive is unshared
Drive Properties > Sharing > Advanced > 'Share this folder' is unchecked - As part of installing Docker you should have a DockerNAT interface setup. Uncheck the File and Printer sharing property and press OK.
Adapter Properties > Networking > Uncheck File and Printer Sharing for Microsoft Networks - Now reverse what you did i.e. check the same file and printer sharing property and hit OK.
After the following the outlined steps above, I was able to share my target drive with Docker with no issues. Seems a bit voodoo no? I hope the tooling will improve to side step this issue altogether in the near future.
Kubernetes local volumes go beta. However, what is it, a Kubernetes local volume? Last time, we have discovered, how to use Kubernetes hostPath volumes. However, we also have seen, that hostPath volumes work well only on single node clusters. Here, Kubernetes local volumes help us to overcome the restriction and we can work in a multi-node environment with no problems.
„Local volumes' are similar to hostPath volumes, but they allow to pin-point PODs to a specific node, and thus making sure that a restarting POD always will find the data storage in the state it had left it before the reboot. They also make sure that other restrictions are met before the used persistent volume claim is bound to a volume.
Note, the disclaimer on the announcement that local volumes are not suitable for most applications. They are much easier to handle than clustered file systems like glusterfs, though. Still, local volumes are perfect for clustered applications like Cassandra.
Let us start:
Contents
- Step 7 (optional): LifeCycle of a Local Volume
- Katacoda persistent Volumes Hello World with an NFS Docker container
- Other Kubernetes Series posts in this blog:
- (1) Installing Minikube on CentOS
- (2) Kubernetes Service on Minikube
- (3) Kubernetes Cluster with Kubeadm
- (4) Kubernetes Persistent Volumes (a hello world a la hostPath)
- We need a multi-node Kubernetes Cluster to test all of the features of „local volumes'. A two-node cluster with 2 GB or better 4 GB RAM each will do. You can follow the instructions found on (3) Kubernetes Cluster with Kubeadm in order to install such a cluster on CentOS.
Docker Mount Permission Denied
According to the docs, persistent local volumes require to have a binding mode of WaitForFirstConsumer. the only way to assign the volumeBindingMode to a persistent volume seems to be to create a storageClass with the respective volumeBindingMode and to assign the storageClass to the persistent volume. Let us start with
The output should be:
Since the storage class is available now, we can create local persistent volume with a reference to the storage class we have just created: Craftopia - soundtrack download.
Note: You might need to exchange the hostname value „node1' in the nodeAffinity section by the name of the node that matches your environment.
The „hostPath' we had defined in our last blog post is replaced by the so-called „local path„.
Similar to what we have done in case of a hostPath volume in our last blog post, we need to prepare the volume on node1, before we create the persistent local volume on the master:
The output should look like follows:
Similar to hostPath volumes, we now create a persistent volume claim that describes the volume requirements. One of the requirement is that the persistent volume has the volumeBindingMode: WaitForFirstConsumer. We can assure this by referencing the previously created a storageClass:
With the answer:
From point of view of the persistent volume claim, this is the only difference between a local volume and a host volume.
However, different to our observations about host volumes in the last blog post, the persistent volume claim is not bound to the persistent volume automatically. Instead, it will remain „Available' until the first consumer shows up:
This should change in the next step.
The Kubernetes Architects have done a good job in abstracting away the volume technology from the POD. As with other volume technologies, the POD just needs to reference the volume claim. The volume claim, in turn, specifies its resource requirements. One of those is the volumeBindingMode to be WairForFirstCustomer. This is achieved by referencing a storageClass with this property:
Once a POD is created that references the volume claim by name, a „best match' choice is performed under the restriction that the storage class name matches as well.
Okay, let us perform the last required step to complete the described picture. The only missing piece is the POD, which we will create now:
This should yield:
Before, we have seen that the persistent volume claim was not bound to a persistent volume yet. Now, we expect the binding to happen, since the last missing piece of the puzzle has fallen in place already:
Yes, we can see that the status is bound to claim named „default/my-claim'. Since we have not chosen any namespace, the claim is located in the „default' namespace.
The POD is up and running:
We now can create an index file in the local persistent volume:
Now, since the index file is available, we can access the index file of the POD. For that, we need to retrieve the POD IP address:
Now we can access the web server's index file with a cURL command:
Perfect.
Note: as long as the index file is not present, you will receive a 403 Forbidden message here. In that case, please check that you have created the index file in the correct host and path.
Step 7.1: Exploring Local Volume Binding after POD Death
Here, we want to explore what happens to an orphaned Kubernetes local volume. For that, we delete a POD with a local volume and observe, whether or not the binding state changes. My guess is, that once a local volume is bound to a persistent volume claim, the binding will persist, even if the corresponding POD has died.
Enough guessing, let us check it! See below the binding state if a POD is up and running and it is using the local volume:
Now let us delete the POD and check again:
This should yield:
Before, we have seen that the persistent volume claim was not bound to a persistent volume yet. Now, we expect the binding to happen, since the last missing piece of the puzzle has fallen in place already:
Yes, we can see that the status is bound to claim named „default/my-claim'. Since we have not chosen any namespace, the claim is located in the „default' namespace.
The POD is up and running:
We now can create an index file in the local persistent volume:
Now, since the index file is available, we can access the index file of the POD. For that, we need to retrieve the POD IP address:
Now we can access the web server's index file with a cURL command:
Perfect.
Note: as long as the index file is not present, you will receive a 403 Forbidden message here. In that case, please check that you have created the index file in the correct host and path.
Step 7.1: Exploring Local Volume Binding after POD Death
Here, we want to explore what happens to an orphaned Kubernetes local volume. For that, we delete a POD with a local volume and observe, whether or not the binding state changes. My guess is, that once a local volume is bound to a persistent volume claim, the binding will persist, even if the corresponding POD has died.
Enough guessing, let us check it! See below the binding state if a POD is up and running and it is using the local volume:
Now let us delete the POD and check again:
Yes, I was right: the status is still „Bound', even though the POD is gone.
Step 7.2: Attach a new POD to the existing local volume
Let us try to attach a new POD to the existing local volume. For that, we create a new POD with reference to the same persistent volume claim (named „my-claim' in our case).
This will an output
Docker Volume Mount Permission
It is just a simple CentOS container that is sending the content of the data to the log every 10 sec. Let us retrieve the log now:
Cool, that works fine.
Step 7.3: Verifying Multiple Read Access
How about attaching more than one container to the same local volume? Let us create a second centos container named „centos-local-volume2':
This will an output
Let us retrieve the log once again:
Docker Volume Mount Permission Denied
Here, we can see that both containers have read access to the volume.
Step 7.4: Verifying Multiple Write Access
Now let us check the write access by entering the first centos container and changing the index file:
Now let us check the log of the second container:#
That works! And it works also the other way round:
Docker Volume Mount Permission Denied 2018
As you can see, I had been quick enough to still see two of the old lines, but the last line is showing that the content has been changed by centos-local-volume2.
In this blog post, we have shown that Kubernetes local volumes can be run on multi-node clusters without the need to pin PODs to certain nodes explicitly. Local volumes with their node affinity rules make sure that a POD is bound to a certain node implicitly, though. Kubernetes local volumes have following features:
- Persistent volume claims will wait for a POD to show up before a local persistent volume is bound
- Once a persistent local volume is bound to a claim, it remains bound, even if the requesting POD has died or has been deleted
- A new POD can attach to the existing data in a local volume by referencing the same persistent volume claim
- Similar to NFS shares, Kubernetes persistent local volumes allow multiple PODs to have read/write access
Kubernetes local persistent volume they work well in clustered Kubernetes environments without the need to explicitly bind a POD to a certain node. However, the POD is bound to the node implicitly by referencing a persistent volume claim that is pointing to the local persistent volume. Once a node has died, the data of all local volumes of that node are lost. In that sense, Kubernetes local persistent volume cannot compete with distributed solutions like Glusterfs and Portworx volumes.
