ecryptfs does not work properly over nfs, cifs, samba, WebDAV, or aufs

Date: 2007-06-08 16:57
Sender: mhalcrowProject Admin
Logged In: YES
user_id=729064
Originator: NO

FYI, Erez Zadok recently confirmed a problem with stacked filesystems,
writepages, and NFS and has proposed a fix, which we need to evaluate.

Date: 2007-02-23 00:06
Sender: mhalcrowProject Admin
Logged In: YES
user_id=729064
Originator: NO

Dmitriy Monakhov moved the O_LARGEFILE into the flags under all
circumstances. This may have fixed the problem.

Date: 2007-01-19 14:49
Sender: mhalcrowProject Admin
Logged In: YES
user_id=729064
Originator: NO

This is an explicit call to BUG by NFS; while trying to do a radix insert
of an encrypted page, it looks like the page already exists in the NFS
inode page map when it shouldn't. If NFS were to just ignore the issue
rather than bug out, this may work fine. In any event, this is yet another
NFS corner case that we will need to scrutinize.

Tyler-

What's the status of this bug in the upstream kernel code?

I seem to remember you saying that you had a handle on the fixes. Have those been merged upstream? If so, can you point to the git commits, in case others want to try and backport them to their kernels?

:-Dustin

Unfortunately, I haven't gotten to work on this bug much in the last month. I do not yet have a working patch set, but things were getting close for NFS support. I will try to devote some time this week to more work on NFS support.

Setting to in-progress.

Tyler has some working prototypes for this now.

Don't know if this helps, but I hit a different BUG (in inode.c) every time I try to use rsync/SSH to back up data to an ecryptfs directory. Trace is attached.

I have found a workaround (at least for the last 5 minutes):

1) Uninstall ecryptfs-utils in the client machine and install sshfs
2) Mount Private using sshfs.

It works for me up to now.

I'm not sure if this is related, but I tried ecryptfs with .private on a Rackspace cloud storage filesystem (cloudfuse) and it would crash the file system every time (http://github.com/redbo/cloudfuse/issues#issue/4 ). I'm sure this is a problem on the file system side (which shouldn't just crash), but this obviously sounds related due to the observations for other remote file systems. At worst, consider it just another data point.

Let me know if I can help testing in any way.

Not sure if this is the same bug or a different one, but I find ecryptfs generally works smoothly over NFS, except that the second and subsequent "ls" in any folder doesn't return any files. I don't see anything in dmesg to suggest a problem.

This sounds like the bug reported two years ago here: http://<email address hidden>/msg00265.html

I see the exact same behavior as Dan above me. In any folder the first ls (or any other command which requires file system data) works fine where the second and all subsequent calls fail. Nothing in any log file nor in dmesg as far as I can tell.
Reading through the comments I'm also not sure if this is a separate bug or not.

I'm running:
Client:
Kernel: 2.6.36-ARCH
nfs-utils: 1.2.2-3
ecryptfs-utils: 83

Server:
Kernel: 2.6.32-25-generic
nfs-utils: 1.2.0-4
ecrytpfs-utils 83

Hey Tyler,

What's the current state of your eCryptfs-on-NFS work-in-progress?

Dustin

Hello,

I tried running "ls" on a directory mounted over NFS. This works fine for the first time but for subsequent attempts "ls" does not show any output.

However, when I repeat the same operation over a mount over local filesystems, ls works fine.

Is this a known problem?

The ecryptfs README says that stability with NFS is still under progress. So is the problem with stability related to the stacking model that ecryptfs is based on, or is it due to the encryption/decryption complexity?

Thanks,
Pallavi

Confirming the problem as reported by Dan (#10), Corni (#11) and Pallavi (#13):
Mounting over NFS: First "ls" after the mount works, subsequent "ls" return an empty directory.

FYI, Tyler is getting closer to solving this bug, with a few patches
having been pushed to Linus' kernel tree.

Tyler, could you perhaps give us a brief update on where we are now?

--
:-Dustin

On the NFSv3 client front, I have all of the needed eCryptfs patches upstream (as of 2.6.39-rc5), but there is still an NFS client patch that needs to be accepted upstream. The NFS client maintainer has nack'ed it, but I plan on making a couple small changes and resubmitting it soon.

http://www.spinics.net/lists/linux-nfs/msg20820.html

@Tyler, what's the status on that NFS client patch? Any progress?

eCryptfs on top of NFS isn't going to happen in the short term. Nearly all of the eCryptfs fixes are now in place, except for one last sticking point. The NFS maintainer is requiring that eCryptfs fix up a struct nameidata and pass it down into NFS, but that can get ugly and is something that the VFS maintainer is against. The VFS maintainer has alluded to a patch set that he is working on to do away with the nameidata passing, but it is not currently posted anywhere. When that happens, eCryptfs has a much better chance at supporting NFS.

@Tyler, thanks for your status update!

@Tyler: does this last remaining NFS issue also effect the other protocols like WebDAV or CIFS? If not, would it be possible to release a fix which supports all the other protocols except NFS?

On 2012-01-16 20:47:27, Chris wrote:
> @Tyler: does this last remaining NFS issue also effect the other
> protocols like WebDAV or CIFS? If not, would it be possible to release a
> fix which supports all the other protocols except NFS?

I'm not certain as my focus was just on NFS. CIFS looks to make heavy
use of nameidata, which is the basis of the NFS holdup. I have no idea
about WebDAV.

Will this bug be fixed, or it is already fixed?

On 2012-12-13 19:10:06, Don wrote:
> Will this bug be fixed, or it is already fixed?

It isn't fixed. Most of the work for NFS is done, but there's still more
to do. It is marked wishlist because no one is working on it and it
isn't considered a priority at this time.

In the future, please check the bug status rather than asking if it is
fixed. Thanks!

FYI it does seem to work over afp/netatalk. It isn't a workaround for most uses (including mine unfortunately) but it might help some people.

for me doing a bindfs mount on top of the webdav mount and then mounting the ecryptfs on the bindfs mount worked.

mount.davfs https://dav.box.com box
bindfs box/dav /crypt
mount -t ecryptfs /crypt /decrypt

Hi zombi

>for me doing a bindfs mount on top of the webdav mount and then mounting the ecryptfs on the bindfs mount worked.

does this work with "distributed editing"?
e.g
- mounting from device A
- mounting from device B
- edit on device A

are the changes properly reflected on device B ?

As a work around I successfully shared my encrypted partition using CIFS instead. I do have to share the partition with Windows boxes as well and so far everything seems to work fine. Performance is good and usability is seemless for both linux and windows clients. I just need to look into security, but the configurability has been simple so far.

By the way the title of this bug seems misleading as ecryptfs seems to work fine for cifs, samba, and even sshfs. Not sure about WebDAV or aufs .

@Johnny: You mean eCryptFS runs better on SAMBA than on NFS on your side? Can you give us some informations about your setup?

Hello KrautOS. Here's some of my setup info, please let me know if you need more details. Thanks.

vmhost:
vSphere 5.5

vmmachine (nfs-server):
>uname -a
Linux xxx 3.11.0-18-generic #32-Ubuntu SMP Tue Feb 18 21:11:14 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
>lsb_release -a
Distributor ID: Ubuntu
Description: Ubuntu 13.10
Release: 13.10
Codename: saucy
>pvdisplay
  --- Physical volume ---
  PV Name /dev/md0
  VG Name RaidVG
  PV Size 3.63 TiB / not usable 4.81 MiB
  Allocatable yes (but full)
  PE Size 4.00 MiB
  Total PE 951549
  Free PE 0
  Allocated PE 951549
  PV UUID EIJ1f2-N30Y-TRRk-dkW7-qtqC-MgIx-UXyx4n
>vgdisplay
  --- Volume group ---
  VG Name RaidVG
  System ID
  Format lvm2
  Metadata Areas 1
  Metadata Sequence No 2
  VG Access read/write
  VG Status resizable
  MAX LV 0
  Cur LV 1
  Open LV 1
  Max PV 0
  Cur PV 1
  Act PV 1
  VG Size 3.63 TiB
  PE Size 4.00 MiB
  Total PE 951549
  Alloc PE / Size 951549 / 3.63 TiB
  Free PE / Size 0 / 0
  VG UUID vfKSY2-RbHq-ZXih-EGXD-JCYw-IKEq-7J2DZZ
>lvdisplay
  --- Logical volume ---
  LV Path /dev/RaidVG/NFSLV
  LV Name NFSLV
  VG Name RaidVG
  LV UUID Ykdggh-D769-sQmp-ilzL-1bpL-a8eF-MEOvqw
  LV Write Access read/write
  LV Creation host, time nfs-server, 2014-03-02 12:54:24 -0700
  LV Status available
  # open 1
  LV Size 3.63 TiB
  Current LE 951549
  Segments 1
  Allocation inherit
  Read ahead sectors auto
  - currently set to 256
  Block device 252:2
>/etc/fstab
/dev/mapper/RaidVG-NFSLV /srv/nfs ext4 defaults 0 2
/srv/nfs /srv/nfs ecryptfs defaults 0 0
/dev/mapper/cryptswap1 none swap sw 0 0
>package versions (amd64):
mdadm 3.2.5-5ubuntu2
lvm2 2.02.98-6ubuntu1
ecryptfs 103-0ubuntu2
nfs-kernel-server 1:1.2.8-2ubuntu2
>/etc/exports
/srv/nfs 192.168.2.0/25(secure,rw,sync,wdelay,hide,no_subtree_check,secure_locks)
>exportfs
/srv/nfs 192.168.2.0/25
>syslog
kernel: [ 14.559732] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
kernel: [ 14.584691] NFSD: starting 90-second grace period (net ffffffff81cd2dc0)
rpc.mountd[1209]: Version 1.2.8 starting

nfs-client:
>/etc/fstab
nfs-server:/srv/nfs /mnt/nfs nfs4 rsize=8192,wsize=8192,timeo=14,intr 0 0
>package versions
nfs-common 1:1.2.5-3ubuntu3.1
>sudo showmount -e nfs-server
Export list for nfs-server:
/srv/nfs 192.168.2.0/25
>sudo mount /mnt/nfs
mount.nfs4: mounting nfs-server:/srv/nfs failed, reason given by server:
  No such file or directory
> nfs-server syslog
nfs-server rpc.mountd[1209]: qword_eol: fp...

I just done a server with NFS(w/o Kerberos)+NIS on basis of the upcoming Ubuntu 14.04 with server and desktop installations. eCryptfs seems to work, as i can log into an eCryptfs protected home directory on the graphical login screen (lightdm needs some config changes though).

At first i thought it would be just able with an extra bind mount like:
mount -t nfs4 -o proto=tcp,port=2049 SERVER:/home /home.nfs
mount -o bind /home.nfs /home

But it's also working with a direct mount of NFSv4 on /home, i only missed to install "ecryptfs-utils" on the second desktop i was testing on, as i seems to not be installed when you don't vote for encrypting the users home folder.

Now i need some rough test cases, as i just did login with a fresh account and that may not be the corner cases. Does somebody have any clues on?

@Johny: as i just looked at your setup it seems, you did something like a mixed thing between NFSv3 and NFSv4. That's you can export the folder is really strange as eCryptfs should be transparent to NFS, it's only that seems to have problems on top of NFS. For your setup have a look on https://help.ubuntu.com/community/NFSv4Howto

@Tyler: It's been a while since your last comment, so do you mind to give us some update on your point of view?

@KrautOS: It's been a long while since you commented, and still I do not have any clue about what the bug status is? Is Tyler still working on that? I'm going to work ecryptfs on samba for the personal project in the lab...

@wlxing: Sorry didn't see the notification until now. But actually when i worked on this years ago it was for a SOHO demo project with desktop clients bound via LAN to a centralized NFS Server for keeping clients dump and still having encryption. In the end for production we switched to laptops with no realm integration, gave up on LAN dependency and just did push backups of eCryptFS encrypted files. Didn't try to integrate eCryptFS over NFS any further since then :(

In server scenarios eCryptFS would be still a nice way of interactively working with file based encryption on remote storage. EncFS is doomed and using GPG is non-interactive. The idea of streaming a big file through remote storage protocols and looping on top it with either LUKS or Ext4 encryption doesn't seem too fancy although it worked when i tested it briefly with LUKS and btrfs.

@kirkland @tyhicks Guess you're guys quite busy with a lot of stuff at Canonical ;) But do you have an short update on this? Or do you know any alternative to eCryptFS for using file level based encryption on remote storage through the mentioned protocols?

You should probably take a look at gocryptfs.