Error bind mounting lms container

Hi @regis,

I experimented with the bind mounting features of tutor as discussed here. However, I encountered the error below (I’m no docker master so sorry).

MacOS Mojave 10.14.6
Tutor 11.1.0

I tried the command with the lms and cms dev servers down and running but keep getting the same error. If you could share ideas on how to fix this, would appreciate it greatly.

Eragons-MacBook-Pro:themes Eragon$ tutor dev bindmount lms /openedx/venv
docker-compose -f /Users/Eragon/playground/site1/env/local/docker-compose.yml -f /Users/Eragon/playground/site1/env/dev/docker-compose.yml --project-name tutor_dev run --rm --no-deps --volume /Users/Eragon/playground/site1/volumes:/tmp/volumes lms sh -e -c rm -rf /tmp/volumes/venv
cp -r /openedx/venv /tmp/volumes/venv
chown -R 501 /tmp/volumes/venv
Creating tutor_dev_lms_run ... done
Setting file permissions for user openedx...
File permissions set.
cp: cannot create directory '/tmp/volumes/venv/lib/python3.8/site-packages/poll/public/js/translations/ja_jp': File exists
cp: cannot create directory '/tmp/volumes/venv/lib/python3.8/site-packages/poll/public/js/translations/zh_CN': File exists
cp: cannot create directory '/tmp/volumes/venv/lib/python3.8/site-packages/poll/public/js/translations/fr_CA': File exists
cp: cannot create directory '/tmp/volumes/venv/lib/python3.8/site-packages/poll/public/js/translations/pt_BR': File exists
cp: cannot create directory '/tmp/volumes/venv/lib/python3.8/site-packages/poll/public/js/translations/ko_kr': File exists
cp: cannot create directory '/tmp/volumes/venv/lib/python3.8/site-packages/poll/public/js/translations/de_de': File exists
Error: Command failed with status 1: docker-compose -f /Users/Eragon/playground/site1/env/local/docker-compose.yml -f /Users/Eragon/playground/site1/env/dev/docker-compose.yml --project-name tutor_dev run --rm --no-deps --volume /Users/Eragon/playground/site1/volumes:/tmp/volumes lms sh -e -c rm -rf /tmp/volumes/venv
cp -r /openedx/venv /tmp/volumes/venv
chown -R 501 /tmp/volumes/venv

Now this is interesting. The poll xblock contains translations for multiple languages: xblock-poll/poll/public/js/translations at master · open-craft/xblock-poll · GitHub
In particular, there are six folder pairs that will look the same for a case-insensitive filesystem:

de_DE and de_de
fr_CA and fr_ca
ja_JP and ja_jp
ko_KR and ko_kr
pt_BR and pt_br
zh_CN and zh_cn

The Mac OS filesystem is case-insensitive, but Docker does not know that. When Docker attempts to copy these directories from a case-sensitive Ubuntu filesystem to the Mac OS case-insensitive filesystem, these failures occur.

I don’t know how to resolve this, but what I do know is that Tutor cannot fix this for you. Here are a few suggestions:

  1. Get in touch with the xblock-poll authors via a Github issue on their repo (I know some people from Opencraft are here :eyes: @arbrandes :wink: )
  2. Don’t work directly on the /openedx/venv bind-mounted volume, but work on a subfolder of the virtualenv which does not include the poll/ folder.
  3. Work on a case-sensitive filesystem (I have no idea if this is possible in Mac OS. Maybe take a look at this?).
1 Like

Huh. This is a funny one. Though I’m not a Mac user, there are plenty at OpenCraft, and I’m sure they’ll start hitting this issue once Tutor is popularized as a devstack. :wink:

Though I think most Mac users over here develop on Ubuntu VMs. Is that a possibility for you, @eragon?

In any case, please do open an issue on Github. Once you do, ping me here (or there - my GH username is the same, @arbrandes), and I’ll try and prioritize it.

Thanks for pitching in @arbrandes.

:crossed_fingers:

More seriously though, don’t they already face this issue when cloning the xblock-poll repo?

Okay thanks @regis.

I’ll try to keep tabs with the OpenCraft guys working on xblock-poll. But I’ll experiment with approach 2 once I’m done installing Devstack Docker.

Hi @arbrandes ,

I started installing the DevStack Docker set-up. But I encounter connection issues with some repos most likely due to my bad connection. I’ll let you know if I see the same issues with Devstack Docker.

I avoided using Ubuntu VMs because of resource constraints (already have two VMs set-up). But if the Devstack Docker setup will face more issues, I’ll have to go the VM Ubuntu route.

Thanks a lot!

@eragon You should really try out the machine plugin: https://overhang.io/tutor/plugin/machine
I live in a remote village of the French Alps, where my bandwidth is quite poor (~500kB/s), so I primarily developed this plugin to resolve my own badnwidth and performance issues. The Machine plugin allows you to hack on Open edX on a low-end laptop with poor connectivity.

@regis,

Okay I’ll check out the machine plugin. Just finished pulling Devstack Docker and provisioning it for NFS.

Your 500kB/s is paradise compared to a download speed of 30kB/s (most of the time - but it could reach more than 500kB/s at times). My hard disk space has like 60GB space left (of 1TB). But I’ll probably archive a lot of old and unused files and code ce week-end :-D.

Merci beaucoup!

Great finding! I had the same issue here.

Newer versions of MacOS support case-sensitive file systems, although it’s not the default. So the best solution would be to use one of these if you are provisioning a new Mac. I guess it’s not possible to change the FS type without loosing all the data.

I’ve found a hack:

  1. Create a case-sensitive APFS file system using the disk utility. I named it APFS_case_sensitive, so the mount point is /Volumes/APFS_case_sensitive
  2. Move the volumes directory inside the tutor root directory (created by a previous run of bindmount) to the new volume.
cd $(tutor config printroot)
mv volumes /Volumes/APFS_case_sensitive 
  1. Create a symlink in the tutor root directory
ln -nsf /Volumes/APFS_case_sensitive/volumes .
  1. Rerun the tutor dev bindmount commands. Now the errors should be gone.

Check that the volumes directory is a link to the new APFS case sensitive volume

% ls -la                                        
total 8
drwxr-xr-x   6 andres  staff   192 Jan 26 13:18 .
drwxr-xr-x  13 andres  staff   416 Jan 26 11:57 ..
-rw-r--r--   1 andres  staff  2142 Jan 26 11:47 config.yml
drwxr-xr-x   9 andres  staff   288 Jan 26 11:11 data
drwxr-xr-x  10 andres  staff   320 Jan 26 11:41 env
lrwxr-xr-x   1 andres  staff    36 Jan 26 13:18 volumes -> /Volumes/APFS_case_sensitive/volumes
1 Like