Import Courses via Tutor for Development

[misilot]

Hello,

I am looking to start using tutor to hand to our course developers. So, I am looking for the recommended best practices in regards to this that way we can easily move the courses from development to production, ideally via Git/GitHub.

I looked through the documentation but did not see anything, and the only command I saw in tutor was for importing a demo course.

Thanks!

[tony-h]

Hello @misilot,

Having an understanding of docker volumes helps. The problem to overcome is that the content to import must be inside of the docker container (the same problem exists for exporting; the exported course contents remain inside of the container).

The logic is simple:

1. Start an lms or cms docker stack with a mounted volume
2. Call ./manage.py cms import /volume/dir/course_contents
3. Exit the docker stack

Here is our import script if you can follow my bash script. You can call this from the host with the course export as the arg. It should work if you first call git clone and then use that directory as the arg.

import-course.sh

#!/bin/bash

###############################################################################
# Script by Anthony
# Imports a single course in Open edX from a .tar.gz file or course directory
###############################################################################

if [ $# -eq 0 ]
then

    echo ""
    echo "-----------------------------------------------"
    echo "'import-course.sh' imports an Open edX course"
    echo " The course can be in one two formats: "
    echo "  - file.tar.gz (an export from the web interface)"
    echo "  - directory_name (a directory containing the course contents)"
    echo ""
    echo "usage: "
    echo "    import-course.sh file_name.tar.gz [user]"
    echo "    import-course.sh path/to/course [user]"
    echo ""
    echo "examples:"
    echo "    import-course.sh IT_IS+PP_01+2021_SP.tar.gz"
    echo "    import-course.sh imports/IT_IS+PP_01+2021_SP"
    echo "    import-course.sh IT_IS+PP_01+2021_SP.tar.gz sysadmin"
    echo ""
    echo "-----------------------------------------------";
    echo ""
    exit
fi

echo "-----------------------------------------------"
echo "Importing '$(basename $1)'..."
echo ""

#-----------------------------------------------------------
# Copies the shell script to the volume directory
# Need to execute the 'import' command inside of the container
#-----------------------------------------------------------
write_script() {

#
# Write commands to a shell script
#
# !!NOTE!! These variables are evaluated on the host, not in the docker-compose containers
# These variables become literals when written inside of the container.
#
cat <<EOT > temp.sh
    #!/bin/bash

    # Import the course extracted to a directory
    if [[ -z "$FILE_NAME" ]]; then
        ./manage.py cms import ../data /tmp/volume/ > /tmp/volume/$(basename $VOLUME_DIR).log 2>&1

        # Clean up
        chown $UID:$UID /tmp/volume/$(basename $VOLUME_DIR).log

    # Import a file.tar.gz file
    else
        # Create the import directory and extract the course data
        mkdir -p /tmp/import_data/
        tar -xzf /tmp/volume/$FILE_NAME -C /tmp/import_data/ --strip-components=1

        # Determine if the "course.xml" file exists in the current directory
        # If not, then change to the first directory.
        # The export file will contain the course as a directory if exported export_all_courses
        if [ ! -f "/tmp/import_data/course.xml" ]; then
            
            # Import the course in sub directory
            ./manage.py cms import ../data /tmp/import_data/*/ > /tmp/volume/$FILE_NAME.log 2>&1
            
        else
            # Import the course in current directory
            ./manage.py cms import ../data /tmp/import_data/ > /tmp/volume/$FILE_NAME.log 2>&1
        fi
        

        # Clean up
        chown $UID:$UID /tmp/volume/$FILE_NAME.log
        rm -r /tmp/import_data/
    fi

    # Clean up
    rm /tmp/volume/temp.sh
EOT
}

#-----------------------------------------------------------
# If specified user (allows another sudo users to execute this script)
if [ -n "$2" ]; then
    USER="$2"
fi

# If arg is directory, set VOLUME_DIR
if [[ -d $1 ]]; then
    VOLUME_DIR=$(realpath $1) 
# If arg is a file, set VOLUME_DIR and FILE_NAME
elif [[ -f $1 ]]; then
    VOLUME_DIR=$(dirname $(realpath $1))
    FILE_NAME=$(basename $1)
# Otherwise, invalid input
else
    echo "'$1' is not valid a file or directory."
    exit 1
fi

#-----------------------------------------------------------

# Change to dir to use relative paths
cd $VOLUME_DIR

# writes the script file to the mount point for the volume
write_script

# Start the docker containers and execute the script
docker-compose \
     -f /home/"$USER"/.local/share/tutor/env/local/docker-compose.yml \
     -f /home/"$USER"/.local/share/tutor/env/local/docker-compose.prod.yml \
     --project-name tutor_local run --rm \
     --volume "$VOLUME_DIR":/tmp/volume cms \
     sh -c "/bin/bash /tmp/volume/temp.sh $FILE_NAME"

#-----------------------------------------------------------

echo ""
echo "All done! Check the log for errors"
echo "less $VOLUME_DIR/$(basename $1).log"
echo "-----------------------------------------------"

echo ""