Skip to content

Latest commit

 

History

History
124 lines (82 loc) · 3.83 KB

README.md

File metadata and controls

124 lines (82 loc) · 3.83 KB
Raw

xtract-container-service

This is the repository for the Xtract Container Service (XCS), an application for pushing and pulling Docker and Singularity containers.

Getting started for development

These instructions will get the XCS application running on your local machine for development and testing purposes.

Prerequisites

  • Docker (available here)
  • Singularity (available here)
  • AWS CLI (available here)
  • AWS RDS, S3, SQS
  • PostgreSQL

Installation

  1. Clone this repository and activate a virtual environment:

     git clone https://github.com/xtracthub/xtract-container-service
 cd xtract-container-service
 virtualenv venv
 source venv/bin/activate

  2. Install the requirements:

     pip install -r requirements.txt

Setting up AWS

  1. Configure your AWS CLI:

     aws configure

  2. Next, create a PostgreSQL database in AWS RDS and store the following in a file named database.ini:

     [postgresql]
 host=YOUR_HOST_ADRESS
 database=postgres
 user=postgres
 password=YOUR_PASSWORD

  3. Create an AWS S3 bucket and a SQS queue named xtract-container-service:

Running XCS

  1. Save your Globus Auth. Client ID and Client Secret as environment variables:

     export GL_CLIENT=YOUR_GL_CLIENT
 export GL_CLIENT_SECRET=YOUR_GL_CLIENT_SECRET

  2. Save the path of the flask app. as an environment variable:

     export FLASK_APP=application.py

  3. Start the application with root privelages:

     sudo flask run

  4. Ensure the Docker daemon is running using sudo dockerd for Ubuntu or starting Docker Desktop for Mac.

Getting started for production

These instructions will get the XCS application running on Ubuntu for production.

Prerequisites

  • Docker (available here)
  • Singularity (available here)
  • AWS CLI (available here)
  • AWS RDS, S3, SQS
  • PostgreSQL
  • Python3

Installation and Configuration

  1. Install Apache:

     sudo apt install apache2 libapache2-mod-wsgi-py3

  2. Clone this repository and activate a virtual environment:

     git clone https://github.com/xtracthub/xtract-container-service
 cd xtract-container-service
 virtualenv venv
 source venv/bin/activate

  3. Configure the AWS CLI:

     aws configure

  4. Create a symbolic link to the /var/www/html/flaskapp folder:

     sudo ln -sT ~/xtract-container-service /var/www/html/flaskapp

  5. In flaskapp.wsgi, set your Globus Auth and AWS credentials.

  6. In /etc/apache2/sites-enabled/000-default.conf, add the following after the DocumentRoot /var/www/html line:

     WSGIDaemonProcess flaskapp threads=5 python-path=/var/www/html/flaskapp/venv/lib/python3.6/site-packages
 WSGIScriptAlias / /var/www/html/flaskapp/flaskapp.wsgi
 WSGIPassAuthorization On

 <Directory flaskapp>
     WSGIProcessGroup flaskapp
     WSGIApplicationGroup %{GLOBAL}
     Order deny,allow
     Allow from all
 </Directory>

  7. Allow Apache to run Docker as a non-root user:

     sudo groupadd docker
 sudo usermod -aG docker www-data
 sudo newgrp docker

  8. Restart Apache:

     sudo service apache2 restart

Running XCS

  1. In one terminal, ensure that Docker is stopped and start the Docker daemon:

     sudo systemctl stop docker
 sudo dockerd

Interacting with the server

XCS is a REST API so all interactions can be made with Python's request library. Examples of how to make requests can be found in app_demo.ipynb (New SDK coming soon)