This document describes the current method for running/testing BD Box Flask application.

Step-by-step guide

Getting read and write access tokens for a Box file

  1. Download the PEM file for the host (bd-box) that is responding to the skills invocations from LastPass Browndog shared folder

  2. Save the PEM file in your ~/.ssh directory and set file permissions to 600

  3. SSH into the bd-box host by running ssh -i ~/.ssh/bd-box.pem ubuntu@www.bdbox.ndslabs.org

  4. Run tail -f nohup.out in the ubuntu user’s home dir and you can see a log message that is produced on the skills invocation.

  5. Upload a file to DES shared Box folder https://uofi.app.box.com/folder/51605357386

  6. From running the tail command in bd-box host, get the file ID and the read and write access keys which will be used for submitting a POST request

Simulating a Box Skills invocation and running bd-box flask application to receive the invocation.

  1. Run the flask app bd-box after setting up the virtual environment and installing required packages from requirements.txt. You also need to create a conf.cfg from the template provided in the source code. Set the environment variable CONFIG to conf.cfg. Optionally you can also set the environment variable LOG_CFG to a logging configuration json document. The default one is provided in the source code. 

    1. Setting up conf.cfg: You can get the CLIENT_ID and CLIENT_SECRET from 61647055_6hw17xyv_config.json in the bd-box host home directory. Scratch directory path can be set to any path in the your machine and the file from Box will be downloaded to this path.

  2. Send a POST request with the following JSON payload to http://0.0.0.0/skills/<extractor>. Here <extractor> should be replaced by an extractor name and the read and write access tokens and the file ID obtained from running the tail command on bd-box host

    {
  "token":{
    "write":{
      "access_token":"<READ-WRITE TOKEN>"
    },
    "read":{
      "access_token":"<READ-ONLY TOKEN>"
    }
  },
     "source":{
       "id":"<FILE ID>"
     }
}

  3. This should successfully simulate the skills invocation from Box.

Running Extractor

To run the extractor from www.bdbox.ndslabs.org:

  1. Login to the machine:

    ssh -i ~/.ssh/bd-box.pem ubuntu@www.bdbox.ndslabs.org
  2. Check if extractor is already running:
    ps aux | grep extractor.py

  3. Kill the running extractor if needed

    kill <PID>
  4. Start the extractor: 
    cd /opt/rank_galaxies/im_rank
    env PYTHONPATH=/home/ubuntu/pyclowder2 python3 extractor.py

    Optionally you can use nohup to run the extractor without interruption.

    nohup env PYTHONPATH=/home/ubuntu/pyclowder2 python3 extractor.py

  5. Check log file:

    tail -f nohup.out