Create VMs on Nebula
- If the is a new instance for gltg and you intend to use the current database VMs, you will need to build 3 new VMS
- Nginx proxy server
- Clowder Server
- Geodashboard Server
- VMs on Nebula are created with a script that uses python-openstackclient (tested on python-openstackclient==3.4.1) (probably some other pip libraries as well). It is recommended to create a virtualenv
- set up the environment. In a linux (mac) shell within the viritualenv
export OS_AUTH_URL=http://nebula.ncsa.illinois.edu:5000/v2.0 export OS_TENANT_ID=c4121a001a8240d4a8b701d664ef4bf0 export OS_TENANT_NAME="GLTG" # This is for GLTG project, change to your project name export OS_PROJECT_NAME="GLTG" # This is for GLTG project, change to your project name export OS_USERNAME="username" # Your Nebula username export OS_PASSWORD="password" # Your Nebula password export OS_REGION_NAME="RegionOne"
Run the script
Get the script https://opensource.ncsa.illinois.edu/bitbucket/snippets/6b41ea2cfea041cb822d66b909a7bf31
run script (make sure you have correct permissions if it fails to run (use chmod 755 makevm.sh)):
./makevm.sh -n <name of new VM> -k <name of key>
for example, creating a new vm named "ilnlrs-dev" with nebula key pair "gltg"
./makevm.sh -n ilnlrs-dev -k gltg
- set up the environment. In a linux (mac) shell within the viritualenv
Setup Nginx Server
Login to the Nginx server. If you are using the project key pair it will look like this (make sure you have the key in your .ssh folder, get it from Nebula interface https://nebula.ncsa.illinois.edu/dashboard/project/access_and_security/:
ssh -i ~/.ssh/<key> ubuntu@<vm floating ip address>
Install Nginx:
apt-get install nginx
Edit nginx config
create and edit
sudo vim /etc/nginx/sites-available/gltg # creates file name gltg and opens vim editor (change to your key name)
populate config (this is bare min without ssl, more docs coming)
server { listen 80; client_max_body_size 0; proxy_read_timeout 300; # answer from server, 5 min proxy_send_timeout 300; # chunks to server, 5 min proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_http_version 1.1; port_in_redirect off; root /usr/share/nginx/html; index index.html index.htm; # Deny all attempts to access hidden files # such as .htaccess, .htpasswd, .DS_Store (Mac). location ~ /\. { deny all; } location / { try_files $uri $uri/ /index.html; } rewrite ^/geodashboard$ /geodashboard/ permanent; location /geodashboard { proxy_pass http://<geodashboard floating IP>:9000; # replace <geodashboard floating IP> with the floating IP of your geodashboard machine } rewrite ^/clowder$ /clowder/ permanent; location /clowder/ { proxy_pass http://<clowder floating IP>:9000; # replace <clowder floating IP> with the floating IP of your geodashboard machine } }
delete (soft link) of default config and enable new config
sudo rm /etc/nginx/sites-enabled/default sudo ln -s /etc/nginx/sites-available/gltg /etc/nginx/sites-enabled/gltg
Add (Edit) index file to root
For the tutorial we will only put a redirect to the /geodashboard route. If you want to put in a static landing page with links to /geodashboard, /clowder, etc, please do - everyone will love you.
sudo vim /usr/share/nginx/html/index.html # This is the root path in the config we just created
Add this text to file
<meta http-equiv="refresh" content="0; url=http:/geodashboard/" />
Install geodashboard-v3 https://opensource.ncsa.illinois.edu/bitbucket/projects/GEOD/repos/geodashboard-v3/browse (as of this documentation, geodashboard-v2 runs on the geodashboard server, but uses the v3 search page which is installed on the nginx server)
In the nginx root directory (where we put the redirect above), create a directory 'gd3' which contains the build files bundle.js, config.js, index.html. If v3 doesn't work, there is an issue that on a build an extra "}" needs to be added at the end of config.js
ls /usr/share/nginx/html/gd3 # if gd3 directory is in correct nginx root path, 'ls' will show you the v3 build files bundle.js config.js index.html
Setup Clowder Server
- Setup Puppet (Getting started - Marcus will need to do before docs will be good)
- Login to foreman https://gonzo-foreman.ncsa.illinois.edu/hosts
- verify existence of host - the name will be <name of vm>.os.ncsa.edu
- puppet env
- production
- puppet classes
- clowder
or install java
sudo apt-get install default-jre
- maybe service puppet restart on host machine
- Login to foreman https://gonzo-foreman.ncsa.illinois.edu/hosts
- configure clowder
login to clowder machine
Edit /home/clowder/clowder/custom/custom.conf
clowder should (as in must) have a security token if exposed to internet. If using ssl, set:
securesocial.ssl=true
permissions = public application.context="/clowder/" initialAdmins="" # add admin emails between quotes smtp.host="smtp.ncsa.illinois.edu" # securesocial customization securesocial.onLoginGoTo=/clowder/ securesocial.onLogoutGoTo=/clowder/login securesocial.ssl=false securesocial.cookie.idleTimeoutInMinutes=1440 # rabbitmq clowder.rabbitmq.uri="amqp://clowder:***********@rabbitmq.ncsa.illinois.edu/clowder" # you'll need the security code clowder.rabbitmq.exchange="gltg-clowder-dev" # mongodb These are the IP addresses for the current mongodb servers mongodbURI="mongodb://141.142.209.172:27017,141.142.209.173:27017,141.142.209.174:27017/gltg?replicaSet=GLTG&maxpoolsize=100" # postgres postgres.user="***********" # you'll need the postgres username postgres.password="**************" # you'll need the postgres user password postgres.host="141.142.209.176" # This is the IP of the current postgres vm postgres.db="geostream-dev" # you can start by using one of the existing databases - this is for gltg-dev # cache geostream.cache=/home/clowder/cache # security options application.secret="******************************************************" # you'll need to create and application.secret (random) commKey=************ # you'll need to create commKey (random) # storage service.byteStorage=services.filesystem.DiskByteStorageService clowder.diskStorage.path="/home/clowder/data"
Edit /home/clowder/clowder/custom/play.plugins
9992:services.RabbitmqPlugin 10005:services.PostgresPlugin
Setup Geodashboard Server
- Setup Puppet (Getting started - Marcus will need to do before docs will be good)
- Login to foreman https://gonzo-foreman.ncsa.illinois.edu/hosts
- verify existence of host - the name will be <name of vm>.os.ncsa.edu
- puppet env
- production
- puppet classes
- clowder
- what about java?
- maybe service puppet restart on host machine
- Login to foreman https://gonzo-foreman.ncsa.illinois.edu/hosts