This guide assumes you have a Linux machine. We use Ubuntu as our Linux installation so some files, or commands to install software, might be at a different location if you use another flavor of Linux. We will try and mark them as Ubuntu specific. If you use another flavor of Linux, please send us any updates so we can add them to this guide.
Pre-requisites
The system will store all files in the filesystem and all meta-data in a MySQL database. The extraction service is dependent on FFMPEG.
Installing Apache and Tomcat
The web application is written as a servlet using GWT. The following command will install Apache HTTPD as well as Tomcat.
Code Block |
---|
title | Ubuntu |
---|
borderStyle | solid |
---|
|
apt-get -qq -y install apache2 tomcat6
|
Updating Tomcat Script
We need to make two modifications to the tomcat script under Linux. The first one will allow Medici to read from the local file system, the second change will allow the extraction service to run as tomcat (otherwise tomcat gets confused and thinks it is already running). We also bump the memory of tomcat. While this is not required, it might improve performance.
Code Block |
---|
title | Ubuntu |
---|
borderStyle | solid |
---|
|
sed -i -e 's#TOMCAT6_SECURITY=yes#TOMCAT6_SECURITY=no#g' \
-e "s#-x /bin/bash --#--pidfile /tmp/tomcatsh.pid -x /bin/bash --#g" \
-e 's#-Xmx128M#-Xmx256M#' /etc/init.d/tomcat6
|
Installing additional JAR files
Two jar files will need to be installed in the Tomcat lib folder (in case of Ubuntu this is /usr/share/tomcat6/lib).
Download the MySQL Connector/J JDBC driver v5.1 and add the MySQL JDBC driver jar (mysql-connector-java-5.x.x-bin.jar) to the tomcat/lib folder. Internally, we use version 5.0.4.
The second jar file is the Xerces implementation, after downloading add the xercesImpl.jar to the tomcat/lib folder. Internally, we use version 2.6.2.
Once the jar files are installed you will need to restart tomcat
Code Block |
---|
title | Ubuntu |
---|
borderStyle | solid |
---|
|
/etc/init.d/tomcat6 restart
|
Installing MySQL
MySQL is used to store the metadata, the database will be created in the next section.
Code Block |
---|
title | Ubuntu |
---|
borderStyle | solid |
---|
|
apt-get -qq -y install mysql-server-5.1
|
Installing FFmpeg
FFmpeg is used to convert video to FLV video to be played in the browser as well to extract a single frame from the video.
Code Block |
---|
title | Ubuntu |
---|
borderStyle | solid |
---|
|
wget http://www.medibuntu.org/sources.list.d/`lsb_release -cs`.list --output-document=/etc/apt/sources.list.d/medibuntu.list
apt-get -qq update
apt-get -qq -y --allow-unauthenticated install medibuntu-keyring
apt-get -qq update
apt-get -qq -y install ffmpeg libavcodec-extra-52
|
Installing UnZip
UnZip is needed to unzip the war file, modifying the file telling Medici where to find the database as well as the datafolder. The following command will install Unzip.
Code Block |
---|
title | Ubuntu |
---|
borderStyle | solid |
---|
|
apt-get -qq -y install unzip
|
Additional Fonts
To enable the conversion from PDF to preview images some fonts need to be installed.
Code Block |
---|
title | Ubuntu |
---|
borderStyle | solid |
---|
|
apt-get -qq -y install ttf-dejavu-core ttf-baekmuk ttf-kochi-gothic ttf-kochi-mincho ttf-wqy-zenhei ttf-indic-fonts-core ttf-telugu-fonts ttf-oriya-fonts ttf-kannada-fonts ttf-bengali-fonts
|
Install NTP (Optional)
The time on the server is used for many interactions with the system and thus it is important for this to be correct. This can be achieved using NTP.
Code Block |
---|
title | Ubuntu |
---|
borderStyle | solid |
---|
|
apt-get -qq -y install ntp
|
Installation of Medici
Once the pre-requisites are installed, we can install Medici. The system consists of two major components, the web component as well as the extraction service. The extraction service can be shared by multiple installations. Both the web component as well as the extraction service will need to write to the same folder on disk. To enable this, we will run the extraction service as the same user as tomcat (in case of Ubuntu this will be tomcat6
).
The first step is to create a folder where the extraction service will be installed.
Code Block |
---|
mkdir -p /home/medici
cd /home/medici
|
Next, download the latest version of the extraction service. The following will download the 64 bit version of the 1.0 version of the extraction service. Install the code and change the permission to tomcat.
Code Block |
---|
wget -q -O extractor.zip "http://isda.ncsa.illinois.edu/download/get.php?project=Medici&category=extractor&version=v1.1&file=Extractor.gtk.linux.x86_64.zip"
unzip extractor.zip
mv Extractor.gtk.linux.x86_64 extractor
chmod -R g+w extractor
chown -R tomcat6.users extractor
|
Based on the machine and expected data it might be worth it to increase the memory from the default 1GB to more (we use 9GB).
Code Block |
---|
sed -i -e 's#-Xmx1G#-Xmx9G#g' extractor/ExtractionServer.ini
|
Next, install extractor as a Unix service and start it.
Code Block |
---|
title | Ubuntu |
---|
borderStyle | solid |
---|
|
cp extractor/service/extractor.ubuntu /etc/init.d/extractor
sed -i -e "s#DIR=/home/kooper/Extractor#DIR=/home/medici/extractor#" -e "s#USER=kooper#USER=tomcat6#" /etc/init.d/extractor
chmod 755 /etc/init.d/extractor
update-rc.d extractor defaults
/etc/init.d/extractor start
|
Code Block |
---|
title | Redhat (not tested) |
---|
borderStyle | solid |
---|
|
cp extractor/service/extractor.redhat /etc/init.d/extractor
sed -i -e "s#DIR=/home/kooper/Extractor#DIR=/home/medici/extractor#" -e "s#USER=kooper#USER=tomcat#" /etc/init.d/extractor
chmod 755 /etc/init.d/extractor
/sbin/chkconfig extractor on
/etc/init.d/extractor start
|
If you have a firewall and want access from outside the server to the extraction service you will need to allow for traffic on port 9856. To see the status of the extraction service as well as the log files you can connect with a web browser to: http://<hostname>:9856/extractor/status
Web Component
Next, the web component will be installed. Each installation of Medici needs its own installation. In this document we will describe the installation of one installation called medici
and second one called onlyme
.
Folder creation
The first step is to create a data folder as well as a folder to hold the query optimizations (we use lucene for this). This folder will contain all data associated with a single installation of Medici (in this case called medici).
Code Block |
---|
mkdir -p /home/medici/medici/data
mkdir -p /home/medici/medici/lucene
chown -R tomcat6 /home/medici/medici
|
The name of the folder is not important. For simplicity we recommend to name it after the name of the installation. To create multiple installations on the same machine we need to create additional folders to hold the data.
Code Block |
---|
mkdir -p /home/medici/onlyme/data
mkdir -p /home/medici/onlyme/lucene
chown -R tomcat6 /home/medici/onlyme
|
Creating the Mysql Database
Following code will create the database and give privileges to the user medici to the database.
Code Block |
---|
wget --no-check-certificate -q -O mysql.sql https://opensource.ncsa.illinois.edu/confluence/download/attachments/589837/mysql.sql
sed -e "s#@DB_SCHEMA@#medici#" \
-e "s#@DB_USER@#medici#" \
-e "s#@DB_PASS@#medici#" mysql.sql > medici.sql
mysql -u root -h localhost -p < medici.sql
|
To create a second database for the second installation we use the same code except change the database name to the second installation, for example:
Code Block |
---|
|
wget --no-check-certificate -q -O mysql.sql https://opensource.ncsa.illinois.edu/confluence/download/attachments/589837/mysql.sql
sed -e "s#@DB_SCHEMA@#onlyme#" \
-e "s#@DB_USER@#medici#" \
-e "s#@DB_PASS@#medici#" mysql.sql > onlyme.sql
mysql -u root -h localhost -p < onlyme.sql
|
Creating Context Definition
Next, we need to create a definition file that binds database and folder together. This needs to use the same values as the MySQL script.
Code Block |
---|
wget --no-check-certificate -q -O context.xml https://opensource.ncsa.illinois.edu/confluence/download/attachments/589837/context.xml
sed -e "s#@DB_SCHEMA@#medici#" \
-e "s#@DB_USER@#medici#" \
-e "s#@DB_PASS@#medici#" \
-e "s#@FOLDER@#/home/medici/medici#" context.xml > medici.xml
|
To create the defintion file for the second installation we use the values passed into the second MySQL script.
Code Block |
---|
wget --no-check-certificate -q -O context.xml https://opensource.ncsa.illinois.edu/confluence/download/attachments/589837/context.xml
sed -e "s#@DB_SCHEMA@#onlyme#" \
-e "s#@DB_USER@#medici#" \
-e "s#@DB_PASS@#medici#" \
-e "s#@FOLDER@#/home/medici/onlyme#" context.xml > onlyme.xml
|
Installation of the web component
The final step is to install the Medici web component. This involves downloading the latest version, configuration and installation. Since the application uses Google Maps a new Google Map key needs to be requested.
NOTE: You must generate an appropriate key for "http" or "https". An http key will not work with a server running in https: mode and visa versa.
Code Block |
---|
wget -q -O medici.war "http://isda.ncsa.illinois.edu/download/get.php?project=Medici&category=web&version=v1.1&file=medici.war"
unzip -q -d war medici.war
cp medici.xml war/WEB-INF/classes/context.xml
YOUREMAIL="your email address here"
GOOGLEKEY="your google key here"
sed -i -e "s#mail.from=lmarini@ncsa.illinois.edu#mail.from=$YOUREMAIL#g" \
-e "s/#user.0.email=/user.0.email=$YOUREMAIL/g" \
-e "s@search.index=.*@search.index=/home/medici/medici/lucene@g" war/WEB-INF/classes/server.properties
sed -i -e "s#key=ABQIAAAASEElYb9IDDsAc5ZKA3a2sRQmgYtTImkVBc-VhblDgOLOdwhVaBSGMDSn-_9k3bx4tYolchXvrvB8Ag#key=$GOOGLEKEY#g" war/mmdb.html
chown -R tomcat6 war
mv war /var/lib/tomcat6/webapps/medici
|
The second installation follows almost the same process but with a few changes.
Code Block |
---|
unzip -q -d war medici.war
cp onlyme.xml war/WEB-INF/classes/context.xml
YOUREMAIL="your email address here"
GOOGLEKEY="your google key here"
sed -i -e "s#mail.from=lmarini@ncsa.illinois.edu#mail.from=$YOUREMAIL#g" \
-e "s/#user.0.email=/user.0.email=$YOUREMAIL/g" \
-e "s@search.index=.*@search.index=/home/medici/medici/lucene@g" war/WEB-INF/classes/server.properties
sed -i -e "s#key=ABQIAAAASEElYb9IDDsAc5ZKA3a2sRQmgYtTImkVBc-VhblDgOLOdwhVaBSGMDSn-_9k3bx4tYolchXvrvB8Ag#key=$GOOGLEKEY#g" war/mmdb.html
chown -R tomcat6 war
mv war /var/lib/tomcat6/webapps/onlyme
|
Add proxy to Apache Httpd (Optional)
Instead of having people remember the tomcat URL, we added a proxy to the Apache Httpd allowing it to proxy the requests to Tomcat. This can be achieved by adding the redirects. If the system runs using virtual hosts it needs to be added the virtual host section otherwise a new file can be created in the conf.d folder (/etc/apache2/conf.d in case of Ubuntu).
Code Block |
---|
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyRequests Off
ProxyPreserveHost On
ProxyPass /medici http://localhost:8080/medici
ProxyPassReverse /medici http://localhost:8080/medici
|
If we want to add the second installation to the proxy as well we need to add 2 more lines to the file.
Code Block |
---|
ProxyPass /onlyme http://localhost:8080/onlyme
ProxyPassReverse /onlyme http://localhost:8080/onlyme
|
Finally the proxy http module needs to be enabled and Apache Httpd needs to be restarted.
Code Block |
---|
title | Ubuntu |
---|
borderStyle | solid |
---|
|
a2enmod proxy_http
/etc/init.d/apache2 restart
|