This guide assumes you have a Linux machine. We use Ubuntu as our Linux installation so some files, or commands to install software, might be at a different location if you use another flavor of Linux. We will try and mark them as Ubuntu specific. If you use another flavor of Linux, please send us any updates so we can add them to this guide.
Pre-requisites
The system will store all files in the filesystem and all meta-data in a MySQL database. The extraction service is dependent on FFMPEG.
Installing Apache and Tomcat
The web application is written as a servlet using GWT. The following command will install Apache HTTPD as well as Tomcat.
apt-get -qq -y install apache2 tomcat6
Updating Tomcat Script
We need to make two modifications to the tomcat script under Linux. The first one will allow Medici to read from the local file system, the second change will allow the extraction service to run as tomcat (otherwise tomcat gets confused and thinks it is already running). We also bump the memory of tomcat. While this is not required, it might improve performance.
sed -i -e 's#TOMCAT6_SECURITY=yes#TOMCAT6_SECURITY=no#g' \
-e "s#-x /bin/bash --#--pidfile /tmp/tomcatsh.pid -x /bin/bash --#g" \
-e 's#-Xmx128M#-Xmx256M#' /etc/init.d/tomcat6
Installing additional JAR files
Two jar files will need to be installed in the Tomcat lib folder (in case of Ubuntu this is /usr/share/tomcat6/lib).
Download the MySQL Connector/J JDBC driver v5.1 and add the MySQL JDBC driver jar (mysql-connector-java-5.x.x-bin.jar) to the tomcat/lib folder. Internally, we use version 5.0.4.
The second jar file is the Xerces implementation, after downloading add the xercesImpl.jar to the tomcat/lib folder. Internally, we use version 2.6.2.
Once the jar files are installed you will need to restart tomcat
/etc/init.d/tomcat6 restart
Installing MySQL
MySQL is used to store the metadata, the database will be created in the next section.
apt-get -qq -y install mysql-server-5.1
Installing FFmpeg
FFmpeg is used to convert video to FLV video to be played in the browser as well to extract a single frame from the video.
wget http://www.medibuntu.org/sources.list.d/`lsb_release -cs`.list --output-document=/etc/apt/sources.list.d/medibuntu.list apt-get -qq update apt-get -qq -y --allow-unauthenticated install medibuntu-keyring apt-get -qq update apt-get -qq -y install ffmpeg libavcodec-extra-52
Installing UnZip
UnZip is needed to unzip the war file, modifying the file telling Medici where to find the database as well as the datafolder. The following command will install Unzip.
apt-get -qq -y install unzip
Additional Fonts
To enable the conversion from PDF to preview images some fonts need to be installed.
apt-get -qq -y install ttf-dejavu-core ttf-baekmuk ttf-kochi-gothic ttf-kochi-mincho ttf-wqy-zenhei ttf-indic-fonts-core ttf-telugu-fonts ttf-oriya-fonts ttf-kannada-fonts ttf-bengali-fonts
Install NTP (Optional)
The time on the server is used for many interactions with the system and thus it is important for this to be correct. This can be achieved using NTP.
apt-get -qq -y install ntp
Installation of Medici
Once the pre-requisites are installed, we can install Medici. The system consists of two major components, the web component as well as the extraction service. The extraction service can be shared by multiple installations. Both the web component as well as the extraction service will need to write to the same folder on disk. To enable this, we will run the extraction service as the same user as tomcat (in case of Ubuntu this will be tomcat6).
Extraction Service
The first step is to create a folder where the extraction service will be installed.
mkdir -p /home/medici cd /home/medici
Next, download the latest version of the extraction service. The following will download the 64 bit version of the 1.0 version of the extraction service. Install the code and change the permission to tomcat.
wget -q -O extractor.zip "http://isda.ncsa.illinois.edu/download/get.php?project=Medici&category=extractor&version=v1.1&file=Extractor.gtk.linux.x86_64.zip" unzip extractor.zip mv Extractor.gtk.linux.x86_64 extractor chmod -R g+w extractor chown -R tomcat6.users extractor
Based on the machine and expected data it might be worth it to increase the memory from the default 1GB to more (we use 9GB).
sed -i -e 's#-Xmx1G#-Xmx9G#g' extractor/ExtractionServer.ini
Next, install extractor as a Unix service and start it.
cp extractor/service/extractor.ubuntu /etc/init.d/extractor sed -i -e "s#DIR=/home/kooper/Extractor#DIR=/home/medici/extractor#" -e "s#USER=kooper#USER=tomcat6#" /etc/init.d/extractor chmod 755 /etc/init.d/extractor update-rc.d extractor defaults /etc/init.d/extractor start
cp extractor/service/extractor.redhat /etc/init.d/extractor sed -i -e "s#DIR=/home/kooper/Extractor#DIR=/home/medici/extractor#" -e "s#USER=kooper#USER=tomcat#" /etc/init.d/extractor chmod 755 /etc/init.d/extractor /sbin/chkconfig extractor on /etc/init.d/extractor start
If you have a firewall and want access from outside the server to the extraction service you will need to allow for traffic on port 9856. To see the status of the extraction service as well as the log files you can connect with a web browser to: http://<hostname>:9856/extractor/status
Web Component
Next, the web component will be installed. Each installation of Medici needs its own installation. In this document we will describe the installation of one installation called medici and second one called onlyme.
Folder creation
The first step is to create a data folder as well as a folder to hold the query optimizations (we use lucene for this). This folder will contain all data associated with a single installation of Medici (in this case called medici).
mkdir -p /home/medici/medici/data mkdir -p /home/medici/medici/lucene chown -R tomcat6 /home/medici/medici
The name of the folder is not important. For simplicity we recommend to name it after the name of the installation. To create multiple installations on the same machine we need to create additional folders to hold the data.
mkdir -p /home/medici/onlyme/data mkdir -p /home/medici/onlyme/lucene chown -R tomcat6 /home/medici/onlyme
Creating the Mysql Database
Following code will create the database and give privileges to the user medici to the database.
wget --no-check-certificate -q -O mysql.sql https://opensource.ncsa.illinois.edu/confluence/download/attachments/589837/mysql.sql
sed -e "s#@DB_SCHEMA@#medici#" \
-e "s#@DB_USER@#medici#" \
-e "s#@DB_PASS@#medici#" mysql.sql > medici.sql
mysql -u root -h localhost -p < medici.sql
To create a second database for the second installation we use the same code except change the database name to the second installation, for example:
wget --no-check-certificate -q -O mysql.sql https://opensource.ncsa.illinois.edu/confluence/download/attachments/589837/mysql.sql
sed -e "s#@DB_SCHEMA@#onlyme#" \
-e "s#@DB_USER@#medici#" \
-e "s#@DB_PASS@#medici#" mysql.sql > onlyme.sql
mysql -u root -h localhost -p < onlyme.sql
Creating Context Definition
Next, we need to create a definition file that binds database and folder together. This needs to use the same values as the MySQL script.
wget --no-check-certificate -q -O context.xml https://opensource.ncsa.illinois.edu/confluence/download/attachments/589837/context.xml
sed -e "s#@DB_SCHEMA@#medici#" \
-e "s#@DB_USER@#medici#" \
-e "s#@DB_PASS@#medici#" \
-e "s#@FOLDER@#/home/medici/medici#" context.xml > medici.xml
To create the defintion file for the second installation we use the values passed into the second MySQL script.
wget --no-check-certificate -q -O context.xml https://opensource.ncsa.illinois.edu/confluence/download/attachments/589837/context.xml
sed -e "s#@DB_SCHEMA@#onlyme#" \
-e "s#@DB_USER@#medici#" \
-e "s#@DB_PASS@#medici#" \
-e "s#@FOLDER@#/home/medici/onlyme#" context.xml > onlyme.xml
Installation of the web component
The final step is to install the Medici web component. This involves downloading the latest version, configuration and installation. Since the application uses Google Maps a new Google Map key needs to be requested.
NOTE: You must generate an appropriate key for "http" or "https". An http key will not work with a server running in https: mode and visa versa.
wget -q -O medici.war "http://isda.ncsa.illinois.edu/download/get.php?project=Medici&category=web&version=v1.1&file=medici.war"
unzip -q -d war medici.war
cp medici.xml war/WEB-INF/classes/context.xml
YOUREMAIL="your email address here"
GOOGLEKEY="your google key here"
sed -i -e "s#mail.from=lmarini@ncsa.illinois.edu#mail.from=$YOUREMAIL#g" \
-e "s/#user.0.email=/user.0.email=$YOUREMAIL/g" \
-e "s@search.index=.*@search.index=/home/medici/medici/lucene@g" war/WEB-INF/classes/server.properties
sed -i -e "s#key=ABQIAAAASEElYb9IDDsAc5ZKA3a2sRQmgYtTImkVBc-VhblDgOLOdwhVaBSGMDSn-_9k3bx4tYolchXvrvB8Ag#key=$GOOGLEKEY#g" war/mmdb.html
chown -R tomcat6 war
mv war /var/lib/tomcat6/webapps/medici
The second installation follows almost the same process but with a few changes.
unzip -q -d war medici.war
cp onlyme.xml war/WEB-INF/classes/context.xml
YOUREMAIL="your email address here"
GOOGLEKEY="your google key here"
sed -i -e "s#mail.from=lmarini@ncsa.illinois.edu#mail.from=$YOUREMAIL#g" \
-e "s/#user.0.email=/user.0.email=$YOUREMAIL/g" \
-e "s@search.index=.*@search.index=/home/medici/medici/lucene@g" war/WEB-INF/classes/server.properties
sed -i -e "s#key=ABQIAAAASEElYb9IDDsAc5ZKA3a2sRQmgYtTImkVBc-VhblDgOLOdwhVaBSGMDSn-_9k3bx4tYolchXvrvB8Ag#key=$GOOGLEKEY#g" war/mmdb.html
chown -R tomcat6 war
mv war /var/lib/tomcat6/webapps/onlyme
Add proxy to Apache Httpd (Optional)
Instead of having people remember the tomcat URL, we added a proxy to the Apache Httpd allowing it to proxy the requests to Tomcat. This can be achieved by adding the redirects. If the system runs using virtual hosts it needs to be added the virtual host section otherwise a new file can be created in the conf.d folder (/etc/apache2/conf.d in case of Ubuntu).
<Proxy *> Order deny,allow Allow from all </Proxy> ProxyRequests Off ProxyPreserveHost On ProxyPass /medici http://localhost:8080/medici ProxyPassReverse /medici http://localhost:8080/medici
If we want to add the second installation to the proxy as well we need to add 2 more lines to the file.
ProxyPass /onlyme http://localhost:8080/onlyme ProxyPassReverse /onlyme http://localhost:8080/onlyme
Finally the proxy http module needs to be enabled and Apache Httpd needs to be restarted.
a2enmod proxy_http /etc/init.d/apache2 restart