This guide assumes you have a Linux machine. We use Ubuntu as our Linux installation so some files, or commands to install software, might be at a different location if you use another flavor of Linux. We will try and mark them as Ubuntu specific. If you use another flavor of Linux please send us any updates so we can add them to this guide.
Prerequisites
The system will store all files in the filesystem and all meta-data in a MySQL database. The extraction service is dependent on FFMPEG.
Installing Apache and Tomcat
The web application is written as a servlet using GWT. The following command will install Apache HTTPD as well as Tomcat.
apt-get -qq -y install mysql-server-5.1 apache2
Updating Tomcat Script
We need to make two modifications to the tomcat script under Linux. The first one will allow Medici to read from the local file system, the second change will allow the extraction service to run as tomcat (otherwise tomcat gets confused and thinks it is already running). We also bump the memory of tomcat this is not required but might improve performance.
sed -i -e 's#TOMCAT6_SECURITY=yes#TOMCAT6_SECURITY=no#g' \ -e "s#-x /bin/bash --#--pidfile /tmp/tomcatsh.pid -x /bin/bash --#g" \ -e 's#-Xmx128M#-Xmx256M#' /etc/init.d/tomcat6
Installing additional JAR files
Two jar files will need to be installed in the Tomcat lib folder (in case of Ubuntu this is /usr/share/tomcat6/lib).
Download the MySQL Connector/J JDBC driver v5.1 and add the MySQL JDBC driver jar (mysql-connector-java-5.x.x-bin.jar) to the tomcat/lib folder. Internally we use version 5.0.4.
The second jar file is the Xerces implementation, after downloading add the xercesImpl.jar to the tomcat/lib folder. Internally we use version 2.6.2.
Installing MySQL
MySQL is used to store the metadata, the database will be created in the next section.
apt-get -qq -y install mysql-server-5.1 unzip apache2
Installing FFmpeg
FFmpeg is used to convert video to FLV video to be played in the browser as well to extract a single frame from the video.
wget http://www.medibuntu.org/sources.list.d/`lsb_release -cs`.list --output-document=/etc/apt/sources.list.d/medibuntu.list apt-get -qq update apt-get --yes -qq --allow-unauthenticated install medibuntu-keyring apt-get -qq update apt-get -qq -y ffmpeg libavcodec-extra-52
Installing UnZip
UnZip is needed to unzip the war file, modify the file telling Medici where to find the database as well as the datafolder. The following command will install Unzip.
apt-get -qq -y install unzip
Additional Fonts
To enable the conversion from PDF to preview images some fonts need to be installed.
apt-get -qq -y install ttf-dejavu-core ttf-baekmuk ttf-kochi-gothic ttf-kochi-mincho ttf-wqy-zenhei ttf-indic-fonts-core ttf-telugu-fonts ttf-oriya-fonts ttf-kannada-fonts ttf-bengali-fonts
Installation of Medici
One the prerequisites are installed we can install Medici. The system consists of two major components, the web component as well as the extraction service. The extraction service can be shared by multiple installation. Both the web component as well as the extraction service will need to write to the same folder on disk. To enable this we will run the extraction service as the same user as tomcat (in case of Ubuntu this will be tomcat6
).
Extraction Service
First step is to create a folder where the extraction service will be installed.
mkdir -p /home/medici
Next download the latest version of the extraction service, the following will download the 64 bit version of the 1.0 version of the extraction service. Install the code and change the permission to tomcat.
wget -q -O extractor.tar.gz "http://isda.ncsa.illinois.edu/download/get.php?project=MMDB&category=extractor&version=v1.0&file=Extractor-linux.gtk.x86_64.tar.gz" tar zxf extractor.tar.gz chmod -R g+w Extractor chown -R tomcat6.users Extractor
Based on the machine and expected data it might be worth it to increase the memory from the default 1GB to more (we use 9GB).
ed -i -e 's#-Xmx1G#-Xmx8=9G#g' Extractor/ExtractionServer.ini
Next install extractor as a Unix service and start it.
cp Extractor/service/extractor.ubuntu extractor sed -i -e "s#DIR=/home/kooper/Extractor#DIR=/home/medici/Extractor#" \ -e "s#USER=kooper#USER=tomcat6#" extractor cp extractor /etc/init.d chmod 755 /etc/init.d/extractor update-rc.d extractor defaults /etc/init.d/extractor start
cp Extractor/service/extractor.redhat extractor sed -i -e "s#DIR=/home/kooper/Extractor#DIR=/home/medici/Extractor#" \ -e "s#USER=kooper#USER=tomcat#" extractor cp extractor /etc/init.d chmod 755 /etc/init.d/extractor /sbin/chkconfig extractor on /etc/init.d/extractor start
If you have a firewall and want access from outside the server to the extraction service you will need to allow for traffic on port 9856. To see the status of the extraction service as well as the log files you can connect with a web browser to: http://<hostname>:9856/extractor/status
Web Component
Next the web component will be installed. Each installation of Medici needs its own installation. In this document we will describe the installation of one installation called medici
and second one called onlyme
.
Folder creation
First step is to create a data folder as well as a folder to hold the query optimizations (we use lucene for this). This folder will contain all data associated with a single installation of Medici (in this case called medici).
mkdir -p /home/medici/medici/data mkdir -p /home/medici/medici/lucene
The name of the folder is not important. For simplicity we recommend to name it after the name of the installation. To create multiple installations on the same machine we need to create additional folders to hold the data.
mkdir -p /home/medici/onlyme/data mkdir -p /home/medici/onlyme/lucene
Creating the Mysql Database
Following code will create the database and give privileges to the user medici to the database.
wget --no-check-certificate -q -O mysql.sql https://opensource.ncsa.illinois.edu/confluence/download/attachments/589837/mysql.sql sed -e "s#<at:var at:name="DB_SCHEMA" />#medici#" \ -e "s#<at:var at:name="DB_USER" />#medici#" \ -e "s#<at:var at:name="DB_PASS" />#medici#" mysql.sql > medici.sql mysql -u root -h localhost -p < medici.sql
To create a second database for the second installation we use the same code except change the database name to the second installation, for example:
wget --no-check-certificate -q -O mysql.sql https://opensource.ncsa.illinois.edu/confluence/download/attachments/589837/mysql.sql sed -e "s#<at:var at:name="DB_SCHEMA" />#onlyme#" \ -e "s#<at:var at:name="DB_USER" />#medici#" \ -e "s#<at:var at:name="DB_PASS" />#medici#" mysql.sql > onlyme.sql mysql -u root -h localhost -p < onlyme.sql
Creating Context Definition
Next we need to create a definition file that binds database and folder together. This needs to use the same values as the MySQL script.
wget --no-check-certificate -q -O context.xml https://opensource.ncsa.illinois.edu/confluence/download/attachments/589837/context.xml sed -e "s#<at:var at:name="DB_SCHEMA" />#medici#" \ -e "s#<at:var at:name="DB_USER" />#medici#" \ -e "s#<at:var at:name="DB_PASS" />#medici#" \ -e "s#<at:var at:name="FOLDER" />#/home/medici/medici#" context.xml > medici.xml
To create the defintion file for the second installation we use the values passed into the second MySQL script.
wget --no-check-certificate -q -O context.xml https://opensource.ncsa.illinois.edu/confluence/download/attachments/589837/context.xml sed -e "s#<at:var at:name="DB_SCHEMA" />#onlyme#" \ -e "s#<at:var at:name="DB_USER" />#medici#" \ -e "s#<at:var at:name="DB_PASS" />#medici#" \ -e "s#<at:var at:name="FOLDER" />#/home/medici/onlyme#" context.xml > onlyme.xml
Installation of the web component
The final step is to install the Medici web component. This involves downloading the latest version, configuration and installation. Since the application uses Google Maps a new Google Map key needs to be requested.
wget -q -O medici.war "http://isda.ncsa.illinois.edu/download/get.php?project=MMDB&category=web&version=v1.0&file=medici.war" unzip -q -d medici medici.war cp medici.xml medici/WEB-INF/classes/context.xml YOUREMAIL="your email address here" GOOGLEKEY="your google key here" sed -i -e "s#mail.from=lmarini@ncsa.illinois.edu#mail.from=$YOUREMAIL#g" \ -e "s/#user.0.email=/user.0.email=$YOUREMAIL/g" \ -e "s@search.index=.*@search.index=/home/medici/medici/lucene@g" medici/WEB-INF/classes/server.properties sed -i -e "s#key=ABQIAAAASEElYb9IDDsAc5ZKA3a2sRQmgYtTImkVBc-VhblDgOLOdwhVaBSGMDSn-_9k3bx4tYolchXvrvB8Ag#key=$GOOGLEKEY#g" medici/mmdb.html mv medici var/lib/tomcat6/webapps/
The second installation follows almost the same process but with a few changes.
wget -q -O medici.war "http://isda.ncsa.illinois.edu/download/get.php?project=MMDB&category=web&version=v1.0&file=medici.war" unzip -q -d onlyme medici.war cp onlyme.xml onlyme/WEB-INF/classes/context.xml YOUREMAIL="your email address here" GOOGLEKEY="your google key here" sed -i -e "s#mail.from=lmarini@ncsa.illinois.edu#mail.from=$YOUREMAIL#g" \ -e "s/#user.0.email=/user.0.email=$YOUREMAIL/g" \ -e "s@search.index=.*@search.index=/home/medici/medici/lucene@g" onlyme/WEB-INF/classes/server.properties sed -i -e "s#key=ABQIAAAASEElYb9IDDsAc5ZKA3a2sRQmgYtTImkVBc-VhblDgOLOdwhVaBSGMDSn-_9k3bx4tYolchXvrvB8Ag#key=$GOOGLEKEY#g" onlyme/mmdb.html mv onlyme var/lib/tomcat6/webapps/
Add proxy to Apache Httpd
Instead of having people remember the tomcat URL, we added a proxy to the Apache Httpd allowing it to proxy the requests to Tomcat. This can be achieved by adding the redirects. If the system runs using virtual hosts it needs to be added the virtual host section otherwise a new file can be created in the conf.d folder (/etc/apache2/conf.d in case of Ubuntu).
<Proxy *> Order deny,allow Allow from all </Proxy> ProxyRequests Off ProxyPreserveHost On ProxyPass /medici http://localhost:8080/medici ProxyPassReverse /medici http://localhost:8080/medici
If we want to add the second installation to the proxy as well we need to add 2 more lines to the file.
ProxyPass /onlyme http://localhost:8080/onlyme ProxyPassReverse /onlyme http://localhost:8080/onlyme
Finally the proxy http module needs to be enabled and Apache Httpd needs to be restarted.
a2enmod proxy_http /etc/init.d/apache2 restart
Additional information
wget --no-check-certificate https://opensource.ncsa.illinois.edu/svn/mmdb/trunk/edu.illinois.ncsa.mmdb.web/war/WEB-INF/lib/mysql-connector-java-5.0.4.jar
wget --no-check-certificate https://opensource.ncsa.illinois.edu/svn/mmdb/trunk/edu.illinois.ncsa.mmdb.web/war/WEB-INF/lib/xercesImpl-2.6.2.jar
into /usr/share/tomcat6/lib
also add --pidfile to /etc/init.d/tomcat6