Interacting with DataOne Node
Using REST interface
You can interact with dataone using an REST API. Most of the calls will require some payload in xml. The REST calls can be found at http://mule1.dataone.org/ArchitectureDocs-current/apis/MN_APIs.html. Here are some examples that have been testes against dataone.ncsa.illinois.edu
PING
- if host is alive will return no answer but return code 200 in case of success.
- curl http://dataone.ncsa.illinois.edu/metacat/d1/mn/v1/monitor/ping
OBJECT LIST
Get a list of objects
curl "http://dataone.ncsa.illinois.edu/metacat/d1/mn/v1/object?count=5"
GET DATA
- Get either data or metadata of an object
curl http://dataone.ncsa.illinois.edu/metacat/d1/mn/v1/object/XYZ332
curl http://dataone.ncsa.illinois.edu/metacat/d1/mn/v1/meta/XYZ332
Using R
This assumes you have installed the package dataone from cran (install.packages('dataone')
) Following are commands that can be pasted into R. Right now I am stuck on getting the package.
require(dataone)
# create certificate manger
certmanager <- CertificateManager()
# Following will open web browser and ask you to login
# after you login you will need to download the certificate
# using a java webstart application.
# On the mac, once you click on the jnlp file it will say
# it can not start the application. You will need to go to
# system properties and allow the application in security.
downloadCert(certmanager)
# create a dataone client connection. This will use the
# certificate that was downloaded earlier
client <- D1Client()
# list member nodes
listMemberNodes(client)
# get datasets (as an R list)
datasets <- xmlParse(d1SolrQuery(client,list(q="+species population diversity", fl="*")))
# get the first document
dataset <- xpathApply(x, "//doc")[[1]]
# get data url of first dataset
identifier <- xpathApply(dataset, "//str[@name='identifier']", xmlValue)[[1]]
# get package (this does not work)
package <- getPackage(client, identifier)
# get the data (maybe)
getData(package)
Installing a DataOne Node
MetaCat
apt-get -y install libwww-perl libload-perl libxml-libxml-perl libxml-libxslt-perl libtemplate-perl libcgi-session-perl libdigest-sha-perl
cpan -i Config::Properties
replace Digest::SHA1 with Digest::SHA in register-dataset.cgi
METACAT
http://knb.ecoinformatics.org/knb/docs/install.html#installing-on-linux
- create account metacat in postgresql
- download metacat http://knb.ecoinformatics.org/software/dist/metacat-bin-2.4.0.tar.gz
# install packages
apt-get install postgresql tomcat6 apache2 libapache2-mod-jk
# setup database
vi /etc/postgresql/9.1/main/pg_hba.conf
sudo -u postgres createuser -D -E -l -P -R -S metacat
sudo -u postgres createdb metacat -O metacat
# extract metacat
wget http://knb.ecoinformatics.org/software/dist/metacat-bin-2.4.0.tar.gz
mkdir metacat
cd metacat
tar zxvf ../metacat*.tar.gz
# enable AJP in tomcat
vi /etc/tomcat6/server.xml
# enable some specifics for dataone
echo "org.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true” >> /etc/tomcat6/catalina.properties
echo "org.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true” >> /etc/tomcat6/catalina.properties
# create the admin group
./authFileManager.sh groupadd -g cn=admin,o=isda,dc=ncsa,dc=illinois,dc=edu -d "MetaCat Admin"
# create regular users group
./authFileManager.sh groupadd -g cn=dataone,o=isda,dc=ncsa,dc=illinois,dc=edu -d "BrownDog Users”
# create a user
./authFileManager.sh useradd -i -dn uid=kooper,o=isda,dc=ncsa,dc=illinois,dc=edu -e kooper@illinois.edu -s Kooper -f Rob -g cn=dataone,o=isda,dc=ncsa,dc=illinois,dc=edu
# add the user to the group of admins
./authFileManager.sh usermod -group -a -dn uid=kooper,o=isda,dc=ncsa,dc=illinois,dc=edu -g cn=admin,o=isda,dc=ncsa,dc=illinois,dc=edu
# change the password
./authFileManager.sh usermod -password -dn uid=kooper,o=isda,dc=ncsa,dc=illinois,dc=edu -i
# setup AJP
cp debian/jk.conf /etc/apache2/mods-available
cp debian/workers.properties /etc/apache2
a2dismod jk
a2enmod jk
# setup site
Alias /metacat/ "/var/lib/tomcat6/webapps/metacat"
<Directory "/var/lib/tomcat6/webapps/metacat">
Options Indexes MultiViews FollowSymLinks
Order allow,deny
Allow from all
</Directory>
ScriptAlias /metacat/cgi-bin/ /var/lib/tomcat6/webapps/metacat/cgi-bin/
<Directory "/var/lib/tomcat6/webapps/metacat/cgi-bin/">
AllowOverride All
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>
<IfModule mod_jk.c>
JkMount /metacat ajp13
JkMount /metacat/* ajp13
JkMount /metacat/metacat ajp13
JkMount /*.jsp ajp13
JkUnMount /metacat/cgi-bin/* ajp13
</IfModule>
cp debian/metacat-site /etc/apache2/sites-available
a2ensite metacat-site
# install metacat
mkdir /var/metacat
chown tomcat6.tomcat6 /var/metacat
sudo cp metacat.war /var/lib/tomcat6/webapps
DataOne GMN
http://pythonhosted.org/dataone.generic_member_node/setup-d1-gmn.html
After installing pip run following:
pip install setuptools --no-use-wheel —upgrade
Before running pip install data one run
pip install -v pyxb==1.2.1; pip install dataone.generic_member_node
make sure to change CATOP as well in CA.pl
Ubuntu 12.04 uses python 2.7 so
cd /var/local/dataone/gmn/lib/python2.6/site-packages/deployment
Ubuntu 12.04 uses postgresql 9.1
vi /etc/postgresql/9.1/main/pg_hba.conf
settings_site.py
STAND_ALONE = True
NODE_IDENTIFIER = ‘urn:node:TestBrownDogNCSA’
NODE_NAME = 'BrownDog NCSA’
NODE_DESCRIPTION = 'Test Node BrownDog NCSA’
NODE_BASEURL = 'https://dataone.ncsa.illinois.edu/mn'
NODE_SUBJECT = 'CN=urn:node:TestBrownDogNCSA,DC=dataone,DC=org'
NODE_CONTACT_SUBJECT = 'CN=BrownDog,O=NCSA,C=US,DC=cilogon,DC=org’
TIER = 3
NODE_REPLICATE = False
DATAONE_ROOT = 'https://cn.dataone.org'
replace 2.6 with 2.7 in /etc/apache2/sites-enabled/gmn-ssl
Install dataone client
http://pythonhosted.org/DataONE_ClientLib/setup-d1-client.html#