This page will describe how to create an extractor using java. A lot of this code can be copied and pasted. The only major part is the processFile function. You can find this example also in our code repository at: https://opensource.ncsa.illinois.edu/stash/projects/MMDB/repos/extractors-examples/browse/java
To get started we define all required jar files needed in a maven pom file
Code Block | ||||
---|---|---|---|---|
| ||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>edu.illinois.ncsa.medici</groupId>
<artifactId>example-extractor</artifactId>
<version>1.0.0-SNAPSHOT</version>
<properties>
<exec.mainClass>edu.illinois.ncsa.medici.extractor.example.WordCount</exec.mainClass>
</properties>
<dependencies>
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>3.0.2</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.4.1.1</version>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.3</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
<scope>runtime</scope>
</dependency>
</dependencies>
</project> |
The first step in the example is to setup some global variables that will be used in the rest of the code.
...
Code Block | ||||
---|---|---|---|---|
| ||||
private String postMetaData(String host, String key, String fileid, Map<String, Object> metadata) throws IOException { URL url = new URL(host + "api/files/" + fileid + "/metadata?key=" + key); HttpURLConnection conn = (HttpURLConnection) url.openConnection(); conn.setRequestMethod("POST"); conn.setRequestProperty("Content-Type", "application/json"); conn.setDoOutput(true); DataOutputStream wr = new DataOutputStream(conn.getOutputStream()); mapper.writeValue(wr, metadata); wr.flush(); wr.close(); int responseCode = conn.getResponseCode(); if (responseCode != 200) { throw (new IOException("Error uploading metadata [code=" + responseCode + "]")); } BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream())); String inputLine; StringBuffer response = new StringBuffer(); while ((inputLine = in.readLine()) != null) { response.append(inputLine); } in.close(); return response.toString(); } |
...