This page will describe how to create an extractor using java. A lot of this code can be copied and pasted. The only major part is the processFile function. You can find this example also in our code repository at: https://opensource.ncsa.illinois.edu/stash/projects/MMDB/repos/extractors-examples/browse/java
To get started we define all required jar files needed in a maven pom file
| Code Block | ||||
|---|---|---|---|---|
| ||||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>edu.illinois.ncsa.medici</groupId>
<artifactId>example-extractor</artifactId>
<version>1.0.0-SNAPSHOT</version>
<properties>
<exec.mainClass>edu.illinois.ncsa.medici.extractor.example.WordCount</exec.mainClass>
</properties>
<dependencies>
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>3.0.2</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.4.1.1</version>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.3</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
<scope>runtime</scope>
</dependency>
</dependencies>
</project> |
The first step in the example is to setup some global variables that will be used in the rest of the code.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
private String postMetaData(String host, String key, String fileid, Map<String, Object> metadata) throws IOException {
URL url = new URL(host + "api/files/" + fileid + "/metadata?key=" + key);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/json");
conn.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
mapper.writeValue(wr, metadata);
wr.flush();
wr.close();
int responseCode = conn.getResponseCode();
if (responseCode != 200) {
throw (new IOException("Error uploading metadata [code=" + responseCode + "]"));
}
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
return response.toString();
} |
...