...
Internal Working
The Java Classes
- Software Server:
- SoftwareServer.java: handles wrapper scripts;
- SoftwareServerRestlet.java: handles the Restlet service;
- SoftwareServerRESTUtilities.java: the rabbitMQHandler() method handles all the interaction with RabbitMQ.
- Polyglot:
- The entry point is PolyglotRestlet.java.
java -cp polyglot.jar:lib/* -Xmx1g edu.illinois.ncsa.isda.softwareserver.polyglot.PolyglotRestlet
- Polyglot.java: abstract class;
- PolyglotStewardAMQ.java: handles IOGraph and interaction with RabbitMQ;
- PolyglotRestlet.java: handles the Restlet interface.
process_jobs(): at the end, writes the ".url" file.
- The entry point is PolyglotRestlet.java.
- Software Server:
SS Registration
...
- When Polyglot starts, it does:
- read the configuration file;
- start the PolyglotStewardAMQ thread;
- start a thread to update Mongo; call PolyglotRESTUtilities.updateMongo(). By default updates every 2 sec.
- start the restlet service.
- When PolyglotStewardAMQ thread starts, it starts 3 threads:
- discoveryAMQ(), every 30 s.
- process_jobs(), every 3 s,
- heartbeat(), to remove unresponsive SSs. every Heartbeat secs, default to 10 s.
- Two queryEndpoint() methods.
In SoftwareServerRESTUtilities.java: returns pure text; in PolyglotStewardAMQ.java, returns json. kgm.utility.Utility source code:
https://isda.ncsa.illinois.edu/svn/isda/trunk/kgm/Utilities/src/kgm/utility/Utility.java
Testing
Testing conversion on dap-dev.
- Commands:
Send the conversion request:
curl http://browndog.user:password1@dap-dev.ncsa.illinois.edu:8184/convert/jpg/http%3A%2F%2Fbrowndog.ncsa.illinois.edu%2Fexamples%2Fbrowndog.pngGet the converted file:
curl -u browndog.user:password1 -O http://dap-dev.ncsa.illinois.edu:8184/file/200598567_browndog.jpg
- Used a png to test.
http://browndog.ncsa.illinois.edu/examples/browndog.png
The converion URL requires an escaped URL.
To generate an escaped URL using perl:
perl -e 'use URI::Escape; print uri_escape("http://browndog.ncsa.illinois.edu/examples/browndog.png")'
http://www.perlhowto.com/encode_and_decode_url_strings
man URI::Escape Bi-hourly tests.py.
On dap-dev in /var/www/html/dap/tests/. It used the Polyglot GET API
using the URL to convert, got a returned URL, then downloaded that URL to
"tmp/<count>_<file_basename>.<output_format>", then checked
whether that file existed and was not empty.
Python requests:
r = requests.get(api_call, auth=(username, password),
headers=headers, timeout=timeout)
result = r.text
contains only the text, not the HTML tags.
That is, the actual content returned is:
<a href=http://dap-dev.ncsa.illinois.edu:8184/file/200598567_browndog.jpg>http://dap-dev.ncsa.illinois.edu:8184/file/200598567_browndog.jpg</a>
but result is only:
http://dap-dev.ncsa.illinois.edu:8184/file/200598567_browndog.jpg
python requests API:
http://docs.python-requests.org/en/master/api/
...