Versions Compared

Old Version 23

changes.mady.by.user Rui Liu

Saved on

compared with

New Version Current

changes.mady.by.user Rui Liu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Internal Working

  • The Java Classes
    1. Software Server:
      1. SoftwareServer.java: handles wrapper scripts;
      2. SoftwareServerRestlet.java: handles the Restlet service;
      3. SoftwareServerRESTUtilities.java: the rabbitMQHandler() method handles all the interaction with RabbitMQ.
    2. Polyglot:
      1. The entry point is PolyglotRestlet.java.
        java -cp polyglot.jar:lib/* -Xmx1g edu.illinois.ncsa.isda.softwareserver.polyglot.PolyglotRestlet
      2. Polyglot.java: abstract class;
      3. PolyglotStewardAMQ.java: handles IOGraph and interaction with RabbitMQ;
      4. PolyglotRestlet.java: handles the Restlet interface.
        process_jobs(): at the end, writes the ".url" file.
  • SS Registration

...

Code Block
languagebash
titleSS conf and .aliases.txt content
collapsetrue
SoftwareServer.conf:
#BatchScripts=scripts/bat
#ShellScripts=/home/polyglot/scripts/sh
#RScripts=scripts/R
#AHKScripts=scripts/csr-configured
#AppleScripts=scripts/applescript
#SikuliScripts=scripts/sikuli
PythonScripts=scripts/py

$ grep -v ^# */.aliases.txt
ahk/.aliases.txt:A3DReviewer
ahk/.aliases.txt:ImgMgk
ahk/.aliases.txt:IrfanView
py/.aliases.txt:demclip_convert.py
py/.aliases.txt:streamclip_convert.py
R/.aliases.txt:PEcAn
sh/.aliases.txt:avconv-audio
sh/.aliases.txt:avconv-video

...

  • Polyglot REST APIs
  • POL REST endpoints that POL handles on its own without accessing/redirecting to SSes:
    GET:
    /           Returns a list of supported endpoints.
    /alive      Returns "yes".
    /checkin
    /convert    Returns all supported output formats
    /convert/output_format1   returns all supported input formats that can be converted to output_format1
    /convert/output_format1/file_url1   do the conversion: download file_url1
    /form
    /image
    /inputs
    /inputs/<format1>
    /outputs
    /requests
    /servers
    /software
  • POL REST endpoints that access or redirect to SSes:
    GET:
    /file/<file1>      # If file1 doesn't exist and file1.url exists.
    /servers/<server1_ip>[/...]   # Redirects to server1_ip:8182/software/...
    /software/<sw1>    # Accesses all SS:8182/software until finding one that contains sw1 and redirects to <ss_ip1>:8182/software/<sw1>.
 
    POST:
    /servers/<ip1>[/...]   # Redirects to ip1:8182/software/<...>.
    /software/<sw1>        # Accesses all SS:8182/software until finding one that contains sw1 and redirects to <ss_ip1>:8182/software/<sw1>.
  • Implementation Details
  • Configuration files
...
  • When Polyglot starts, it does:
    1. read the configuration file;
    2. start the PolyglotStewardAMQ thread;
    3. start a thread to update Mongo; call PolyglotRESTUtilities.updateMongo(). By default updates every 2 sec.
    4. start the restlet service.
  • When PolyglotStewardAMQ thread starts, it starts 3 threads:
    1. discoveryAMQ(), every 30 s.
    2. process_jobs(), every 3 s,
    3. heartbeat(), to remove unresponsive SSs. every Heartbeat secs, default to 10 s.
  • Two queryEndpoint() methods.
    In SoftwareServerRESTUtilities.java: returns pure text; in PolyglotStewardAMQ.java, returns json.
  • kgm.utility.Utility source code
    https://isda.ncsa.illinois.edu/svn/isda/trunk/kgm/Utilities/src/kgm/utility/Utility.java
...