...
- Docker containers
- JSONLD
- Extractor info registration
- Use pyclowder (for python extractors)
- Add status messages to all extractors and fix level granularity
- Make status constants (DONE, ERROR)
- Arcgis multiprocessing extractor
- Register on on demand execution queues
- Add on demand key binding to configuration file: messageType = "*.file.text.plain", "extractors."+extractorName
- Standardize around python logging
- Figure out what to log and what format to follow
Add logstash to docker compose- Add sample input/ouput to git repository
- Add icon for tools catalog to git repository
- Add entry to Tools catalog, with icon
ID (Extractor Name from config file, |
---|
same as queue name) | Programming |
---|
Language | Software | OS | Can be Dockerized? |
---|
Assigned To |
---|
Repo | Author | ||||||
---|---|---|---|---|---|---|---|
DEPLOYED |
ncsa.image.ocr | Python | Tesseract | Linux |
---|
Rui | ocr | ||
ncsa.cv.faces | Python | OpenCV | Linux |
---|
Rui | https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/opencv | Liana | ||
ncsa.cv.eyes | Python | OpenCV | Linux |
---|
Rui | https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/opencv | Liana | |
ncsa.cv.closeups | Python | OpenCV | Linux |
---|
Rui | https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/opencv | Liana | ||
ncsa.cv.profiles | Python | OpenCV | Linux |
---|
Rui | https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/opencv | Liana | ||
ncsa.cellprofiler.fluorescentcomet | Python | pymedici | Windows | No |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler | Liana | ||||
ncsa.cellprofiler.fly | Python | Windows | No |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler | Liana | |||
ncsa.cellprofiler.human | Python | Windows | No |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler | Liana | |||
ncsa.cellprofiler.silvercomet | Python | Windows | No |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler | Liana | |||
ncsa.cellprofiler.speckle | Python | Windows | No |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler | Liana | |||
ncsa.cellprofiler.trackobject | Python | Windows | No |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler | Liana | |||
ncsa.cellprofiler.tumor | Python | Windows | No |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler | Liana | |||
ncsa.cellprofiler.yeast | Python | Windows | No |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler | Liana | ||
ncsa.image.sphog | Python | Matlab, mnist-sphog | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/handwritten/HandwrittenNumbers | ||||
|
---|
ncsa.bisque.histogram (notes: disabled) | Python | Linux |
---|
ncsa.bisque.metadata (notes: disabled) | Python | Linux |
---|
census-section-segmentor | Java | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/census | Liana, Inna | |||
ncsa.cv.river | Python | OpenCV (python), convert (from imagemagick), and Gdal | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/river | Liana | ||
ncsa.geo.shpExtractor | Python | gdal | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-geo/browse | Jong Lee | |||
ncsa.geo.tiffExtractor | Python | gdal | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-geo/browse | Jong Lee | |||
ncsa.image.geotiff | Python | GDAL, Cython, numpy, | Linux |
---|
Rui | https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-geotiff/browse | Rui, Mostafa Elag | ||
ncsa.image.ponddetect | Python | Matlab | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-maps/browse/feature_detection | Marcus, Ankit | ||
ncsa.image.humanpref | Python | Matlab | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-maps/browse/humanpref | Marcus, Ankit | |||
ncsa.xml.greenindexroute, ncsa.csv.greenindexroute | Python | OpenCV | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-maps/browse/greenroute | Marcus | |||
ncsa.image.knn_numerals | Python | OpenCV | Linux |
---|
Marcus | ||||
ncsa.audio.speech2text | Java | CMU Sphinx, ffmpeg, sox | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-core/browse/audio/speech2text | Marcus | |||
ncsa.audio.preview | Python |
---|
Inna | https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-core/browse/audio/preview | |||||
ncsa.nlp.simplelanguage | Python | numpy |
---|
Inna | https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-nlp/browse/SimpleLanguage | Liana | |||
ncsa.nlp.simplesummary | Python | Natural Language Toolkit (NLTK) for Python, NLTK Data or at least: nltk.corpus,nltk.stem.porter and nltk.tokenize.punkt. |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-nlp/browse/SimpleSummary | Liana | ||||
ncsa.nlp.SNLPSentiment | Java | Stanford CoreNLP tool, java, maven |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-nlp/browse/SNLP/SNLPSentimentExtractor | Liana, Marcus(?) | ||||
ncsa.nlp.wordtables | Python | requests, pika, win32com |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-nlp/browse/WordTablesExtractor | Liana | ||||
siegfried | Python |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-siegfried/browse | Gregory Jansen | |||
ncsa.versus.image | Java | Versus | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-versus/browse | Kenton, Smruti | |
ncsa.image.preview (note: check if really deployed. there is an extractor in Hosted VMs list with a similar name.) | Python |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-core/browse/image/preview | Rob, Sandeep | |||||
ncsa.pdf.preview (note: check if really deployed. there is an extractor in Hosted VMs list with a similar name.) | Python |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-core/browse/pdf/preview | Rob |
ncsa.video.preview (note: check if really deployed. there is an extractor in Hosted VMs list with a similar name.) | Python |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-core/browse/video/preview | Rob |
NOT DEPLOYED |
---|
ncsa.image.digitpy | Python | opencv |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/handwritten/SimpleDigitPython | |||||
ncsa.cv.pdfimages | pdfimages, from poppler-utils |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/poppler | |||||
ncsa.cv.caltech101 | Python | Matlab and VLFeat | 64-bit Mac OS |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/vlfeat | ||||
dbpedia | Python | Natural Language Toolkit (NLTK) and rdflib. |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-dbpedia/browse | |||||
digest | Python |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-digest/browse | ||||||
ncsa.hpc | Python |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-hpc/browse | ||||||
LSVA | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-lsva/browse | Liana, Constantinos | |||
LSVA integrated |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-lsva-integrated/browse | |||||
ncsa.movieslice | Python |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-movieslice/browse | Sandeep | |
mri2mesh | Python | pymedici, subprocess, logging, os, numpy, shutil, zipfile |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-mri/browse/mri2mesh | Marcus | ||||
msc-ChemCBCExtractor | Python | requests, pika, openpyxl, xlrd, pymongo | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-msc/browse/ChemCBCExtractor | Yan | ||
msc-IsletExtractor | Python | requests, pika, openpyxl, xlrd, pymongo | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-msc/browse/IsletExtractor | Yan | |||
msc-MonitorExtractor | Python | requests, pika, openpyxl, xlrd, pymongo | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-msc/browse/MonitorExtractor | Yan | |||
ncsa.msc.dailymonitor | Python | requests, pika, openpyxl, xlrd, pymongo |
---|
not used | https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-msc/browse/OldMonitorExtractor | Ashwini | |||
msc-PhenotypeExtractor | Python | requests, pika, openpyxl, xlrd, pymongo | Linux |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-msc/browse/PhenotypeExtractor | Yan | |
ncsa.nlp.SNLP | Java | Stanford CoreNLP tool, java, maven |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-nlp/browse/SNLP/SNLPExtractor | Liana | ||||
ncsa.nlp.tika | Python | Tika project page, pymedici |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-nlp/browse/tika | Liana | ||||
person-detector | Python | MATLAB, FFMPEG, requests and pika |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-person-detector/browse/python | Sandeep | ||||
ncsa.person-tracker | Python | python, MATLAB, FFMPEG requests and pika |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-person-tracking/browse/python | Sandeep | ||||
terra.plantcv | Python | pika |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-plantcv/browse | Yan | ||||
medici_PTM_thumbnails | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-ptm/browse/PTMThumbnailExtractor | Constantinos | |||||
medici_PTM_metadata | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-ptm/browse/PTMMetadataExtractor | Constantinos | |||||
Name not clear PtmMetadata(?) | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-ptm/browse/PTMMetadata | Constantinos | |||||
medici_ptm_maps | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-ptm/browse/PTMMapsExtractor | Constantinos | ||||
medici_ptm_3d | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-ptm/browse/PTM3DExtractor | Constantinos | ||||
medici_images_ptm | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-ptm/browse/ImagesPTMExtractor | Constantinos |
extractors-rabbitmq (look like examples) |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-rabbitmq/browse | |||||||
Name not clear extractors-seabird/ | Scala |
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-seabird/browse | Luigi |
medici_3d_x3d (one of extractors-3d) | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-3d/browse/ObjJSONExtractor | Constantinos | |||||
medici_3d_obj_merger (one of extractors-3d) | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-3d/browse/OBJMergerExtractor | Constantinos | |||||
medici_oni (one of extractors-3d) | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-3d/browse/OniExtractor | Constantinos | |||||
medici_ply_obj (one of extractors-3d) | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-3d/browse/PlyObjExtractor | Constantinos |
medici_3d_metadata (one of extractors-3d) | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-3d/browse/ThreeDMetadataExtractor | Constantinos | |||||
medici_x3d_html (one of extractors-3d) | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-3d/browse/X3DhtmlExtractor | Constantinos | |||||
ncsa.arcgis.landsat7mosaic | Python | ArcGIS | Windows | No |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-bd-cz/browse/ndviextractor | Smruti | |||
ncsa.arcgis.floodplain | Python | ArcGIS | Windows | No |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-bd-cz/browse/terex_floodplain/config.py | Smruti | ||
medici_book | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-books/browse/BookPreviewExtractor | Theerasit Issaranon | |||||
medici_image_pyramid | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-books/browse/ImagePreviewPyramidExtractor-shebook | Theerasit Issaranon | |||||
shebook | Java |
---|
| Theerasit Issaranon | |||||
lsva-cedd | Java |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cedd/browse | Constantinos | ||||
ncsa.cinemetrics | Python |
---|
https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cinemetrics/browse | Constantinos | ||||
ncsa.image.metadata | Python |
---|