Page History

...

1. Docker containers
2. JSONLD
3. Extractor info registration
4. Use pyclowder (for python extractors)
5. Add status messages to all extractors and fix level granularity
   1. Make status constants (DONE, ERROR)
   2. Arcgis multiprocessing extractor
6. Register on on demand execution queues
   1. Add on demand key binding to configuration file: messageType = "*.file.text.plain", "extractors."+extractorName
7. Standardize around python logging
   1. Figure out what to log and what format to follow
8. Add logstash to docker compose
9. Add sample input/ouput to git repository
10. Add icon for tools catalog to git repository
11. Add entry to Tools catalog, with icon

ID (Extractor Name from config file,

same as queue name)	Programming

Language	Software	OS	Can be Dockerized?

Can be upload to Docker Hub ?DEPLOYED        

Assigned To

Link to repoWho wrote or worked on the code

Repo	Author
DEPLOYED

ncsa.image.ocr	Python	Tesseract	Linux

 

Rui	ocr
ncsa.cv.faces	Python	OpenCV	Linux

 

		Rui	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/opencv	Liana
ncsa.cv.eyes	Python	OpenCV	Linux

 

Rui	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/opencv	Liana
ncsa.cv.closeups	Python	OpenCV	Linux

 

		Rui	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/opencv	Liana
ncsa.cv.profiles	Python	OpenCV	Linux

 

		Rui	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/opencv	Liana
ncsa.cellprofiler.fluorescentcomet	Python	pymedici	Windows	No

 

		https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler	Liana
ncsa.cellprofiler.fly	Python		Windows	No

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler	Liana
ncsa.cellprofiler.human	Python		Windows	No

 

		https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler	Liana
ncsa.cellprofiler.silvercomet	Python		Windows	No

 

		https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler	Liana
ncsa.cellprofiler.speckle	Python		Windows	No

 

		https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler	Liana
ncsa.cellprofiler.trackobject	Python		Windows	No

 

		https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler	Liana
ncsa.cellprofiler.tumor	Python		Windows	No

 

		https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler	Liana
ncsa.cellprofiler.yeast	Python		Windows	No

 

		https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/cellprofiler	Liana
ncsa.image.sphog	Python	Matlab, mnist-sphog	Linux

 

		Gregory Jansen	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/handwritten/HandwrittenNumbers
ncsa.image.caltech101

 

Gregory Jansen
ncsa.bisque.histogram (notes: disabled)	Python		Linux

 

ncsa.bisque.metadata (notes: disabled)	Python		Linux

 

census-section-segmentor	Java		Linux

 

		Sandeep Puthanveetil Satheesan	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/census	Liana, Inna
ncsa.cv.river	Python	OpenCV (python), convert (from imagemagick), and Gdal	Linux

 

Smruti Padhy	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/river	Liana
ncsa.geo.shpExtractor	Python	gdal	Linux

 

		Jong Lee	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-geo/browse	Jong Lee
ncsa.geo.tiffExtractor	Python	gdal	Linux

 

		Jong Lee	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-geo/browse	Jong Lee
ncsa.image.geotiff	Python	GDAL, Cython, numpy, pygeoprocessing	Linux

 

		Rui	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-geotiff/browse	Rui, Mostafa Elag
ncsa.image.ponddetect	Python	Matlab	Linux

 

Marcus Slavenas	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-maps/browse/feature_detection	Marcus, Ankit
ncsa.image.humanpref	Python	Matlab	Linux

 

		Marcus Slavenas	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-maps/browse/humanpref	Marcus, Ankit
ncsa.xml.greenindexroute, ncsa.csv.greenindexroute	Python	OpenCV	Linux

 

		Marcus Slavenas	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-maps/browse/greenroute	Marcus
ncsa.image.knn_numerals	Python	OpenCV	Linux

 

		Marcus Slavenas		Marcus
ncsa.audio.speech2text	Java	CMU Sphinx, ffmpeg, sox	Linux

 

		Marcus Slavenas	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-core/browse/audio/speech2text	Marcus
ncsa.audio.preview	Python

 

				Inna	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-core/browse/audio/preview
ncsa.nlp.simplelanguage	Python	numpy

 

			Inna	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-nlp/browse/SimpleLanguage	Liana
ncsa.nlp.simplesummary	Python	Natural Language Toolkit (NLTK) for Python, NLTK Data or at least:  nltk.corpus,nltk.stem.porter and nltk.tokenize.punkt.

 

			Gregory Jansen	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-nlp/browse/SimpleSummary	Liana
ncsa.nlp.SNLPSentiment	Java	Stanford CoreNLP tool, java, maven

 

			Marcus Slavenas	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-nlp/browse/SNLP/SNLPSentimentExtractor	Liana, Marcus(?)
ncsa.nlp.wordtables	Python	requests, pika, win32com

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-nlp/browse/WordTablesExtractor	Liana
siegfried	Python

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-siegfried/browse	Gregory Jansen
ncsa.versus.image	Java	Versus	Linux

 

Smruti Padhy	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-versus/browse	Kenton, Smruti
ncsa.image.preview (note: check if really deployed. there is an extractor in Hosted VMs list with a similar name.)	Python

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-core/browse/image/preview	Rob, Sandeep
ncsa.pdf.preview (note: check if really deployed. there is an extractor in Hosted VMs list with a similar name.)	Python

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-core/browse/pdf/preview	Rob
ncsa.video.preview (note: check if really deployed. there is an extractor in Hosted VMs list with a similar name.)	Python

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-core/browse/video/preview

Rob

   

NOT DEPLOYED

 

ncsa.image.digitpy	Python	opencv

 

				https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/handwritten/SimpleDigitPython
ncsa.cv.pdfimages		pdfimages, from poppler-utils

 

				https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/poppler
ncsa.cv.caltech101	Python	Matlab and VLFeat	64-bit Mac OS

 

			https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cv/browse/vlfeat
dbpedia	Python	Natural Language Toolkit (NLTK) and rdflib.

 

			Luigi Marini	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-dbpedia/browse	Luigi Marini
digest	Python

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-digest/browse
ncsa.hpc	Python

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-hpc/browse	Sandeep Puthanveetil Satheesan
LSVA	Java

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-lsva/browse	Liana, Constantinos
LSVA integrated

 

Sandeep Puthanveetil Satheesan	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-lsva-integrated/browse	Sandeep Puthanveetil Satheesan
ncsa.movieslice	Python

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-movieslice/browse	Sandeep
mri2mesh	Python	pymedici, subprocess, logging, os, numpy, shutil, zipfile

 

				https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-mri/browse/mri2mesh	Marcus
msc-ChemCBCExtractor	Python	requests, pika, openpyxl, xlrd, pymongo	Linux

 

Yan Zhao	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-msc/browse/ChemCBCExtractor	Yan
msc-IsletExtractor	Python	requests, pika, openpyxl, xlrd, pymongo	Linux

 

		Yan Zhao	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-msc/browse/IsletExtractor	Yan
msc-MonitorExtractor	Python	requests, pika, openpyxl, xlrd, pymongo	Linux

 

		Yan Zhao	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-msc/browse/MonitorExtractor	Yan
ncsa.msc.dailymonitor	Python	requests, pika, openpyxl, xlrd, pymongo

  

			not used	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-msc/browse/OldMonitorExtractor	Ashwini
msc-PhenotypeExtractor	Python	requests, pika, openpyxl, xlrd, pymongo	Linux

Yan Zhao	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-msc/browse/PhenotypeExtractor	Yan
ncsa.nlp.SNLP	Java	Stanford CoreNLP tool, java, maven

 

				https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-nlp/browse/SNLP/SNLPExtractor	Liana
ncsa.nlp.tika	Python	Tika project page, pymedici

 

			Kenton McHenry	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-nlp/browse/tika	Liana
person-detector	Python	MATLAB, FFMPEG, requests and pika

 

				https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-person-detector/browse/python	Sandeep
ncsa.person-tracker	Python	python, MATLAB, FFMPEG requests and pika

 

				https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-person-tracking/browse/python	Sandeep
terra.plantcv	Python	pika requests wheel

 

				https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-plantcv/browse	Yan
medici_PTM_thumbnails	Java

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-ptm/browse/PTMThumbnailExtractor	Constantinos
medici_PTM_metadata	Java

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-ptm/browse/PTMMetadataExtractor	Constantinos
Name not clear PtmMetadata(?)	Java

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-ptm/browse/PTMMetadata	Constantinos
medici_ptm_maps	Java

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-ptm/browse/PTMMapsExtractor	Constantinos
medici_ptm_3d	Java

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-ptm/browse/PTM3DExtractor	Constantinos
medici_images_ptm	Java

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-ptm/browse/ImagesPTMExtractor	Constantinos
extractors-rabbitmq (look like examples)

 

						https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-rabbitmq/browse
Name not clear extractors-seabird/	Scala

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-seabird/browse	Luigi
medici_3d_x3d (one of extractors-3d)	Java

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-3d/browse/ObjJSONExtractor	Constantinos
medici_3d_obj_merger (one of extractors-3d)	Java

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-3d/browse/OBJMergerExtractor	Constantinos
medici_oni (one of extractors-3d)	Java

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-3d/browse/OniExtractor	Constantinos
medici_ply_obj (one of extractors-3d)	Java

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-3d/browse/PlyObjExtractor	Constantinos
medici_3d_metadata (one of extractors-3d)	Java

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-3d/browse/ThreeDMetadataExtractor	Constantinos
medici_x3d_html (one of extractors-3d)	Java

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-3d/browse/X3DhtmlExtractor	Constantinos
ncsa.arcgis.landsat7mosaic	Python	ArcGIS	Windows	No

 

	Smruti Padhy	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-bd-cz/browse/ndviextractor	Smruti
ncsa.arcgis.floodplain	Python	ArcGIS	Windows	No

 

	Smruti Padhy	https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-bd-cz/browse/terex_floodplain/config.py	Smruti
medici_book	Java

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-books/browse/BookPreviewExtractor	Theerasit Issaranon
medici_image_pyramid	Java

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-books/browse/ImagePreviewPyramidExtractor-shebook	Theerasit Issaranon
shebook	Java

 

					https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-books/browse/SheBookPreviewExtractor/src/BookPreviewExtractor   https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-books/browse/SheBookPreviewExtractor/src/bookpreviewextractor	Theerasit Issaranon
lsva-cedd	Java

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cedd/browse	Constantinos
ncsa.cinemetrics	Python

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-cinemetrics/browse	Constantinos
ncsa.image.metadata	Python

 

https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/extractors-core/browse/image/metadata	Max. Rob
ncsa.debod.segmentor						https://opensource.ncsa.illinois.edu/bitbucket/projects/DEBOD/repos/extractors-cellsegmentor/browse
ncsa.image.dmp						https://opensource.ncsa.illinois.edu/bitbucket/projects/DEBOD/repos/extractors-debod/browse https://opensource.ncsa.illinois.edu/bitbucket/projects/DEBOD/repos/extractors-dmp/browse
ncsa.image.sphog.debod						https://opensource.ncsa.illinois.edu/bitbucket/projects/DEBOD/repos/extractors-handwrittendecimals/browse