...
- Polyglot refactoring
- Update extractors to latest techs
- JSONLD
- Docker containers
- Extractor metadata registration
- pyclowder
- Add status messages to all extractors and fix level granularity
- Make status constants (DONE, ERROR)
- Arcgis multiprocessing extractor
- Register on on demand queues
- Standardize around python logging
- Polyglot information loss
- Provenance
- Data wolf
- Polyglot: add file.jpg.log and file.jpg.wf to id
- Clowder: each step is one of the extractors executed on the specific file
- Check file format at every step
- Data wolf
- Add new tools
- Look at the ones in Jira labeled as "Extractors" and "Converters"
- Praveen's new extractor
- Support students into doing this
- Move data vs move computation
- Long hanging fruit implementation?
- Host large files local?
- Logstash and Kibana
- Add log stash to the docker file
- Make extractors and software servers logs consistent
- Standardize around python logging
- Don't forget java extractors (versus, audio)
- BDFiddle
Automatic Process Adjustments
Multiple results panes
Extraction Results
Conversions Results
Remove colon on Extractors/Converters
Extract
Convert To
Flip conversion and extractors boxes for real estate
Website Security
Use an anonymous token/key with limits on file size and submissions. (Long Term - Not In Scope)
Login using user/name and password
Sign-In page first
Get key
Fetch token
Key and token displayed on top of page
Indent code snippet buttons to line up with code pane
Links for setup by code snippets
- Manual Process
- Metadata (Extraction)
- Allow selection of multiple metadata tools
- Pick only one tool to start
- Display error from extractor if it fails -> Need clear errors in the extractors
- List each tool specifically -> Get tools from tool catelog
- Conversion
- Populate output (conversion) based on the input type of the file
- User will then select conversion format, which will then populate a list of tools to do the conversion
- Polyglot will give the list of available tools by conversion format
- Metadata (Extraction)