This is a description of the extractors and converters that we plan to implement next at the UMD iSchool. These reflect priorities around curating mostly born digital archival collections.
| tool | action | input
| output format / fields | use case | Effort | notes |
|---|---|---|---|---|---|---|
| XML indexing | Extract |
|
| Meaningful search over XML files in the archives will hinge on the schema employed. By extracting at the schema we can index it. This is analogous to file characterization within the XML world. | ?? | |
| Locations | Extract |
|
| Allows geospatial search and discovery of relevant archives. | Medium | |
| Web links | Extract |
|
| Allows us to create an index of all of the web pages in a web archive of a site for a federal agency, etc.. Text of links can be used to describe the page referenced, becoming additional keywords. | Low | Let's us try pagerank scoring in archives. |
| Creation date | Extract |
|
| Often files that have been moved through archival deposit workflows or have been moved from computer to computer prior to deposit will no longer have good metadata on the creation date of a document. The algorithm would produce a best guess at a document creation date, based on the various dates used in the text. | Low | Concerned this won't work very often... |
| Provenance | ... As a result duplicates and/or revised versions of documents are present within an archive. A Versus signature based on the File2Learn code will allow us to compare documents to see how similar they are based on text, images, and vector graphics. Can be used to potetially reconstruct the order by which the documents were edited. | Medium | Would need to get File2learn code from Rob Kooper. Was in our SVN repo I believe... | |||
| Digitized documents | Extract |
|
| A federal agency will have legacy paper records and often these are scanned into digital form, but rarely become useful as structured data records. Layout recognition metrics, such as the offsets of horizontal and vertical lines, can be used to classify images as depicting a particular type of paper form. These might be tax forms, census forms, or any kind of routine paper record from the pre/post-digital era. Recognition of the layout lets you apply a template that identified document regions for OCR processing. | Medium/High | Would probably depending on the layout of the document, i.e. would need a tempalte for each type? Gregory Jansen is there a type of scanned document we might start with? Sandeep Puthanveetil Satheesan I believe we could throw in the Census forms here (as they are in CIBER). Any others though? Sandeep: Yes, I can only think of those as well. |
| Converter |
|
| The archive is filled with files that are not easily examined. To help with this we will create converters from these difficult formats to formats that are more readily openable either on ones machine or better yet over the web. | Low | Probably have to use the original or modern project management software, running in a VM, or else parse an XML file (within a ZIP) | |
| Access | Converter |
| The archive is filled with files that are not easily examined. To help with this we will create converters from these difficult formats to formats that are more readily openable either on ones machine or better yet over the web. | Low | Gregory Jansen can you put a few here? | |
| Access | Converter |
| The archive is filled with files that are not easily examined. To help with this we will create converters from these difficult formats to formats that are more readily openable either on ones machine or better yet over the web. | Low | Gregory Jansen can you put a few here? |