...
- Create a Python virtual environment in your computer using virtualenv, virtualenvwrapper or any other tool.
- Activate the created Python virtual environment
- Run the following command:
pip install bdcli This will install bdcli and all the dependent packages and you should see a new application bd from your terminal.
Working with Terminal
This section is intended only to those users who are working in a terminal for the very first time or are not very experienced using it. Others, please free to skip this section.
TODO: Write content for this section
Example Transformations
Below we are listing out some example transformations that you can perform using BD-CLI. For a complete list, please refer to the help section below.
Extraction
Extractions
Extraction In the context of Brown Dog, extraction refers to the process of extracting relevant deriving additional useful information from the contents of one or more data filesfiles/datasets that helps a user or application in determining what's in the file/dataset or collection of files/datasets. For example, finding out the location of people's faces using machine learning to find an object in an image.You can find an example of the extraction command below For example:
Here we are using the development instance of Brown Dog by passing in the API URL using -b option. This is an optional flag and by default BD-CLI uses the production API instance at at https://bd-api.ncsa.illinois.eduYou can then read the contents of . The results of the extraction are by default placed ina JSON file, in this case ncsa.json:
| Code Block | ||||||||
|---|---|---|---|---|---|---|---|---|
| ||||||||
examples $ cat ncsa.json | jq .
{
"metadata.jsonld": [
{
"content": {
"Human Preference Extractor": {
"Definitions": {
"Human Preference": "A Computer Vision model that uses the spectral data of an image to get a human preference value ranging from 1 to 5.",
"Green Index": "The green index is the estimated percentage of green pixels within an image."
},
"Data": {
"Green Index": "2.3"
}
}
},
"created_at": "Tue Jun 20 10:34:06 CDT 2017",
"agent": {
"extractor_id": "http://clowder.ncsa.illinois.edu/extractors/deprecatedapi",
"@type": "cat:extractor",
"name": "http://clowder.ncsa.illinois.edu/extractors/deprecatedapi"
}
},
{
"content": {
"y1": 288,
"y0": 88,
"x0": 920,
"x1": 1220
},
"@context": [
"https://clowder.ncsa.illinois.edu/contexts/metadata.jsonld",
{
"y1": "http://clowder.ncsa.illinois.edu/gi_detector#y1",
"y0": "http://clowder.ncsa.illinois.edu/gi_detector#y0",
"x0": "http://clowder.ncsa.illinois.edu/gi_detector#x0",
"x1": "http://clowder.ncsa.illinois.edu/gi_detector#x1"
}
],
"created_at": "Tue Jun 20 10:33:54 CDT 2017",
"agent": {
"extractor_id": "https://clowder.ncsa.illinois.edu/clowder/api/extractors/gi_detector",
"@type": "cat:extractor",
"name": "https://clowder.ncsa.illinois.edu/clowder/api/extractors/gi_detector"
}
},
{
"content": {
"ocr_text": "WEB BROWSER MOSAIC THE FIRST POPULAR GRAPHICAL BROWSER FOR THE WORLD WIDE WEB WAS CREATED BY MARC ANDREESSEN AND ERIC BINA AT THE NATIONAL CENTER FOR COMPUTING APPLICATIONS NCSA UPON ITS 1993 RELEASE To THE PUBLIC MOSAIC GAVE INTERNET USERS EASY ACCESS TO MULTIMEDIA SOURCES OF INFORMATION WEB BROWSERS HAVE TRANSFORMED THE EXCHANGE OP INFORMATION UNIVERSITY OF ILLINOIS"
},
"@context": [
"https://clowder.ncsa.illinois.edu/contexts/metadata.jsonld",
{
"ocr_text": "http://clowder.ncsa.illinois.edu/ncsa.image.ocr#ocr_text"
}
],
"created_at": "Tue Jun 20 10:33:50 CDT 2017",
"agent": {
"extractor_id": "https://clowder.ncsa.illinois.edu/clowder/api/extractors/ncsa.image.ocr",
"@type": "cat:extractor",
"name": "https://clowder.ncsa.illinois.edu/clowder/api/extractors/ncsa.image.ocr"
}
},
{
"content": {
"basic_caltech101_score": "-0.696487",
"basic_caltech101_category": "bonsai"
},
"@context": [
"https://clowder.ncsa.illinois.edu/contexts/metadata.jsonld",
{
"basic_caltech101_score": "http://clowder.ncsa.illinois.edu/metadata/ncsa.cv.caltech101#basic_caltech101_score",
"basic_caltech101_category": "http://clowder.ncsa.illinois.edu/metadata/ncsa.cv.caltech101#basic_caltech101_category"
}
],
"created_at": "Tue Jun 20 10:33:50 CDT 2017",
"agent": {
"extractor_id": "http://141.142.211.76:9000/clowder-dev/extractors/ncsa.cv.caltech101/1.0",
"@type": "cat:extractor",
"name": "http://141.142.211.76:9000/clowder-dev/extractors/ncsa.cv.caltech101/1.0"
}
},
{
"content": {
"versus_descriptors": [
{
"descriptor_id": "class edu.illinois.ncsa.versus.descriptor.impl.RGBHistogramDescriptor",
"bands": 3,
"extractor_id": "edu.illinois.ncsa.versus.extract.impl.RGBHistogramExtractor",
"bins": 8,
"descriptor": [
[
0.5913427300480177,
0.10906269800126721,
0.29938380696545547,
0.00021076498525954203,
0,
0,
0,
0
],
[
0.6151575370082368,
0.18266582360672562,
0.2015070350995701,
0.000669604285467427,
0,
0,
0,
0
],
[
0.5999196213037582,
0.1472782516717198,
0.2524234700556629,
0.0003786569688591462,
0,
0,
0,
0
]
]
}
]
},
"@context": [
"https://clowder.ncsa.illinois.edu/contexts/metadata.jsonld",
{
"descriptor_id": "http://clowder.ncsa.illinois.edu/ncsa.versus.image#descriptor_id",
"versus_descriptors": "http://clowder.ncsa.illinois.edu/ncsa.versus.image#versus_descriptors",
"bands": "http://clowder.ncsa.illinois.edu/ncsa.versus.image#bands",
"descriptor": "http://clowder.ncsa.illinois.edu/ncsa.versus.image#descriptor",
"extractor_id": "http://clowder.ncsa.illinois.edu/ncsa.versus.image#extractor_id",
"bins": "http://clowder.ncsa.illinois.edu/ncsa.versus.image#bins"
}
],
"created_at": "Tue Jun 20 10:33:49 CDT 2017",
"agent": {
"extractor_id": "https://clowder.ncsa.illinois.edu/clowder/api/extractors/ncsa.versus.image",
"@type": "cat:extractor",
"name": "https://clowder.ncsa.illinois.edu/clowder/api/extractors/ncsa.versus.image"
}
},
{
"content": {
"Border color": "srgb(223,223,223)",
"Compression": "Zip",
"Elapsed time": "0:01.219",
"Artifacts": {
"verbose": true,
"filename": "/tmp/tmpzdHVT2.png"
},
"height": 1929,
"Interlace": "None",
"Compose": "Over",
"Tainted": false,
"Filesize": "3.797MB",
"Pixels per second": "13.89MB",
"Units": "PixelsPerCentimeter",
"Chromaticity": {
"red primary": "(0.64,0.33)",
"green primary": "(0.3,0.6)",
"blue primary": "(0.15,0.06)",
"white point": "(0.3127,0.329)"
},
"Type": "TrueColor",
"Rendering intent": "Perceptual",
"Print size": "22.3539x27.2227",
"Channel depth": {
"blue": "8-bit",
"green": "8-bit",
"red": "8-bit"
},
"Background color": "white",
"Transparent color": "black",
"Format": "PNG (Portable Network Graphics)",
"Image statistics": {
"Overall": {
"skewness": 0.67031,
"min": "0 (0)",
"max": "255 (1)",
"standard deviation": "73.7679 (0.289286)",
"kurtosis": -1.19763,
"mean": "91.9025 (0.360402)"
}
},
"Number pixels": "3.056M",
"width": 1584,
"Channel statistics": {
"Blue": {
"skewness": 0.771814,
"min": "0 (0)",
"max": "255 (1)",
"standard deviation": "70.8725 (0.277931)",
"kurtosis": -0.940633,
"mean": "87.3909 (0.342709)"
},
"Green": {
"skewness": 0.627148,
"min": "0 (0)",
"max": "255 (1)",
"standard deviation": "73.9244 (0.2899)",
"kurtosis": -1.27569,
"mean": "90.7096 (0.355724)"
},
"Pixels": 3055536,
"Red": {
"skewness": 0.59793,
"min": "0 (0)",
"max": "255 (1)",
"standard deviation": "76.4027 (0.299618)",
"kurtosis": -1.38753,
"mean": "97.607 (0.382772)"
}
},
"Class": "DirectClass",
"Mime type": "image/png",
"Page geometry": "1584x1929+0+0",
"Geometry": "1584x1929+0+0",
"Colorspace": "sRGB",
"Depth": "8-bit",
"User time": "0.130u",
"Matte color": "grey74",
"Resolution": "70.86x70.86",
"Gamma": 0.45455,
"Image": "/tmp/tmpzdHVT2.png",
"Properties": {
"photoshop": {
"DateCreated": "2014-01-13T09:13:13"
},
"exif": {
"LightSource": 0,
"FlashPixVersion": "48, 49, 48, 48",
"ExposureMode": 0,
"Flash": 16,
"SceneCaptureType": 0,
"DateTime": "2014:01:13 09:13:13",
"MeteringMode": 5,
"ExifVersion": "48, 50, 51, 48",
"ExposureBiasValue": "0/1",
"Saturation": 0,
"ExposureProgram": 2,
"FocalLengthIn35mmFilm": 33,
"ColorSpace": 1,
"ExifImageWidth": 1584,
"Contrast": 0,
"DateTimeOriginal": "2014:01:13 09:13:13",
"WhiteBalance": 0,
"CompressedBitsPerPixel": "4/1",
"DateTimeDigitized": "2014:01:13 09:13:13",
"SensingMethod": 2,
"FNumber": "7/2",
"CustomRendered": 0,
"FocalLength": "28/5",
"ComponentsConfiguration": "1, 2, 3, 0",
"ExifOffset": 188,
"ISOSpeedRatings": 100,
"Model": "DMC-FZ200",
"Software": "Ver.1.0",
"ExposureTime": "1/125",
"FileSource": 3,
"MaxApertureValue": "95/32",
"Make": "Panasonic",
"Sharpness": 0,
"GainControl": 0,
"DigitalZoomRatio": "0/1",
"ExifImageLength": 1929
},
"icc": {
"model": "IEC 61966-2.1 Default RGB colour space - sRGB",
"description": "sRGB IEC61966-2.1",
"copyright": "Copyright (c) 1998 Hewlett-Packard Company",
"manufacturer": "IEC http://www.iec.ch"
},
"signature": "2decf705da4c929f51db201f646f51f5dcd9dcfd42aac6997cf825e37593ca8d",
"date": {
"create": "2017-06-20T15:33:48+00:00",
"modify": "2017-06-20T15:33:48+00:00"
},
"aux": {
"SerialNumber": "O071309260157"
},
"xmp": {
"CreateDate": "2014-01-13T09:13:13",
"ModifyDate": "2014-01-13T09:13:13",
"CreatorTool": "Ver.1.0"
},
"png": {
"bKGD": "chunk was found (see Background color, above)",
"IHDR": {
"color-type-orig": 2,
"color_type": "2 (Truecolor)",
"bit_depth": 8,
"width,height": "1584, 1929",
"bit-depth-orig": 8,
"interlace_method": "0 (Not interlaced)"
},
"pHYs": "x_res=7086, y_res=7086, units=1",
"text": "47 tEXt/zTXt/iTXt chunks were found",
"sRGB": "intent=0 (Perceptual Intent)",
"gAMA": "gamma=0.45455 (See Gamma, above)",
"tIME": "2017-05-08T17:53:53Z",
"cHRM": "chunk was found (see Chromaticity, above)"
}
},
"Version": "ImageMagick 6.8.9-9 Q16 x86_64 2017-05-26 http://www.imagemagick.org",
"Iterations": 0
},
"@context": {
"@vocab": "http://www.w3.org/2003/12/exif/ns"
},
"created_at": "Tue Jun 20 10:33:48 CDT 2017",
"agent": {
"extractor_id": "http://141.142.211.76:9000/clowder-dev/api/extractors/ncsa.image.metadata",
"@type": "cat:extractor",
"name": "http://141.142.211.76:9000/clowder-dev/api/extractors/ncsa.image.metadata"
}
},
{
"content": {
"conformsTo": [
{
"sf:mime": "image/png",
"sf:version": "1.1",
"@id": "info:pronom/fmt/12",
"sf:name": "Portable Network Graphics",
"sf:basis": "extension match png; byte match at [[[0 16]] [[37 4]] [[3796773 12]]] (signature 3/3)"
}
],
"extent": 3796785
},
"@context": [
"https://clowder.ncsa.illinois.edu/contexts/metadata.jsonld",
{
"conformsTo": "http://purl.org/dc/terms/conformsTo",
"extent": "http://purl.org/dc/terms/extent"
}
],
"created_at": "Tue Jun 20 10:33:48 CDT 2017",
"agent": {
"extractor_id": "http://141.142.211.76:9000/clowder-dev/extractors/siegfried/2.0",
"@type": "cat:extractor",
"name": "http://141.142.211.76:9000/clowder-dev/extractors/siegfried/2.0"
}
},
{
"content": {
"mean grayscale value": 92.37236772860801
},
"@context": [
"https://clowder.ncsa.illinois.edu/contexts/metadata.jsonld",
{
"mean grayscale value": "http://clowder.ncsa.illinois.edu/ncsa.cv.meangrey#mean_grayscale_value"
}
],
"created_at": "Tue Jun 20 10:33:48 CDT 2017",
"agent": {
"extractor_id": "https://clowder.ncsa.illinois.edu/clowder/api/extractors/ncsa.cv.meangrey",
"@type": "cat:extractor",
"name": "https://clowder.ncsa.illinois.edu/clowder/api/extractors/ncsa.cv.meangrey"
}
},
{
"content": {
"sha1": "cd89ff5f85078b3a98f4f92dc25894a69bd89181",
"sha224": "9b9f30bede741663dcb7c3d1f1d9427a3b059d490831c3a6cd6b62ca",
"sha384": "5ec428a0f040935aca0ff21fa72478864626ecd8ce326db12a7ede223464126a1a91ff5712192af8df9051ea7ab3ee4c",
"sha256": "12adc20571b87d90350238f16a5fcaeaa9d5ea56cd3a4e57616d84b8d2799a32",
"sha512": "c7dd3508b864904de193b105d4d2347ec591fa31d0b6ab54bcd3fa2c2ecef9967bb14a452d434d21fb5cfacc849d74c9e098648671e1ec1776599ebb0931b28e",
"md5": "2a2d9dc555876abfd4641c4529eb7960"
},
"@context": [
"https://clowder.ncsa.illinois.edu/contexts/metadata.jsonld",
{
"sha1": "http://www.w3.org/2001/04/xmldsig-more#sha1",
"sha224": "http://www.w3.org/2001/04/xmldsig-more#sha224",
"sha384": "http://www.w3.org/2001/04/xmldsig-more#sha384",
"sha256": "http://www.w3.org/2001/04/xmldsig-more#sha256",
"sha512": "http://www.w3.org/2001/04/xmldsig-more#sha512",
"md5": "http://www.w3.org/2001/04/xmldsig-more#md5"
}
],
"created_at": "Tue Jun 20 10:33:48 CDT 2017",
"agent": {
"extractor_id": "http://141.142.211.76:9000/clowder-dev//api/extractors/ncsa.file.digest",
"@type": "cat:extractor",
"name": "http://141.142.211.76:9000/clowder-dev//api/extractors/ncsa.file.digest"
}
}
],
"versusmetadata": [
{
"descriptor_id": "class edu.illinois.ncsa.versus.descriptor.impl.RGBHistogramDescriptor",
"bands": 3,
"extractor_id": "edu.illinois.ncsa.versus.extract.impl.RGBHistogramExtractor",
"bins": 8,
"descriptor": [
[
0.5913427300480177,
0.10906269800126721,
0.29938380696545547,
0.00021076498525954203,
0,
0,
0,
0
],
[
0.6151575370082368,
0.18266582360672562,
0.2015070350995701,
0.000669604285467427,
0,
0,
0,
0
],
[
0.5999196213037582,
0.1472782516717198,
0.2524234700556629,
0.0003786569688591462,
0,
0,
0,
0
]
]
}
],
"tags": [],
"filename": "ncsa.png",
"technicalmetadata": [
{
"Human Preference Extractor": {
"Definitions": {
"Human Preference": "A Computer Vision model that uses the spectral data of an image to get a human preference value ranging from 1 to 5.",
"Green Index": "The green index is the estimated percentage of green pixels within an image."
},
"Data": {
"Green Index": "2.3"
}
}
}
],
"id": "5949405b4f0c4db82da2da84"
} |
...