pyclowder2

• 

  Marcus Slavenas 26 Jul 2017 40dc0ad87d5c9a921be56e71ec2011fb51fd7e32

  Merge branch 'master' into seven-miles-changes
• 

  Max Burnette <mburnet88@gmail.com> 31 May 2017 f2b3a6093ad106753fd1a5b0ab98c2de5c35cbc7 m

  Squashed commit of the following:
  commit 9466fae6a10da7f2a618dd80291ecd7097244c7f
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Wed May 31 14:22:39 2017 -0500
  
      must appease our PEP-8 overlord masters
  
  commit 57134641b4ae3f205bb1c72d84ec602c7bd41f16
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Wed May 31 13:38:58 2017 -0500
  
      Move default resource_type out of it statements
  
  commit 5422d1d8c8b526d97a935767ecbbf07aeb9b8ad2
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Wed May 31 11:00:56 2017 -0500
  
      Send self to check/process message
  
  commit a140d0bdc3f41eb293346d5ac5af4b5a12df3240
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Tue May 30 13:31:52 2017 -0500
  
      Fix doubling of self
  
  commit 328685970945c071076ff7778f8cfda5b62dafaa
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Tue May 30 11:28:14 2017 -0500
  
      Update README
  
  commit 7c0e61e750727b55f1688be2e560a88bb3d42159
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Tue May 30 10:33:40 2017 -0500
  
      change default to action=store_true
  
  commit ebd54243b4458bdf4393904cc91e47f7525badb9
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Tue May 23 14:52:05 2017 -0500
  
      pep style fix
  
  commit 6ecba2cd986f546f25e71ff76fe00b71521f6a8f
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Tue May 23 14:50:23 2017 -0500
  
      Remove extra line
  
  commit e780fb6e82532cf51b082f709f0860d0a5ad4b5c
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Tue May 23 14:49:23 2017 -0500
  
      Refactor _process_message()
  
  commit 73cca6b2ed6e8abd420fff068ada042942ed2732
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Tue May 23 13:26:38 2017 -0500
  
      Clean up resource generation code (round 1)
  
  commit 1479b050a261fd7339b1a611f425ae4472019b74
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Tue May 23 12:50:45 2017 -0500
  
      Change unbound to --no-bind
  
  commit f4132334a52486d206ad2eb0b0f1f1ef9c5a3582
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Tue May 23 12:48:47 2017 -0500
  
      add check_duplicate flag
  
  commit 58e73a74cd50f172f55ec795850d9e719ba74212
  Author: Max Burnette <mburnet88@gmail.com>
  Date:   Fri Apr 21 09:11:00 2017 -0500
  
      Add unbound flag
  
      Allow extractors to run ‘unbound’ at runtime - this will run them
      without binding the process file type bindings, only bound by extractor
      name. This means extractor will only be accessible to manual triggers
      or rule_checker extractor, but can also have a version w/ file type
      bindings committed to git for use other times.

  • README.md (version f2b3a6093ad106753fd1a5b0ab98c2de5c35cbc7) (diffs)
  • pyclowder/connectors.py (version f2b3a6093ad106753fd1a5b0ab98c2de5c35cbc7) (diffs)
  • pyclowder/extractors.py (version f2b3a6093ad106753fd1a5b0ab98c2de5c35cbc7) (diffs)
  • pyclowder/files.py (version f2b3a6093ad106753fd1a5b0ab98c2de5c35cbc7) (diffs)
• 

  Sandeep Puthanveetil Satheesan 30 May 2017 097cc79ff55c805fa1507d23e0d4e77d65b032c5 m

  Replaced empty strings in local_parameters object with None

  • pyclowder/connectors.py (version 097cc79ff55c805fa1507d23e0d4e77d65b032c5) (diffs)
• 

  Sandeep Puthanveetil Satheesan 26 May 2017 029ace144b39bb06a999deefedadb72c397dfe68 m

  Fixed merge conflicts by incorporating changes from master
• 

  Max Burnette <mburnet88@gmail.com> 25 May 2017 1a48101e362c0b82f303b16580ed48a1677b0fd9 m

  style fix

  • pyclowder/files.py (version 1a48101e362c0b82f303b16580ed48a1677b0fd9) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 25 May 2017 c686e1c32fa63af9bfe404766f11c37882895594 m

  More cleanup for file extraction

  • pyclowder/files.py (version c686e1c32fa63af9bfe404766f11c37882895594) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 25 May 2017 91f3cf79b3ffb24e000bc17ae23aa6a6fbb45c7a m

  Fix missing ssl check in files

  • pyclowder/files.py (version 91f3cf79b3ffb24e000bc17ae23aa6a6fbb45c7a) (diffs)
• 

  Sandeep Puthanveetil Satheesan 24 May 2017 19e51b7ef1a248b1263166f05a55d6054071044f m

  Removed duplicate line and added task number to TODO item

  • pyclowder/connectors.py (version 19e51b7ef1a248b1263166f05a55d6054071044f) (diffs)
• 

  Sandeep Puthanveetil Satheesan 23 May 2017 643e0098eae0c41eb2bcb476d7c943c2ee28af29 m

  Fix for PEP8 line length limit

  • pyclowder/extractors.py (version 643e0098eae0c41eb2bcb476d7c943c2ee28af29) (diffs)
• 

  Sandeep Puthanveetil Satheesan 23 May 2017 ac0c5f5db78e35dc304bf3e2efffb1aca24feaa7 m

  Updated README. Fixed typos.

  • README.md (version ac0c5f5db78e35dc304bf3e2efffb1aca24feaa7) (diffs)
• 

  Sandeep Puthanveetil Satheesan 23 May 2017 cca4015025b7f27e1481506043cffa9ead05018d m

  Append extension '.json' to input filename to genetate output filename

  • pyclowder/connectors.py (version cca4015025b7f27e1481506043cffa9ead05018d) (diffs)
• 

  Sandeep Puthanveetil Satheesan 23 May 2017 449f309af2de3bd4db5c41ef1acababa1ba11cc2 m

  Added code to do local processing

  • pyclowder/connectors.py (version 449f309af2de3bd4db5c41ef1acababa1ba11cc2) (diffs)
  • pyclowder/extractors.py (version 449f309af2de3bd4db5c41ef1acababa1ba11cc2) (diffs)
• 

  Sandeep Puthanveetil Satheesan 22 May 2017 dd5a4844554f6c02e568e72a8b091d50ca325c1b m

  Intendation fix for conforming to PEP8 standards

  • pyclowder/files.py (version dd5a4844554f6c02e568e72a8b091d50ca325c1b) (diffs)
• 

  Sandeep Puthanveetil Satheesan 22 May 2017 e41acf79f400940a9128ecb9f6ee3ccf0df92ede m

  Corrected property name to get status code

  • pyclowder/files.py (version e41acf79f400940a9128ecb9f6ee3ccf0df92ede) (diffs)
• 

  Sandeep Puthanveetil Satheesan 22 May 2017 037d987cfedb01a1af392902b82e4bed6f159cae m

  Integrated new post methods in wrappers for Clowder Files API

  • pyclowder/files.py (version 037d987cfedb01a1af392902b82e4bed6f159cae) (diffs)
• 

  Sandeep Puthanveetil Satheesan 22 May 2017 fc30585389fd188fdc3d97b87d20f978a214a3c1 m

  Integrated new get methods in wrappers for Clowder Files API

  • pyclowder/files.py (version fc30585389fd188fdc3d97b87d20f978a214a3c1) (diffs)
• 

  Sandeep Puthanveetil Satheesan 22 May 2017 4736b5e59bbcae8155d4141720dcc23c934c8700 m

  Added get, post, put and delete methods to Connector class

  • pyclowder/connectors.py (version 4736b5e59bbcae8155d4141720dcc23c934c8700) (diffs)
• 

  Sandeep Puthanveetil Satheesan 22 May 2017 090e5cd43497c832a0258385e84a1eb575751914 m

  Merge pull request #35 in CATS/pyclowder2 from catch-closed-RMQ-connection to master
  * commit 'ceb3f2e7b4286f8c081cae4250b78c1a5222ca92':
    Add additional try/catch on connection close
• 

  Max Burnette <mburnet88@gmail.com> 22 May 2017 ceb3f2e7b4286f8c081cae4250b78c1a5222ca92 m

  Add additional try/catch on connection close

  • pyclowder/connectors.py (version ceb3f2e7b4286f8c081cae4250b78c1a5222ca92) (diffs)
• 

  Rob Kooper 12 May 2017 483946612bc5b5d6ea47c2e26c0587c37202a65b m

  Merge pull request #34 in CATS/pyclowder2 from preserve-reply_to to master
  * commit '11d494ee6da550e67e6f755f2a2a38132823f837':
    preserve reply_to in error queue, make sure reply_to exists before sending message
• 

  Rob Kooper 11 May 2017 f31267684f05300488da411251cae4883fd8dfa1 m

  Merge pull request #33 in CATS/pyclowder2 from better-error-catching-on-process-message to master
  * commit 'cf5fdbaa02a67aeda82db48c7c788cf880606abd':
    Send status update on download fail
    error message on download fail
    differentiate manual vs migrated messages
    add try/catch to info download
• 

  Rob Kooper 11 May 2017 11d494ee6da550e67e6f755f2a2a38132823f837 m

  preserve reply_to in error queue, make sure reply_to exists before sending message

  • pyclowder/connectors.py (version 11d494ee6da550e67e6f755f2a2a38132823f837) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 11 May 2017 cf5fdbaa02a67aeda82db48c7c788cf880606abd m

  Send status update on download fail

  • pyclowder/connectors.py (version cf5fdbaa02a67aeda82db48c7c788cf880606abd) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 11 May 2017 e269daad9b40380cccab6a994863190f6bcd26e1 m

  error message on download fail

  • pyclowder/connectors.py (version e269daad9b40380cccab6a994863190f6bcd26e1) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 11 May 2017 203a9e6073f5a7d0950eecf807e2fd69cbaa030f m

  differentiate manual vs migrated messages

  • pyclowder/connectors.py (version 203a9e6073f5a7d0950eecf807e2fd69cbaa030f) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 11 May 2017 5ad19cc57fdc2dd45c730a6b1679b3498c92b3eb m

  add try/catch to info download

  • pyclowder/connectors.py (version 5ad19cc57fdc2dd45c730a6b1679b3498c92b3eb) (diffs)
• 

  Rob Kooper 10 May 2017 6ab150d2fa4c10989f50d3abeeb09450692b42a8 m

  remove file if not succesfully donwloaded

  • pyclowder/files.py (version 6ab150d2fa4c10989f50d3abeeb09450692b42a8) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 05 May 2017 9142bfb37725c69cc1b1884f3ac521f0d34c375e m

  Allow mounts to be overridden by ENV

  • pyclowder/extractors.py (version 9142bfb37725c69cc1b1884f3ac521f0d34c375e) (diffs)
• 

  Sandeep Puthanveetil Satheesan 02 May 2017 d1071c3c998a39df9a7bdfbb27fc1370d9ba8703 m

  Process messages in first-in, first-out order

  • pyclowder/connectors.py (version d1071c3c998a39df9a7bdfbb27fc1370d9ba8703) (diffs)
• 

  Rob Kooper 27 Apr 2017 78f3a41d7f9c7edc7f952171a23f552962227f52 m

  Merge pull request #29 in CATS/pyclowder2 from fix-typecheck-amend to master
  * commit '162136a74e84f5104c66bffcf42b7cc862edde8d':
    Remove extractors. prefix
• 

  Max Burnette <mburnet88@gmail.com> 27 Apr 2017 162136a74e84f5104c66bffcf42b7cc862edde8d m

  Remove extractors. prefix

  • pyclowder/connectors.py (version 162136a74e84f5104c66bffcf42b7cc862edde8d) (diffs)
• 

  Rob Kooper 27 Apr 2017 a3ae56f46f26b0565020300186dbbddf14033d0c m

  Merge pull request #28 in CATS/pyclowder2 from fix-typecheck-on-migrated-messages to master
  * commit '683dfa7323f5f429633c3f99aca3196e7b0c1600':
    Fix type check on migrated messages
• 

  Max Burnette <mburnet88@gmail.com> 27 Apr 2017 683dfa7323f5f429633c3f99aca3196e7b0c1600 m

  Fix type check on migrated messages
  Messages moved from existing queue do not have “Extractors.” at the
  prefix and so were being incorrectly flagged as ‘file’ events and
  processing was failing.

  • pyclowder/connectors.py (version 683dfa7323f5f429633c3f99aca3196e7b0c1600) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 19 Apr 2017 95a2a1d7f16e4e4656c0f4781a4b825cc730fe21 m

  style fix

  • pyclowder/utils.py (version 95a2a1d7f16e4e4656c0f4781a4b825cc730fe21) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 19 Apr 2017 b297a6363e1730a1b1fd5952603ae5f7722e970f m

  Move log file download to setup_logging

  • pyclowder/extractors.py (version b297a6363e1730a1b1fd5952603ae5f7722e970f) (diffs)
  • pyclowder/utils.py (version b297a6363e1730a1b1fd5952603ae5f7722e970f) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 19 Apr 2017 e9acf943c4b23162d5e30bf29ccaa6e17e04cc49 m

  If logging arg is a url, download the file

  • pyclowder/extractors.py (version e9acf943c4b23162d5e30bf29ccaa6e17e04cc49) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 19 Apr 2017 5476e0dae4fdabb486656ffff2018ff5d45082c1 m

  Get logging info from environment variable if not provided in args

  • pyclowder/extractors.py (version 5476e0dae4fdabb486656ffff2018ff5d45082c1) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 18 Apr 2017 b499397415765b21a531fc944f3a1f8a4f7f9bc8 m

  Fix extractor registration

  • pyclowder/connectors.py (version b499397415765b21a531fc944f3a1f8a4f7f9bc8) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 18 Apr 2017 7eddf07eebf891be1c0892eed6939537cdbe05c9 m

  Fix typo

  • pyclowder/geostreams.py (version 7eddf07eebf891be1c0892eed6939537cdbe05c9) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 11 Apr 2017 fb9afac08ab8f2b0788dc10965ea242f92dc1258 m

  pep-8 silliness

  • pyclowder/datasets.py (version fb9afac08ab8f2b0788dc10965ea242f92dc1258) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 11 Apr 2017 4272e2522a560c91d12c2a3f88a28fb57fec1211 m

  response_code -> status_code

  • pyclowder/datasets.py (version 4272e2522a560c91d12c2a3f88a28fb57fec1211) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 11 Apr 2017 a913fe1f5391a6b29ee585bb3450068995120990 m

  GET -> POST on submit

  • pyclowder/datasets.py (version a913fe1f5391a6b29ee585bb3450068995120990) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 11 Apr 2017 46d93081dae7fe5ce6f6aec3a7048d838ed4a5d7 m

  Allow connector to be None

  • pyclowder/collections.py (version 46d93081dae7fe5ce6f6aec3a7048d838ed4a5d7) (diffs)
  • pyclowder/datasets.py (version 46d93081dae7fe5ce6f6aec3a7048d838ed4a5d7) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 11 Apr 2017 5dfa9991876dfae8c5fbdda70d1a36427eb57422 m

  Merge branch 'master' into feature-improvements
• 

  Max Burnette <mburnet88@gmail.com> 04 Apr 2017 7867a71977094d4812f87681d6c99031dc840dcd m

  Allow connector to be None
  …allows calling of pyclowder2 functions outside of extractor context by
  allowing one to pass None as connector

  • pyclowder/collections.py (version 7867a71977094d4812f87681d6c99031dc840dcd) (diffs)
  • pyclowder/datasets.py (version 7867a71977094d4812f87681d6c99031dc840dcd) (diffs)
  • pyclowder/files.py (version 7867a71977094d4812f87681d6c99031dc840dcd) (diffs)
  • pyclowder/geostreams.py (version 7867a71977094d4812f87681d6c99031dc840dcd) (diffs)
  • pyclowder/sections.py (version 7867a71977094d4812f87681d6c99031dc840dcd) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 31 Mar 2017 50257d2f25ef7b09ec94aa32e69eea159f0cb9eb m

  Change docker hub location to correct one

  • docker-compose.yml (version 50257d2f25ef7b09ec94aa32e69eea159f0cb9eb) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 31 Mar 2017 97771fda9aae335691419fc1d67cd767fb6a7517 m

  PEP-8 pickiness

  • pyclowder/datasets.py (version 97771fda9aae335691419fc1d67cd767fb6a7517) (diffs)
  • pyclowder/files.py (version 97771fda9aae335691419fc1d67cd767fb6a7517) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 31 Mar 2017 73583ca4e3a4ef7e0282d4d962816d4474f8412a m

  Add dataset extraction

  • pyclowder/datasets.py (version 73583ca4e3a4ef7e0282d4d962816d4474f8412a) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 31 Mar 2017 84f88cb42ac9b4b71dc31740ea86d181b1fa57fa m

  Add extractor submit method to ds/file

  • pyclowder/files.py (version 84f88cb42ac9b4b71dc31740ea86d181b1fa57fa) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 31 Mar 2017 07a146361350cca77d3f077330b53b1fdceabb65 m

  add get_datasets to collection

  • pyclowder/collections.py (version 07a146361350cca77d3f077330b53b1fdceabb65) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 30 Mar 2017 8822c3d4df34fd98a429f84b85a3fa58db0ea78e m

  Clean up after temp downloads better

  • pyclowder/connectors.py (version 8822c3d4df34fd98a429f84b85a3fa58db0ea78e) (diffs)
• 

  Max Burnette <mburnet88@gmail.com> 07 Mar 2017 361489af9a997ea123d6155c5c29fee2cc3536f4 m

  Fix typo in geostreams code

  • pyclowder/geostreams.py (version 361489af9a997ea123d6155c5c29fee2cc3536f4) (diffs)
• 

  Todd Nicholson <tcnichol@illinois.edu> 23 Feb 2017 aa3bb8d31c3ac0155d719040bf70a9cfad1d8824 m

  added space for datasets/collections     create empty collection added.     collection or dataset have optional parentId for parent collection

  • pyclowder/collections.py (version aa3bb8d31c3ac0155d719040bf70a9cfad1d8824) (diffs)
  • pyclowder/datasets.py (version aa3bb8d31c3ac0155d719040bf70a9cfad1d8824) (diffs)