...
The team seems to be familiar, more comfortable with Java, Scala or Python, so we consider using these. This module mainly operates by itself, instead of mainly serving client requests, so it does not have to be a web app. A stand-alone app seems a better fit. Can start with no UI or simple CLI. We plan to start with a stand-alone service implementation, which writes part of the data structure / status into a data storage, such as Redis, Mongo, Postgres, so later another module such as a dashboard, or web app could be developed to display or visualize them, without knowing the details of this module.
- Set up this module as a service.
As an overall monitoring module in the DTS, this module is important. We plan to use upstart to watch and ensure this module itself is up and running.
- Testing
- Input
We plan to use scripts to generate high request rates. We can use different input files/configuration to generate high request rates for a certain set of extractors to test the scaling up part. We can either reduce the request rate, or stop sending the requests to test the scaling down part. - Output
We plan to use the OpenStack CLI tool or web UI (for the VM part), and RabbitMQ CLI tool or web UI (for the extractor part) to verify that the extractors are started, and VMs are started/resumed/suspended as expected.
...