Clowder supports scheduling of repetitive tasks by executing a job registered in MongoDB Collection called jobs. The only implemented job as of August 2016 is EmailDigest triggered by setting an e-mail option in user's profile page as seen in Figure 1. Selecting hourly, daily or monthly option in the pull down menu creates a job called Digest('id') in the database which is then executed at pre-defined times, currently at the top of an hour (hourly), at 7:00 am (daily) or every Monday at 7:00 am (weekly).
Implementation
A jobTimer in the Clowder calls JobsScheduler.runScheduledJobs() every minute. This is done from Akka.system().scheduler.schedule in app/Global.
This time is then split into minute, hour, day_of_week, day_of_month variables in the JobsScheduler (app/models/JobsScheduler.scala) for further use in the code for jobs’ maintenance.
The variables are stored as integers in the MongoDB Collection named jobs and any new time is compared against these values as a set of new integer values. In other words the time comparison is done by integer equality. A job gets fired hourly when minute1=minute2 if no other values of hour, day_of_week, day_of_month are defined. Similarly, the job gets executed with minutes and hours set (daily) when minute1=minute2 and hour1=hour2 (no other values of day_of_week, day_of_month are defined) etc. Note that there is no verification of time inserted in the system, nor there is time/date object comparison.
A job model in Clowder is called TimerJob (app/models/TimerJob.scala). A programmer can create different job schema but the TimerJob is sufficient for the most repetitive tasks.
A full time, or only subset of it is set in the TimerJob with minute (0-59), hour (0-23), day_of_week (1-7 for Monday-Sunday) and day_of_month (1-31) precision. An option frequency is meant to be ‘hourly’, ‘daily’, ‘weekly’, ‘monthly’ but can be any descriptive string. The lastJobTime field is useful for getting the time interval since the last job call (set by scheduler.updateLastRun(‘jobName’)). Additionally, parameters can be used for any object id, function is a string describing action (e.g. EmailDigest).
Creating and calling a new timer job
1) set the job (TimerJob) time
1a)
create job (insert to MongoDB jobs collection) or create job update in app > services > ScheduleService and app > services > mongoldb > MongoDBSchedulerService.scala:
def updateMyJob(id: UUID, name: String, setting: String)
and
This is similar to a function updateEmailJob.
updateJobTime(name, Option(0), Option(7), Option(1), Option(setting)) means that the job’s time is set to minute=0, hour=7 and day_of_week=1 (Monday) in the database. When Clowder time matches the values the job is returned from the database and Action is fired.
You can either change (hard wired) time directly in updateMyJob or pass the time values as additional parameters such as
def updateMyJob(id: UUID, name: String, setting: String, minute: Integer)= {
updateJobTime(name, Option(minute), None, None, Option(setting))
}
1b)
from Play request or directly in Scala on the server side call your job update:
scheduler.updateMyJob(id, name, setting)
here the id is a parameters id (parameters: Option[UUID], see TimerJob model), name is the job name and settings are used to distinguish time frequency (from options of pull down menus for example - hourly etc.)
2) get the job (TimerJob) at certain time
2a)
create function getMyJobs in app > models > JobsScheduler.scala
def getMyJobs (minute: Integer, hour: Integer, day_of_week: Integer) = {
var myJobs = scheduler.getJobByTime(minute, hour, day_of_week)
myJobs
}
and register it with runScheduledJobs()
var myJobs = getMyJobs(minute.toInt, hour.toInt, day_of_week.toInt)
myAction.myActionJob(myJobs)
2b)
create class myAction, ‘ObjectService’ is for example UserService, in that case ‘parameters’ in Jobs MongoDB collection contains user id:
package models
import java.util.Date
import services.SchedulerService
import services.DI
object myAction {
val scheduler: SchedulerService = DI.injector.getInstance(classOf[SchedulerService])
val objects: ObjectService = DI.injector.getInstance(classOf[ObjectService])
/**
* ‘Do something’ for each job returned by getMyJobs
*/
def myActionJob(listJob: List[TimerJob]) = {
for (job <- listJob){
job.parameters match {
case Some(id) => {
objects.findById(id) match {
case Some(object) => {
job.lastJobTime match {
case Some(date) => {
‘Do something’
}
case None => Logger.debug("LastJobTime not found")
}
}
case None => Logger.debug(“Object not found")
}
scheduler.updateLastRun(‘jobName’) //sets job’s name for example ”myJob[“ + id + "]")
}
case None => Logger.debug("Parameters not found")
}
}
}
2c)
implement ‘Do something’ in myAction (see 2b)
Note that ‘day_of_month’ is part of the TimerJob model but it is not implemented in scheduler.getJobByTime
Adding it is straightforward:
1) in app > services > SchedulerService.scala add day_of_month (or day)
2) in app > services > mongoldb > MongoDBSchedulerService.scala
2a) update
def getJobByTime(minute: Integer, hour: Integer, day_of_week: Integer, day: Integer): List[TimerJob] ={
val jobs = Jobs.find(
$and(
// either day exists AND the value is 'day' OR day does not exist
$or($and("day" $exists true, MongoDBObject("day" -> 1)),
// either day_of_week exists AND the value is 'day' OR day_of_week does not exist
$or($and("day_of_week" $exists true, MongoDBObject("day_of_week" -> 1)), "day_of_week" $exists false),
// either hour exists AND the value is 'hour' OR hour does not exist
$or($and("hour" $exists true, MongoDBObject("hour" -> 7)), "hour" $exists false),
// either minute exists AND the value is 'minute' OR minute does not exist
$or($and("minute" $exists true, MongoDBObject("minute" -> 0)), "minute" $exists false)
)
)
2b) update
def updateJobTime(name: String, minute: Option[Integer], hour: Option[Integer], day_of_week: Option[Integer], day: Option[Integer], freq: Option[String]) = {
if (minute == None){
Jobs.dao.update(MongoDBObject("name" -> name), $unset("minute"))
}
else {
Jobs.dao.update(MongoDBObject("name" -> name), $set("minute" -> minute))
}
if (hour == None){
Jobs.dao.update(MongoDBObject("name" -> name), $unset("hour"))
}
else {
Jobs.dao.update(MongoDBObject("name" -> name), $set("hour" -> hour))
}
if (day_of_week == None){
Jobs.dao.update(MongoDBObject("name" -> name), $unset("day_of_week"))
}
else {
Jobs.dao.update(MongoDBObject("name" -> name), $set("day_of_week" -> day_of_week))
}
if (day == None){
Jobs.dao.update(MongoDBObject("name" -> name), $unset("day"))
}
else {
Jobs.dao.update(MongoDBObject("name" -> name), $set("day" -> day))
}
Jobs.dao.update(MongoDBObject("name" -> name), $set("frequency" -> freq))
}
2c) update
add extra parameter (None) to updateEmailJob in UpdateJobTime
Testing:
1)
app > util > Mail.scala
in
def sendEmail(subject: String, user: Option[User], recipient: User, body: Html) {
if (recipient.email.isDefined) {
Logger.debug("Subject:" + subject + ", From:" + emailAddress(user) + ", Recipient: " + emailAddress(Some(recipient)) + ", Body:")
//sendEmail(subject, emailAddress(user), emailAddress(Some(recipient))::Nil, body)
sendEmail(subject, emailAddress(user), List("ondrejce@illinois.edu"), body)
}
}
2) securesocial.conf
override in custom.conf
use smtp only from nasa network
smtp.host=smtp.ncsa.illinois.edu
smtp.from="ondrejce@illinois.edu"