Fence User Quotas

Requirements

• Rate Limiting (against flooding)
  • Time stamp each request and make sure there is not more than a fixed amount over the last time interval (seconds/minutes/hours/etc)
• Requests Quota (against user overusing the system)
  • Keep a counter of how many requests a user can make
  • Decrease counter with every new request
  • When at 0 notify user they are out of requests
  • This could be a fixed number or could be reset every year/month/week
• Byte count 
  • We could do the same as with Requests Quota, but by counting bytes
  • Or we could to byte size limit for individual files
• Limits are by user
  • If we want them by key we need to limit keys as well

Tasks

• Move storing of events to MongoDB or Postgres
  • It's taking up too much space in Redis
• Add key users:{user}:uploaded:bytes and users:{user}:uploaded:count
• Add check when uploading. If current users:{user}:uploaded:bytes > config value, then send error
• More accurate byte counts
  • Change Clowder to return file size after upload
  • Change Polyglot to return file size after upload

Other

• Return appropriate message when token has expired so user can request a new one and retry the request

Background Resources

• https://tyk.io/tyk-documentation/rate-limiting/