Page tree
Skip to end of metadata
Go to start of metadata

Requirements

  • Rate Limiting (against flooding)
    • Time stamp each request and make sure there is not more than a fixed amount over the last time interval (seconds/minutes/hours/etc)
  • Requests Quota (against user overusing the system)
    • Keep a counter of how many requests a user can make
    • Decrease counter with every new request
    • When at 0 notify user they are out of requests
    • This could be a fixed number or could be reset every year/month/week
  • Byte count 
    • We could do the same as with Requests Quota, but by counting bytes
    • Or we could to byte size limit for individual files
  • Limits are by user
    • If we want them by key we need to limit keys as well

Tasks

  • Move storing of events to MongoDB or Postgres
    • It's taking up too much space in Redis
  • Add key users:{user}:uploaded:bytes and users:{user}:uploaded:count
  • Add check when uploading. If current users:{user}:uploaded:bytes > config value, then send error
  • More accurate byte counts
    • Change Clowder to return file size after upload
    • Change Polyglot to return file size after upload

Other

  • Return appropriate message when token has expired so user can request a new one and retry the request

Background Resources

  • No labels

3 Comments

  1. Luigi Marini, all options seem good.  Would you support all of them or pick one or two?

  2. Kenton McHenry I am thinking of doing them in this order 2 > 1 > 3. I don't think doing quotas by key buys us much. I think by user should be enough.