Requirements
- Rate Limiting (against flooding)
- Time stamp each request and make sure there is not more than a fixed amount over the last time interval (seconds/minutes/hours/etc)
- Requests Quota (against user overusing the system)
- Keep a counter of how many requests a user can make
- Decrease counter with every new request
- When at 0 notify user they are out of requests
- This could be a fixed number or could be reset every year/month/week
- Byte countÂ
- We could do the same as with Requests Quota, but by counting bytes
- Or we could to byte size limit for individual files
- Limits are by user
- If we want them by key we need to limit keys as well
Tasks
- Move storing of events to MongoDB or Postgres
- It's taking up too much space in Redis
- Add key users:{user}:uploaded:bytes and users:{user}:uploaded:count
- Add check when uploading. If current users:{user}:uploaded:bytes > config value, then send error
- More accurate byte counts
- Change Clowder to return file size after upload
- Change Polyglot to return file size after upload
Other
- Return appropriate message when token has expired so user can request a new one and retry the request
Background Resources