July 16, 2012

How DropBox Scales Up in Amazon AWS Cloud

How DropBox scales up in Amazon AWS cloud.
 
Rajiv Eranki on the infrastructure used for scaling up DropBox.
 
  • Python for virtually everything; not more than a couple thousand lines of C
  • MySQL
  • Paster/Pylons/Cheetah (web framework — minimal use beyond templating and handling form input)
  • S3/EC2 for storing and serving file blocks
  • memcached
  • in front of the database and for handling inter-server coordination
  • ganglia
  • for graphing, with drraw for custom graphs like the stack graph mentioned above
  • nginx
  • for the frontend server
  • haproxy
  • for load balancing to app servers, after nginx (better configurability than nginx’s balancing modules)
  • nagios
  • for internal health checks
  • Pingdom
  • for external service monitoring and paging
  • GeoIP
  • for mapping IPs to locations


Related, how Instagram scales up in Amazon AWS. And, when AWS had problems, it led to Instragram 503 errors.


Tags: how dropbox scales up, how instagram scales up, cloud infrastructure behind dropbox, cloud infrastructure behind instagram