Last week I attended the local AWS users group meet-up. This was the first time with this group and naturally I was a little apprehensive. Luckily the pizza was amazing, the group was welcoming and the presentation was well executed and informative. Great job Scott! I've already got the next meet-up on my calendar.
ElastiCache is hosted/managed Redis and Memcached by Amazon. This allows developers to immediately use these technologies without provisioning and configuring servers. One negative though is that it must be used within Amazon's VPC (Virtual Private Cloud) and cannot be secured except for with instance level access lists.
For the demonstration our presenter, Scott, wrote a simple web application (in Ruby) that allowed for Widgets to be built by workers and then processed. Finally a CEO dashboard broke out a bunch of statistics about the work done, both today and historically.
Naturally in the small scale demo (< 10,000 widgets) the system ran great! Everyone was happy and the CEO could get their statistics. Enter a huge number of widgets. Suddenly worker productivity plummets. It takes tens of seconds to create a widget, over a minute to load the process screen and the CEO dashboard times out with an error. Now we have an upset CEO. Let's solve this by caching. By storing simple counts and last created widgets in ElastiCache Scott was able to simply fix both the worker and process pages (once the cache was warmed-up). The CEO dashboard proved to be a little more interesting as it required much more data in order for the cache to warm up. Once it was loaded (~3 minutes for a large number of widgets) then the CEO dashboard was once again happy.