From SQL to AWS Kinesis, EMR and Elasticsearch [Video, Hebrew]

That thing that happens when your company is scaling so fast you have to replace your infrastructure within weeks or the whole thing crashes. This is what happened to our customer, and this is the story of how we replaced good-ol' SQL with streams and batch processing technologies on AWS.

Using Terraform, Packer, Elasticsearch, AWS Kinesis, Lambda and EMR we were able to provide a truly scalable solution in a matter of weeks, without hurting production and while dealing with a huge scale.

In this talk we will quickly go through the design phase, the various stake holders and how we improved their usage of the system, and introduce the tools that helped us make this happen.

The video is from a talk I gave (in Hebrew) at the ReversimX meetup in Israel.