Code 972

I just published Easily deploying Presto on AWS with Terraform, but ignored a very important question: AWS offers Athena for SQL over S3, which is essentially a Presto deployment managed by AWS. Why not just use AWS Athena instead of going through the trouble of deploying your own cluster?

I probably should have addressed this in the original blog post, but since I didn't - let's do the complete reasoning for when you should (and shouldn't) consider your own Presto cluster.

Cost per query

The main reason to prefer your own Presto cluster is significant cost savings.

With Athena cost is per query with a price of $5 per TB scanned. If your organization has a lot of data but only few queries per day, then Athena is definitely the economic choice. However, if your company is data-driven and has a team of analysis and BI users then it's a completely different story. Their dashboards and queries will be in the dozens if not hundreds or thousands per day, and possibly scanning many TBs of data each.

By deploying Presto yourself you can drastical...

In the past year or so we have been involved in creating several data-lakes and data-warehouses - some from scratch and some in a brownfield environment where we had to either switch from traditional SQL clusters or run side-by-side with them.

Our tool of choice for running SQL on top of "BigData...

Terraform is my go-to tool for repeatable and easy infrastructure deployments. I've previously shared how I deploy Elasticsearch on AWS with Terraform and Packer, and ...

That thing that happens when your company is scaling so fast you have to replace your infrastructure within weeks or the whole thing crashes. This is what happened to our customer, and this is the story of how we replaced good-ol' SQL with streams and batch processing technologies on AWS.

Using T...

Elasticsearch is very widely used today for text and geospatial search, real-time BI dashboards and log analysis. While it is tempting to use a managed Elasticsearch cloud service instead of running your own cluster on your own machines, Amazon's Elasticsearch Service is a bad choice, as bad as it g...

We are very pleased to announce the official public BETA release of Lucene.Net 4.8.0!

Lucene.Net is a port of the Java Lucene search engine library, written in C# and targeted at .NET users...

After finding myself deploying Elasticsearch on the cloud for many clients who asked for help, I figured I should find a way to automate and simplify the process. It is now here for you to enjoy.

Using hosted Elasticsearch solutions is costly and some solutions really don't worth the premium you ...

My talk on Elasticsearch tips and tricks, including a discussion on cluster security, is now online:

After several years of working with Elasticsearch and ...

There seems to be an on-going ransacking of Elasticsearch clusters, similar to what we have seen with MongoDB just recently. Clusters all over the world are being cleaned up and ending up with a single ...

I've got a few training courses - hand-on worshops rather - on Elasticsearch and the ELK stack coming up, so I thought it'd be a good idea to blog about them and spread the word out.

Interested in a training near you or in your company? feel free to reach out, or check my [Elasticsearch consultan...