Top 10 Reasons Snowflake Rocks!12-08-2019 0 Comment(s)
Using the following SQL statement on Snowflake, I managed to stand up a 4 node (MEDIUM) virtual compute cluster, fully installed, and ready to accept queries against an existing multi-petabyte data warehouse.
Even if you’re lucky enough to be running a relatively modern multi-node MPP system, it could take days to install an additional node, and even in the cloud it can take anything from a few
An incredible benefit of the Snowflake solution is that every workload can be executed against it's own separate virtual compute cluster (called a virtual warehouse), which avoids the usual tug of war between batch loading ELT processes and low latency dashboards. In addition to being independent, each virtual warehouse can be individually resized to fit the workload and budget available.
Independently Sized Elasticity
Having started a new dedicated ELT compute cluster, you then discover there’s been an unexpectedly large increase in data volume, and the batch ELT processes will not complete on time. You now need to adjust the resources to complete the workload faster.
The above SQL statement took just 118 milliseconds on the production system to resize the dedicated ELT warehouse. Without any interruption to the executing loading process, all queued, and newly executed SQL statements were automatically started on the larger 32 node cluster with eight times the compute capacity.
To keep it simple, Snowflake clusters come in a range of T-Shirt Sizes which start at extra-small (one node), through, Small, Medium and large, up to a massive 128 node 4X Large cluster. Each increment is designed to be double the capacity of the previous cluster.
In this case, as it’s not clear how long the ETL process will take to complete, it’s possible to set the cluster to automatically suspend after five minutes inactivity, but automatically re-start when the next batch job is executed. This happens transparently to the application, and like other operations typically executes within milliseconds.
No Comments to Show