Provision on-demand Spark clusters on Docker using Azure Batch's infrastructure

added by DotNetKicks
10/4/2017 5:09:23 PM

1 Kicks, 333 Views

Since its release 3 years ago, Apache Spark has soared in popularity amongst Big Data users, but is also increasingly common in the HPC space. However, spinning up a Spark cluster, on-demand, can often be complicated and slow. Instead, Spark developers often share pre-existing clusters managed by their company's IT team.