At Bitfusion, our job is to know how well various compute-intensive workloads scale on different infrastructures and to help people maximize performance. Since we launched our Deep Learning and CUDA AMIs in the AWS Marketplace we’ve heard many of our customers ask for bigger GPU instances, but the largest Amazon EC2 instance, the g2.8xlarge, currently maxes out at just 4 GPUs.
With a combination of our Bitfusion Boost remoting technology and the use of CloudFormation templates (step-by step Caffe CFN tutorial), we now allow you to easily spin up virtual instances with a lot more GPU power. For example, you can combine a large memory machine with an unlimited number of GPU nodes into a single more powerful virtual instance:
Currently we support OpenCL and CUDA based applications, but support for other APIs like OpenGL is just around the corner. Here is how it works:
The Boost runtime intercepts API calls, splits up the compute and data, and forwards the requests to a fast run-time scheduler to dispatch computation to both local and remote GPUs. As a result, you can combine the compute resources of various nodes into a single giant node. In fact, the GPU application doesn’t see any of this complexity; it just sees itself running on a single giant machine as shown in the figure below.
Attention: Since this post was published we have simplified the launching of the Monster Machines mentioned below even more. While you can still use the CFNs below if you would like, we strongly recommend that you refer to our latest tutorial post titled: Deploy Bitfusion Boost on AWS faster than ever to start these machines directly from the AWS Marketpalce.
Ready to take one of these big machines for a spin? You can try out the new G2 instance with 8 GPUs:
Feeling adventurous? Try out the massive R3 instance with 16 GPUs, 32 CPUs, and 244 GB of Memory:
To build a custom GPU instance type with a different combination of instances, we have you covered:
You can start any of these systems with pre-configured AMIs that feature Caffe or Torch, or you can start with a clean client AMI and install whatever GPU application you want. Over the next few weeks, we’ll share some case studies, performance results you can expect across several GPU apps, and several new product announcement.