Amazon EC2 Spot Fleet API – Manage Thousands of Spot Instances with one Request

It has been really interesting to watch Amazon Elastic Compute Cloud (EC2) evolve over the last eight or nine years. At first you could launch a single instance type, in one region, at a predetermined (On-Demand) price. Today, you can launch a plethora of instance types, in any one of eleven regions (twelve including AWS GovCloud (US)), with your choice of On-Demand, Reserved, or Spot pricing (currently in nine regions). Along the way, we have added many features to EC2 and have also used it as a building block for other services including Amazon EMR, AWS Elastic Beanstalk, Amazon WorkSpaces, EC2 Container Service, and AWS Lambda.

New Spot Fleet API
Today we are making EC2’s Spot Instance model even more useful with the addition of a new API that allows you to launch and manage an entire fleet of Spot Instances with one request (a fleet is a collection of Spot Instances that are all working together as part of a distributed application. A fleet could be a batch processing job, a Hadoop workflow, or an HPC grid computing job). Many AWS customers launch fleets of Spot Instances (in sizes ranging from one instance up to thousands), using custom-written code that is responsible for discovering capacity, monitoring market prices across instance types and availability zones, and managing bids, all with the goal of running their workloads (ranging from large scale molecular dynamics simulations to continuous integration environments) at the lowest possible cost.

With today’s launch, this custom code is no longer necessary! Instead, a single API function (RequestSpotFleet) does all of the work on your behalf. You simply specify the fleet’s target capacity, a bid price per hour, and tell Spot what instance types you would like to launch. Spot will find the lowest priced spare EC2 capacity available, and work to achieve and maintain the fleet’s target capacity. One call does it all, as they say…

Making the Request
You can have up to 1,000 active Spot fleets per region, with a per-fleet and a per-region limit of 3,000 instances (the usual EC2 per-account and per-region limits are still in effect and will govern the number of instances that you can launch, the number of Amazon Elastic Block Store (EBS) volumes that you can create, and so forth).

Each request (via the API or the CLI) must include the following values:

• Target Capacity – The number of EC2 instances that you want in your fleet.
• Maximum Bid Price – The maximum bid price that you are willing to pay.
• Launch Specifications - The quantities and types of instances that you would like to launch, and how you want them to be configured (AMI Id, VPC, subnets or availability zones, security groups, block device mappings, user data, and so forth). In general, launch specifications that do not target a particular subnet or availability zone are more economical.
• IAM Fleet Role – The name of an IAM role. It must allow EC2 to terminate instances on your behalf.

Each request can also include any or all of the following optional values:

• Client Token – A unique, case-sensitive identifier for the request. You can use this to ensure idempotency for your Spot fleet requests.
• Valid From -The start date and time of the request.
• Valid Until – The end date and time of the request.
• Terminate on Expiration – If set to TRUE, all Spot instances in the fleet will be terminated when the Valid Until time is reached. If set to FALSE (the default), running Spot instances will be left as-is, but no new ones will be launched.

The RequestSpotFleet function will return a Spot Fleet Request Id if all goes well, or an error if the request is malformed. You will also receive an error if you ask for instance types that are not available in Spot form. You can use the Id to call other Spot fleet functions including DescribeSpotFleetRequests, DescribeSpotFleetInstances, DescribeSpotFleetRequestHistory, and CancelSpotFleetRequests (there are also command-line equivalents to each of these functions).

Behind the Scenes
Once your request has been accepted and the start date and time has been reached, EC2 will attempt to reach and then maintain the desired target capacity, even as Spot prices change. It will start by looking up the current Spot price for each launch specification in your request. Then it will launch Spot Instances using the launch specification(s) that result in the lowest price, until capacity, Spot limits, or bid price limits are reached. As instances in the fleet are terminated due to rising prices, replacement instances will be launched using whatever specification(s) result in the lowest price at that point in time.

The request remains active until it expires or you cancel it. The Spot Instances in the fleet will remain running unless you indicated that you wanted them to be terminated. As I mentioned earlier, you need to include an IAM role so that EC2 can terminate instances that are running on your behalf.

Things to Know
As is often the case with new AWS features, this is an initial release and we have a healthy backlog of features in the queue. For example, we plan to add a weighting system. It will allow you to express the relative power of each of your launch specifications in numeric form. The target capacity will also be expressed in these units; this will allow you to indicate that you need a certain amount of “horsepower” in a fleet.

Each fleet is run within a particular AWS region. In the future we would like to support fleets that span two or more regions.

Available Now
You can launch Spot fleets today in all public AWS regions where Spot is available. There is no charge for the Spot fleet; you pay Spot prices for the EC2 instances that you launch and any other resources that they consume.

— Jeff;