Expanding the Cloud: Amazon Machine Learning Service, the Amazon Elastic Filesystem and more

Today was a big day for the Amazon Web Services teams as a whole range of new services and functionality was delivered to our customers. Here is a brief recap of it:

The Amazon Machine Learning service As I wrote last week machine learning is becoming an increasingly important tool to build advanced data driven applications. At Amazon we have hundreds of teams using machine learning and by making use of the Machine Learning Service we can significantly speed up the time they use to bring their technologies into production. And you no longer need to be a machine learning expert to be able to use it.

Amazon Machine Learning is a service that allows you easily to build predictive applications, including fraud detection, demand forecasting, and click prediction. Amazon ML uses powerful algorithms that can help you create machine learning models by finding patterns in existing data, and using these patterns to make predictions from new data as it becomes available. The Amazon ML console and API provide data and model visualization tools, as well as wizards to guide you through the process of creating machine learning models, measuring their quality and fine-tuning the predictions to match your application requirements. Once the models are created, you can get predictions for your application by using the simple API, without having to implement custom prediction generation code or manage any infrastructure. Amazon ML is highly scalable and can generate billions of predictions, and serve those predictions in real-time and at high throughput. With Amazon ML there is no setup cost and you pay as you go, so you can start small and scale as your application grows. Details on the AWS Blog

The Amazon Elastic File System

AWS has been offering a range of storage solutions: objects, block storage, databases, archiving, etc. for a while already. Customers have been asking to add file system functionality to our set of solutions as much of their traditional software required a broad accessible shared file system. When we designed Amazon EFS we decided to build along the AWS principles: Elastic, scalable, highly available, consistent performance, secure, and cost-effective.

Amazon EFS is a fully-managed service that makes it easy to set up and scale shared file storage in the AWS Cloud. With a few clicks in the AWS Management Console, customers can use Amazon EFS to create file systems that are accessible to EC2 instances and that support standard operating system APIs and file system semantics. Amazon EFS file systems can automatically scale from small file systems to petabyte-scale without needing to provision storage or throughput. Amazon EFS can support thousands of concurrent client connections with consistent performance, making it ideal for a wide range of uses that require on-demand scaling of file system capacity and performance. Amazon EFS is designed to be highly available and durable, storing each file system object redundantly across multiple Availability Zones. With Amazon EFS, there is no minimum fee or setup costs, and customers pay only for the storage they use. Details on the AWS Blog.

The Amazon ECS Container Service

Containers are an important building block in modern style of software development and since the launch of Amazon ECS last year November it has become a very important tool for architects and developer. Today Amazon ECS moves into General Availability (GA) so you can use it for your certified production systems.

With going GA Amazon ECS also delivers a new scheduler to support long running application, see my detail block post over here: State Management and Scheduling with the Amazon EC2 Container Service Also read the details on the AWS Blog

Amazon Lambda One of the most exciting technologies we have built lately at AWS is Amazon Lambda. Developers really have flocked to using this serverless programming technology to build event driven services. A great example by Valentino in the AWSSummit today was the use of Lambda by Adroll to deliver real-time updates around the word to their DynamoDB instances. Today Amazon Lambda is entering General Availability. Two areas where Lambda is driving a lot of innovation is Mobile and the Internet of Things (IoT). We have taken feedback from our customers and driven much innovation to extend it with great new functionality:

• Synchronous Events – You can now create AWS Lambda functions that respond to events in your application in real time (synchronous) as well as asynchronously. Synchronous requests allow mobile and IoT apps to move data transformations and analysis to the cloud and make it easy for any application or web service to use Lambda to create back-end functionality. Synchronous events operate with low latency so you can deliver dynamic, interactive experiences to your users. To learn more about using synchronous events, read Getting Started: Handling Synchronous Events in the AWS Lambda Developers Guide.
• AWS Mobile SDK support for AWS Lambda (Android, iOS) – AWS Lambda is now included in the AWS Mobile SDK, making it easy to build mobile applications that use Lambda functions as their app backend. When invoked through the mobile SDK, the Lambda function automatically has access to data about the device, app, and end user identity, making it easy to create rich, personalized responses to in-app activity. To learn more, visit the AWS Mobile SDK page.
• Target, Filter, and Route Amazon SNS Notifications with AWS Lambda – You can now invoke a Lambda function by sending it a notification in Amazon SNS, making it easy to modify or filter messages before routing them to mobile devices or other destinations. Apps and services that already send SNS notifications, such as Amazon CloudWatch, gain automatic integration with AWS Lambda through SNS messages without needing to provision or manage infrastructure.
• Apply Custom Logic to User Preferences and Game State – Amazon Cognito makes it easy to save user data, such as app preferences or game state, in the AWS Cloud and synchronize it among all the user’s devices. You can now use AWS Lambda functions to validate, audit, or modify data as it is synchronized and Cognito will automatically propagate changes made by your Lambda function to the user’s devices. End user identities created using Amazon Cognito are also included in Lambda events, making it easy to store or search for customer-specific data in a mobile, IoT, or web backend.
• AWS CloudTrail Integration – Lambda now supports AWS CloudTrail logging for API requests. You can also use AWS Lambda to automatically process CloudTrail events to add security checks, auditing, or notifications for any AWS API call.
• Enhanced Kinesis Stream Management – You can now add, edit and remove Kinesis streams as event sources for Lambda functions using the AWS Lambda console, as well as view existing event sources for your Lambda functions. Multiple Lambda functions can now respond to events in a single Kinesis or DynamoDB stream.
• Increased Default Limits – Lambda now offers 100 concurrent executions and 1,000 TPS as a default limit and you can contact customer service to have these limits quickly raised to match your production needs.
• Enhanced Metrics and Logging – In addition to viewing the number of executions of your Lambda function and its error rate and duration, you can now also see throttled attempts through a CloudWatch metric for each function. Amazon CloudWatch Logs now also support time-based sorting, making it easier to search Lambda logs and correlate them with CloudWatch metrics. API enhancements make it easier to distinguish problems in your code (such as uncaught top-level exceptions or timeouts) from errors you catch and return yourself.
• Simplified Access Model and Cross-Account Support – Lambda now supports resource policies and cross-account access, making it easier to configure event sources such as Amazon S3 buckets and allowing the bucket owner to be in a separate AWS account. Separate IAM roles are no longer required to invoke a Lambda function, making it faster to set up event sources.

We will also launch Java as a programming language to be used in lambda in a few weeks. More details on the AWS Blog

For more details on these new services visit the official AWS Blog and the What's New Section on the AWS website