1. technology and computing
2. software
3. databases

STOCKHOLM
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
AWS Lambda
Amazon Aurora
Danilo Poccia, Technical Evangelist, AWS –
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
@danilop
INNOVATION
AWS Lambda
Stateless Functions in the Cloud
AWS Lambda “Run stateless functions in the cloud and scale without any servers to manage”
Zero Administration
Focus on business logic,
not infrastructure. Upload
your code; AWS Lambda
handles everything else
Auto Scaling
(Never under or over
provision)
Lambda scales the
infrastructure as needed
to match the event rate
and pay as you go
Bring Your Own Code
Starting with JavaScript but
later any language, Create
threads and processes, run
batch scripts or other
executables
How can you put AWS Lambda to work? Serverless
backend
Data triggers
Mobile/IoT
Stream
processing
Indexing &
synchronization
Case Study: AWS Lambda on Social Media Aggrega>on CommandPost is CMP.LY’s patented
Monitoring, Measurement & Insights (MMI)
tool for managed social communications.
Increase the amount of data
processed while reducing(!)
the resources (instances)
required to do it
AWS Lambda Now Ready for Production at Scale
Respond to events in real-­‐0me Map local func0ons to cloud func0ons from within the SDK Target, Filter, and Route Amazon SNS No0fica0ons
Kinesis
events
S3 event
notifications
DynamoDB
Streams
Cognito
events
Apply Custom Logic to User Preferences and Game State SNS
events
Custom
events
Java Support (Coming soon), CloudTrail integra0on, Enhanced metrics and logging via CloudWatch
J
Map local
functions to
invoke
Lambda
Functions
synchronously
Amazon
Cognito Sync
Dataset
SNS
Personalize your
notification for
every user
Push notification
Maintain
Intelligence
in the cloud
and not the
device
Amazon
DynamoDB
Table
Chain
multiple
Functions or
call them in
Parallel
AWS Lambda Event Sources
Amazon
S3
Amazon
DynamoDB
Amazon
Kinesis
Amazon
SNS
Amazon
Cognito
AWS Lambda Event Sources
Amazon
S3
Amazon
DynamoDB
Amazon
Kinesis
Amazon
SNS
Content / Media Upload
(Thumbnails, Video Transcoding, OCR)
Anything logging on S3
(Elastic Load Balancer, AWS CloudTrail)
Amazon
Cognito
AWS Lambda Event Sources
Amazon
S3
Amazon
DynamoDB
Amazon
Kinesis
Amazon
SNS
Amazon
Cognito
React to a change on a Table, including:
changes from another Lambda function
direct changes from a Mobile / JavaScript Application
(Fine-grained Access Control via a IAM Policy)
AWS Lambda Event Sources
Amazon
S3
Amazon
DynamoDB
Amazon
Kinesis
Amazon
SNS
Analyze your data stream in almost real-time
React quickly with custom logic
(Inject data into CloudWatch Logs)
Amazon
Cognito
AWS Lambda Event Sources
Amazon
S3
Amazon
DynamoDB
Amazon
Kinesis
Amazon
SNS
Anything publishing on SNS, including:
Amazon CloudWatch Alarms
Amazon RDS Events
Amazon
Cognito
AWS Lambda Event Sources
Amazon
S3
Amazon
DynamoDB
Amazon
Kinesis
Amazon
SNS
Get all changes on a Dataset
Can be your asynchronous mobile backend
Amazon
Cognito
Calling AWS Lambda Directly – Custom Events
Native
Mobile
Applications
Client-side
JavaScript
Web Application
Server-based
Backend
Applications
Adding a Lambda Backend
to your Mobile App is simple
Initialize the LambdaFactory and define the Interface for the functions
lambda = new LambdaInvokerFactory(context, Regions.US_WEST_2, provider);
//interface
@LambdaFunction(functionName="cloudFunction”)
String localFunction(String nameInfo);
Create/Upload the Lambda Function to the AWS Management Console
exports.handler = function(event, context) {
context.done(null, event + 'Lambda'); // SUCCESS with message
};
Call synchronize on the dataset
lambda.localFunction(“Hello From “); // this will return “Hello From Lambda”
Customer Driven Innovation
F-Secure Telemetry
Service
Juhana Enqvist, Lead Architect, Big Data
Heikki Nousiainen, Lead Architect, Cloud
OVER 25 YEARS
NORDIC HERO ON
SECURITY AND
PRIVACY
.
PROBLEM STATEMENT
Customer Lifecycle
Acquire Free use Pay Use Renewal § Better understand the Customer using our Products and Services, while
§ Respect and Protect their Privacy and Identity
21
© F-Secure
3-Stage Real-Time Pipeline
Unified
Raw Data
Zone
S
D
K
Mobile
Clients
Analysed
Data
Zone
Product Teams
Dashboards
Comms
Service
Provider
DynamoDB
S
D
K
Desktop
Clients
S
D
K
Analytics
Service
ELB
Ingestion
Kinesis
ASG
Analysis
State
Routing
Kinesis
Aggregated
Feeds
ASG
Backend
Services
22
Formatting
Forwarding
ASG
ASG
S3
S3
© F-Secure
ASG
Identity
Management
Service
Identity Decoupling
Service
No User Identifiers
Stateful
Data
Realtime
Data
Pipeline
Internal Tools &
Dashboards
Aggregated
Partner feeds
Aggregated
Partner feeds
Another
Marketing
Partner?
Another
Analytics
Product
Backends
S3
Access Control
23
Messaging
Service
Only essential data
No Personal Data
Opt-In Required
Product
Clients
Analytics
Service
© F-Secure
Tool?
More Internal
Tools?
AWS BENEFITS
§ Customer Experience
§  People expect rely on 100% availability
1)  Multiple availability zones with low latencies
2)  Programmable responses to a wide range of failure conditions
3)  Leveraging AWS solutions that have been engineered for these requirements
24
© F-Secure
AWS BENEFITS
§ Productivity
§  Focus on differentiation, value creation; buy “plumbing” from the experts
§  Everything-as-Code. Quality Assurance for all of infrastructure.
§  Deploy small, deploy often. React to customer and market needs faster.
25
© F-Secure
Thank You
Juhana Enqvist, Lead Architect, Big Data
Heikki Nousiainen, Lead Architect, Cloud
Amazon Aurora
Amazon’s New Relational Database Engine
Current DB architectures are monolithic
SQL
Transactions
Caching
Logging
Multiple layers of
functionality all on a
single box
Current DB architectures are monolithic
Application
SQL
SQL
Transactions
Transactions
Caching
Caching
Logging
Logging
Even when you scale
it out, you’re still
replicating the same
stack
Current DB architectures are monolithic
Application
SQL
SQL
Transactions
Transactions
Caching
Caching
Logging
Logging
Even when you scale
it out, you’re still
replicating the same
stack
Current DB architectures are monolithic
Application
SQL
SQL
Transactions
Transactions
Caching
Caching
Logging
Logging
Storage
Even when you scale
it out, you’re still
replicating the same
stack
This is a problem.
For cost. For flexibility. And for availability.
Reimagining the relational database
What if you were inventing the database today?
You wouldn’t design it the way it was done in 1970.
At least not entirely.
You’d build something that can scale out, that is self-healing,
and that leverages existing AWS services.
A service-oriented architecture applied to the database
Data Plane
• 
• 
• 
Moved the logging and storage layer
into a multi-tenant, scale-out
database-optimized storage service
Integrated with other AWS services
like Amazon EC2, Amazon VPC,
Amazon DynamoDB, Amazon SWF,
and Amazon Route 53 for control
plane operations
Control Plane
SQL
Transactions
Caching
Amazon
DynamoDB
Logging + Storage
Amazon SWF
Integrated with Amazon S3 for
continuous backup with
99.999999999% durability
Amazon S3
Amazon Route 53
Aurora preview
•  Sign up for preview access at:
https://aws.amazon.com/rds/aurora/preview
•  Soon available in US West (Oregon) and EU (Ireland),
in addition to US East (N. Virginia)
•  Thousands of customers already invited to the limited preview
•  Now moving to unlimited preview; accepting all requests in 2–3 weeks
•  Full service launch in the coming months
Aurora Works with Your Existing Apps
Establishing our ecosystem
“It is great to see Amazon Aurora remains MySQL compatible; MariaDB connectors
work with Aurora seamlessly. Today, customers can take MariaDB Enterprise with
MariaDB MaxScale drivers and connect to Aurora, MariaDB, or MySQL without worrying about
compatibility. We look forward to working with the Aurora team in the future to further
accelerate innovation within the MySQL ecosystem.”— Roger Levy, VP Products, MariaDB
Business Intelligence
Data Integration
Query and Monitoring
SI and Consulting
Amazon Aurora Is Easy to Use
Simplify database management
•  Create a database in minutes
•  Automated patching
•  Push-button scale compute
•  Continuous backups to S3
•  Automatic failure detection and failover
Amazon RDS
Simplify storage management
•  Read replicas are available as failover targets—no data loss
•  Instantly create user snapshots — no performance impact
•  Continuous, incremental backups to S3
•  Automatic storage scaling up to 64 TB — no performance
or availability impact
•  Automatic restriping, mirror repair, hot spot management, encryption
Simplify data security
• 
Encryption to secure data at rest
– 
– 
– 
• 
• 
AES-256; hardware accelerated
All blocks on disk and in Amazon S3 are encrypted
Key management via AWS KMS
AZ 2
AZ 1
Customer
VPC
MySQL App
SSL to secure data in transit
Primary
Instance
Network isolation via Amazon VPC by
default
• 
No direct access to nodes
• 
Supports industry standard security
and data protection certifications
Replica
Instance
Internal
VPC
Amazon S3
AZ 3
Amazon Aurora Is Highly Available
Aurora storage
• 
Highly available by default
– 
– 
6-way replication across 3 AZs
4 of 6 write quorum
• 
– 
AZ 1
AZ 2
Automatic fallback to
3 of 4 if an AZ is unavailable
3 of 6 read quorum
SQL
Transactions
• 
SSD, scale-out, multi-tenant storage
– 
– 
– 
• 
Seamless storage scalability
Up to 64 TB database size
Only pay for what you use
Caching
Log-structured storage
– 
– 
– 
Many small segments, each with
their own redo logs
Log pages used to generate data pages
Eliminates chatter between
database and storage
Amazon S3
AZ 3
Consistent, low-latency writes
MySQL with standby
AZ 1
2 phase commit
Primary
Instance
PiTR
Amazon Aurora
AZ 2
AZ 1
AZ 2
Standby
Instance
Primary
Instance
Replica
Instance
async
4/6 quorum
Amazon Elastic
Block Store
(EBS)
EBS
Sequential
write
EBS
mirror
Sequential
write
Distributed
writes
EBS
mirror
Amazon S3
Improvements
• 
Consistency—tolerance to outliers
• 
Latency—two-phase commit vs. asynchronous replication
Amazon S3
Type of writes
Log records
Binlog
Data
• 
Significantly more efficient use of network I/O
Double-write buffer
FRM files,
metadata
AZ 3
Self-healing, fault-tolerant
• 
Lose two copies or an AZ failure without read or write availability impact
• 
Lose three copies without read availability impact
• 
Automatic detection, replication, and repair
AZ 1
AZ 2
AZ 3
AZ 1
SQL
SQL
Transaction
Transaction
Caching
Caching
Read availability
AZ 2
AZ 3
Read and write availability
Instant crash recovery
Traditional databases
Amazon Aurora
• 
Have to replay logs since the last
checkpoint
• 
Underlying storage replays redo
records on demand as part of a
disk read
• 
Single-threaded in MySQL;
requires a large number of disk
accesses
Crash at T requires
• 
Parallel, distributed, asynchronous
0
a re-application of the
SQL in the redo log since
last checkpoint
Checkpointed Data
Crash at T0 will result in redo
logs being applied to each segment
on demand, in parallel, asynchronously
Redo Log
T0
T0
Survivable caches
• 
• 
We moved the cache out of
the database process
Cache remains warm in the
event of a database restart
• 
Lets you resume fully loaded
operations much faster
• 
Instant crash recovery +
survivable cache = quick and
easy recovery from DB
failures
Caching process is outside the DB process
and remains warm across a database restart
SQL
SQL
SQL
Transactions
Transactions
Transactions
Caching
Caching
Caching
Multiple failover targets — no data loss
MySQL Replica
Aurora Master
70% Write
70% Write
30% Read
30% New Reads
30% Read
Data Volume
Data Volume
MySQL Master
70% Write
Single-threaded
binlog apply
MySQL read scaling
•  Replicas must replay logs
•  Replicas place additional load on master
•  Replica lag can grow indefinitely
•  Failover results in data loss
Page cache
invalidation
Aurora Replica
100% New
Reads
Shared Multi-AZ Storage
Simulate failures using SQL
• 
To cause the failure of a component at the database node:
ALTER SYSTEM CRASH [{INSTANCE | DISPATCHER | NODE}]
• 
To simulate the failure of disks:
ALTER SYSTEM SIMULATE percent_failure DISK failure_type IN
[DISK index | NODE index] FOR INTERVAL interval
• 
To simulate the failure of networking:
ALTER SYSTEM SIMULATE percent_failure NETWORK failure_type
[TO {ALL | read_replica | availability_zone}] FOR INTERVAL interval
Amazon Aurora Is Fast
Write performance (console screenshot)
•  MySQL Sysbench
•  R3.8XL with 32 cores
and 244 GB RAM
•  4 client machines with
1,000 threads each
Read performance (console screenshot)
•  MySQL Sysbench
•  R3.8XL with 32 cores
and 244 GB RAM
•  Single client with
1,000 threads
Read replica lag (console screenshot)
•  Aurora Replica with 7.27 ms replica lag at 13.8 K updates/second
•  MySQL 5.6 on the same hardware has ~2 s lag at 2 K updates/second
Writes scale with table count
Tables
Amazon
Aurora
MySQL
I2.8XL
Local SSD
MySQL
I2.8XL
RAM Disk
RDS MySQL
30K IOPS
(Single AZ)
10
60,000
18,000
22,000
25,000
100
66,000
19,000
24,000
23,000
1,000
64,000
7,000
18,000
8,000
10,000
54,000
4,000
8,000
5,000
Thousands of Writes per Second
Write Performance and Table Count
70
60
50
40
30
20
10
10
Write-only workload
1,000 connections
Query cache (default on for Amazon Aurora, off for MySQL)
100
1,000
Number of Tables
Aurora
MySQL on I2.8XL
MySQL on I2.8XL with RAM Disk
RDS MySQL with 30,000 IOPS (Single AZ)
10,000
Better concurrency
Write Performance and Concurrency
Connections
Amazon
Aurora
RDS MySQL
30K IOPS
(Single AZ)
50
40,000
10,000
500
71,000
21,000
5,000
110,000
13,000
Thousands of Writes per Second
120
100
80
60
40
20
-
OLTP Workload
Variable connection count
250 tables
Query cache (default on for Amazon Aurora, off for MySQL)
50
500
5,000
Concurrent Connections
Aurora
RDS MySQL with 30,000 IOPS (Single AZ)
INNOVATION
STOCKHOLM
@danilop