Life on the Edge with ESI Kit Chan( ) Yahoo!

Life on the Edge with ESI
Kit Chan([email protected])
Yahoo!
Life on the Edge
Source: http://img.shockblast.net/2012/01/ShockBlast-4-803109.jpg
Agenda





Introduction
Use Cases
Performance Characteristics
Future
Q&A
Agenda





Introduction
Use Cases
Performance Characteristics
Future
Q&A
Edge Computing/CDN
Source: http://www.excitingip.com/wp-content/uploads/2011/01/contentdeliverynetworkblockdiagram.jpeg
CDN in Yahoo!
User Location Latency - 100K file in US Through Y! Edge
California, USA 0.2s
0.02s
Singapore
2s
0.02s
Korea
1.3s
0.02s
Ireland
1.2s
0.2s
Switzerland
1.0s
0.2s
- Pages: avg 30.2 Gbps send, 28.48 Gbps recv, 20.2% cache ratio
- Assets: avg 38.7 Gbps send, 4.1 Gbps recv. 96.7% cache hit ratio
Edge Computing in Yahoo!
User Location Latency - 100K file in US Through Y! Edge
California, USA 0.2s
0.02s
Singapore
2s
0.02s
Korea
1.3s
0.02s
Ireland
1.2s
0.2s
Switzerland
1.0s
0.2s
- Pages: avg 30.2 Gbps send, 28.48 Gbps recv, 20.2% cache ratio
- Assets: avg 38.7 Gbps send, 4.1 Gbps recv. 96.7% cache hit ratio
ESI – Edge Side
Includes
Syntax Example
Markup in Response
<esi:include src=http://xyz.com/content.html/>
Syntax Example
Can handle response error and timeout
<esi:try><esi:attempt>
<esi:include src=http://xyz.com/content.html/ >
</esi:attempt><esi:except>
Sorry there is an error!
</esi:except></esi:try>
ESI Support in Proxy Server – Proven & Popular
ESI Support in CDN Vendor
Agenda





Introduction
Use Cases
Performance Characteristics
Future
Q&A
Use Case 1 – Content Assembly
Use Case 1 – Content Assembly
Use Case 1 – Content Assembly
Use Case 1 - Content Assembly
Why? Performance!
Scenario
No server cache +
No ESI
Origin Server
Handling Full Page
requests each time
and executing every
module every time.
Server cache + No
ESI
Origin Server
handling full page
requests, only
executing non
cacheable modules
FE throughput per
box
Server latency
45 queries per
140 queries per
second
second
900 ms per request 80 ms per page
ATS Cache + ESI
modules
Origin Server only
serving requests for
non cacheable
modules
190 queries per
second
55 ms per page
Why? Availability
Why it goes down?
- Releases
- Outage
How ESI Help?
- Template cached
- Fallback for each ESI module
Why? Legacy Module Support
Use Case 2 – Combo Handler
 Allow us to fetch multiple JS/CSS in one HTTP request
 E.g. http://l.yimg.com/zz/combo?d/1.js&d/2.js
- It combines http://l.yimg.com/d/1.js and http://l.yimg.com/d/2.js
into one response in the server and returns one response
 Minimize HTTP requests from browser to increase performance
Combo Handler Sequence Diagram
Sample ESI Doc from Combo URL Decode Service
<esi:try><esi:attempt>
<esi:include src=http://l.yimg.com/d/1.js/>
</esi:attempt><esi:except>
/* error fetching 1.js */
</esi:except></esi:try>
<esi:try><esi:attempt>
<esi:include src=http://l.yimg.com/d/2.js/ >
</esi:attempt><esi:except>
/* error fetching 2.js */
</esi:except></esi:try>
Why?
 ATS with ESI Plugin
- Provide sophisticated features for HTTP resource caching
- Provide other features such as timeout, collapse forwarding for request
- Provide capability of fetching contents simultaneously and assembling
them together
Use Case 3 - Timely Launch
Before Launch
After Launch
Timely Launch – Before Launch
Timely Launch – Before Launch
Timely Launch – After Launch
Timely Launch – After Launch
Usage Pattern
ESI Service
Client
Proxy
Server/CDN
Origin Server
for ESI Include
Agenda





Introduction
Use Cases
Performance Characteristics
Future
Q&A
Concurrent Requests
First Byte Flush
ESI Support in Varnish
No concurrent requests for includes
First Byte Flush Support
ESI Support in Apache Traffic Server
Concurrent requests for includes
First Byte Flush Support*
Agenda





Introduction
Use Cases
Performance Characteristics
Future
Q&A
History
The spec (ESI 1.0) is published over 10 years ago.
- W3C spec
- Editor - Mark Nottingham from Akamai
Mark Nottingham
http://www.mnot.net/
Future – Deep HTTP Integration
Origin Server
returing ESI
Document
Client
Proxy Server
Supporting
ESI
Origin Server
for ESI Include
Future – Deep HTTP Integration
Origin Server
returing ESI
Document
Client
Proxy Server
Supporting
ESI
Origin Server
for ESI Include
Final response
header to user
cannot be changed
Future – Deep HTTP Integration
Request Header,
method, timeout
cannot be changed
for includes
Client
Origin Server
returing ESI
Document
Proxy Server
Supporting
ESI
Origin Server
for ESI Include
Final response
header to user can
be changed
Future – Deep HTTP Integration
Request Header,
method, timeout can
be changed for
includes
Client
Origin Server
returing ESI
Document
Proxy Server
Supporting
ESI
Origin Server
for ESI Include
Final response
header to user can
be changed
Request/Response
Header cannot be
retrieved as variables
Future – Smart Assembly
One request for multiple
includes (e.g. one Ad call
fetching multiple ad position)
Future – More Use cases
Device Detection
Bucket Testing
https://doazrm3qeh7n2.cloudfront.net/images/features/quick-a-vs-b-testing.jpg
http://illusionmedia.com/wp-content/uploads/2012/03/mobileDevices1300x200.jpg
Agenda





Introduction
Use Cases
Performance Characteristics
Future
Q&A
Q&A
Thank you
[email protected]
Reference
 ESI Language Specification – http://www.w3.org/TR/esi-lang
 Edge Architecture Specification - http://www.w3.org/TR/edge-arch
 ATS ESI plugin https://github.com/apache/trafficserver/blob/master/plugins/experimental/esi/README
 Varnish ESI - https://www.varnish-cache.org/docs/3.0/tutorial/esi.html
 Akamai ESI - http://www.akamai.com/dl/technical_publications/akamai_esi_extensions.pdf
 F5 - http://support.f5.com/kb/enus/products/wa/manuals/product/wa9_4policy/WA_Policy_9_4-19-1.html
Extra: Moving to the Real Edge? – the Clients
 Problem with SEO
 Performance - More HTTP requests between clients and edge servers