Pavan Podila
Director, Technology
Sapient Global Markets
Java AWT/Swing
WPF
iOS
Win32/MFC
Web
Eclipse SWT
WinForms
2002
2015
2005
KDE
Nokia QT
2011
Adobe Flex
Mac
UI
Fundamentals of
UI Engineering
Event Loop
Events
Drawing
Modes
Display Pipeline
Event Loop
Events
Drawing
Modes
Display Pipeline
Single Thread
Drawing
Application
Window Server
Operating System
Hardware
Events
Single Thread
Event Loop
5
4
3
2
1
Event Loop
5
4
3
2
1
Event Loop
Main
3
2
1
Event Loop
3
2
1
Main
Event Loop
60 FPS
10
16 ms
5 FPS
15 FPS
60 FPS
https://frames-per-second.appspot.com/
Event Loop
Events
Drawing
Modes
Display Pipeline
Event Loop
Events
Drawing
Modes
Display Pipeline
Drawing Modes
Immediate
Retained
Immediate
Retained
Immediate
Retained
<s
<
<
Imperative and State-less
<
<
<
<
Declarative and State-full
(tree-model)
Redraw
Layer-based
drawing
Manual Invalidation
Property-based Invalidation
Canvas, WebGL, CoreGraphics
DOM, SVG, SceneKit
Immediate VS Retained
Event Loop
Events
Drawing
Modes
Display Pipeline
Event Loop
Events
Drawing
Modes
Display Pipeline
Window Server
Operating System
Events
Immediate
Manual Hit-Testing
Retained
Event Bubbling/Capturing
Event Loop
Events
Drawing
Modes
Display Pipeline
Event Loop
Events
Drawing
Modes
Display Pipeline
Display Pipeline
Display Pipeline
Affine
Transforms
Drawing
Image Decoding
Geometry
Colorspace
Conversion
GPU
Texture
Generation
Build Visual Tree
Style
Recalculation
Blending
Font Metrics
+ Glyphs
Vector
Rasterization
Antialiasing
Video Decoding
Layout
Painting
Style
Recalculation
Drawing
Geometry
Video Decoding
Font Metrics
+ Glyphs
Antialiasing
Build Visual Tree
Image Decoding
Vector
Rasterization
Colorspace
Conversion
Compositing
Affine
Transforms
GPU
Texture
Generation
Blending
Display Pipeline
Layout
Painting
Compositing
Display Pipeline
Layout
Painting
Animation
Compositing
Layout
•
First step in the pipeline
•
Measure recursively down
•
Arrange in space with origin + size
•
Generally done on the main-thread, but could be
offloaded to a separate thread.
Painting
•
Generally accounts for 2D painting
•
Primitives include
•
Fills and Gradients
•
Strokes
•
Paths (Shapes)
•
Images, Patterns
•
Immediate-mode operations (cursor driven)
•
Examples: CoreGraphics, Skia (Chrome), Direct2D
•
Historically painting was done on main-thread but now moved on to a Renderthread. This is done by recording operations from main-thread and playing
them on render-thread. Also gives parallelism for more complex drawing.
Compositing
Textures
CPU
GPU
Compositing
Textures
CPU
DOM
GPU
Render Objects
Render Layers
Graphic Layers
GPU
Event Loop
Events
Drawing
Modes
Display Pipeline
Learn more…
The Pursuit of 60 FPS
https://www.youtube.com/watch?v=3Bq521dIjCM
Why Multiple Threads are a Bad Idea!
http://web.stanford.edu/~ouster/cgi-bin/papers/threads.pdf
Jank Free
http://jankfree.org/
Chromium Design Documents
https://www.chromium.org/developers/design-documents
Immediate vs Retained mode
https://msdn.microsoft.com/en-us/library/windows/desktop/ff684178%28v=vs.85%29.aspx
@pavanpodila
blog.pixelingene.com
www.quicklensapp.com