Enhancing Your Unity Mobile Games
Roberto Lopez Mendez – Senior Engineer, ARM
Angelo Theodorou – Senior Engineer, ARM
Tony Prosser – Managing Director, RealtimeUK
Carl Callewaert – Americas Director, Unity Technologies
1
Agenda





2
Introduction (Roberto)
Notes About Enhancing Your Mobile Games (Angelo)
Reflections and shadows based on local cubemaps (Roberto)
Project Ice Cave: from hours to milliseconds (Tony)
Unity 5 and Physically Based Shading (Carl)
Notes About Enhancing Your Mobile Games
Angelo Theodorou
Senior Engineer, ARM
3
Agenda
 Some differences when developing for mobile
 Various bottlenecks
 Notes About Enhancing Your Mobile Games





4
Best practices for ARM® Mali™ GPUs
General optimization tips
Coding guidelines
Profiling considerations
Contents of The ARM Guide to Unity
Introduction
Some differences when developing for mobile
 Bandwidth:
 Desktop: huge amount of bandwidth (>100 GB/s) between GPU and dedicated video memory
 Mobile: reduced amount of bandwidth (> 5 GB/s) available
 Power efficiency:
 Desktop: constantly plugged to a power source
 Mobile: portable devices with batteries
 Render architecture:
 Desktop: immediate mode – graphics commands are executed when issued
 Mobile: mostly deferred mode – graphics commands are collected by the driver and issued later
 Memory subsystem:
 Desktop: most of the time the CPU and the GPU have physically separated memories
 Mobile: both the CPU and the GPU access to a unified memory system
5
Introduction
System elements and application bottlenecks
 CPU
 Amount of draw calls
 Scripts or physics complexity
 Vertex processing
 Vertex count
 Computation per vertex
 Fragment processing
 Fragment count and overdraw
 Computation per fragment
 Bandwidth
 Size and format of textures
 Framebuffer resolution
6
First bottleneck is the CPU
Bottlenecks equally distributed
Notes About Enhancing Your Mobile Games
Mali GPU specific notes
 On Mali GPUs 4x multi sampling comes virtually for free
 Mali-T62X GPUs and onward support Adaptive Scalable Texture Compression (ASTC)
7
Notes About Enhancing Your Mobile Games
General optimization tips
 Avoid the use of GameObject.tag property
GameObject mainCamera = GameObject.Find("Main Camera");
// Gameobject.tag property
if(mainCamera.tag == "MainCamera") { // Do something }
 Prefer the GameObject.CompareTag() method
// Gameobject.CompareTag() method
if(mainCamera.CompareTag("MainCamera")) { // Do something }
8
Notes About Enhancing Your Mobile Games
Coding guidelines
 Avoid the use of hard-coded strings, for example when dealing with tag names
 Easy to misspell, hard to consistently modify: the very opposite of robustness and scalability 
if(gameObject.CompareTag("Player"))
 Gather all your tag names in a public class that exposes constant strings
public class Tags
{
public const string Player = "Player";
public const string Enemy = "Enemy";
}
if(gameObject.CompareTag(Tags.Player))
9
Notes About Enhancing Your Mobile Games
Profiling considerations
 Mark a region of your script with Profiler.BeginSample()/EndSample()
 This way you avoid the overhead of the “Deep Profile” option
void Update()
{
Profiler.BeginSample("ProfiledSection");
[...]
Profiler.EndSample();
}
10
Notes About Enhancing Your Mobile Games
Contents of The ARM Guide to Unity
 Developing for mobile is not the same as developing for desktop
 Unity eases the deployment on different platforms…
 …but sometimes this automatic process is not enough
 The Unity guide by ARM provides you with:





Mali GPU specific notes
General optimization tips
Coding guidelines
Profiling considerations
Screenshots, code snippets, useful links…
 http://malideveloper.arm.com/develop-for-mali/arm-guide-unity-enhancing-mobile-games/
11
Reflections and Shadows Based on Local
Cubemaps
Roberto Lopez Mendez
Senior Engineer, ARM
12
Content
 Reflections based on local cubemaps
 Reflections with infinite cubemaps
 Local correction to reflections based on cubemaps
 Combining reflections based on local cubemaps with reflections rendered at runtime
 New shadows technique: dynamic soft shadows based on local cubemaps





13
The foundations of shadows based on local cubemaps
Why dynamic? Why soft?
Benefits and limitations of shadows based on local cubemaps
Combining shadows based on local cubemaps with shadows rendered at runtime
Wrap up
Reflections with Infinite Cubemaps
Normal N and view vector D are
passed to fragment shader from the
vertex shader.
cubemap
R
N
R
In the fragment shader the texture
colour is fetched from the cubemap
using the reflected vector R:
D
float3 R = reflect(D, N);
float4 col = texCUBE(Cubemap, R);
Reflective surface
14
Incorrect Reflections
Reflection generated using a cubemap without any local binding
15
Local Correction Using a Bounding Box as a Proxy Geometry
cubemap
C
R’
float3 R = reflect(D, N);
P
N
R
D
float4 col = texCUBE(Cubemap, R);
Find intersection point P
Find vector R’ = CP
Float4 col = texCUBE(Cubemap, R’);
Reflective surface
Bounding Box
GPU Gems. Chapter 19. Image-Based Lighting. Kevin Bjork, 2004. http://http.developer.nvidia.com/GPUGems/gpugems_ch19.html
Cubemap Environment Mapping. 2010. http://www.gamedev.net/topic/568829-box-projected-cubemap-environment-mapping/?&p=4637262
Image-based Lighting approaches and parallax-corrected cubemap. Sebastien Lagarde. SIGGRAPH 2012. http://seblagarde.wordpress.com/2012/09/29/image-based-lightingapproaches-and-parallax-corrected-cubemap/
16
Correct Reflections
Reflection generated after applying the “local correction”
17
Reflection generated without “local correction”
Infinite and Local Cubemaps
Infinite Cubemaps
• They are used to represent the
lighting from a distant environment.
• Cubemap position is not relevant.
Local Cubemaps
• They are used to represent the
lighting from a finite local
environment.
• Cubemap position is relevant.
• The lighting from these cubemaps is
right only at the location where the
cubemap was created.
• Local correction must be applied to
get the right local reflections.
18
Filtering Cubemaps to Achieve Visual Effects
Development Stage
The cost of the process depends on
filter complexity and cubemap resolution
19
Runtime
Just use the filtered cubemap
Reflections Based on Filtered Local Cubemap
Reflection as rendered with frosty effect
20
Handling Reflections From Different Types of Geometries
Reflection from
Skybox
Reflection from
Static Geometry
Reflection from
Dynamic Geometry
Use Infinite
Cubemap technique
Use Local Cubemap
technique
Use Virtual Reflection
Camera technique
All Reflections
Combine all types of
reflections
21
Combined reflections
22
Dynamic Soft Shadows
23
Dynamic Soft Shadows Based on Local Cubemaps
Generation stage
Top
+Y
Render the
transparency of the
scene in the alpha
channel
Y
Left –X
Front
-Z
Bottom
-Y
X
Back
+Z
Opaque geometry is rendered with alpha = 1.
We have a map of the
zones where light rays
can potentially come
from and reach the
geometry.
Semi-transparent geometry is rendered with
alpha different from 1.
No light information is
processed at this stage.
Z
Camera background alpha colour = 0.
Fully transparent geometry is rendered with
alpha 0.
24
Right
+X
Dynamic Soft Shadows Based on Local Cubemaps
Runtime stage
• Create a vertex to light source L vector in the vertex shader.
• Pass this vector to the fragment shader to obtain the vector
from the pixel to the light position piL.
L
Q
P
cubemap
• Find the intersection of the vector piL with the bounding box.
C
shadowed
pixel
pk
lit pixel
pi
• Build the vector CP from the cubemap position C to the
intersection point P.
• Use the new vector CP to fetch the texture from the
cubemap.
float texShadow = texCUBE(_CubeShadows, CP).a;
Bounding Box
Source code in the update of ARM Guide to Unity at Unite Europe 2015.
25
Dynamic Soft Shadows Based on Local Cubemaps
Why dynamic?
If the position of the light source
changes the shadows are generated
correctly using the same cubemap.
26
Dynamic Shadows
27
Dynamic Soft Shadows Based on Local Cubemaps
Why soft?
The further from
the object the softer
the shadows
28
Dynamic Soft Shadows Based on Local Cubemaps
Why soft?
float texShadow = texCUBE(_CubeShadows, CP).a;
L
Q
P
float4 newVec = float4(CP, factor * length(piP))
cubemap
C
float texShadow = texCUBElod(_CubeShadows, newVec ).a;
shadowed
pixel
pk
lit pixel
pi
Bounding Box
Source code in the update of ARM Guide to Unity at Unite Europe 2015.
29
Soft shadows
30
Dynamic Soft Shadows Based on Local Cubemaps
Benefits
1. Simple to implement.
2. Very realistic and physically correct.
Limitations
1. Works fine in open space with no geometry
in the centre from where the cubemap will
more likely be generated.
3. High quality of shadows.
4. Cubemap texture can be compressed.
5. Offline filtering effects can be applied which
could be very expensive at run time.
6. Resource saving technique compared with
runtime generated shadows.
7. Very tolerant of deviations from the
bounding box shape when compared with
reflections.
31
2. Objects in the scene must be close to the
proxy geometry when generating static
texture for good results.
3. Does not reflect changes in dynamic objects
unless we can afford to update the cubemap
at runtime.
Handling Shadows from Different Types of Geometries
Shadows from Static
Geometry
Shadows from
Dynamic Geometry
Use Local Cubemap
technique
Use Shadow Mapping
technique
All Shadows
Combine both types
of shadows
32
Combined shadows
33
Wrap Up
 Reflections and shadows based on local cubemaps are resource saving
techniques that work great in mobile devices where available resources
must be carefully balanced.
 These techniques can be effectively combined with runtime techniques to
render static and dynamic objects together.
 Both techniques are currently used in the Ice Cave demo together with
other FX as fog, light shafts, procedural skybox, Enlighten in custom
shaders, fireflies, dirty lens, etc.
Source code in the update of ARM Guide to Unity at Unite Europe 2015.
34
Any Questions?
Ask the best question and win a PiPO P4 tablet!
 Rockchip RK3288 processor
 ARM Cortex-A17 MP4 CPU
 ARM Mali-T760 MP4 GPU
35
Project Ice Cave
From Hours to Milliseconds
Tony Prosser
Managing Director Real Time
36
Who is RealtimeUK
 Our Mission:
“To be a leading creative
production studio by
delivering exceptional quality
& pioneering cutting edge
solutions for the video games
industry”
37
Demo Reel
http://www.realtimeuk.com/video/games-showreel/
38
ARM Brief
 Target devices
 Glacier cave inspired by
Chinese culture
 Ghost tiger
 Refractive/reflective
surfaces
 Glowing assets and VFX
 Texture and Tri budgets
39
Art Direction - Concept Art
 Technical meetings before
concept art
 Adapt concepts based on tri
budget
 Agreed on colour palette
 Experimenting with
scales/placements on proxy
geometries
40
Asset Creation
 Use similar techniques with
our pre-rendered world
 Generate maps such as
normal and diffuse maps
 Rig and animate characters
 Create in-game effects with
a more artistic lead
approach
41
Visual Benchmark
 We rendered the cave using
pre-rendered techniques
 Shared our shader with
ARM in order to help them
replicate a similar look
within Unity
42
Visual Benchmark Movie
43
Demo Ice Cave
44
Demo Ice Cave
45
Unity 5 and Physically Based Shading
Carl Callewaert
Americas Director
46
Any Questions???
Ask the best question and
win a Mali-T760 GPU
based PiPO P4 tablet!
More chances to win a tablet
available on Twitter @ARMMobile
Tweet the best thing you’ve learned from
ARM at GDC #ARMatGDC to enter
47
Thank You
The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM Limited (or its subsidiaries) in the EU
and/or elsewhere. All rights reserved. Any other marks featured may be trademarks of their respective owners
48
To Find Out More….
 ARM Booth #1624 on Expo Floor
 Live demos of the techniques shown in this session
 In-depth Q&A with ARM engineers
 More tech talks at the ARM Lecture Theatre
 http://malideveloper.arm.com/GDC2015
 Revisit this talk in PDF and video format post GDC
 Download the tools and resources
49
More Talks from ARM at GDC 2015
Available post-show online at Mali Developer Center
 Unreal Engine 4 mobile graphics and the latest ARM CPU and GPU architecture - Weds 9:30AM; West Hall 3003
This talk introduces the latest advances in features and benefits of the ARMv8-A and tile-based Mali GPU architectures on Unreal Engine 4, allowing
mobile game developers to move to 64-bit’s improved instruction set.
 Unleash the benefits of OpenGL ES 3.1 and Android Extension Pack (AEP) – Weds 2PM; West Hall 3003
OpenGL ES 3.1 provides a rich set of tools for creating stunning images. This talk will cover best practices for using advanced features of OpenGL ES
3.1 on ARM Mali GPUs using recently developed examples from the Mali SDK.
 Making dreams come true – global illumination made easy – Thurs 10AM; West Hall 3014
In this talk, we present an overview of the Enlighten feature set and show through workflow examples and gameplay demonstrations how it enables
fast iteration and high visual quality on all gaming platforms.
 How to optimize your mobile game with ARM Tools and practical examples – Thurs 11:30AM; West Hall 3014
This talk introduces you to the tools and skills needed to profile and debug your application by showing you optimization examples from popular
game titles.
 Enhancing your Unity mobile game – Thurs 4PM; West Hall 3014
50
Learn how to get the most out of Unity when developing under the unique challenges of mobile platforms.
Any Questions?
Ask the best question and win a PiPO P4 tablet!
 Rockchip RK3288 processor
 ARM Cortex-A17 MP4 CPU
 ARM Mali-T760 MP4 GPU
51