Mirko Damiani
How to build a minimal
GNU/Linux embedded system
. . . from scratch!
Better Embedded Conference
Firenze, Sept. 24-25, 2012
Terms of Interest
●
●
●
GNU/Linux
●
GNU software
●
Linux kernel
Minimal
●
Size and number of utilities
●
Complexity of scripts
Embedded system
●
Custom boards
●
Low resources
2 / 35
Mirko Damiani – [email protected]
Prerequisites
●
●
Entry level
●
GNU/Linux kernel
●
Hardware schematics
●
Operating systems
●
Computer architectures
Intermediate level
●
GNU/Linux utilities
●
SW development
3 / 35
Mirko Damiani – [email protected]
Reference board
●
●
LeopardBoard 365
●
DM365 SoC (Texas Instruments)
●
ARM processor
Linux support
●
TI evaluation board (DaVinci)
●
LeopardBoard differs from TI evaluation board
●
Board Support Package available
4 / 35
Mirko Damiani – [email protected]
Overview
●
Board flashing
●
●
System startup
●
●
Bootloader
Kernel
●
●
Strategy
How to load and start it
Root filesystem
●
How to create it
5 / 35
Mirko Damiani – [email protected]
Board flashing
●
Common methods
●
Flashing strategy
●
Rootfs archive
●
Read-only rootfs
●
Flashing on the fly
6 / 35
Common methods
●
●
Utilities to program the NAND flash
●
SD/MMC card
●
USB (memory stick or protocol)
●
Serial port
Custom boards may need a specific
flashing strategy
●
SD/MMC or USB may be missing
●
Flashing utilities might not work
●
Serial port is almost always present
7 / 35
Mirko Damiani – [email protected]
Flashing strategy
●
●
●
Let's suppose we are able to flash
at least the bootloader
We load the entire system in RAM
●
Kernel is always loaded in RAM
●
Rootfs is passed as an initramfs archive
●
Kernel extracts its initramfs in RAM when it boots up
System is flashed for permanent setup
●
Linux starts from NAND flash
8 / 35
Mirko Damiani – [email protected]
Flashing strategy
●
●
Small components
●
Kernel is less than 4MB
●
Rootfs fits in 32MB (unpacked)
No different configurations
●
●
●
PC
(tftp server)
The same system is used both for
flashing and production
Software development is easier
Need more space?
●
Board
NAND
Minimal rootfs is able to mount
other NAND partitions
9 / 35
Mirko Damiani – [email protected]
Rootfs archive
●
●
The same rootfs must be saved in 2 different formats
●
Initramfs (gzipped cpio archive)
●
SquashFS image (read-only archive)
Different kernel parameters, for instance:
●
●
Initramfs
rdinit=/sbin/init initrd=0x82000000,0x82a190
SquashFS
init=/sbin/init root=/dev/mtdblock3 rootfstype=squashfs
10 / 35
Mirko Damiani – [email protected]
Read-only rootfs
●
Still need to have some R/W folders
●
●
/etc, /tmp, /var, /dev
Move contents in RAM and remount from there
●
mount -n -t tmpfs none /mnt/rwfs -o size=4096k
●
mkdir -p /mnt/rwfs/etc
●
cp -a /etc/* /mnt/rwfs/etc
●
mount -n --bind /mnt/rwfs/etc /etc
●
Now we don't have write errors anymore
●
Read-only filesystems are fast
Mirko Damiani – [email protected]
11 / 35
Flashing on the fly
●
●
Kernel and rootfs are downloaded via tftp server again
It's not convenient to save files in RAM,
since we might not have enough room
RAM
●
NAND
Both files are directly flashed,
without storing them in RAM
NAND
12 / 35
Mirko Damiani – [email protected]
System startup
●
DM365 as reference
●
Boot overview
●
Boot loading stages
●
Debug at early stages
13 / 35
Boot overview
●
Board initialization
●
●
Persistent configuration (kernel params)
kernel
Kernel loading
●
●
bootloader
Bootloader execution
●
●
Low level, hardware related
Operating system and devices setup
rootfs
System startup
●
/sbin/init and other scripts
14 / 35
Mirko Damiani – [email protected]
Bootloader stages (DM365)
●
ROM boot loader (RBL)
●
●
●
ARM subsystem initialization
User boot loader (UBL)
●
RBL
(1st stage)
External devices initialization
UBL
(2nd stage)
U-Boot
●
Power on
Kernel loading
U-Boot
(3rd stage)
15 / 35
Mirko Damiani – [email protected]
Bootloader stages (DM365)
Power on
NAND
UART
RBL
(1st stage)
Get UBL
USB
Store in IRAM
(32KB)
UBL
(2nd stage)
U-Boot
(3rd stage)
16 / 35
Mirko Damiani – [email protected]
Bootloader stages (DM365)
Power on
NAND
UART
RBL
(1st stage)
Get U-Boot
USB
Store in RAM
(128MB)
UBL
(2nd stage)
U-Boot
(3rd stage)
17 / 35
Mirko Damiani – [email protected]
Bootloader stages (DM365)
Power on
RBL
(1st stage)
Get Kernel
Scripts
Parameters
Environment
Run Linux
UBL
(2nd stage)
U-Boot
(3rd stage)
18 / 35
Mirko Damiani – [email protected]
RBL NAND loading (DM365)
●
Loading algorithm
●
●
●
●
NAND memories are organized
in sectors and pages
RBL searches for a valid block
In case of read errors,
next block is searched for
a valid UBL (up to #24)
Robustness
●
●
Multiple UBL / U-Boot pairs
may be saved in NAND
Flashing utilities do this for you
19 / 35
Mirko Damiani – [email protected]
Debug at early stages
●
U-Boot also provides an UBL for DaVinci boards
●
●
●
UBL sets the ARM processor frequency,
so i might want to change that values
Very hard to debug
●
●
●
TI UBL may be replaced with U-Boot UBL
Commons methods (printf, debugger, gdb, valgrind...)
are not available
Boards simply don't boot,
without any evidence of errors
A useful approach: exploit GPIO pins
20 / 35
Mirko Damiani – [email protected]
LeopardBoard GPIO
●
Access some registers (memory mapped I/O)
●
Setup pin multiplexing
●
Select GPIO direction
●
Drive GPIO as output high or low
21 / 35
Mirko Damiani – [email protected]
Kernel
●
Board Support Packages
●
Configuration and build
●
Loading and parameters
●
Patches
●
NAND partitioning
22 / 35
Board Support Packages
●
●
●
●
●
Kernel and drivers are patched to work with the
specified development board
We must understand how they works,
e.g. a camera driver would use video4linux
Custom boards may differ from development boards,
so additional SW patches are required
Ad-hoc drivers are not easy to write from scratch,
e.g a camera driver we would support only a specific
sensor and very few operation modes
Minimal drivers lead to an higher code maintenance
23 / 35
Mirko Damiani – [email protected]
Configuration and build
●
●
Compilation
●
Download from www.kernel.org
●
Toolchain: Sourcery Codebench Lite
Select the correct system and board, e.g.:
●
ARM system type
●
DaVinci 365 based system
●
Disable all which is not needed
●
Vanilla
●
We have a mainstream kernel
●
Easy to maintain (update)
●
Recent version
Mirko Damiani – [email protected]
24 / 35
Kernel loading
●
●
●
●
●
Loading is performed
by the bootloader
Arguments are passed
to the kernel
bootloader
args
Kernel sets up its devices
When booting is completed,
/sbin/init is called
(from within the rootfs)
/sbin/init is responsible for
bringing up the rest of the
system (the userland)
/sbin/init
pid = 1
Mirko Damiani – [email protected]
25 / 35
Kernel patches
●
●
DaVinci RTC on LeopardBoard 365
●
Driver is tested on DM360 EVM
●
LeopardBoard has a different schematic
Mux pins and drive some GPIOs
26 / 35
Mirko Damiani – [email protected]
Kernel parameters
●
Serial console device and options
●
●
console=ttyS0,115200n8
LeopardBoard 365
●
●
MAC address changes at every reboot
We can save it on U-Boot environment,
and pass it to the Linux kernel
–
davinci_emac.macaddr=AA:BB:CC:DD:EE:FF
–
davinci_emac.macaddr=${ethaddr}
27 / 35
Mirko Damiani – [email protected]
NAND partitioning
●
●
●
Bootloader is
never touched
Environment holds
kernel arguments
This layout may be
statically written
in kernel sources,
or passed through
arguments
30 blocks
bootloader
256 KB
environment
4 MB
kernel
32 MB
rootfs
28 / 35
Mirko Damiani – [email protected]
Root filesystem
●
Busybox
●
Accessing bootloader environment
●
Building the root filesystem
29 / 35
Busybox
●
Provides almost a complete environment
for small systems
●
●
Tiny version of common utilities (fewer options)
Single executable
●
Size optimization
●
Static or dynamic library linking
●
Symlinks to busybox executable
–
●
e.g: /bin/ls -> /bin/busybox
Default configuration
●
Static: 2MB
●
Dynamic: 936KB
30 / 35
Mirko Damiani – [email protected]
Busybox
●
Packages
●
Coreutils, Shells, Editors, Network, …
●
Login files
–
●
System logs
–
●
●
/etc/passwd, /etc/shadow, /etc/groups, …
syslogd, klogd
...
Init program (/sbin/init)
●
Reads /etc/inittab
●
Runs /etc/rc.d/rcS
●
Other scripts, /etc/rc.d/init.d/network, ...
31 / 35
Mirko Damiani – [email protected]
Bootloader environment
●
Recall: boot env is stored into the 2nd partition
●
Example: U-Boot
●
●
Provides some tools: fw_printenv, fw_setenv
Generally speaking, it's possible to
access them from userspace.
●
●
Within our flashing procedure,
IP address of tftp server
may be read from the
bootloader environment
System might not save at all
kernel and rootfs into NAND,
while being still configurable
192.168.10.101
Mirko Damiani – [email protected]
32 / 35
Building the root filesystem
folders
structure
rootfs
toolchain
libraries
clean
binaries
busybox
kernel
modules
skeleton
files
33 / 35
Mirko Damiani – [email protected]
Let's Talk
office
+39 055 3984627
e-mail / jabber
[email protected]
web
www.develer.com
License
Creative Commons Attribution 3.0 Unported