SCHED_DEADLINE How to use it Juri Lelli Retis Lab – SSSUP Pisa (Italy), June 26th 2014 TeCIP Insitute, Scuola Superiore Sant'Anna Area della Ricerca CNR, Via G. Moruzzi 1 56127 Pisa, Italy Outline ● ● Basics and status What SCHED_DEADLINE is implementing ● API and usage ● Demos Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 2/39 Outline ● ● Basics and status What SCHED_DEADLINE is implementing ● API and usage ● Demos Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 3/39 OS kernels and schedulers ● Linux is an OS kernel (not a full OS as it doesn't include FS utilities, WM, GUIs, compilers, etc.) ● Composed by several sub-systems ● One of which is the Scheduler ● ● Decides which process to run where and when, achieving an apparent simultaneous execution of multiple processes on the same shared HW Traditionally General Purpose: didn't include proper Real-Time scheduling features Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 4/39 A new scheduling policy/class SCHED_DEADLINE SCHED_DEADLINE SCHED_FIFO SCHED_FIFO SCHED_RR SCHED_RR SCHED_BATCH SCHED_BATCH SCHED_NORMAL SCHED_NORMAL SCHED_IDLE SCHED_IDLE deadline.c rt.c fair.c Linux scheduler Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 5/39 An history of LKML posts ● ● started within the ACTORS EU project (2008) originally developed and maintained by Dario Faggioli et al. ● then I switched in (v4 April 6th, 2012) ● 9 versions posted ● Merged by Linus on Sun, 2 Feb 2014 17:12:22 ● WE ARE IN MAINLINE!!! YAY!!! ● Current stable is Linux 3.14.2 Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 6/39 Outline ● ● Basics and status What SCHED_DEADLINE is implementing ● API and usage ● Demos Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 7/39 Predictability and Isolation 4 CPU-hog processes (i.e. while(1)) on 4 CPUs SCHED_NORMAL (default Linux scheduling policy) Jun 26, 2014 SCHED_DEADLINE ● finer-grained control over tasks scheduling ● tasks don't interfere with each other Juri Lelli - SCHED_DEADLINE @ Retis Lab 8/39 Sporadic task model activation/wakeup absolute deadline start time computation time relative deadline period Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 9/39 Real-time scheduling algorithms ● ● ● tasks are executed in priority order Rate Monotonic (RM) → tasks with shorter periods have higher priorities Earliest Deadline First (EDF) → tasks with earlier deadlines have higher priorities Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 10/39 Static vs. Dynamic (UP) τ1 → 2 every 5 2 4 + ≃0.97 5 7 τ2 → 4 every 7 97% overall load τ1 2 4 6 8 10 12 14 16 18 t 2 4 6 8 10 12 14 16 18 t 2 4 6 8 10 12 14 16 18 t 2 4 6 8 10 12 14 16 18 t τ2 RM → not more than 69% overall load (when n is big) τ1 EDF → can achieve 100% τ2 Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 11/39 Deadline scheduling (EDF) τ1 → 5 every 9 τ2 → 2 every 6 ~88.9% overall load τ1 t τ2 t Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 12/39 Deadline scheduling (EDF) τ1 → blocks just after the second activation τ1 → resumes with the third instance of τ2 τ1 t τ2 t Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 13/39 Deadline scheduling (EDF) τ1 → second job behaves bad τ1 t τ2 t Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 14/39 Reservation Based scheduling ● Concurrent real-time tasks compete for resources (CPU time) ● Resource Reservation mechanism ● A task is allowed to execute for: ● – Q time units (runtime) – in every interval of length P (period) Task utilization is U=Q/P Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 15/39 Reservation Based scheduling ● ● ● ● ● Each reservation has its own dynamic deadline (Constant Bandwidth Server) Reservations are scheduled by EDF EDF gives higher priority to more urgent tasks CBS slows down (or throttles) misbehaving tasks EDF + CBS ensures temporal isolation Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 16/39 EDF + CBS plain EDF (block/unblock) → τ1 t τ2 t τ1 → blocks just after the second activation τ1 → resumes with the third instance of τ2 (CBS “unblock rule” applied) τ1 t τ2 t Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 17/39 EDF + CBS plain EDF (bad task) → τ1 t τ2 t τ1 → second job behaves bad → once budget exausted, delay until next activation period τ1 t τ2 t Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 18/39 Outline ● ● Basics and status What SCHED_DEADLINE is implementing ● API and usage ● Demos Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 19/39 Adds ● ● ● ● new syscalls: sched_setattr(), sched_getattr() system wide deadline bandwidth management coupled with RT – /proc/sys/kernel/sched_rt_runtime_us – /proc/sys/kernel/sched_rt_period_us rt_bw is enforced at runtime (rt throttling) dl_bw is reserved with AC, and enforced at runtime Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 20/39 Symmetric Multi-Processors ● ● DL tasks are kept into CPU-specific runqueues implements Global EDF (DL tasks are migrated among runqueues to achieve): – on an M-CPU system the M earliest DL ready tasks are always running; – affinity/cpusets settings of all the DL tasks is always respected Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 21/39 SMP ready! 4 CPU-hog processes (i.e. while(1)) on 4 CPUs + 1 more after a while... new cpuhog5 arrival causes a migration for cpuhog2 Jun 26, 2014 system stabilizes into a new condition, similar to partitioned scheduling Juri Lelli - SCHED_DEADLINE @ Retis Lab 22/39 Interface struct sched_attr { u32 size; u32 sched_policy; u64 sched_flags; /* SCHED_NORMAL, SCHED_BATCH */ s32 sched_nice; /* SCHED_FIFO, SCHED_RR */ u32 sched_priority; /* SCHED_DEADLINE */ u64 sched_runtime; u64 sched_deadline; u64 sched_period; }; int sched_setattr(pid_t pid, const struct sched_attr *attr, unsigned int flags); int sched_getattr(pid_t pid, const struct sched_attr *attr, unsigned int size, unsigned int flags); Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 23/39 Example #include <sched.h> ... struct sched_attr attr; attr.size = sizeof(struct attr); attr.sched_policy = SCHED_DEADLINE; attr.sched_runtime = 30000000; attr.sched_period = 100000000; attr.sched_deadline = attr.sched_period; ... if (sched_setattr(gettid(), &attr, 0)) perror("sched_setattr()"); ... Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 24/39 RR over sporadic task model simple rule of thumb, see literature for optimal settings... average runtime [ns] deadline [ns] period [ns] Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 25/39 Outline ● ● Basics and status What SCHED_DEADLINE is implementing ● API and usage ● Demos Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 26/39 Clone and Compile ● Clone Linus' repo git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ● Configure, compile and install cp /boot/config`uname r` .config yes "" | make oldconfig make j `cat /proc/cpuinfo | grep processor | wc l | awk '{print $1 + 1}'` sudo make modules_install install updateinitramfs c k VERSION && updategrub or wait for next Ubuntu, Fedora, etc. release... Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 27/39 Test ● rt-app git clone [email protected]:gbagnoli/rtapp.git cd rtapp ./autogen.sh (requires autotools) ./configure –withdeadline –withjson make && make install rtapp t 100000:10000:d D10 or rtapp taskset.json ● schedtool git clone [email protected]:jlelli/schedtooldl.git cd schedtooldl make && make install schedtool E t 10000000:100000000 e yes Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 28/39 encapsulate a “yes” process inside a 10 ms, 100ms reservation schedtool E t 10000000:100000000 e yes see execution trace with kernelshark Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 29/39 encapsulate a “yes” process inside a 10 ms, 100ms reservation schedtool E t 10000000:100000000 e yes 10 ms Jun 26, 2014 100 ms Juri Lelli - SCHED_DEADLINE @ Retis Lab 30/39 Experiments with random tasks Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 31/39 Static vs. Dynamic (UP) τ1 → 2 every 5 2 4 + ≃0.97 5 7 τ2 → 4 every 7 97% overall load τ1 2 4 6 2 4 6 2 4 2 4 τ2 8 10 12 14 16 18 t RM → not more than 69% overall load (when n is big) WITH rt-app! 8 10 12 14 16 18 t 6 8 10 12 14 16 18 t 6 8 10 12 14 16 18 t τ1 EDF → can achieve 100% τ2 Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 32/39 Provide QoS to VMs ● ● KVM creates one thread per VPCU and one thread per VM doing I/O encapsulate VPCU threads inside reservations to provide QoS guarantees Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 33/39 Provide QoS to VMs ● ● a single-core VM runs a CPU-bound activity: sysbench, calculation of prime numbers up to a certain value we vary the reservation budget of its VCPU Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 34/39 Provide QoS to VMs ● ● a single-core VM runs a network activity: iperf server with external requests from a client running on different HW we vary the reservation budget of its VCPU Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 35/39 Provide QoS to VMs ● ● vm1 (visualized) runs iperf server, vm2 sysbench load of vm2 doesn't affect vm1 performance vm2 reservation vm1 reservation Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 36/39 Provide QoS to VMs ● ● both vm1 (visualized) and vm2 run iperf servers we still have a “certain degree” of isolation (probably better with dedicated HW or different configurations) vm2 reservation vm1 reservation Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 37/39 Outline ● ● Basics and status What SCHED_DEADLINE is implementing ● API and usage ● Demos Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 38/39 Thanks! Questions ? [email protected] Jun 26, 2014 Juri Lelli - SCHED_DEADLINE @ Retis Lab 39/39
© Copyright 2024