Multimedia and Sense Sense of Mode of Sense vision hear smell taste balance "5−Sense" visual auditory olifactive gustatorisch vestibulär pressure vibration cold warmth pain skin touch tactil position power Propriorezeptoren kinesthetic Display optical accoustic − − − haptic [additional information] $LOGOIMAGE Multimedia and Sense 1 Videos about our senses Videos taken from TV: "nano", 3sat: (language: german) • sehen • hören • riechen • tasten • schmecken $LOGOIMAGE Videos about our senses 2 Environment − Perception $LOGOIMAGE Environment − Perception 3 Environment mediated by media $LOGOIMAGE Environment mediated by media 4 General Assumption The precondition to realize multimedia systems is, that the environmental signals can be digitized: Example: The voice signal in the telephone system is limited to 4000 Hertz. With a sampling rate of 8000 (125µs) and 8 bit per sample we have 64 Kbit/s Nyquist Theorem $LOGOIMAGE General Assumption 5 Enabling Multimedia • Not all signals of the real environment have to be reproduced. ♦ only those signals which cause the same perception in the sense organs as the original signals must be reproduced. • Limited resolution of the human sensory system ♦ artificial signals only have to be transmitted in a proper resolution. • The systems of senses is widely modular. ♦ e.g.: Text needs only a picture; Speaker needs voice and pictures • Limitation of intensity ♦ Stimulus crossing a certain limit of intensity cause damage $LOGOIMAGE Enabling Multimedia 6 Limitations of sense organs All human sensory systems have limited spectrum of perception. Signals beyond this spectrum can´t be perceived or lead to damage of the sense organ. Examples: • Audio: Sound between 20 and 20000 Hertz • Visual: Electromagnetic waves between 380 and 720 nm • Vibration: oscillations between 10 and 500 Hertz $LOGOIMAGE Limitations of sense organs 7 Sense of Vision The interior of the eyeball is covered with photosensitive cells known as the retina. The cells, known as rods and cones, form the layer of cells at the back of the retina. The backward−design of the retina results in the so−called blindspot. Rods (110−125 Mio.) are sensitive to very low levels of illumination and are responsible for our ability to see in dim light (scotopic vision). They contain a pigment with a maximum sensitivity at about 510 nm, in the green part of the spectrum. Scotopic vision is completely lacking in color. Color vision is provided by the cones (photopic vision) (5−7 Mio.), of which there are three distinct classes with absorptions at about 430, 530, and 560 nm. $LOGOIMAGE Sense of Vision 8 Range of visibility $LOGOIMAGE Range of visibility 9 Dynamic of see sense Candela per square meter $LOGOIMAGE Dynamic of see sense 10 Chromatic Aberration (1) refraction power of the eye wavelength (nm) 687 656 reference point−−> 589 527 486 431 $LOGOIMAGE color description dark red medium red yellow yellow−green blue blue−violet Chromatic Aberration (1) refraction difference + 0.34 <−− +0.25 can not be 0 fokused −0.30 simultaneously −0.58 −1.07 <−− 11 Chromatic Aberration (2) $LOGOIMAGE Chromatic Aberration (2) 12 Sense of Hearing • 16.000 receptors (hair cells) in a human cochlea ♦ 3500 inner cells ♦ 12000 – 20000 outer hair cells • Hearing range 20 Hz to 20000 Hz • Number of fibers in auditory nerve 28000 • Detection threshold 10–12 W/m2 ♦ Dynamic range 120 dB (12 orders of magnitude) $LOGOIMAGE Sense of Hearing 13 Human Aural Properties Human ear can detect sounds between ~20Hz and 20kHz: • Total audio range: 5Hz − 50 kHz $LOGOIMAGE Human Aural Properties 14 Limitations of audio−frequencies $LOGOIMAGE Limitations of audio−frequencies 15 Dynamic of hearing $LOGOIMAGE Dynamic of hearing 16 Decibel The decibel (dB) is a common unit of measurement for the relative loudness of a sound or, in electronics, for the relative difference between two power levels. • In sound, decibels measure a scale from the threshold of human hearing, 0 dB, upward towards the threshold of pain, about 120−140 dB. • I := 2*I0 −> 3 db • I := 10*I0 −> 10 db • I := 100*I0 −> 20 db • I := 1012*I0 −> 120 db [What is a decibel] $LOGOIMAGE Decibel 17 The Physics of Acoustics Sound can be considered in one of two ways: • In the time domain • In the frequency domain • Transformation is accomplished by a Fourier transform Fourier: • Any waveform can be created by a series of sine waves summed together $LOGOIMAGE The Physics of Acoustics 18 Sense of Smell Electronic Nose • Analysis of smells • Synthesis of smells $LOGOIMAGE Sense of Smell 19 The fragrance of the Internet The Ruetz company together with France Telecom will develop a device which will extend the audio−visual experience in cinema or even in the internet by a smelling component. The prototype has 64 fragrances which can be changed for special events. A future device can be attached to the PC and is equipped with fragrance−chips, which can be adopted to the individual demands. The Sniffman consist of different fragrance−chambers and a heating−chamber. From the heating−chamber the fragrance will reach the nose. $LOGOIMAGE The fragrance of the Internet 20 Sense of Taste Fungiform papillae are located on the most Circumvallate papillae are sunken papillae, with a trough anterior part of the tongue and generally contain separating them from surrounding wall. They confer a one to several taste buds per papilla. They appear sour/bitter sensitivity to the posterior 2/3 of the tongue. as red spots on the tongue. Foliate papillae are situated on the edge of the tongue slightly anterior of the circumvallate line. They are predominantly sensitive to sour tastes. $LOGOIMAGE Sense of Taste 21 Sense of Touch The mechanics of electronic touch devices are neither purely electronic nor simply mechanical. They're both. As electro−mechanical devices, they translate digital information into physical sensations. For example, when you push on a mouse or a joystick, the device pushes back—using magnetic actuators and sensors built into the device. Technically speaking, this process is called force feedback. And resistance is only one of the hundreds of sensations like springs, liquids, textures, vibrations and so on. One can simulate the sensation, as long as it can be translated into a mathematical equation. http://www.immersion.com/ $LOGOIMAGE Sense of Touch 22 Vibration thresholds $LOGOIMAGE Vibration thresholds 23 Additional Information (1) MIT encyclopedia of cognitive science http://cognet.mit.edu/ Mark Newbold's Animated Necker Cube http://dogfeathers.com/java/necker.html Visual Cognition Demonstrations http://www.viscog.net/demos.html Perception http://psych.la.psu.edu/clip/Perception.htm $LOGOIMAGE Additional Information (1) 24 Additional Information (2) Your Sense of Hearing http://tqjunior.thinkquest.org/3750/hear/hear.html The Human Body's Senses: Hearing Theme Page http://www.cln.org/themes/hearing.html The Human Body's Senses: Sight Theme Page http://www.cln.org/themes/sight.html The Human Body's Senses: Smell Theme Page http://www.cln.org/themes/smell.html The Human Body's Senses: Taste Theme Page http://www.cln.org/themes/taste.html $LOGOIMAGE Additional Information (2) 25 Additional Information (3) The Human Body's Senses: Touch Theme Page http://www.cln.org/themes/touch.html That's Tasty http://faculty.washington.edu/chudler/tasty.html ARTIFICIAL RETINAS http://www.optobionics.com/ TOUCHING..... http://sln.fi.edu/qa97/me10/me10.html Introduction to the Skin http://www.meddean.luc.edu/lumen/MedEd/medicine/dermatology/ skinlsn/skini.htm $LOGOIMAGE Additional Information (3) 26 Additional Information (4) Retina Reference http://retina.anatomy.upenn.edu/~lance/retina/retina.html Artificial hearing http://www.allhear.com/monographs/m−96−htm.html $LOGOIMAGE Additional Information (4) 27