How to get started on your own FTIR Multi-touch device Chris Delbuck Spring 2007 DAI627: Digital Media 4 Prof. Paul Catanese Some things you should know first. Multi-touch is a term used for a touch based interface system that allows for multiple input points (in this case derived from touching a surface). Some of the inclusive benefits of a multi-touch system are: 1) suitable for multiple users 2) freedom from the cursor 3) higher quality input. FTIR: Frustrated Total Internal Reflection (or the frustration of Total Internal Reflection) This is the primary physical property that allows for this multi-touch approach to be possible. Total Internal Reflection is an “optical phenomenon” that occurs in transparent materials, in this case clear acrylic. Different mediums reflect, bend or pass light depending the angle at which the light approaches the surface. In this usage we want to keep the light utilzed for this method trapped inside the acrylic. The frustration occurs when a finger or other material with a higher refractive index come in contact with the medium conducting the total internal reflection. Acrylic: For the FTIR Multi-touch approach, clear acrylic sheet (”plexiglass”) is an optimal medium to use to achieve total internal reflection. However there are some individuals who have managed to achieve the TIR with polycarbonate materials. 1/4” thick Acrylic sheet is preferable because it will remain rigid enough for touch interaction and will be narrow enough to create more dense internally reflected light. Infrared Light (IR): Since this approach will use computer sourced projection and computer vision it is important to use a combination visible and non-visible (or near visible) light with each being designated to one of the components. The visible light will be reserved for display so the user has visual feedback to their engagement with the device. The IR (non-visible) will be used for the total internal reflection and a video camera to capture the points (touches) where the light is frustrated. IR light is light that exceeds a 750nm wavelength (visible light exists from 380 - 750nm). Since we cannot see it, the IR light can exist simultaneous to the visible light of the projection. The range of IR light used for this can range from 750 - 950nm, the decision will be made of the application and setting of its use and surrounding ambient light. A list of essential materials. Acrylic Sheet: 1/4” in thickness is optimal. The greater the thickness, the less number of reflections that occur, though this can be compensated with more IR light (would not suggest anything over 1/2“). Anything below 1/4” becomes potentially too flexible for touch and counterproductive to successful total internal reflection. Width and height are completely up to the application and usage of the multi-touch system. Camera: A CCD based camera that has the ability to send a digital signal to the computer is necessary to capture the points of frustrated light. A black and white camera is absolutely fine if not optimal for this application because more precise capture from black to white (or 0.0 - 1.0) can be achieved. Some USB webcams can be used for this, however higher quality Firewire based cameras are optimal. Lens: A lens that does not have an IR coating or filter integrated into it. Some webcams have a small layer of film that acts as an IR filter which can be removed. Visible Light Blocker: A means to prevent visible light from entering the camera is necessary, only the frustrated IR light from the acrylic is required to be captured. Common 35mm photographic negative film is a perfect material. Pull out any length from the roll (cartridge) and expose to direct sunlight for at least 5 - 10 minutes, then have the film processed at any lab. The returned material should be completely visibly opaque and dark in nature. Cut the film negative to size to fit between the lens and the image sensor of the camera (sometimes double layering is necessary). Projector: For the user to have a direct visual correlation to where their fingers engage the surface, a projection onto that surface is necessary. Any typical computer sourced projector should work in this application. Keep in mind different projectors have varying angles, so calculate the desired size of projection to the require distance the projector must have from the surface. If you are projecting a 4:3 ratio image you will probably want a piece of acrylic that will have a visible area also close to 4:3. IR LEDs: Light Emitting Diode’s will be the light source for the total internal reflection within the acrylic. You can choose LEDs with a visible light range from 750-950nm, depending on how much IR light might be ambient in the area. 840-850nm is typically a safe option. LEDs are typically 5mm in diameter, only go larger if the acrylic is thicker than 1/4”. The most important specification you must look for is the beam angle. Total angle of 80° is the maximum allowable for this process. Make sure when purchasing of the beam angle is referenced as total or half angle, half angle should be 40°. You will also need an adequate power source, resistors and length of quality copper wire for the LED configuration you choose. The number of LEDs needed depends on how large the area of touch surface is required. To achieve a high level of light, use one LED for every inch on the vertical (left and right) edges. Computer: One that can interface with the camera and projector being used. As well as adequate enough to process the video signal and drive a resulting projection. Two computers can be used, one for each task, with some communication bridge between them. Materials subject to variation. Framing and Structure: Something will be required to hold the Acrylic in place as well as hold the IR LEDs flush to the edges of the acrylic. Either wood or metal can work. Something sturdy enough to hold up the acrylic and keep it in place when used. I found some very cheap wood frames at IKEA that worked perfectly. I lined up the acrylic in between the two frames, secured the pieces with tape and clamps, then drilled through all three pieces uniformly. This allowed me to secure all pieces with four bolts and nuts. Adhesives: Some various tapes and glue are always helpful to keep things in place either permanently or temporarily. Projection Surface: One of the largest areas subject to variation is how to place the projection onto the relatively transparent surface. Since the nature of TIR is subject to frustration, its important to have a surface for projection that does not set off the FTIR effect. If between the projector and the acrylic, allows for IR light to pass through. Other Surfaces: Since bare acrylic is not necessarily the easiest to drag fingers across it becomes troublesome as an effortless touch surface, in fact the FTIR method requires some force from the fingers to activate. Ideally a “zero-force” approach with easy gliding would be the best for a worthwhile experience with the multi-touch system. FTIR in further depth. As light passes into the sides of the acrylic, it will remain reflected as long as the angle which it approaches the surfaces is less than 42.1° (the angle of total internal reflection of acrylic to air). Once something comes in contact to the surface of the acrylic with a higher refractive index, the internal reflection is frustrated and light will leave the acrylic at that point. Acrylic LED Internal Reflection Light beyond critical angle Frustrated Light Projection and other material layering. In my work I ended up using 4mm thick sheet of Mylar with a single matte side. This could lay flat against the back side of the acrylic and not set off the FTIR, it is also a very nice projection material as well. Some individuals have found a combination of layers that achieve this. Silicone Rubber and Rear Projection Screen allow for easy gliding and near zero-force FTIR. Utilizing clear silicone rubber (either a poured from a two part curing compound or pre-made sheet) no more than 1mm thick directly on the topside of the surface and then gray rear projection screen material on top of that seems to be one of the best configurations for an easy to use multi-touch system. The nature of the Silicone Rubber allows for a great deal of projected light to pass through, and only sets of the FTIR upon touch. The Projection Screen delivers great image quality from the projection and as very easy surface to engage with fingers. My Approach Another Approach Projection Screen Silicone Rubber Acrylic Mylar Acrylic Projection Projection LED configuration and acrylic edge preparation. Various approaches can be tackled in order to conduct TIR of IR light within the acrylic medium. Some individuals arrange LEDs on all 4 sides of the material, while others only do 1. I decided to do the left and right sides of the surface, placing an LED at every inch in the length of the surface that would be utilized for a touch area. What is also very important, is the preparation of the acrylic edges. Typically when the acrylic is cut with a saw, the edges are left very rough, which is not optimal for light transmission. There are various approaches to purifying the edge of any grain or roughness and have an appropriate clarity for the IR LEDs. One option is to use various grades of sand paper (in the order by grit, 100, 200, 400 and then 800) then polishing with either copper or brass polish. With mine, I opted to have the supplier use a router and flame polish the edges. This produced a very good edge quality for the infrared light. Hardware configuration. Most calibration to achieve an accurate visual correspondence to the physically touched area will occur in software, it is important to understand the relation where all of the components will exist. Though this is not the only configuration possible (mirror to alter placement can work) this is the arrangement in basic principle. The camera and projector must be placed on the opposite side of the surface from the user, being able to project and capture, visible and IR light respectively. 12VDC IR LIGHT IR CAMERA VIEWPOINT FRUSTRATIONS DIGITAL PROJECTOR SENDS IMAGE TO PROJECTOR IMAGE ANALYSIS SENDS IMAGE TO COMPUTER VISUAL GENERATED COMPUTER ESCAPED LIGHT Image Analysis & Visual Generation. For this proof of concept, it is necessary to illustrate the connection between where a user touches and how the device visually reacts to engagement. It is necessary for the furstrated areas (touches) to become seen by the IR sensitive camera. Then that signal is sentto the computer. The image is analyzed and based on development can include calibration, the resulted analysis is the generation of a visual response back to the surface where the user interacts with it. The left is a still from the IR Camera The Right is the projected image Image Analysis & Visual Generation. (cont.) The image being delivered from the camera to the computer is a grayscale image that will have white spots on a mostly dark background. The white spots come from where the internal reflection of the IR light in the acrylic is frustrated. The camera captures the touch areas as bright white blobs. These blobs captured by the camera are the main component to how the computer will interpret the image and generate a resulting visual response to the touches. Using blobDetection by v3ga (processing library), I was able to capture the image coming from the camera. In addition to camera capture, the software also allows for finding bright areas apart from dark areas in the image. These blobs are found based on areas in the image that fall above a set brightness level. Each blob detected is analyzed and has certain attributes computed. Each blob has an overall width and height, which is used to also find a center. The edge of the blob is broken up into edge points. The software can return the edge points one at a time, or two at a time to be used to create a line. There is also the functionality to access blobTriangles which takes two edge points at a time and the center point to return three points which can be used as vertices. Amongst the aforementioned types of data the blobDetection will return there are others as well. In most cases, centers will typically be the primary information be used, just to most accurately pin-point where the user is interacting. However the edges and triangle information could easily be used as a means to interpret pressure being applied by the user. Perhaps a blob generated by the palm of a hand touching the surface could create a different response than a fingertip. The Final Product. (Conclusion) Essentially the goal of this project was to at least get to a stage where the hardware was working with the FTIR approach and generating something basically responsive. This was achieved, and even on a simple level of reaction, still was very engaging for the small audience that was the first to witness the project in its “completed” form. Going through two prototypes in hardware was definitely a consumer of energy and work hours. The second prototype (shown below) was much more managable since I used prefabricated framing materials which were modified for this project. If this had been done as part of a group project with the knowledge and resources of the product designers in DAI, the end product could be something more substantial and applicable for public use. The possible software development for this hardware approach is almost endless. Since there is such a freedom for software development, the applications are almost equally broad. I believe the freedom from the tether of the mouse and keyboard are part of the fundamental quality to this technology, how that freedom is used is up to the developer. A Budget and Suggested Purchases. There are a varying array of materials that can be used for this project. There are the necessary items along with the options for low-end to high-end technology. I’ve also included a couple of alternative materials which would lead toward a better multi-touch experience. Essential components 1/4” Acrylic Flame Polishing Framing (wood or metal) Screws and Braces IR LEDs + Resistors Power Supply Mylar Copper Two-Wire $5.50 / sq ft. $1 per linear foot $25 - $75 $20 (72) 20° 850nm with Resistors | $38 $10 - $20 $3 - $5 per foot | 36”(w) 24¢ per foot The following can change dramatically on selection Camera / Lens / Housing $200 Optoma EP747 Projector $1499 Personal Computer $2000 Other suggested materials Rear Projection Screen $15.70 per yard / 55”(w) Silicone Rubber Sheet $36 per foot / 36”(w) Purchasing Resrouces 1/4” Acrylic IR LEDs + Resistors Mylar TAP Plastics www.tapplastics.com www.electronicpartsforless.com (custom packages) Arch www.archsupplies.com Rear Projection Screen Rosebrand www.rosebrand.com Silicone Rubber Sheet Acme Rubber www.acmerubber.com Online Literature Resources. NUI Group - http://www.nuigroup.com Features wiki and a message board with global members discussing their own multi-touch experiments and projects. Compliant Surface http://nuigroup.com/forums/viewthread/6/ Multi-Touch Interaction Research - http://cs.nyu.edu/~jhan/ftirtouch/index.html Low-cost multi-touch sensing through frustrated total internal reflection by Jeff Han http://portal.acm.org/citation.cfm?doid=1095034.1095054 Synthesis and control on large scale multi-touch sensing displays by Jeff Han http://portal.acm.org/citation.cfm?id=1142269 Multi-Touch Systems that I have Known and Loved by Bill Buxton http://www.billbuxton.com/multitouchOverview.html
© Copyright 2024