Welcome to Accurate Robotic 3D Vision An educational webinar sponsored by
Welcome to Accurate Robotic 3D Vision An educational webinar sponsored by Universal Robotics and Yaskawa Motoman Robotics Speakers Hob Wubbena, Director of Marketing, Universal Robotics • • • • Engineering, technical planning and product marketing for Hewlett-Packard and Agilent Technologies for 25 years Aerospace & defense, telecommunications, test & measurement, electronic manufacturing, and the chemical process industries 3 patents, numerous marketing awards, ~12 published articles B.S. Civil Engineering, University of Wisconsin; Masters Business, Denver University Erik Nieves, Technology Director, Yaskawa Motoman Robotics • • • • Management and engineering, for Motoman Robotics for 20 years; currently focused on corporate strategic technology roadmap and emerging applications Standards Development & Education Committees for the Robotics Industries Association (RIA), “Ask the Experts” forum, RIA Many published robot technology articles including control and metrology B.S. Mathematical Physics, Southwestern Adventist University 7/21/2011 Copyright © 2011 Universal Robotics Page 2 Experts Aditya Nawab, R&D Manager, Universal Robotics • • • • Extensive design experience in computer vision, hardware/software integration, autonomous vehicle development and industrial robotics Worked on numerous Department of Defense and NASA robotic projects His work at Universal focuses on dexterous manipulation, force control, sensory-motor coordination, forward and inverse kinematics, dynamics of articulated manipulators, and R&D project management B.S. Mechanical Engineering, Florida Atlantic University; M. S. Mechanical Engineering, University of Florida Greg Garmann, Technology Leader, Yaskawa Motoman Robotics • • • Technology Leader at Yaskawa America, Motoman Robotics Division, and has been involved in automation for more than 25 years. Developed vision capabilities for robotics guidance using 2D and 3D technologies B.S. Computer Engineering, Wright State University 7/21/2011 Copyright © 2011 Universal Robotics Page 3 AGENDA: Accurate Robotic 3D Vision • Introduction to Robotic 3D Vision • Types of 2D, 2½D & 3D robotic vision systems • Elements of Accuracy • Choosing a 3D vision system • Q&A 7/21/2011 Copyright © 2011 Universal Robotics Page 4 Robotic 3D Vision Introduction • Beyond scope of webinar: – – – – – – – 3D vision (movies) anaglyphs for depth perception Photometric stereo from telecentric camera Depth From Focus (DFF) 3D Mosaicking Vision analysis tools (blob analysis, face recognition) Time of Flight (TOF) Future webinar Laser line projector • Robotic vision delivers real-time data about objects – – Part Inspection Vision Guidance (X, Y, Z position & Rx, Ry, Rz pose) • 3D accuracy requires both intrinsic camera calibration and hand-eye calibration 7/21/2011 Copyright © 2011 Universal Robotics Page 5 Repeatability vs. Accuracy • Repeatability is important for automated tasks where robot picking and placing to same locations • Relative Accuracy is important for random tasks where the spatial location is constantly changing: – Random box depalletization – Random box moving – Random part picking – Random bin picking 7/21/2011 Copyright © 2011 Universal Robotics Page 6 Intrinsic Camera Calibration Calibration process • • • Camera sensor information Fiducial in full FOV 15 varying fiducial images Y Z Data in 3 coordinate systems • Image, Camera, Object X Image rectification (ΔY) • Removes perspective and/or lens distortions If multiple cameras • • Computes disparity, distance, 3D coordinates Aligns image on common plane 7/21/2011 Camera Object Y Z Y Z X Image Copyright © 2011 Universal Robotics X Focal length Page 7 FOV Field of View 3D Vision & Robotic 3D Vision 3D Vision: stereo vision with depth (z) perception resulting from the disparity (Δx) of different images of the same object 3D Accuracy: resolution in the Δz depth of an object viewed from stereoscopic vision. It is based on quality and geometry of cameras. Δy distance difference between the cameras (placement and orientation) is corrected during intrinsic camera calibration, resulting in a rectified image. The Δx disparity of each camera's image of same pixel point in space is computed through 3D software algorithms. Robotic 3D Vision • 3D Vision with vision guidance for a robot delivers in real-time: – – – – 7/21/2011 Copyright © 2011 Universal Robotics Position (X, Y, Z) Pose/orientation (Rx, Ry, Rz) Interactively, with tool offsets Choice of robot tool affects vision requirements (vacuum and grippers can work within 2-3 mm) Page 8 Hand–Eye Calibration • First, calibrate cameras • Then Hand-Eye calibration enables data transform across coordinate systems – Robot “hand” tool guided by camera “eye” • RESULTS FOR: – – – VISION GUIDANCE Part Inspection (optional) 3D Part Model creation (optional) 7/21/2011 Copyright © 2011 Universal Robotics Page 9 AGENDA: Accurate Robotic 3D Vision • Introduction to Robotic 3D Vision • Types of 2D, 2½D & 3D robotic vision systems • Elements of Accuracy • Choosing a 3D vision system • Q&A 7/21/2011 Copyright © 2011 Universal Robotics Page 10 Types of 2D, 2½D & 3D Vision Systems 2½D 2D Rz Rz X 3D Z Rz #1 #2 Z X X Rx •Single Camera •Single camera with known calibration plate, OR multiple images to obtain depth, OR add TOF/laser •Move camera OR add 2nd camera with shape-matching (model) OR TOF/laser with surface-matching (point cloud) •Vision data in same plane •Z data added •Rx, Ry data added •X, Y, Rz(angle) only; no Z •Postion X, Y, Z, Rz (angle) only •Position X, Y, Z, & Pose Rx, Ry, Rz •No tilt (Rx, Ry) •No tilt (Rx, Ry) •Allows for part tilt •Camera distance constant •Camera distance can change •Camera distance can change 7/21/2011 Copyright © 2011 Universal Robotics Page 11 Types of 3D Vision Camera System Vision Methodology Required Comments Single Stationary 2D, 2½D, 3D 2½: ONLY if height is known & fixed. Height can be variable with TOF 3D: Shape-match Good if plane is fixed Single Moving 3D 3D: with shape-match or surface-match Can measure depth, Slower than Stereo Binocular Stereo 3D Shape-match (model) or surface-match (point cloud) Surface-match requires TOF or laser 3D Mosaicking Shape-match (model) Surface-match requires TOF or laser, Slower than binocular Multiple Stereo 7/21/2011 Copyright © 2011 Universal Robotics Page 12 Single Stationary Camera • Using 3D Model • Shape-Based 3D Matching – Speed up by using only a region of interest, eliminate unnecessary edges • Surface-Based 3D matching 7/21/2011 Copyright © 2011 Universal Robotics Page 13 Single Camera, Multiple Positions • Parts must be stationary for measurement (e.g. Stacked parts) • May require multiple images of same object from different camera locations • Good for vision guidance and part inspection 7/21/2011 Copyright © 2011 Universal Robotics Page 14 Binocular Cameras • Only one image required • Faster than multiple image approach • Can find randomly located objects • Use of standard off-theshelf cameras • Can provide full field of view of robot operating envelope 7/21/2011 Copyright © 2011 Universal Robotics Page 15 AGENDA: Accurate Robotic 3D Vision • Introduction to Robotic 3D Vision • Types of 2D, 2½D & 3D robotic vision systems • Elements of Accuracy • Choosing a 3D vision system • Q&A 7/21/2011 Copyright © 2011 Universal Robotics Page 16 3D Vision Accuracy and Robot Operating Envelope Geometry 3D Accuracy, Δz, is affected by: Object Z = object distance Δz Virtual Image 1 P1 (x1,y1,z1) P2 (x2,y2,z2) Z X Camera 1 Virtual Image 2 f=focal length Camera 2 b=baseline distance 7/21/2011 •Distance, b, between pair of cameras •Distance, Z, from object to cameras •Disparity, Δd, offset in x between image points, P1 & P2 •Focal length, (pixels, not mm) 2 Δz = Z f·b · Δd where f (pixels) = f of Lens (mm) * Camera Horiz. Resolution (px) / CCD Horiz. sensor width (mm) Copyright © 2011 Universal Robotics Page 17 3D Camera Positions for Objects on Flat Surface 2nd pair orthogonal ~100mm apart @ 45˚ 1st pair cameras just above robot at ~100mm apart, 45˚ below horizontal NOTE: Irregular objects require about 300px by 300px for determining position & pose Robot ±150mm depending on part & envelope dimensions Robot Work Envelope Robot Envelope Height Robot Envelope Depth Parts / Boxes on Flat Surface 7/21/2011 Copyright © 2011 Universal Robotics Page 18 NOTE: Rectangular objects require about 200px by 200px for determining position & pose Camera Positions for Objects in Bin 1st pair cameras just above robot at ~100mm apart, 60˚ below horizontal 2nd pair orthogonal ~100mm apart @ 60˚ NOTE: Irregular objects require about 300px by 300px for determining position & pose Robot Operating Envelope Robot operating envelope Height ±150mm depending on part & envelope dimensions Robot Envelope Depth Parts or Boxes in Bin 7/21/2011 Copyright © 2011 Universal Robotics Page 19 NOTE: Rectangular objects require about 200px by 200px for determining position & pose Robot Accuracy & Repeatability • • • Accuracy: how close a robot can reach a commanded 3D position Accuracy varies with robot speed, robot reach, and with payload Accuracy of a robot is determined by three elements of the system: – Resolution of the control system – Error operating the robot arm under closed loop servo operation – Imprecision of mechanical linkages, gears & deflections under load • • Repeatability: the ability to duplicate an action or a result every time See ISO 9283 7/21/2011 Copyright © 2011 Universal Robotics Page 20 Typical Robot Repeatability • Plotted over 100 robots from 3 companies – – • Repeatability loosely a function of: – – – • 2Kg to 1,200Kg SCARA, material handling, welding, assembly… Payload (Kg) Reach (M) Speed (˚/sec) Best repeatability approaches limit line 7/21/2011 Copyright © 2011 Universal Robotics Page 21 Robot Accuracy • Accuracy is typically worse than repeatability, not constant over workspace • Full robot calibration: – Accuracy = 2X – 4X of Repeatability; can approach 1X • Typical robot calibration: – Accuracy = 3X – 5X of Repeatability Best (2X) Repeatability Accuracy 0.015mm 0.03mm 0.06mm 0.1mm 0.2mm 0.03mm 0.06mm 0.12mm 0.2mm 0.4mm 7/21/2011 Typ. (4X) Accuracy 0.06mm 0.12mm 0.24mm 0.4mm 0.8mm Payload, Reach, Speed 2kg 5kg 35kg 50kg 275kg Copyright © 2011 Universal Robotics Page 22 0.6M 0.8M 1.3M 1.6M 2.5M 375 deg/s 270 deg/s 170 deg/s 170 deg/s 90 deg/s Overall 3D Robotic Vision System Accuracy System Accuracy results (go to Universal Robotics 3D Made Easy Calculator at www.universalrobotics.com/calc 7/21/2011 Copyright © 2011 Universal Robotics Page 23 Camera Selection Scalable vision systems enable a wide variety of cameras to fit the robot operating envelope geometry and accuracy needs USB Webcams & GigE Camera Selection Medium Size Parts & Boxes (workspace < 1.0M wide x 1.0M deep x 1.0M high) Boxes, Parts, & Pallets (workspace < 1.5M x 1.5M x 1.5M) Working Distance from Cameras to Objects (Camera selection provides <1mm camera accuracy, resulting in a system accuracy of 1-3mm within the workspace. Consult Universal Robotics for final configuration) 0.3 - 0.5M 1.0M 2.0M 2.5M USB 1.0 MP, f3.7 USB 1.3 MP, f3.5 GigE 1.4 MP, f3.5 (0.5M workspace) USB 1.3 MP, f4.5-f6 USB 2.0 MP, f3.5-4.5 GigE 1.2 MP, f3.5- 4.5 GigE 1.4 MP, f4.5 –f6 USB 2.0 MP, f6 – f8 GigE 1.2 MP, f6 - f8 GigE 1.4 MP, f8 -f12 USB 2.0 MP, f8 - f12 GigE 1.2 MP, f8 –f12 GigE 1.4 MP, f12 -f16 Consult with Universal Robotics Consult with Universal Robotics USB 1.3 MP, f6 - f8 USB 2.0 MP, f4.5 – f6 USB 2.0 MP, f6 –f8 GigE 1.4 MP, f8 – f12 7/21/2011 Copyright © 2011 Universal Robotics Page 24 AGENDA: Accurate Robotic 3D Vision • Introduction to Robotic 3D Vision • Types of 2D, 2½D & 3D robotic vision systems • Elements of Accuracy • Choosing a 3D vision system • Q&A 7/21/2011 Copyright © 2011 Universal Robotics Page 25 Attributes of Low-Cost 3D Vision WORKSPACE: __ Robot operating envelope within 1.5M x 1.5M? __ Cameras mounted within 2.5M of workspace? ROBOT: __ Part-box picked up with vacuum or grippers? (sets 3mm accuracy) __ Part or box weighs < 50 Kg? (robot size) __ ≤10 parts-boxes/min or 600/hour? (can USB2.0 cameras) PARTS OR BOXES: __ Part has visible edge and/or boundary? (enables shape-matching) YES: White egg with black background; surface with reflectivity < 75 NO: Dull black molded rubber ball with black background __ Part/box have CAD model? (shape-matching; no special lighting) __ Only one part? (scalable to multiple parts, but additional work) 7/21/2011 Copyright © 2011 Universal Robotics Page 26 3D Vision at 2D Prices • Standard off-the-shelf components scalable to multiple parts with world-class software Vision System $K 3D Vision at 2D Prices $60 $50 $40 $30 $20 $10 $1985 1990 1995 2000 2005 2D Machine Vision 3D Machine Vision 7/21/2011 Copyright © 2011 Universal Robotics Page 27 2010 Robotic Vision Options MotoSight 3D Spatial Vision 2½ D Surface Traditional 3D Precise 3D Machine Vision Inspection Random Part Pick Automated Assembly Machine Vision Precise Part Inspection/Guidance $8-$10k $8k $12k - $18k $20k - $25k Two Fixed Cameras One Camera on Robot One – Multiple Arm Fixed or on Robot One Camera on Robot Arm 3D: complex parts 2½ D: planar parts only 3D: complex parts 2½ D: planar parts only 3D shape match 3D surface match 3D geometric pattern match Stationary & Moving Parts Stationary Parts Stationary & Moving Stationary Parts Parts Three – Ten Parts One Part One Part One Part 1 - 5 mm accuracy @ 0.5 – 2.5M 1 - 5 mm @ 0.5 - 2.0M ~0.2 - 2 mm @ 0.1 - 0.5M ~ 0.12 mm @ 0.08M 7/21/2011 Copyright © 2011 Universal Robotics Page 28 3D geometric pattern match Key Benefits of Robotic 3D Vision • Locates 3D Objects in 3D Space – – Identifies 6 degrees of freedom X, Y, Z, Rx, Ry, Rz Not just 2D flat parts in space • 3D Shape Matching – – – Identifies complex objects Handles varying lighting conditions Identifies partially hidden objects • Scalable Precision – – Choose just the right cameras for your needs Only ~3mm accuracy needed for vacuum pick or grippers • 3D Vision at 2D Prices – MotoSight 3D Spatial Vision U.S. List $10,000 7/21/2011 Copyright © 2011 Universal Robotics Page 29 Robotic 3D Vision Summary • Use 3D where part complexity, location randomness, or motion require it – – Vision guidance Part inspection • 3D is a mainstream solution; requires choosing right tool & approach – Only pay for what you need; end effectors can manage from 2-3mm • Motoman robots are some of the most accurate robots in the world • Universal Robotics 3D Made Easy Calculator at www.universalrobotics.com/calc 7/21/2011 Copyright © 2011 Universal Robotics Page 30 AGENDA: Accurate Robotic 3D Vision • Introduction to Robotic 3D Vision • Types of 2D, 2½D & 3D robotic vision systems • Elements of Accuracy • Choosing a 3D vision system • Q&A 7/21/2011 Copyright © 2011 Universal Robotics Page 31 End 7/21/2011 Copyright © 2011 Universal Robotics Page 32 3D Calculator Inputs 3D Calculator Inputs 1st Pair Cameras (0,H+Offset,0) 2nd pair parallel cameras ~100mm apart; orthogonal to 1st 1st pair cameras ~100mm apart; pointed at center 2nd Pair Cameras (-½W, H+Offset,½D) Robot Envelope Height Robot Work Envelope Robot Center (0, ½H, ½D) Object Height Obj. Depth Object (-½W, 0, D ) Center Edge (0,0,0) (½W, 0, D ) Robot Envelope Depth 5/31/201 2 Copyright © 2011 Universal Robotics Page 33
© Copyright 2024