A Non-prominent Head Mounted Face Capture System Chandan K. Reddy Master\'s Thesis Defense Thesis Committee: Dr. George C. Stockman (Main Advisor) Dr. Forthcoming Biocca (Co-Advisor) Dr. Charles Owen Dr. Jannick Rolland (External Faculty)Slide 2
Modes of Communication Text just - e.g. Mail, Electronic Mail Voice just – e.g. Phone PC camera based conferencing – e.g. Web cam Multi-client Teleconferencing through Virtual Environments Augmented Reality Based TeleconferencingSlide 3
Problem Definition Face Capture System ( FCS ) Virtual View Synthesis Depth Extraction and 3D Face Modeling Head Mounted Projection Displays 3D Tele-immersive Environments High Bandwidth Network ConnectionsSlide 4
Thesis Contributions Complete equipment setup for the FCS. Camera-reflect parameter estimation for the ideal arrangement of the FCS. Era of value frontal recordings from two side recordings Reconstruction of surface mapped 3D confront demonstrate from two side perspectives Evaluation components for the produced frontal perspectives.Slide 5
Existing Face Capture Systems FaceCap3d - an item from Standard Deviation Optical Face Tracker – an item from Adaptive Optics Courtesy : Advantages : Freedom for Head Movements Drawbacks : Obstruction of the client\'s Field of view Main Applications : Character Animation and Mobile situationsSlide 6
Existing Face Capture Systems Courtesy: Sea of Cameras (UNC Chappel Hill) National tele-drenching Initiative Advantages : No weight for the client Drawbacks : Highly prepared situations and limited head movement Main Applications : Teleconferencing and Collaborative workSlide 7
Proposed Face Capture System (F. Biocca and J. P. Rolland, "Teleportal eye to eye framework", Patent Filed, 2000.) Novel Face Capture System that is being produced. Two Cameras catch the relating side perspectives through the mirrorsSlide 8
Advantages User\'s field of view is unhampered Portable and simple to utilize Gives exceptionally precise and quality face pictures Can prepare continuously Simple and easy to understand framework Static concerning human head Flipping the mirror – cameras see the client\'s perspectiveSlide 9
Applications Mobile Environments Collaborative Work Multi-client Teleconferencing Medical Areas Distance Learning Gaming and Entertainment industry OthersSlide 10
System DesignSlide 11
Equipment RequiredSlide 12
Optical Layout Three Components to be viewed as Camera Mirror Human FaceSlide 13
Specification Parameters Camera Sensing region: 3.2 mm X 2.4 mm (¼"). Pixel Dimensions: Image detected is of measurements 768 X 494 pixels. Digitized picture size is 320 X 240 because of limitations of the RAM measure. Central Length(Fc): 12 mm (VCL – 12UVM). Field of View (FOV): 15.2 0 X 11.4 0 . Distance across (Dc): 12mm Fnumber (Nc): 1 - accomplish most extreme daintiness. Least Working Distance (MWD)- 200 mm. Profundity of Field (DOF): to be evaluatedSlide 14
Specification Parameters (Contd.) Mirror Diameter (Dm)/Fnumber (Nm) Focal Length (fm) Magnification consider (Mm) Radius of arch (Rm) Human Face Height of the face to be caught (H~ 250mm) Width of the face to be caught (W~ 175 mm) Distances Distance between the camera and the mirror. (D cm ~150mm) Distance between the mirror and the face. (D mf ~200mm)Slide 15
Customization of Cameras and Mirrors Off-the-rack cameras Customizing camera focal point is a repetitive assignment Trade-off must be made between the field of view and the profundity of field Sony DXC LS1 with 12mm focal point is appropriate for our application Custom planned mirrors A plano-raised focal point with 40mm measurement is covered with dark on the planar side. The span of ebb and flow of the raised surface is 155.04 mm. The thickness at the focal point of the focal point is 5 mm. The thickness at the edge is 3.7 mm.Slide 16
Block graph of the frameworkSlide 17
Experimental setupSlide 18
Virtual Video SynthesisSlide 19
Problem Statement Generating virtual frontal view from two side perspectivesSlide 20
Data preparing Two synchronized recordings are caught progressively (30 outlines/sec) at the same time. For viable catching and preparing, the information is put away in uncompressed arrange. Machine Specifications (Lorelei @ metlab.cse.msu.edu): Pentium III Processor speed: 746 MHz RAM Size: 384 MB Hard Disk compose Speed (functional): 9 MB/s MIL-LITE is designed to utilize 150 MB of RAMSlide 21
Data preparing (Contd.) Size of 1 second video = 30 * 320 * 240 *3 = 6.59 MB Using 150 MB RAM, just 10 seconds video from two cameras can be caught Why does the handling need to be disconnected? Adjustment system is not programmed Disk composing speed must be no less than 14 MB/S. To catch 2 recordings of 640 * 480 determination, the Disk composing speed must be no less than 54 MB/S ???Slide 22
Structured Light system Projecting a network on the frontal perspective of the face A square framework in the frontal view shows up as a quadrilateral (with bended edges) in the genuine side viewSlide 23
Color Balancing Hardware based approach White adjusting of the cameras Why this is more strong ? – why not programming based ? There is no adjustment in the info camera Better treatment of shifting lighting conditions No pre - information of the skin shading is required No extra overhead Its enough if both cameras are shading adjusted moderatelySlide 24
Off-line Calibration Stage Left Calibration Face Image Right Calibration Face Image Projector Transformation TablesSlide 25
Operational Stage Left Face Image Right Face Image Transformation Tables Left Warped Face Image Right Warped Face Image Mosaiced Face ImageSlide 26
Virtual video union (Calibration stage)Slide 27
Virtual video union (contd.)Slide 28
Virtual Frontal VideoSlide 29
Comparison of the Frontal Views First column – Virtual frontal perspectives Second line – Original frontal perspectivesSlide 30
Video Synchronization (Eye flickering) First line – Virtual frontal perspectives Second line – Original frontal perspectivesSlide 31
Face Data through Head Mounted SystemSlide 32
3D Face ModelSlide 33
Coordinate Systems There are five facilitate frameworks in our application World Coordinate System (WCS) Face Coordinate System (FCS) Left Camera Coordinate framework (LCCS) Right Camera Coordinate framework (RCCS) Projector Coordinate System (PCS)Slide 34
s L P r C 11 C 12 C 13 C 14 C 21 C 22 C 23 C 24 C 31 C 32 C 33 1 s W P x s W P y s L P c = s W P z s 1 Camera Calibration Conversion from 3D world directions to 2D camera organizes - Perspective Transformation Model Eliminating the scale consider u j = (c 11 – c 31 u j ) x j + (c 12 – c 32 u j ) y j + (c 13 – c 33 u j ) z j + c 14 v j = (c 21 – c 31 vj) xj + (c 22 – c 32 vj) yj + (c 23 – c 33 vj) zj + c 24Slide 35
Calibration circle A circle can be utilized for Calibration focuses on the circle are picked in a manner that the Azimuthal edge is fluctuated in ventures of 45 o Polar edge is differed in ventures of 30 o The area of these alignment focuses is known in the 3D arrange System as for the starting point of the circle The root of the circle characterizes the birthplace of the World Coordinate SystemSlide 36
Projector Calibration Similar to Camera Calibration 2D picture directions can not be gotten specifically from a 2D picture. A "Clear Image" is anticipated onto the circle The 2D directions of the adjustment focuses on the anticipated picture are noted More focuses can be seen from the projector\'s perspective – a few focuses are basic to both camera sees Results seem to have marginally more blunders when contrasted with the camera alignmentSlide 37
3D Face Model Construction Why? To acquire diverse perspectives of the face To create the stereo combine to view it in the HMPD Steps required Computation of 3D Locations Customization of 3D Model Texture MappingSlide 38
Computation of 3D focuses 3d point estimation utilizing Stereo between two cameras is impractical on account of the impediment by the facial components Hence two stereo match calculations Left camera and projector Right camera and projector Using stereo, register 3D purposes of unmistakable facial element focuses in FCSSlide 39
3D Generic Face Model A non specific face display with 395 vertices and 818 triangles Left: front view and Right: side viewSlide 40
Texture Mapped 3D FaceSlide 41
Evaluation Schemes Evaluation of outward appearances and is not examined widely in writing Evaluation should be possible for facial arrangement, confront acknowledgment for static pictures Lip and eye developments in a dynamic occasion Perceptual quality – How are the inclinations passed on? Two sorts of assessment Objective assessment Subjective assessmentSlide 43
Objective Evaluation Theoretical Evaluation No human input required This assessment can give us a measure of Face acknowledgment Face arrangement Facial developments Methods connected Normalized cross relationship Euclidean separation measuresSlide 44
Evaluation Images 5 casings were considered for target assessment First column – virtual frontal perspectives Second line – unique frontal perspectivesSlide 45
Normalized Cross-Correlation Regions considered for standardized cross-relationship ( Left: Real picture Right: Virtual picture)Slide 46
Normalized Cross-Correlation Let V be the virtual picture and R be the genuine picture Let w be the width and h be the tallness of the pictures The Normalized Cross-relationship between\'s the two pictures V and R is given by whereSlide 47
Normalized Cross-CorrelationSlide 48
Euclidean Distance measures Euclidean separation between two focuses i and j is given by Let Rij be the euclidean separation between two focuses i and j in the genuine picture Let Vij be the euclidean separation between two focuses i and j in the virtual picture Dij = | Rij - Vij | .:t
To expand autonomy in working towards individual wellness targets ... I have realized what pract ...
fabricated through hereditary engeneering. Human Carbonic Anhydrase ... Hereditary building bio- ...
produced through hereditary engeneering. Human Carbonic Anhydrase ... Hereditary building bio-im ...
Perusing propensities are for the most part altered by age 30. Since 1972 daily papers have had ...
Blackout. Head/Brain injuryTemporary impedance of cerebrum functionMOI: Direct or Indirect hit t ...
Information Capture. Information catch means acquiring information for a computer.Collecting rep ...
***Please Note***. The data contained in the accompanying presentation is for the general prepar ...
Why do it along these lines?. Catalogers at libraries which have Sirsi as their ILS may want to ...
MANDING How to capture and contrive motivation. OBJECTIVES. Define mand Explain the benefi ...
. La comunicazione non verbale. La comunicazione umana avviene attraverso l\'uso simultaneo e *i ...
Presentation Overview. HistoryVineyardsWines . . . History. Toasted Head was built up in 1995 in ...
Our Role in WESTCARB. Integrate dataStore information on reliable basisEasily show information i ...
Framework of our exercises . 1) streak calcination of limestone2) entrained stream carbonation3) ...
SFAS Number 93-Depreciation (1987). Required that all NPO\'s under FASB locale record deteriorat ...
Building Internet Services. What can non-specialists do today?Can rent infrastructure.Amazon EC2 ...
Console information passage. the most well-known information devicesuitable for an extensive var ...
Bolster Vector Machines. Overlooked last time,Important Extension:Multi-Class SVMsHsu
Presentation diagram. Parts of non-money related disclosureWhy ought to non-budgetary exposures ...