A Non-prominent Head Mounted Face Capture System .

Uploaded on:
Category: News / Events
Methods of Communication. Content just - e.g. Mail, Electronic MailVoice just
Slide 1

A Non-prominent Head Mounted Face Capture System Chandan K. Reddy Master\'s Thesis Defense Thesis Committee: Dr. George C. Stockman (Main Advisor) Dr. Forthcoming Biocca (Co-Advisor) Dr. Charles Owen Dr. Jannick Rolland (External Faculty)

Slide 2

Modes of Communication Text just - e.g. Mail, Electronic Mail Voice just – e.g. Phone PC camera based conferencing – e.g. Web cam Multi-client Teleconferencing through Virtual Environments Augmented Reality Based Teleconferencing

Slide 3

Problem Definition Face Capture System ( FCS ) Virtual View Synthesis Depth Extraction and 3D Face Modeling Head Mounted Projection Displays 3D Tele-immersive Environments High Bandwidth Network Connections

Slide 4

Thesis Contributions Complete equipment setup for the FCS. Camera-reflect parameter estimation for the ideal arrangement of the FCS. Era of value frontal recordings from two side recordings Reconstruction of surface mapped 3D confront demonstrate from two side perspectives Evaluation components for the produced frontal perspectives.

Slide 5

Existing Face Capture Systems FaceCap3d - an item from Standard Deviation Optical Face Tracker – an item from Adaptive Optics Courtesy : Advantages : Freedom for Head Movements Drawbacks : Obstruction of the client\'s Field of view Main Applications : Character Animation and Mobile situations

Slide 6

Existing Face Capture Systems Courtesy: Sea of Cameras (UNC Chappel Hill) National tele-drenching Initiative Advantages : No weight for the client Drawbacks : Highly prepared situations and limited head movement Main Applications : Teleconferencing and Collaborative work

Slide 7

Proposed Face Capture System (F. Biocca and J. P. Rolland, "Teleportal eye to eye framework", Patent Filed, 2000.) Novel Face Capture System that is being produced. Two Cameras catch the relating side perspectives through the mirrors

Slide 8

Advantages User\'s field of view is unhampered Portable and simple to utilize Gives exceptionally precise and quality face pictures Can prepare continuously Simple and easy to understand framework Static concerning human head Flipping the mirror – cameras see the client\'s perspective

Slide 9

Applications Mobile Environments Collaborative Work Multi-client Teleconferencing Medical Areas Distance Learning Gaming and Entertainment industry Others

Slide 10

System Design

Slide 11

Equipment Required

Slide 12

Optical Layout Three Components to be viewed as Camera Mirror Human Face

Slide 13

Specification Parameters Camera Sensing region: 3.2 mm X 2.4 mm (¼"). Pixel Dimensions: Image detected is of measurements 768 X 494 pixels. Digitized picture size is 320 X 240 because of limitations of the RAM measure. Central Length(Fc): 12 mm (VCL – 12UVM). Field of View (FOV): 15.2 0 X 11.4 0 . Distance across (Dc): 12mm Fnumber (Nc): 1 - accomplish most extreme daintiness. Least Working Distance (MWD)- 200 mm. Profundity of Field (DOF): to be evaluated

Slide 14

Specification Parameters (Contd.) Mirror  Diameter (Dm)/Fnumber (Nm) Focal Length (fm) Magnification consider (Mm) Radius of arch (Rm) Human Face Height of the face to be caught (H~ 250mm) Width of the face to be caught (W~ 175 mm) Distances Distance between the camera and the mirror. (D cm ~150mm) Distance between the mirror and the face. (D mf ~200mm)

Slide 15

Customization of Cameras and Mirrors Off-the-rack cameras Customizing camera focal point is a repetitive assignment Trade-off must be made between the field of view and the profundity of field Sony DXC LS1 with 12mm focal point is appropriate for our application Custom planned mirrors A plano-raised focal point with 40mm measurement is covered with dark on the planar side. The span of ebb and flow of the raised surface is 155.04 mm. The thickness at the focal point of the focal point is 5 mm. The thickness at the edge is 3.7 mm.

Slide 16

Block graph of the framework

Slide 17

Experimental setup

Slide 18

Virtual Video Synthesis

Slide 19

Problem Statement Generating virtual frontal view from two side perspectives

Slide 20

Data preparing Two synchronized recordings are caught progressively (30 outlines/sec) at the same time. For viable catching and preparing, the information is put away in uncompressed arrange. Machine Specifications (Lorelei @ metlab.cse.msu.edu): Pentium III Processor speed: 746 MHz RAM Size: 384 MB Hard Disk compose Speed (functional): 9 MB/s MIL-LITE is designed to utilize 150 MB of RAM

Slide 21

Data preparing (Contd.) Size of 1 second video = 30 * 320 * 240 *3 = 6.59 MB Using 150 MB RAM, just 10 seconds video from two cameras can be caught Why does the handling need to be disconnected? Adjustment system is not programmed Disk composing speed must be no less than 14 MB/S. To catch 2 recordings of 640 * 480 determination, the Disk composing speed must be no less than 54 MB/S ???

Slide 22

Structured Light system Projecting a network on the frontal perspective of the face A square framework in the frontal view shows up as a quadrilateral (with bended edges) in the genuine side view

Slide 23

Color Balancing Hardware based approach White adjusting of the cameras Why this is more strong ? – why not programming based ? There is no adjustment in the info camera Better treatment of shifting lighting conditions No pre - information of the skin shading is required No extra overhead Its enough if both cameras are shading adjusted moderately

Slide 24

Off-line Calibration Stage Left Calibration Face Image Right Calibration Face Image Projector Transformation Tables

Slide 25

Operational Stage Left Face Image Right Face Image Transformation Tables Left Warped Face Image Right Warped Face Image Mosaiced Face Image

Slide 26

Virtual video union (Calibration stage)

Slide 27

Virtual video union (contd.)

Slide 28

Virtual Frontal Video

Slide 29

Comparison of the Frontal Views First column – Virtual frontal perspectives Second line – Original frontal perspectives

Slide 30

Video Synchronization (Eye flickering) First line – Virtual frontal perspectives Second line – Original frontal perspectives

Slide 31

Face Data through Head Mounted System

Slide 32

3D Face Model

Slide 33

Coordinate Systems There are five facilitate frameworks in our application World Coordinate System (WCS) Face Coordinate System (FCS) Left Camera Coordinate framework (LCCS) Right Camera Coordinate framework (RCCS) Projector Coordinate System (PCS)

Slide 34

s L P r C 11 C 12 C 13 C 14 C 21 C 22 C 23 C 24 C 31 C 32 C 33 1 s W P x s W P y s L P c = s W P z s 1 Camera Calibration Conversion from 3D world directions to 2D camera organizes - Perspective Transformation Model Eliminating the scale consider u j = (c 11 – c 31 u j ) x j + (c 12 – c 32 u j ) y j + (c 13 – c 33 u j ) z j + c 14 v j = (c 21 – c 31 vj) xj + (c 22 – c 32 vj) yj + (c 23 – c 33 vj) zj + c 24

Slide 35

Calibration circle A circle can be utilized for Calibration focuses on the circle are picked in a manner that the Azimuthal edge is fluctuated in ventures of 45 o Polar edge is differed in ventures of 30 o The area of these alignment focuses is known in the 3D arrange System as for the starting point of the circle The root of the circle characterizes the birthplace of the World Coordinate System

Slide 36

Projector Calibration Similar to Camera Calibration 2D picture directions can not be gotten specifically from a 2D picture. A "Clear Image" is anticipated onto the circle The 2D directions of the adjustment focuses on the anticipated picture are noted More focuses can be seen from the projector\'s perspective – a few focuses are basic to both camera sees Results seem to have marginally more blunders when contrasted with the camera alignment

Slide 37

3D Face Model Construction Why? To acquire diverse perspectives of the face To create the stereo combine to view it in the HMPD Steps required Computation of 3D Locations Customization of 3D Model Texture Mapping

Slide 38

Computation of 3D focuses 3d point estimation utilizing Stereo between two cameras is impractical on account of the impediment by the facial components Hence two stereo match calculations Left camera and projector Right camera and projector Using stereo, register 3D purposes of unmistakable facial element focuses in FCS

Slide 39

3D Generic Face Model A non specific face display with 395 vertices and 818 triangles Left: front view and Right: side view

Slide 40

Texture Mapped 3D Face

Slide 41


Slide 42

Evaluation Schemes Evaluation of outward appearances and is not examined widely in writing Evaluation should be possible for facial arrangement, confront acknowledgment for static pictures Lip and eye developments in a dynamic occasion Perceptual quality – How are the inclinations passed on? Two sorts of assessment Objective assessment Subjective assessment

Slide 43

Objective Evaluation Theoretical Evaluation No human input required This assessment can give us a measure of Face acknowledgment Face arrangement Facial developments Methods connected Normalized cross relationship Euclidean separation measures

Slide 44

Evaluation Images 5 casings were considered for target assessment First column – virtual frontal perspectives Second line – unique frontal perspectives

Slide 45

Normalized Cross-Correlation Regions considered for standardized cross-relationship ( Left: Real picture Right: Virtual picture)

Slide 46

Normalized Cross-Correlation Let V be the virtual picture and R be the genuine picture Let w be the width and h be the tallness of the pictures The Normalized Cross-relationship between\'s the two pictures V and R is given by where

Slide 47

Normalized Cross-Correlation

Slide 48

Euclidean Distance measures Euclidean separation between two focuses i and j is given by Let Rij be the euclidean separation between two focuses i and j in the genuine picture Let Vij be the euclidean separation between two focuses i and j in the virtual picture Dij = | Rij - Vij | .:t

View more...