Cell phones for Control of Universal Mixed media.

Uploaded on:
Human Computer Interaction Institute. School of Computer Science. Carnegie Mellon ... of other media gadgets, as opposed to simply being another contraption to ace? ...
Slide 1

Human Computer Interaction Institute School of Computer Science Carnegie Mellon University Mobile Devices for Control of Ubiquitous Multimedia Brad A. Myers bam@cs.cmu.edu http://www.pebbles.hcii.cmu.edu/

Slide 2

Handheld Devices Typically, handheld gadgets for: Communication As an "Individual Information Manager" As an "Individual Digital Assistant" As a PC substitution "PocketPC" As a media player Handheld Devices for Remote Control of different gadgets and media Remote Control of PCs Remote Control of machines

Slide 3

Problem Appliances are excessively mind boggling

Slide 4

Problem Too numerous remotes

Slide 5

Problem April 29, 1991

Slide 6

Approach Use an individual handheld gadget as an Interface to the PC and to the Appliances

Slide 7

History of Communication ParcTab – constant correspondence Early "coordinators" – no correspondence Newton – additional expense for synchronization Palm – HotSync once every day WindowsCE/PocketPC – ActiveSync once per day

Slide 8

Now, Handhelds are Communicating Toshiba e740 802.11 BlueTooth Cell-telephone system (G3) HP iPaq 3870 Microsoft SmartPhone

Slide 9

Pebbles Project Performed as a feature of the Pebbles Project Overall objective: Use of various gadgets in the meantime Palm and desktop PC Mobile Phone and " Smart Home" Multiple handhelds in a meeting Pocket PC and apparatuses Multiple clients with their gadgets Single client with numerous gadgets

Slide 10

Pebbles is: P E B L E S DAs for ntry of oth ytes and ocations from xternal ources. http://www.pebbles.hcii.edu/

Slide 11

Pebbles for MultiMedia How will handheld gadgets fit into the client\'s general media space? Where does media originate from? Where is the media is shown? How is the media controlled? In what capacity can handheld gadgets enhance the client interfaces of other media gadgets, as opposed to simply being another contraption to ace?

Slide 12

Controlling a PC Remote Commander Slide Show Commander Command Post of the Future Shortcutter http://www.pebbles.hcii.edu/

Slide 13

Remote Commander Allow PDAs to control a PC Can be utilized with any application Uses the standard (single) cursor Don\'t need to bounce up and get mouse All mouse and console capacities Use PDA like touchpad Graffiti or our own pop-up console Word forecast and consummation

Slide 14

PocketPC rendition Get PC\'s screen onto PocketPC Full view, or coordinated zooming Scroll with iPaq\'s catches Control or scrawl

Slide 15

For People with Muscular Disabilities Using handhelds as interface to PCs People with Muscular Dystrophy have fine-engine control yet lose gross engine control Difficulties with mouse and console, however stylus OK So utilize Remote Commander as PC\'s console and mouse

Slide 16

Alphabet for Accurate Gestures Use a square gap in a layout to build precision Just as simple to learn as Graffiti ™ 18% more exact for capable fledglings ( p <.02) Patent Pending Also works with joystick

Slide 17

Power Point Control Use PC to give the presentation Use hand-held to control the PC Two-way correspondence Hand-held shows picture of slide, notes, clock

Slide 18

SlideShow Commander, cont. See rundown of slide titles Scribble Tap on-screen catches and hyper-joins Preview different slides Control exhibitions

Slide 19

Control at a Distance Controlling a PC from over the room Meetings, and so forth. Utilize a laser pointer? Ponders

Slide 20

Command Post of the Future Share media & control over numerous gadgets Big shows, discourse, motion And handhelds

Slide 21

CPOF, cont. Gives remote control of fundamental PC battleboard from handhelds Also, private drill-down of open data Small forms of guide and information representations on handhelds Interact locally, and show changes and explanations when prepared

Slide 22

Shortcutter User-made boards of controls Create custom interfaces and expansions to PC applications And then bring them with you Direct control for alter, then set properties PocketPC or Palm

Slide 23

Shortcutter Widgets Buttons Virtual Joy stick Virtual Knob Sliders Mouse cushion Gesture board

Slide 24

Scenarios of Use Lean-back mail perusing Controlling media players … and numerous others

Slide 25

Controlling Appliances PhD examination of Jeffrey Nichols Assisted by students and MS understudies http://www.pebbles.hcii.edu/puc/

Slide 26

Problem Too numerous mind boggling gadgets, each with its own particular peculiar interface Stereo framework Telephones ATM Fax machine Photocopier Hotel wake up timer Increasingly mechanized Low convenience

Slide 27

Existing "widespread" controls Pre-modified at the industrial facility with a subset, Or, relentlessly hand-customized by the client

Slide 28

Our Approach Handheld is an "Individual Universal Controller" (PUC) Two-way correspondence Appliance portrays its capacities Handheld PUC Automatically makes interface Controls the apparatus Displays criticism about apparatus status Specifications Control Feedback

Slide 29

Automatic Generation of UIs Benefits All interfaces reliable for a client With traditions of the handheld Even from different makers Addresses lodging wake up timer issue Can consider client inclinations Multiple modalities (GUI + Speech UI) A Hard Problem Previous programmed frameworks have not produced superb interfaces

Slide 30

Research Challenges Automatic Design of Graphical User Interfaces Automatic Design of Speech User Interfaces Connection with genuine gadgets Through different conventions X-10, AV/C, HAVi, UPnP, and so on. Additionally, custom equipment and programming

Slide 31

Hand-Generated Graphical Interfaces First, Hand-Designed PocketPC interfaces: AIWA Shelf Stereo (Tape,CD,Tuner) AT&T Telephone/Answering Machine Used Embedded Visual Basic Ensured quality with heuristic investigation and verbally process concentrates on Compared with maker\'s interfaces

Slide 32

Results of Comparison Using PUC, clients took half less time & made half less mistakes All distinctions are critical ( p < 0.05)

Slide 33

Current PUC Specification Language XML Full documentation for the particular dialect and convention: http://www.pebbles.hcii.cmu.edu/puc/Contains test determination for a stereo

Slide 34

Properties of PUC Language State variables & summons Each can have different names Useful when insufficient room Typed variables Base sorts: Boolean, string, listed, numbers, settled point, drifting point, and so forth. Discretionary names for qualities Hierarchical Structure Groups

Slide 35

Dependency Information Crucial for top notch interfaces Expressed as <active-if> conditions Operations: Equals, Less-Than, Greater-Than Combined Logically AND, OR Used for: Dynamic turning gray out Layout Widget choice

Slide 36

Specifications Have working determinations for: Audiophase stereo X-10 lights control Sony CamCorder Windows Media Player Audio ReQuest equipment MP3 player WinAmp Media Player Elevator Parts of GMC Yukon Denali SUV Etc.

Slide 37

Controller Generators iPaq PocketPC SmartPhone No touchscreen Desktop (TabletPC) Speech

Slide 38

Examples of Generated GUIs Shelf stereo Windows Media Player

Slide 39

Generating Speech Interfaces "All inclusive Speech Interface" (USI) venture Prof. Roni Rosenfeld of CMU http://www.cs.cmu.edu/~usi Creates sentence structure, dialect model and articulation word reference from PUC detail Pronunciation from names utilizing phonetic guidelines Can give different elocutions as marks to calibrating Will utilize reliance data to help with disambiguation and clarification Supports questions and talked input Paraphrases as affirmation

Slide 40

"Connectors" give the interface to existing (and future) apparatuses If don\'t bolster particular dialect straightforwardly Custom equipment Custom programming Lutron Windows Media Player X-10 Light switches, and so forth. AV/C (standard convention) Sony CamCorder HAVi UPnP Axis Camera Adaptors

Slide 41

Demonstration Sony CamCorder Windows Media Player Image library Two way correspondence: When state changes from application or apparatus, GUI is overhauled Can question state with discourse

Slide 42

Current Work on PUC Controlling User "Encounters" One summon for various gadgets Play a DVD  sets DVD, link, TV Setting wake up timer  sets heat, espresso, auto Consistency for Users And client inclinations More gadgets UPnP Etc.

Slide 43

For More Information Many papers on the Pebbles site On PUC and other work Most projects accessible for nothing downloading: SlideShow Commander is authorized for business deal See handango.com http://www.cs.cmu.edu/~pebbles

Slide 44

Conclusions As more hardware are mechanized and conveying, cell phones can control them Handheld gadgets can enhance the client interfaces of everything else Not simply be another contraption to be scholarly

Slide 45

Thanks to Pebbles Students This exploration performed by countless since 1997:

Slide 46

The Pebbles examination is upheld by awards from: DARPA NSF Microsoft General Motors NEC Foundation Pittsburgh Digital Greenhouse And gear stipends from: Hewlett Packard Lucent Technologies Lantronix, Inc. Lutron, Inc. Mitsubishi Palm Computing Symbol Technologies IBM SMART Technologies VividLogic Synergy Solutions Handango Thanks to Our Sponsors!

Slide 47

Human Computer Interaction Institute School of Computer Science Carnegie Mellon University Thank You! Cell phones for Control of Ubiquitous Multimedia Brad A. Myers bam@cs.cmu.edu http://www.pebbles.hcii.cmu.edu/

View more...