STYROFOAM Visible Light Communication System

1. STYROFOAM Robert LiKamWa, David Ramirez, Jason Holloway Advisors: Lin Zhong, Behnaam Aazhang, Ashok Veeraraghavan Rice University STYROFOAM VLCS @ Mobicom 2014

3. STYROFOAM Robert LiKamWa, David Ramirez, Jason Holloway Advisors: Lin Zhong, Behnaam Aazhang, Ashok Veeraraghavan Rice University STYROFOAM VLCS @ Mobicom 2014 A Tightly-Packed Coding Scheme for Camera-based Visible Light Communication

4. 4 STYROFOAM VLCS @ Mobicom 2014 Screen Camera Communication

5. STYROFOAM VLCS @ Mobicom 2014 5 Screen Camera Communication

6. STYROFOAM VLCS @ Mobicom 2014 6 Benefits Pairless infrastructure Multi-cast transmission Targeted interaction Use Cases Device-Specific Broadcast Wearable Transmissions Easy Access Media

7. Camera-based VLC Research Challenges Efficient Marker Tracking (COBRA) Blur-aware Ordering (COBRA) Unsynced Camera/Screen rates (LightSync) Increasing block bit-depth (Styrofoam) 7 STYROFOAM VLCS @ Mobicom 2014

8. OFF ON 8 STYROFOAM VLCS @ Mobicom 2014 Increasing block bit-depth (Styrofoam)

9. STYROFOAM Screen -> Camera Communication Understanding the channel Styrofoam Coding Scheme Implementation

10. STYROFOAM VLCS @ Mobicom 2014 10 s D s C s B s A

11. s A s B s C s D 11 STYROFOAM VLCS @ Mobicom 2014 TX:

12. s A s B s C TX: RX: s A s B s C s D 12 STYROFOAM VLCS @ Mobicom 2014

13. s A s B Inter-symbol interference s A s B s C s D 13 STYROFOAM VLCS @ Mobicom 2014 t TX: RX:

14. TX: RX: s D s A s B r j = 0,j s A + 1,j s B s A s B s C TX_ 0 : TX_ 1 : RX_ 0 : RX_ 1 : 0,j 1,j 14 STYROFOAM VLCS @ Mobicom 2014 t

15. STYROFOAM Screen -> Camera Communication Understanding the channel Styrofoam Coding Scheme Implementation 15 STYROFOAM VLCS @ Mobicom 2014

16. TX: t RX: s D s A s B r j = 0,j s A + 1,j s B s A s B s C 16 STYROFOAM VLCS @ Mobicom 2014 Styrofoam Coding Scheme Guaranteeing resolvability

17. Constraint #1 Symbol Visibility TX: RX: s D s A s B s C Capture Period 17 STYROFOAM VLCS @ Mobicom 2014

18. Constraint #1 Symbol Visibility 18 STYROFOAM VLCS @ Mobicom 2014 TX: RX: s D s A s B s C Capture Period Capture Period < Symbol Period

19. Constraint #1 Symbol Visibility Capture Period < Symbol Period This guarantees each symbol is RXed with > 0.5 19 STYROFOAM VLCS @ Mobicom 2014 TX: RX: s D s A s B s C Capture Period

20. Constraint #2 Symbol Color Fidelity Adjacent symbols must match color channels . TX: RX: s D s A s B s A s B s C Color-cross-talk 20 STYROFOAM VLCS @ Mobicom 2014

21. Constraint #2 Symbol Color Fidelity Adjacent symbols must match color channels. TX: RX: s D s A s B s A s B s C Color-cross-talk 21 STYROFOAM VLCS @ Mobicom 2014

22. Constraint #3 Symbol Resolvability TX: RX: s D s A s B r j = 0,j s B + 1,j s C s A s B s C s 1B s C r i = 0,i s A + 1,i s B s C s D r k = 0,k s 1 + 1,k s 2 22 STYROFOAM VLCS @ Mobicom 2014 Mixed Capture: 1 eqn , 2 var

23. s A s B s B s C s C Constraint #3 Symbol Resolvability TX: RX: s D s A s B s C 23 STYROFOAM VLCS @ Mobicom 2014 Mixed Capture: 1 eqn , 2 var Pure Capture: 1 eqn , 1 var r j = 0,j s B + 1,j s C r i = 0,i s A + 1,i s B r k = 1,k s C Ensure pure captures with Styrofoam Blocks

24. Constraint #3 Symbol Resolvability s A s B s C ~ ~ C B A A mixed capture target is always followed by the mixed symbols capture. Spacing => A < B < C 24 STYROFOAM VLCS @ Mobicom 2014 Ensure pure captures with Styrofoam Blocks Constraint #1: Visibility Spacing Spacing => Every symbol appears with >0.5

25. Styrofoam Block Size Styrofoam blocks are only useful when symbol >0.5 Remainder of block is < 0.5. Styrofoam blocks need only be half as wide as symbol blocks s A s B s C ~ ~ 25 STYROFOAM VLCS @ Mobicom 2014 ~ ~

26. Display each symbol for two frames, Styrofoam Block for one frame Display Frame Rate < 2 x Camera Frame Rate e.g. (Display : 30 FPS, Camera : 15 FPS)` Styrofoam Frame Timing 26 STYROFOAM VLCS @ Mobicom 2014 s D s E s F ~ ~ s D s E s F ~ s D s E s F ~ s D s E s F ~ s D s E s F s G s H s I ~ s G s H s I ~ s A s B s C ~ s A s B s C s1 s2 s 3 ~ s1 s2 s 3 s1 s2 s 3 ~ s1 s2 s 3 ~ s1 s2 s 3 ~ s1 s2 s 3

27. Styrofoam Coding s A s N ~ Data Rate s A s N Bit depth/Block/Display Frame ~ Decode Latency 27 STYROFOAM VLCS @ Mobicom 2014 Frame Latency

28. Per-block, Per-Display Frame Rate Datarate Comparisons Traditional (1-bit, blank alternate frames) : 0.5 bits COBRA (5-values, half-frame rate) : 1.16 bit LightSync ( Synchronization interference blocks ): 1 bit Styrofoam: 1.7 bits 1.7 bits * 720 blocks * 60 fps = 73.4 kbps STYROFOAM VLCS @ Mobicom 2014 28

29. STYROFOAM Screen -> Camera Communication Understanding the channel Styrofoam Coding Scheme Implementation 29 STYROFOAM VLCS @ Mobicom 2014

30. Implementation In paper: Transmitter: Blackberry Z30, 60 FPS Receiver: Nokia Lumia, 30 FPS Hardware ISO & Exposure control Decoded offline w/ MATLAB Demo: Nexus 5 + Nexus 5 Android L hardware controls 30 STYROFOAM VLCS @ Mobicom 2014

31. Styrofoam Barcode (Nexus 5) STYROFOAM VLCS @ Mobicom 2014 31

32. Styrofoam Barcode (Simplified Version) 32 STYROFOAM VLCS @ Mobicom 2014 max 1 0

33. Sample TX STYROFOAM VLCS @ Mobicom 2014 33 Even Odd

34. 34 STYROFOAM VLCS @ Mobicom 2014 s1 s2 ~ ~

35. Decoded Pairs 35 STYROFOAM VLCS @ Mobicom 2014

36. Additional Styrofoam Research Challenges Steganography Can we hide the Styrofoam blocks? Power Efficiency Of the display? Of the receiver? Leveraging YUV color channels Mixing colors safely? Geometric problems Screen -> Camera angles? 36 STYROFOAM VLCS @ Mobicom 2014

37. David Ramirez Robert LiKamWa Jason Holloway STYROFOAM Characterize and constrain inter-symbol interference with Styrofoam spacing blocks 37 STYROFOAM VLCS @ Mobicom 2014 Behnaam Aazhang Lin Zhong Ashok Veeraraghavan Advisors: Authors:

38. Constraints Display Frame > Symbol Frame Symbol Visibility for at least half capture Styrofoam block spacing To guarantee pure captures Block at least half width of symbol Color Matching To deter color filter issues STYROFOAM VLCS @ Mobicom 2014 38