Evolution Of MIPI CSI-2 Imaging Conduit
Evolution of MIPI CSI-2 Imaging Conduit October 2018 Haran Thanigasalam Intel IC & Sr. Architect MIPI CWG Chair 1 2018 MIPI Alliance, Inc.
About MIPI Alliance MIPI is a global, collaborative organization founded in 2003 that comprises 300 member companies spanning the mobile and mobileinfluenced ecosystems. MIPI’s mission: To provide the hardware and software interface specifications device vendors need to create state-of-the-art, innovative mobileconnected devices while accelerating time-tomarket and reducing costs 2 2018 MIPI Alliance, Inc.
MIPI for Mobile To date, MIPI has developed more than 45 specifications. In fact, every smartphone on the market today has at least one MIPI specification. 3 2018 MIPI Alliance, Inc.
4 Leveraging MIPI Beyond Mobile Characteristics of MIPI specifications MIPI specifications are suitable for use in a variety of other applications because they've been optimized for: High bandwidth Low power consumption Low electromagnetic interference 4 2018 MIPI Alliance, Inc.
Momentum MIPI CSI-2 Complex problems cannot be solved by an individual company: Autonomous vehicles Paramedic drones Device authentication Machine awareness Need for global alliances 5 2018 MIPI Alliance, Inc.
The Drive Perception l Subjective Experience l Reality Fundamental Drivers: Fear l Desire Maximize Pleasure I Minimize Pain Rapid Technological Advancements Credit: Daniel Siegel https://books.google.com/books/about/The Mindful Therapist A Clinician s Guid.html?id gFo zhxys0AC&printsec frontcover&source kp read button#v onepage&q&f true 6 2018 MIPI Alliance, Inc.
Two different CSI architectures Applications CSI-3 CSI-2 Transport PHY UniPro A-PHY C-PHY D-PHY M-PHY Continuing to evolve and advance CSI-2 imaging conduit targeting multiple platforms CSI-3 is not a next-gen solution, nor superset, nor backwards compatible with CSI-2 7 2018 MIPI Alliance, Inc.
The Bridge Ingredients Photon Collectors, Transistors, Optics, Emitters, Algorithms CSI-2 Imaging Conduit Applications Broad Spectrum of Imaging & Vision Use cases targeting multiple platforms 2018 Global Alliance Engagements: 1. MIPI Alliance (Camera & Imaging WG) 2. Stanford Innovations in Psychiatry and Behavioral health (VR) 3. Embedded Vision Alliance 4. Image Sensor Auto Clear problem definitions Map features to use cases ( 70% reject) Unified Imaging Software Driver End-to-end Conformance Test Suite URL 8 2018 MIPI Alliance, Inc.
Platform Scope Mobile IOT Autonomous Target Platform: Smart Phones Target Platform: Content Creation (AIO, Notebooks, Hub) Target Platform: Automotive Target Imaging Application: Pristine still photography l streaming video Target Imaging Application: Comprehensive E2E imaging conduit to enable machine awareness Target Imaging Application: Real time perception and decicion making 9 2018 MIPI Alliance, Inc.
CSI-2 over C/D-PHYs (2 Lanes) CSI-2 Imaging over D-PHY Image Sensor CSI-2 D-PHY Periodic clock D-PHY TX (6-pins) CSI-2 TX Application Processor Data D-PHY RX (6-pins) CSI-2 RX 1x Effective Pixel BW I2C Compatible 2-wire Camera Control Gyro AF OIS MEMS Flash Pin compatible CSI-2 Imaging over C-PHY Image Sensor CSI-2 C-PHY Application Processor Embedded clock & data C-PHY TX (6-pins) CSI-2 TX C-PHY RX (6-pins) CSI-2 RX 2.3x Effective Pixel BW I2C Compatible 2-wire Camera Control AF Gyro OIS Flash MEMS 10 2018 MIPI Alliance, Inc.
CSI-2 over C-PHY 2.3x Effective BW 3 terminated conductors CSI-2 policy: symbol transition each state for trivial CDR Theoretical BW gain: log2 5 As implemented for CSI-2 16/7 11 2018 MIPI Alliance, Inc.
Algorithm is King Unified Imaging Software Driver Enable Image Sensor Capabilities for Vision Compute Accelerate Bring up on Reference Platforms 12 2018 MIPI Alliance, Inc.
CSI-2 CCS 13 2018 MIPI Alliance, Inc.
Deep Neural Networks Myriad X Hybrid Arch Deep Neural Network 8 Cameras using 16 MIPI Lanes - 700 MP/s Drone Collision Avoidance (DJI SPARK) l-network 14 2018 MIPI Alliance, Inc.
LRTE Dramatically Improve Sensor Aggregation Optimal Transport Preserving Integrity Real-time Perception & Decision Making Phase out EOS &IL Impediments Asymmetric SP/LP Spacers 15 2018 MIPI Alliance, Inc.
16 2018 MIPI Alliance, Inc.
Virtual Channel Expansion System Considerations Proliferation of Image Sensors Require CSI-2 Aggregators and VC Expansion System is limited to 16 or lower VCs: – Supported by CSI-2 over C/D-PHY System requires more than 16 VCs: – Supported by CSI-2 over C-PHY URL URL 17 2018 MIPI Alliance, Inc.
CSI-2 Unified Serial Link CSI-2 USL over D-PHY Image Sensor Module CSI-2 TX Application Processor Periodic FWD clock D-PHY TX Long Reach Wire Reduction Encapsulation Secure Channel CSI-2 USL over C-PHY Data0 D-PHY RX CSI-2 RX Image Sensor Module CSI-2 TX C-PHY RX C-PHY TX Camera Control Interface (CCI) over I2C / I3C / SPI Camera Control Interface (CCI) over I2C / I3C / SPI GPIOs GPIOs USL Application Processor Periodic FWD clock D-PHY TRX Data0 CSI-2 RX USL Image Sensor Module CSI-2 USL Application Processor EMB CD TRIO 0 D-PHY TRX Image Sensor Module Application Processor EMB CD TRIO 0 CSI-2 USL CSI-2 USL C-PHY TRX C-PHY TRX CSI-2 USL 18 2018 MIPI Alliance, Inc.
Interleaved Encryption Enterprise Security Pixel Authentication VC Mapped Interleaved Encryption Threat Model & Use Cases (GCM / AES) 19 2018 MIPI Alliance, Inc.
Machine Awareness (ECV) UID CSI-2 Imaging Conduit APP ECV SAD HAPPY BORED Phase 1: LRTE & PSD (EOY 2016) Phase 2: USL & SROI(EOY 2018) Phase 3: AOSC (EOY 2020) 20 2018 MIPI Alliance, Inc. SNS SNS
DPCM Objective Qualification Enable 10-bit compression of RAW-12 video image with better IQ than prior version 12-8-12 Reduce maximum absolute error of single-bit change in pixel value by a factor of 4.43x Qualified 5 degree slanted edge input image with low, medium, and high illumination levels: – 12-10-12 virtually indistinguishable from original image – MTF frequency response analysis closely tracks the original (HI/MI/LI - LC/MC/HC) Benefits include Link BW reduction cost savings 21 2018 MIPI Alliance, Inc.
PSD Reduction CSI-2 over D-PHY PSD emission reduction with scrambling (data lanes) CSI-2 over C-PHY PSD emission reduction with scrambling (embedded clock and data) 22 2018 MIPI Alliance, Inc.
Smart Region of Interest 3-layer stacked image sensor Phase 1 EOY 2018 Single Frame Phase 2 EOY 2019 Multi Frame ARCH: Edge Hybrid -3layer-stacked-cmos-image-sensor-technology/ 23 2018 MIPI Alliance, Inc.
Functional Safety 24 2018 MIPI Alliance, Inc.
Graceful Link Degradation 25 2018 MIPI Alliance, Inc.
Platform Architectures SNS AGG CSI-2 FEATURES ARCH SROI IENC USL GTA BET FUNC SAFETY F NF ERR LRTE ALP PDQ GLD H F DUPLEX T S SYNC CENTRAL SNS MODULE HEALTH MONITOR IRX VCM GYRO ACC EPROM LED LASER EDGE / DISTRIBUTED HYBRID APP ISP CVE GFX PHY DISCRETE INTEGRATED 26 2018 MIPI Alliance, Inc.
Evolving CSI-2 Architecture for Automotive CSI-2 Aggregators may benefit from Virtual Channel Expansion and LRTE PDQ CSI-2 Vision Long Range Radar CSI-2 streaming data from aggregated image sensors for near real-time perception and decision making platform solutions UltraSonic Video Camera / LiDAR CSI-2 Imaging Conduit - Mobile Devices - Wearables - IoT Appliances - Enterprise - Industrial - Medical - Automotive - Submarines - Exoplanet Rover - 27 2018 MIPI Alliance, Inc.
Interconnect Support Mid-Range solutions targeting EOY 2018 development Long-Range solution targeting EOY 2019 development 28 2018 MIPI Alliance, Inc.
Summary Engage Imaging & Vision Global Alliances One World l One Imaging Conduit l MIPI CSI-2 IOT Platform EOY 2018 – – – – – Automotive Platform EOY 2019 – – – – USL SROI (Single Frame) RAW-24 EOTP Interop Fixes Functional Safety IENC l Pixel Integrity SROI (Multi Frame) Joint Imaging & Security BOF (January 2019) Balance Cleverness with Wisdom – Inclusive Vs Exclusive URL Machine Awareness EOY 2019 29 2018 MIPI Alliance, Inc.
Questions www.mipi.org 30 2018 MIPI Alliance, Inc.
RX CSI - 2 RX D - PHY CSI - 2 TX TX Periodic FWD clock Data 0 Camera Control Interface ( CCI ) over I 2 C / I 3 C / SPI Image Sensor Module Application Processor GPIOs D - PHY TRX CSI - 2 USL D - PHY CSI - 2 TRX USL Image Sensor Module Application Processor C - PHY RX CSI - 2 RX C - PHY CSI - 2 TX TX EMB _ CD _ TRIO _ 0
MIPI CSI- 2 . Front Camera Design a sensor aggregator chip on ECU side - ECU support 1 or 2 sensors System configurable - Sensors One high lane rate Two mid-range lane rate How can MIPI C/D-PHY support this with MIPI CSI-2 controller? MIPI CSI-2 RX. ECU. C/D-PHY. Bridge. MIPI CSI- 2 . Rear Camera. MIPI CSI-2 TX
2.2. MIPI signal CSI-2 uses the MIPI standard for the D-PHY physical layer. This document provides an overview of the MIPI signal format. For more information about the MIPI specification, see MIPI Alliance Standard for Camera Serial Interface 2 documentation at mipi.org. 2.2.1. Lanes CSI-2 is a lane-scalable specification.
MIPI CSI-2 RX Subsystem v2.2 www.xilinx.com 6 PG232 April 05, 2017 Chapter 1: Overview Sub-Core Details MIPI D-PHY The MIPI D-PHY IP core implements a D-PHY RX interface and provides PHY protocol layer support compatible with the CSI-2 RX interface. See the MIPI D-PHY LogiCORE IP Product Guide (PG202) [Ref 3] for details. MIPI D-PHY .
The MIPI CSI-2 RX Controller core consists of multiple layers defined in the MIPI CSI-2 RX 1.1 specification, such as the lane management layer, low level protocol and byte to pixel conversion. The MIPI CSI-2 RX Controller core receives 8-bit data per lane, with support for up to 4 lanes, from the MIPI D-PHY core through the PPI. As shown in .
The MIPI CSI-2 RX Controller core consists of multiple layers defined in the MIPI CSI-2 RX 1.1 specification, such as the lane management layer, low level protocol and byte to pixel conversion. The MIPI CSI-2 RX Controller core receives 8-bit data per lane, with support for up to 4 lanes, from the MIPI D-PHY core through the PPI. As shown in .
VC 2,RAW8 VC 3,RAW8 MIPI CSI-2 Mux Chip MIPI Single Channel VC 0,RAW10 VC 1,RAW10 VC 2,RAW10 VC 3,RAW10 VC 0,RAW12 VC 1,RAW12 VC 2,RAW12 VC 3,RAW12 üTotal bandwidth available üInterleaving as supported MIPI CSI-2 Specification üMaximum input channels supported by mux chip CSI-2Rx Video Processing Visualization Mux Chip How many cam's can .
MIPI CSI-2 is a standard specification defined by a Mobile Industry Processor Interface (MIPI) alliance. The Camera Serial Interface 2 (CSI-2) specification defines an interface between a peripheral device (camera) and a host processor (base-band, application engine). This user guide describes the MIPI CSI-
CSI-2 format, and transmits through four pairs of differential wires. Features AEC-Q100 Qualified (Note 1) Functional Safety Supportive Automotive Products Input Interface: Clockless Link-BD 1 Lane Output Interface: MIPI D-PHY/CSI-2 4 Lane MIPI CSI-2 Ver.1.3 / MIPI D-PHY Ver1.2 Video Formats: YUV422, RAW8/10/12 Supported Repeater Mode
