Advances In CMOS Image Sensors And Associated . - Framework

3m ago
861.26 KB
13 Pages
Last View : 1m ago
Last Download : n/a
Upload by : Jayda Dunning

Advances in CMOS Image SensorsandAssociated Processing (Part 2)First Presented at the Hollywood Professional Alliance (HPA) Tech Retreat in PalmSprings on February 19th, 2016Shin Kikuchi - Semiconductor Device Product Design Division, Canon Inc., JapanDaisuke Kobayashi - Semiconductor Device Product Design Division, Canon Inc., JapanMr. Hitoshi Yasuda - Integrated Design Department, Canon Inc., JapanHajime Ueno - Senior Technical Specialist, ITCG, Canon USA Inc.Laurence Thorpe - Senior Fellow, ITCG, Canon USA Inc.

ABSTRACTPart I of this series focused on Canon technology that exploited the large size of the 35mm Full FrameCMOS image sensor with the modest spatial sampling of 1920 (H) x 1080 (V) to realize a uniquely largephotosite of 19um x 19um.Part II of this series focuses on a new Super 35mm CMOS image sensor specifically developed to supportorigination of High Dynamic Range (HDR) motion imagery. The deployment of two separatephotodiodes within each photosite is central to achieving the 15-stop dynamic range. The dualphotodiode also supports a unique in-sensor phase detection strategy that is followed by powerful dataprocessing that closes a focus control loop around the cine lens. Alternatively, for those who prefermanual focus operation, a separate data process provides a Focus Guide in the form of signaling to theviewfinder that unambiguously indicates achievement of sharpest focus on the chosen subject.1

ALTERNATIVE TO ALGORITHMIC DEBAYERING – DIRECT COMPONENTREADOUTThe traditional single wire output from a Bayer image sensor – as outlined in Figure 1 – entailsformulating the serial data stream into whatever file format the individual camera designers favor. Thismust subsequently be decoded to create the individual RGB frames. Of necessity this decoding entailssophisticated algorithms – and even with the best of these there are inevitable residual reconstructionerrors.FIGURE 1: SHOWING THE TRADITIONAL SINGLE WIRE READOUT OF THE BAYER ENCODED SIGNALIn the original EOS C300 camera the dexterity of the multichannel readout architecture of the speciallydeveloped Canon 4K CMOS image sensor implements a direct parallel read out of the four constituent2K components that constitute the 4K Bayer sampling structure – as simplistically outlined in Figure 2.FIGURE 2: OUTLINING THE PRINCIPLE OF THE PARALLEL READOUT WITHIN THE IMAGE SENSOR OF THE FOUR 2K COMPONENTSTHAT CONSTITUTES A DIRECT DECODING OF THE 4K BAYER COLOR SAMPLING2

A totally new Super 35mm 4K image sensor developed for the second generation C300 Mark II utilizesthe same readout strategy as the earlier image sensor in the C300 camera. The dexterity of the readoutcapability of the CMOS image sensor allows access to the pixel level and this, in turn, allow precisiondismemberment of the 4K Bayer encoding into the four 2K constituent components of R, Gr, Gb, and B.Thus, a debayering process has been implemented that requires no downstream algorithmic decoding –which totally eliminates the traditional associated reconstruction errors [1].DUAL PIXEL STRATEGY – ELEVATION OF DYNAMIC RANGEAmong numerous design strategies in the Super 35mm CMOS image sensor developed for the EOS C300camera was an innovative new photosite design that employed two separate photodiodes – each being6.4 x 3.2 micrometers. For simplicity, this novel design is referred to as the Dual Pixel CMOS imagesensor.FIGURE 3: A REPRESENTATION OF THE DUAL PIXEL CMOS IMAGE SENSOR WITH A SPECIALLY DESIGNED MICROLENS THATOPTIMIZES THE FOCUSING OF THE INCIDENT LIGHT ONTO BOTH PHOTODIODESThe smaller lateral dimension of the individual photodiode supports a higher charge transfer efficiency(see more detailed explanation of this in Part I of this series detailing the ME20F-SH image sensor) whichin turn facilitates a greater speed in totally emptying the accumulated charge from each during theimager reset period. The two charges are later summed following readout and A/D conversion. Thephotodiode was also designed as a higher density N-type which elevates the number of saturationelectrons. The combination of these strategies produces an elevation of the overall dynamic range ofeach photosite.3

FIGURE 4: SHOWING THE DUAL PHOTODIODE STRUCTURE OF A SINGLE PHOTOSITE IN THE 4K CMOS IMAGE SENSOR USED INTHE CINEMA EOS CAMERASSECOND GENERATION DUAL PIXEL CMOS IMAGE SENSOR –15-STOP DRThe C300 Mark II employs a new generation Super 35mm CMOS sensor which is based on the same dualphotodiode per photosite. Additional innovations within the photodiode design in combination withnew on-chip noise cancellation technology have simultaneously lowered the noise floor and furtherelevated the saturation level of the charge well. In addition, a new microlens design heightens theefficiency of light direction onto the two individual photodiodes while also improving the separationbetween the two photodiode outputs.FIGURE 5: SHOWING THE LINEAR ANALOG SIGNAL LEVEL CAPABILITY OF THE NEW CMOS IMAGE SENSORThe combination of these new design strategies contribute to a more than two-fold increase in effectivephotosite dynamic range. This provides a definitive 15-Stop dynamic range capability in this new4

cinematography camera – providing one-stop capability above that of the C300 in the upper region andtwo stops below that of the C300 in the lower region. The reduced noise floor allows the ISO range tobe extended up to ISO 102,400.FIGURE 6: SHOWING THE PRINCIPLE OF THE DUAL OUTPUT FROM EACH INDIVIDUAL PHOTOSITE AND THE ASSOCIATEDANALOG PROCESSING THAT TAKES PLACE WITHIN THE IMAGE SENSOR ITSELFIn 2014, Canon introduced the second generation C100 Mark II which employed the same Super 35mm4K CMOS image sensor as the C300 and C100. But this camera further exploited the two separatephotodiodes within each photosite to empower a phase detection system that identifies the degree ofdefocus in an image – allowing incorporation of an innovative new Auto Focusing system having highprecision. This initial implementation proved very effective under normal shooting conditions. We willfirst describe the basics of what is termed the Dual Pixel CMOS Auto Focus system based upon that firstembodiment in the C100 Mark II camera.DUAL PIXEL CMOS AUTO FOCUS SYSTEMC100 Mark II embodies a powerful auto focus system where the sensing of sharp focus takes placewithin the image sensor photosite itself. It mobilizes the dual photodiodes within each photosite tocreate two separate images that facilitates a phase detection system that indicates the degree ofdefocusing.5

FIGURE 7: PRINCIPLE OF THE AUTO FOCUS CONTROL SYSTEM WHERE DATA FROM THE DUAL PHOTODIODES WITHIN EACHPHOTOSITE CONSTITUTES A PHASE COMPARISON WHICH IS PROCESSED TO CREATE A CONTROL SIGNAL FOR THE LENS FOCUSFigure 8 illustrates the manner in which the sets of dual pixel outputs from the CMOS image sensor aresent to the DIGIC DV5 processing microcircuit that was developed by Canon. Within this processor,these streams are fed to the primary RGB video processing system (where the two photodiode signalsare summed) and separately to a data processing system that makes all of the decision-making and dataprocessing associated with the Auto Focus system.FIGURE 8: SHOWING THE SEPARATE PROCESSING OF THE DUAL PIXEL DATA FROM THE IMAGE SENSOR – FOR VIDEO AND FORAUTO FOCUS – AT THE ENTRY STAGE OF THE DIGIC DV5 PROCESSORWhile all of the eight million photosites are delivering the “dual pixel” data, the operational aspects ofAuto Focus dictate that only a select number of these are activated at any given time. This is becausethe camera operator will make the decision on which particular subject within the overall picture frameis chosen for sharpest focus. Consequently, a cursor type system must be implemented to facilitate thischoice.In this first implementation of Dual Pixel CMOS Auto Focus (for simplicity Auto Focus is referred to as AF)system the “cursor” was fixed in the center of the image frame and had dimensions chosen based uponextensive testing.6

FIGURE 9: SHOWING THE FIXED CENTRAL ACTIVATION OF DUAL PIXELS IN THE C100 MARK II FOR AUTO FOCUS – REQUIRINGTHAT THE CAMERA FRAME THE SELECTED SUBJECT WITHIN THIS RANGEThe phase detection sampling lattice is made up of a number N of selected adjacent vertical samples ofphotosites – with each constituting an AF Sampling LINE – and then M of these Sampling LINES makingup the total vertical sample.FIGURE 10: SHOWING THE SPATIAL STRUCTURE OF THE AUTO FOCUS SAMPLING LATTICE WITHIN THE CMOS IMAGE SENSORThis system worked remarkably well in the C100 Mark II. However, a broadening experience revealedan extensive range of shooting situations that are encountered in the real world of program originationthat challenged the reliable performance of the Dual Pixel CMOS AF system. Analysis of these yielded arange of recommendations:1. Broad request to provide spatial movement of the sampling area – so that different subjectswithin a given scene can be selected for sharpest focus2. Ranging performance improvement is needed in low scene illumination situations3. Improvement in accuracy of the system as ISO setting increased4. Auto Focus should ideally be a real-time action (or as close as possible to real-time) so speed ofcalculations should be increased7

5. Improvement in the calculating algorithm to elevate reliabilityAs part of the development of the new generation Super 35mm 4K image sensor for the secondgeneration C300 Mark II, a totally new Dual Pixel CMOS AF system was developed in concert. A densersampling lattice was developed to increase sensing sensitivity and accuracy over a wider range of sceneillumination and camera ISO settings. The new sampling lattice is actually a matrix of nine adjoiningphotosite arrangements as shown in Figure 11.FIGURE 11: SHOWING THE SUBSTANTIALLY LARGER SAMPLING LATTICE THAT MAKES UP THE AF DETECTION IN THE C300 MARKII IMAGE SENSOREach sub arrangement has been increased to Y Lines (compared to N lines for the earlier system). Withnine such arrangements that becomes a total of 9xY selected lines of photosites. Operational flexibilitywas significantly broadened by allowing that sampling lattice to be repositioned (via a controllingjoystick) across 80% of the total photosite structure of the image sensor – as shown in Figure 12.FIGURE 12: SHOWING THAT THE DUAL PIXEL CMOS AF SELECTION AREA CAN BE MOVED AROUND SOME 80% OF THE ACTIVEIMAGE FRAME TO ALLOW SELECTION OF DIFFERENT SUBJECTS WITHIN THE SCENE FOR SHARP FOCUS8

DATA PROCESSING FOR DUAL PIXEL CMOS AUTO FOCUSThe data processing that is required is quite sophisticated. A parallel feed of the two images createdfrom the individual photodiodes within each photosite is concurrently sent to the Dual Pixel CMOS AutoFocus processing system and the RGB video processing system. A variety of corrections andadjustments are applied to these sampled images A and B as this can aid the precision of the detectiondepending upon lens settings. Correlation processing then takes place that identifies the spatialseparation of those processed images (the phase shift principle underlying this detection). The resultsare sent to a microcomputer that makes the calculations for the requisite correction signal. Experiencesgained with the first Dual Pixel CMOS AF system contributed to an improved algorithm design that teststhe reliability of the detection data and makes appropriate adjustments. The microcomputer alsoaccepts the control signal from the joystick that moves the sampling Dual Pixel CMOS AF area andimplements the associated variable spatial selection of the photosites.OPERATIONAL CONTROL OVER ACTION OF DUAL PIXEL CMOS AUTOFOCUSEarly experiences with Dual Pixel CMOS AF in the C100 Mark II camera exposed the reality that therewas a wide range of creative desires associated with acceptable actuation speed of the lens controlloop. This speed depends upon the type of production and the personal aspirations of the shooter. Itwas explained that different projects sought different lens drive speeds. Many felt that this firstgeneration auto focusing lens drive action was simply too fast. In addition, especially in televisiondrama production and moviemaking – where traditionally the Director and DoP (Director ofPhotography) often like to exercise a “feathering” control over the initial portion of a rack focus – thetakeoff speed of the auto focus drive also requires some choices.The new Dual Pixel CMOS AF system in the C300 Mark II embodies a menu that allows two degrees offreedom in “tuning” the response time. The focusing speed itself has a choice of ten speeds selectedunder SPEED in the menu – consisting of a Standard speed and then a choice of two faster speeds and achoice of seven slower speeds. This capability is only possible with those EF lenses that have slow-speeddrive capabilities. Separately, what is termed the RESPONSE setting is a separate setting of the systemthat offers a choice in how quickly a focusing action is initiated – thus adding a creative dimension to arack focus between two subjects within the scene.FOCUS GUIDE SYSTEMFor the cinematographer who prefers traditional creative manual focus operation, the dual pixel systemcan alternatively be switched from the Auto Focus control loop to an open loop system that utilizes theDual Pixel CMOS AF data processing to instead transfer precision signaling to the camera viewfinder.9

FIGURE 13: OUTLINES THE PRINCIPLE OF THE FOCUS GUIDE SYSTEM: THE IMAGE SENSOR DETECTS MANUAL ACTUATION OFTHE LENS FOCUS CONTROL AND THE DATA PROCESSING SIGNAL IS SENT TO THE VIEWFINDERThe following outlines the nature of the signaling in the camera viewfinder. In this mode, three graycolored arrows appear around a box cursor that is centered on the subject chosen for precision focus.The arrow’s direction signals the direction to turn the focus ring to achieve the sharpest look. Whenprecision focus is achieved, the viewfinder cursor and the indicating arrows snap to a green color.FIGURE 14: THE GUIDE CURSOR DETAIL USES THREE ARROWS TO INDICATE TO THE CAMERA OPERATOR THE DIRECTION TOROTATE THE FOCUS CONTROL. AT THE POINT OF PRECISE FOCUS ON THE CHOSEN SUBJECT THE CURSOR SNAPS TO A GREENCOLOR.10


SUMMARYPart II of this series focused on a 4K Super 35mm sensor that can switch between 4K (4096 x 2160 with a17:9 aspect ratio) and UHD (3840 x 2160 with a 16:9 aspect ratio) spatial sampling. Each of thoseapproximately eight million photosites utilizes two separate photodiodes. By expediting efficiency incharge well readout, this duality ensures an effective elevation of dynamic range. This image sensordelivers 15-stops of dynamic range supporting HDR functionality in the C300 Mark II camera. At thesame time, this photodiode duality also offers an in-sensor phase detection that is subsequentlyprocessed to close a control loop around the camera lens thus providing a very precise auto focussystem.REFERENCESR.B. Wheeler and N. Rodriguez “The Effect of Single-Sensor CFA Captures on Images Intended forMotion Picture and TV Applications”SMPTE J., 200712

defocus in an image - allowing incorporation of an innovative new Auto Focusing system having high precision. This initial implementation proved very effective under normal shooting conditions. We will first describe the basics of what is termed the Dual Pixel CMOS Auto Focus system based upon that first embodiment in the C100 Mark II camera.

Related Documents:

Full-frame vs. APS-C, some other considerations 10 IV. THE ECONOMICS OF IMAGE SENSORS 11 Wafers and sensors 11 V. WHY CMOS? 13 CCD 13 CMOS 14 Power consumption issues 15 Speed issues 16 Noise issues 17 VI. CANON’S UNIQUE R&D SYNERGY 20 Other differences between CMOS and CCD image sensors

CMOS Digital Circuits Types of Digital Circuits Combinational . – Parallel Series – Series Parallel. 15 CMOS Logic NAND. 16 CMOS Logic NOR. 17 CMOS logic gates (a.k.a. Static CMOS) . nMOS and pMOS are not ideal switches – pMOS passes strong 1 , but degraded (weak) 0

8. n-CH Pass Transistors vs. CMOS X-Gates 9. n-CH Pass Transistors vs. CMOS X-Gates 10. Full Swing n-CH X-Gate Logic 11. Leakage Currents 12. Static CMOS Digital Latches 13. Static CMOS Digital Latches 14. Static CMOS Digital Latches 15. Static CMOS Digital Latches . Joseph A. Elias, PhD 2

SOI CMOS technology has been used to integrate analog circuits. In this section, SOI CMOS op amp is discussed. Then, the performance comparison of op amps using bulk and SOI CMOS technologies is presented. 3.1 Analysis on SOI CMOS Op amp Figure 5 shows an SOI CMOS single stage op amp with a symmetrical topology. This circuit has a good .

advances in agronomy adv anat em advances in anatomy embryology and cell biology adv anat pa advances in anatomic pathology . advances in organometallic chemistry adv parasit advances in parasitology adv physics advances in physics adv physl e advances in physiology education adv poly t advances in polymer technology

image sensors first appeared in the late 1960s,'3 most of todays CMOS image sensors are based on work done starting around the early 1980's. Until the early 1990s PPS was the CMOS image sensor technology of choice.'4'8 The feature sizes of the available CMOS technologies were too large to accomodate more than a

CMOS APS: The CMOS pixel sensors use a photogate or photodiode to capture the photon flux and convert it into corresponding electrical charge. The main photodiode type MOS pixel sensors which use the photon flux integration mode can be divided into 2 basic approaches, i.e. of passivepixels (PPS) and active pi

When using either CCD or CMOS APS technology, electronic image sensors are based on analog pixels. With CCD sensors, data conversion is done at board level, and with CMOS APS ones, data conversion is done at either chip or column level. Because digital data is more

CMOS imagers, utilizing Self-Powered Sensors (SPS) is a new approach for ultra low-power CMOS Active Pixel Sensors (APS) implementations. The SPS architecture allows generation of electric power by employing a light sensitive device, located on the same silicon die with an APS and thus redu

Other examples of sensors Heart monitoring sensors "Managing Care Through the Air" » IEEE Spectrum Dec 2004 Rain sensors for wiper control High-end autos Pressure sensors Touch pads/screens Proximity sensors Collision avoidance Vibration sensors Smoke sensors Based on the diffraction of light waves

Used vanilla CMOS process available at many foundries Single-stage "CCD" in each pixel to allow complete charge transfer In-pixel source-follower amplifier for charge gain Allows low noise CDS operation In-column FPN reduction Permitted high performance camera-on-a-chip Basis of all modern CMOS image sensors

CMOS image sensor test module. The experimental results show that the proposed method removes noise while effectively preserves edges. Keywords: CMOS image sensor, noise, image denoi sing, digital camera, camera phone 1. INTRODUCTION Digital imaging devices such as digital cameras or camera phones on the market have been rapidly growing. The .

2B. Guenter et al., ‘Highly curved image sensors: a practical approach for improved optical performance’ Optics Express 25 2 (2017) 13010 Few developments on curved CMOS imagers since first Sony’s work in 2014; no curved microdisplay (as far as we know) CURVED CMOS-BASED DEVICES, STATE-OF-ART CMOS Spherical curvature

4CMOS APS starts diverging from mainstream CMOS to improve pixel performance 1970 1980 1990 2000 CCDs CMOS APS Mainstream CMOS Technology Window of Opportunity System Miniaturization Cost. September 03 22 Buried Photodiodes N P P N P Conventional Photodiode Buried Photodiode.

CMOS APS, or Monolithic Active Pixel Sensors, MAPS 1, have several advantages. 1 The terms MAPS is used to distinguish CMOS APS from hybrid detectors (also called Hybrid Active Pixel Sensors or MAPS are

Iineal circuits, the proposed technique imposes no restriction to the amount of clock skew. The main building blocks of the NORA technique are dynamic CMOS and C2MOS logic functions. Static CMOS functions can ;also be employed. Logic composition rules to mix dynamic CMOS, C 2MOS, and conventional CMOS will be presented. Different from

Circuits-A CMOS VLSI Design Slide 2 Outline: Circuits Lecture A – Physics 101 – Semiconductors for Dummies – CMOS Transistors for logic designers Lecture B – NMOS Logic – CMOS Inverter and NAND Gate Operation – CMOS Gate Design – Adders – Multipliers Lecture C – P

High-Speed CMOS Characteristics Table 1 compares the main characteristics of the high-speed CMOS family with those of standard TTL, LS, S, ALS, AS, and metal-gate CMOS. Table 1. Performance Comparison of High-Speed CMOS With Several Other Logic Families TECHNOLOGY† SILICON-GATE CMOS AHC METAL-GATE

The Mock CMOS process is shown in Figure 2. Using just a metal and oxide film stack on a silicon wafer, one is able to create similar microstructures as those produced in the CMOS-MEMS process, following equivalent post-CMOS fabrication steps. Yet by removing the CMOS component, a designer can place more focus on the

L2: x 0, image of L3: y 2, image of L4: y 3, image of L5: y x, image of L6: y x 1 b. image of L1: x 0, image of L2: x 0, image of L3: (0, 2), image of L4: (0, 3), image of L5: x 0, image of L6: x 0 c. image of L1– 6: y x 4. a. Q1 3, 1R b. ( 10, 0) c. (8, 6) 5. a x y b] a 21 50 ba x b a 2 1 b 4 2 O 46 2 4 2 2 4 y x A 1X2 A 1X1 A 1X 3 X1 X2 X3