Multi-Sensor Integration And Fusion Using PSoC
Multi-Sensor Integration and Fusion using PSoCM.S. FINAL PROJECT REPORTSubmitted byStudent NameMaster of Science in Electrical and Computer EngineeringThe Ohio State University, ColumbusUnder the Guidance ofDr. Lisa FiorentiniAssistant Professor, ClinicalDepartment of Electrical and Computer EngineeringThe Ohio State University, Columbusi
TABLE OF CONTENTSABSTRACT. 1INTRODUCTION . 2PROGRAMMABLE SYSTEM-ON-CHIP (PSoC) . 2MULTI-SENSOR SYSTEM ARCHITECTURE . 5ENCODERS . 7TEMPERATURE SENSOR . 8ACCELEROMETER . 10MAGNETOMETER . 11GPS . 13REAL TIME CLOCK (RTC) . 14DATA LOGGING IN F-RAM . 15COMMUNICATION WITH RASPBERRY PI . 17LIDAR . 21REFERENCES . 23ii
LIST OF TABLESTable 1: List of Hardware Components Used . 5Table 2: Encoder Color Code Description . 6Table 3: PSoC 4200M Pin Connections . 6Table 4: Sensor Characters, Table 5: Special Characters . 18LIST OF FIGURESFigure 1: High Level Block Diagram of PSoC 4200M. 3Figure 2: CY8CKIT-044 PSoC 4 M-Series Pioneer Kit . 3Figure 3: TopDesign Schematics of PSoC Creator Project . 4Figure 4: Design Wide Resources of PSoC Creator Project . 5Figure 5: A and B Channel Encoder Output Waveforms . 7Figure 6: Quadrature Decoder Component Configuration. 7Figure 7: TMP102 Internal Register Structure. 8Figure 8: I2C Component Configuration . 9Figure 9: I2C Timing Diagram to Read data from TMP102 . 9Figure 10: I2C Timing Diagram for ADXL345. 10Figure 11: Internal Schematic diagram of HMC5883L . 12Figure 12: Raw NMEA Sentences transmitted by the GPS . 14Figure 13: RTC Component Configuration . 15Figure 14: Timing Diagram to write data into F-RAM. 15Figure 15: Timing Diagram to read data from F-RAM . 16Figure 16: Bridge Control Panel to read data from F-RAM . 16Figure 17: UART Component Configuration . 17Figure 18: Example communication packets between Raspberry Pi and PSoC . 19Figure 19: PSoC Firmware Flowchart for communication and unparsing . 20Figure 20: LIDAR Lite Block Diagram . 21Figure 21: PSoC 4 I2C Component Configuration for LIDAR . 22iii
ABSTRACTMulti-Sensor Integration and Fusion subsystem is a part of the multidisciplinary research projectUnmanned Ground Vehicle (UGV) and Aerial (UAV) Vehicle Swarms. In order for autonomousnavigation, path planning and target identification of the autonomous vehicles, various sensorsmeasurements are required. A PSoC 4 (Programmable System-on-Chip) from CypressSemiconductors was used to interface various sensors. The PSoC 4 is based on ARM Cortex M0architecture along with the integration of programmable analog and digital blocks. The PSoC 4acts as a coprocessor collecting all the sensor data in real time and communicating it to RaspberryPi which performs high level controls based on the sensor measurements. The sensor data is alsologged in an external memory, a 256 K byte F-RAM available on the PSoC 4 Development Kit. Aspecific communication scheme was also developed on top of UART protocol between the PSoC4 and Raspberry Pi to transfer specific sensor readings requested by Raspberry Pi.Multiple sensors like Motor Encoders, IMU, GPS, Temperature Sensor and LIDAR weresuccessfully interfaced with PSoC 4. The Motor Encoders consists of hall effect sensors whichproduces a pulse every time the motor rotates, the counters in PSoC 4 are used to count thesepulses to determine the rpm of the motor and hence the wheel. The IMU consists of a combinationof Accelerometer and Magnetometer to measure the linear acceleration and orientation of the robotwith respect to the Earth’s Magnetic field essentially a compass pointing towards the northdirection. The temperature sensor is used to measure the ambient temperature. GPS gives theposition of the robot in terms of longitude and latitude. In addition to the position, the speed of therobot can also be measured and the UTC time from the GPS is used to lock the RTC time of PSoC4 for the time stamp of the sensor readings. LIDAR is used to detect obstacles in front of the robot.1
INTRODUCTIONMulti-Sensor integration refers to the process of using of multiple sensors to obtain more accurateand reliable information regarding the system and its environment. Multi-Sensor fusion refers tothe process of combining the information from various sensors and representing it in a commonformat understandable to the main processor which will take decisions based on the sensor data.The acquisition of sensor data must be done in real-time in order to make the decisions at the righttime. The time at which the each sensor data is acquired must also be included in the data formatto be sent to the main processor. In addition to sending the sensor data to the main processor, itmust also be logged in an external memory.In this project, the main processor is a Raspberry Pi 2 and a PSoC 4200M performs the multisensor integration and fusion in real-time and sends the fused sensor data to Raspberry Pi. Thesensors currently integrated with the PSoC 4200M are,1.2.3.4.5.6.Motor EncodersTemperature E SYSTEM-ON-CHIP (PSoC)The PSoC 4200M device from Cypress Semiconductors is a mixed signal system on chip basedon 32-bit Arm Cortex M0 architecture with programmable analog and digital blocks as depictedin the high level block diagram in Figure 1. Multiple sensors, both analog as well as digital sensorscan be interfaced with PSoC 4200M to acquire sensor data in real-time. The programmable analogand digital blocks operate independently along with the CPU of the PSoC [1]. This enablesacquisition of data from multiple sensors, formatting of sensor data and communication with themain processor in parallel. The PSoC 4200M device also contains a Real Time Clock (RTC) whichis used to time stamp the sensor data.2
PSoC 4200MProgrammableDigital BlocksARMCortex M0SensorInputsSensor Data toRaspberry PiProgrammableAnalog BlocksRTCFigure 1: High Level Block Diagram of PSoC 4200MCY8CKIT-044 PSoC 4 M-Series Pioneer Kit shown in Figure 2 features a PSoC 4200M devicewhich is used in this project to interface with the various sensors and the Raspberry Pi. Thisdevelopment kit also contains an on-board programmer and debugger and therefore no additionalhardware is required to program and debug the PSoC 4200M device [2].Figure 2: CY8CKIT-044 PSoC 4 M-Series Pioneer Kit3
The firmware for the PSoC 4200M device was developed using the PSoC Creator IDE (Version –3.2 SP1). The PSoC Creator contain PSoC components which are virtual ICs which users can dragand drop into a design and configure them to meet the application requirements. Every PSoCcomponent comes with its own set of API libraries [3]. PSoC Creator is free to use and can bedownloaded from the following link, www.cypress.com/PSoCCreator.Figure 3 shows the top design schematics of the PSoC Creator developed for Multi-SensorIntegration and Fusion project.Figure 3: TopDesign Schematics of PSoC Creator ProjectFigure 4 shows the design wide resources of the PSoC Creator developed for Multi-SensorIntegration and Fusion project.4
Figure 4: Design Wide Resources of PSoC Creator ProjectMULTI-SENSOR SYSTEM ARCHITECTUREThe Multi-Sensor system architecture with the interconnections between the multiple sensors andPSoC 4200M development kit is shown in figure 5. The Temperature sensor and the Accelerometerboth use I2C interface and are connected in the same I2C bus with the PSoC 4200M. The PSoC4200M identifies the respective sensor with its unique I2C address.Table 1 lists all the hardware components used in the Multi-Sensor system architecture along withtheir web links. The TMP102, ADXL345 and HMC5883L sensor breakout boards from SparkFunare used in this project.Table 1: List of Hardware Components UsedComponentDescriptionWeb linkCY8CKIT-044PSoC 4200M Development Kit www.cypress.com/CY8CKIT-044Raspberry Pi 2Main Processorwww.raspberrypi.orgEncoders48 CPR Quadrature Encoderwww.pololu.com/product/2275TMP102Digital Temperature Sensorwww.sparkfun.com/products/11931ADXL3453-Axis L3-Axis RUltimate GPS moduleLIDAR products/retired/131675
Table 2 describes the color code functionality of connecting wires in the encoder which is obtainedfrom Pololu Robots and Electronics. Table 3 gives the pin connection details between the differentcomponents and PSoC 4200M in the Multi-Sensor system architecture.Table 2: Encoder Color Code DescriptionColorRedFunctionMotor power (connects to one motor terminal)BlackMotor power (connects to the other motor terminal)GreenEncoder GNDBlueEncoder VCC (3.5 – 20 V)YellowEncoder A OutputWhiteEncoder B OutputTable 3: PSoC 4200M Pin ConnectionsComponentEncoder 1Encoder RRaspberry Pi 2PinPSoC 4200M PinEncoder A OutputP2.0Encoder B OutputP2.1Encoder A OutputP2.2Encoder B .0UART TXP1.1 (UART RX)UART RXP1.0 (UART TX)SDAP6.1SCLP6.0GPIO 14 (UART TX)P3.0 (UART RX)GPIO 15 (UART RX)P3.1 (UART TX)6
ENCODERSAn encoder is an electromechanical device used to measure the position and speed of a motor shaft.The motors used in the robot contain a two-channel Hall-effect sensor encoder which outputssquare waves corresponding to the rpm (rotations per minute) of the motor shaft [5]. The outputsof the two channels are 90 degrees out of phase which refers to a quadrature encoder as shown inFigure 5.Figure 5: A and B Channel Encoder Output WaveformsPSoC Creator offers a quadrature decoder component as shown in Figure 6 which is used to acquiredata from the two channel encoders. The encoding mode in the component is set to 4x which offershighest resolution by counting both the rising and falling edges of both the square waves to providethe count value output [4a].Figure 6: Quadrature Decoder Component Configuration7
The Counter X ReadCounter() API is used to read the current counter value from the quadratureencoder. The difference between the encoder values is computed by reading the encoder values attwo different times with a given time interval which is then formatted and sent to the RaspberryPi.TEMPERATURE SENSORThe temperature sensor used in this project is TMP102, which is a digital temperature sensor withI2C interface. The TMP102 has a resolution of 0.0625 C, and accuracy of 0.5 C over thetemperature range of -25 C to 85 C [6].Figure 7 shows the internal register configuration of the temperature sensor. To read the currenttemperature from the sensor, the pointer register must first be initialized to point to the address ofthe temperature register. After pointing to the temperature register, data can be read from thetemperature register. Two bytes of data must be read from the temperature register whichcorresponds to the MSB and LSB of the temperature reading.Figure 7: TMP102 Internal Register StructureThe PSoC 4200M communicates with the TMP102 sensor via I2C interface. The PSoC 4200M isconfigured as an I2C master with 100 Kbps data rate as shown in Figure 8. To communicate withthe slave I2C sensor, the master should know the I2C address of the slave. For TMP102 sensor,the 7-bit I2C address is 0x48 which is provided in the datasheet of the sensor. The I2C timingdiagram to read data from TMP102 is shown in Figure 9.8
Figure 8: I2C Component ConfigurationFigure 9: I2C Timing Diagram to Read data from TMP1029
Following is the sequence of steps followed in the firmware to get the temperature reading:1. Initialize the 8-bit pointer register to “00” to point to the temperature register using the API,I2C I2CMasterWriteBuf2. Read two bytes of data from temperature register using the API, I2C I2CMasterReadBuf [4b]3. Combine the MSB and LSB of the temperature reading4. Shift left the combined data by 4 bits and multiply by 0.0625 to get the current temperature5. Compute the difference between two temperature readings in a given time interval which isthen formatted and sent to the Raspberry PiACCELEROMETERThe accelerometer used in this project is ADXL345, which is a 3-axis MEMS accelerometer with13-bit resolution and measurement at up to /-16 g. Digital output data is formatted as 16-bit twoscomplement and is accessible through either a SPI (3- or 4-wire) or I2C digital interface. TheADXL345 automatically modulates its power consumption in proportion to its output data ratewhich can be configured.Registers 0x32 to 0x37 inside the ADXL345 holds the output data for each axis. Two 8-bit registershold the data for one axis. Register 0x32 and Register 0x33 hold the output data for the x-axis,Register 0x34 and Register 0x35 hold the output data for the y-axis, and Register 0x36 and Register0x37 hold the output data for the z-axis [7].The PSoC 4200M communicates with the ADXL345 sensor via I2C interface. The accelerometeris connected to the same I2C bus as the temperature sensor. For ADXL345 sensor, the 7-bit I2Caddress is 0x53 which is provided in the datasheet of the sensor. The I2C timing diagram forADXL345 is shown in Figure 10.Figure 10: I2C Timing Diagram for ADXL345Following is the sequence of steps followed in the firmware to get the accelerometer reading:1. Initialize the accelerometer by first going into standby mode10
2. Configure the accelerometer in full resolution, 100 Hz data rate, stream mode and measurementmode3. Read data from two registers corresponding to each axis4. Combine the data from the two registers for each axis to get the current reading for therespective axis5. Compute the difference between two accelerometer readings in a given time interval which isthen formatted and sent to the Raspberry PiMAGNETOMETERThe magnetometer used in this project is HMC5883L, which is a 3-axis magnetometer fromHoneywell. The HMC5883L contains a high-resolution HMC118X series magneto-resistivesensors plus an integrated application specific processor for amplification, automatic degaussingstrap drivers, offset cancellation, and a 12-bit ADC that enables 1 to 2 compass headingaccuracy.Digital output data is formatted as 16-bit twos complement and is accessible through e I2C digitalinterface.Registers 0x03 to 0x08 inside the HMC5883L holds the output data for each axis. Two 8-bitregisters hold the data for one axis. Register 0x03 and Register 0x04 hold the output data for thex-axis, Register 0x07 and Register 0x08 hold the output data for the y-axis, and Register 0x05 andRegister 0x06 hold the output data for the z-axis.The PSoC 4200M communicates with the HMC5883L sensor via I2C interface. The magnetometeris connected to the same I2C bus as the temperature sensor and the accelerometer. For HMC5883Lsensor, the 7-bit I2C address is 0x1E which is provided in the datasheet of the sensor [8].Figure 11 shows the internal schematics of the HMC5883L magnetometer sensor along with theexample connection with the I2C Master which in this case is a PSoC 4200M.11
Figure 11: Internal Schematic diagram of HMC5883LFollowing is the sequence of steps followed in the firmware to get the magnetometer reading:1. Initialize the magnetometer by first going into standby mode2. Configure the magnetometer in full resolution, 100 Hz data rate and continuous measurementmode3. Read data from two registers corresponding to each axis4. Combine the data from the two registers for each axis to get the current reading for therespective axis5. Compute the difference between two magnetometer readings in a given time interval which isthen formatted and sent to the Raspberry PiThe magnetometer readings are not tilt compensated, however since the accelerometer reading isalso available to the main processor which is a Raspberry Pi, it can perform tilt compensation usingboth the magnetometer and accelerometer reading to perform tilt compensation and calculate thetrue reading when the magnetometer is not lying flat which is usually the case in off terrainenvironment.12
GPSThe GPS module used in the project is the Ultimate GPS module from Adafruit which is a breakoutboard with the MTK3339 chipset, which is a high-quality GPS module that can track up to 22satellites on 66 channels, has an excellent high-sensitivity receiver (-165 dB), and a built inantenna. The
The time at which the each sensor data is acquired must also be included in the data format . the main processor is a Raspberry Pi 2 and a PSoC 4200M performs the multi-sensor integration and fusion in real-time and sends the fused sensor data to Raspberry Pi. The sensors currently integrated with the PSoC 4200M are, 1. Motor Encoders 2. .
ZigBee, Z-Wave, Wi -SUN : . temperature sensor, humidity sensor, rain sensor, water level sensor, bath water level sensor, bath heating status sensor, water leak sensor, water overflow sensor, fire sensor, cigarette smoke sensor, CO2 sensor, gas s
WM132382 D 93 SENSOR & 2 SCREWS KIT, contains SENSOR 131856 WM132484 B 100 SENSOR & 2 SCREWS KIT, contains SENSOR 131272 WM132736 D 88 SENSOR & HARNESS, contains SENSOR 131779 WM132737 D 100 SENSOR & 2 SCREWS KIT, contains SENSOR 131779 WM132739 D 100 SENSOR & 2 SCREWS KIT, contains SENSOR 132445 WM147BC D 18 RELAY VLV-H.P.-N.C., #WM111526
8. Install VMware Fusion by launching the “Install VMware Fusion.pkg”. 9. Register VMware Fusion when prompted and configure preferences as necessary. 10. Quit VMware Fusion. Create a VMware Fusion Virtual Machine package with Composer 1. Launch VMware Fusion from /Applications. 2. Cre
Accessible Fusion Brochure PDF. Accessible Fusion Brochure PDF. STANDARD FEATURES . FUSION / FUSION HYBRID / FUSION ENERGI. STANDARD FEATURES . GASOLINE ENGINE MODELS. . 2019 Fusion ford.com. 1 After your trial period ends, SiriusXM audio and data services each require a subscription sold separately, or as a package, by SiriusXM Radio Inc. .
2 FUSION SOFTWARE USER GUIDE - V17 FUSION-CAPT FOR FUSION FX7, SL7, PULSE 7, SOLO 7S, SOLO 7X EVOLUTION-CAPT FOR FUSION FX6, SL6, PULSE 6, SOLO 6S, SOLO 6X Thank you Dear Customer, On behalf of Vilber Lourmat, we would like to thank you for choosing the Fusion imaging system. In order to learn the capabilities of your Fusion imaging system, we kindly ask
integrates the two best of breed solutions in the industry. The integration enables the exchange of work structure, candidate and worker data between Fusion HCM and Taleo Recruiting. The integration solution presents both Fusion and Taleo customers the unique opportunity to leverage the Fusion HCM system of record with the Taleo Recruiting .
for utilizing the data. In particular, there are three major aspects to the project, multi-sensor processing, an adaptive approach to object recognition, and distributed sensor system integration. 2 Hyper-Pyramids-Join In a complex mult-robot multi-sensor system, each robot and sensor will have an egoworld model.
Abrasive jet Machining consists of 1. Gas propulsion system 2. Abrasive feeder 3. Machining Chamber 4. AJM Nozzle 5. Abrasives Gas Propulsion System Supplies clean and dry air. Air, Nitrogen and carbon dioxide to propel the abrasive particles. Gas may be supplied either from a compressor or a cylinder. In case of a compressor, air filter cum drier should be used to avoid water or oil .
