Autonomous Targeting Sentry

ATSAutonomous Targeting SentryEthan King - Electrical EngineerDaniel O’Hara – Computer EngineerStephen Rodriguez – Computer EngineerJames Van Gostein – Computer EngineerGROUP12

PROJECT OVERVIEW AND MOTIVATIONPrototype an autonomous turret to defend high priorityareasDetect and track targets using computer vision,determine threat and eliminateTo reduce the amount of human interaction in combatsituations

OBJECTIVES Lightweight Cost efficient Capable of detecting, prioritizing and tracking targets autonomously Track targets moving at high speeds Manual control override Facial recognition Determine when a target is no longer a threat Alert target before firing Adjustable fire rate

SPECIFICATIONSDescriptionConstraintAccuracy Rating 70%Number of Tracked Targets 3Response Time 1.50 secondsLateral Angle140 degreesPitch Angle90 degreesVideo Stream Resolution1280 x 720 pixelsWeight 30 lbsDimensions 2’ x 2’ x 4’Operation Time3600 seconds

HARDWARE BLOCK ServoYawServoRS232to I2CPitchServoCapTouchLaptop

MICROCONTROLLER SELECTIONMSP430 ModelG2231G2203F2471.8 V – 3.6 V1.8 V – 3.6 V1.8 V – 3.6 VGPIO Pins141648Capture & Compares2610CommunicationSPI, I2CSPI, I2C, UARTSPI, I2C, UARTDev Board Cost 4.30 4.30 75.00Operating Voltage

MAIN MCU – MSP430G2231InitializeBusDataTransmit PitchIf ErrorLED1 ONTransmit YawIf ErrorLED2 ONSet TriggerIf ErrorLED3 ONDecode DataSet LaserState FlagSet AlarmTransmit to PC

MAIN MCU – MSP430G2231opCode (Hex)FunctionMethod Description0x00 to 0x8CsetYawAngle(char opCode)Set yawAngle opCode0x90 to 0xEAsetPitchAngle(char opCode)Set pitchAngle opCode – 0x900xEC to 0xEFsetTriggerMode(char opCode)Set trigger to off, auto, burst, single0x8DretYawAngle(char yawAngle)Return yawAngle to PC0xEBretPitchAngle(char PitchAngle)Return pitchAngle to PC0xF0 to 0xF1setAlarm(char opCode)Turn alarm off or on0xF2 to 0xF3setLaser(char opCode)Turn laser off or on0xF4 to 0xF7setLED(char opCode)Turn LEDs 1, 2, 3 off or on0xFA to 0xFFreservedreserved

MAIN MCU – MSP430G2231Return TypeFunctionDescriptionvoidtoBuffer(char data)Send data to the buffervoidencodeData(void)Pull data from the buffer and encode accordingto specifications and push onto buffervoiddecodeData(void)Pull data from the buffer and decode accordingto specificationsvoidtransmit(char address)Pull data from the buffer and transmit to anothermicrocontroller via I2Cvoidreceive(void)Receive data from I2Cbus and push onto thebuffer

CAP TOUCH MCU – MSP430G2452InitializeTouched UppitchAngle EncodeCalibrate CapTouched RightyawAngle BufferRead CapTouched DownpitchAngle--TransmitSet FlagTouched LeftyawAngle--Touched MiddleToggle Trigger

CAP TOUCH MCU – MSP430G2452Return asure capacitance of the air and store intoan arraycharreadTouch(void)Measures capacitance using TI’s API and returnvalue corresponding to up, right, etcvoidtoBuffer(char data)Send data to the buffervoidencodeData(void)Pull data from the buffer and encode accordingto specifications and push onto buffervoidtransmit(char address)Pull data from the buffer and transmit to anothermicrocontroller via I2C

SERVO MOTOR MCU – MSP430G2203InitializeBusDataDetermine AxisReceive AnglesBufferApply PWM

SERVO MOTOR FUNCTIONSReturn TypeFunctionDescriptionVoidfindAxis(void)Determine the axis of received anglevoidsetPosition(void)Pull data from buffer and apply appropriatePWMvoidtoBuffer(char data)Send data to the buffervoidreceive(void)Receive data from I2Cbus and push onto thebuffer

SERVO MOTOR -815HS-81Op. Voltage4.80 V to 6.00 V4.80 V to 6.00 V4.80 V to 6.00 VTorque343 oz-in343 oz-in41.7 oz-inSpeed0.14 S/60 0.14 S/60 0.09 S/60 Current830 mA830 mA280 mADimensions2.26” x 1.18” x 2.59”2.26” x 1.18” x 2.59”1.16” x 0.47” x 1.17”Max Angle90 degrees140 degrees90 degreesCost 39.99 44.99 17.99

RS232 I2C MCU – MSP430G2231InitializeBusDataTransmit to PCTransmit to BusPCData

RS232 I2C MCU – MSP430G2231Return TypeFunctionDescriptionvoidtoBuffer(char data)Send data to the buffervoidtransmit(char address)Pull data from the buffer and transmit to anothermicrocontroller via I2Cvoidreceive(void)Receive data from I2Cbus and push onto thebuffervoidtransmitPC(charaddress)Pull data from the buffer and transmit to anotherto PCvoidreceivePC(void)Receive data from PC and push onto the buffer

AUDIBLE ALARMTurbo series TMC-86-530-WMountingPanel MountedOperating modeLoud – ContinuousOperating Voltage5-30 VdcOperating Frequency2900 250 HzOperating current2mA at 5 Vdc

POWER SOURCE12V/1.3Ah Sealed Lead Acid BatteryModelGH1213Dimensions3.80 “x 2.05” x 1.89”Capacity1300 mAhWeight1.3 lbsRechargeableYesPrice 15.99

HARDWARE BLOCK DIAGRAMAlarmPower Supply5.0 VoltRegLaserLEDsMasterMCU3.3 VoltReg.ServoControlsTriggerServoYawServoRS232to I2CPitchServoCapTouchLaptop

PAINTBALL GUN AirTech E-Matrix electro pneumatic version. Adjustable max rate of fire through DIP switches located on PCB in the grip of gun. Nitrogen tank. Electronic Hopper.

BASE AND MOUNTSketch by Jon Waldmann

LOGITECHC310 HD WEBCAM 1280 x 720 pixels 30 USB 2.0 port Automatic lighting adjustments Fixed position above the turret

SOFTWARE FLOW DIAGRAMWeb nerSubPixtrackTargets

OPENCV Open source computer vision libraries. Using version 2.3.1 (newest released version) Maintained by Intel. Many useful functions for detecting and tracking objects and/or motion.

DETECTION AND TRACKING OPTIONS Iterative Lucas-Kanade Method with Pyramids Gunnar-Farneback Block Matching Method Horn-Schunck Algorithm Blob Tracking Color Tracking Facial Tracking

ITERATIVE LUCAS-KANADE METHODWITH PYRAMIDS Calculates optical flow between 2 images Given a set of points from a previous image, find those points in the newimage Assumes that neighbors of a pixel move in a similar fashion to that pixel itself Pyramids break into sub images. From there, the sub images are analyzed todetect large motions

TARGET CREATION AND THREAT LEVELCALCULATION Keep a history of the movement of each pixel from the optical flowcalculation. Based on all similarly moving points in a neighboring area, create a boundingbox around each target. Place each target into an array of targets. Calculate threat level of each target based on size, velocity, and color. Sort target array based on threat level.

DECISION MAKINGFor no current target: Get target Check threat level Sound Warning FireFor current target: Compare current and new target Choose highest priority If new target, warn Fire

MANUAL CONTROL Windows form application using Visual Studios 2010. The application will contain the video feed from the webcam and buttons toperform the following actions by a user via on-board laptop. Rotate and fire the gun. Sound the alarm and play custom audio files. Rate of fire selection. Emergency stop button.

MANUAL CONTROL FUNCTIONSReturn TypeFunction NameMethod DescriptionVoidrotateRight()Rotates the turret right by a set degree.VoidrotateLeft()Rotates the turret left by a set degree.VoidpanUP()Tilts the turret upwards.VoidpanDown()Tilts the turret downwards.VoidFire()Fires the turret.VoidalertWarning()Plays an audio file to warn intruders that they areentering a dangerous zone and that they shouldproceed to e)Plays an audio file to warn intruders that they are about to befired upon by ATS.sets the firing rate of the turret 0 automatic, 1 burst, 2 single shot.

MANUAL CONTROL – PROTOTYPE GUI

FACIAL RECOGNITION Security system to gain access to the manual controls of the turret. Scans a frame in an attempt to detect a human face using Haar-like features.- Recognizable human facial features such as the dark areas under the eyes or cheekcolor. Classifiers can also be trained to detect movement of the face such asblinking, raising eyebrows, and flared nostrils.Once a face is detected the software will attempt to match the face to a face found in ourface database.

DIVISION OF LABOREthan KingDaniel O’Hara Embedded Software Servo Motors Motion Tracking Threat CalculationsStephen RodriguezJames Van Gostein Power Facial Recognition Manual Control GUI Base and Mount

PROGRESSResearch90%Design85%Parts Aquired75%Parts Testing75%Software42%IntegrationSystem Testing33%10%

BUDGETPartCostAcquisition 102.97Purchased 4.30DonatedMSP430 Capacitive Touch Booster Pack 10.00DonatedLogitec C310 HD Webcam 29.99PurchasedGH1213 Lead Acid Battery 15.99To be purchased 9.64To be purchasedHK – E03358 Laser Pointer 14.99To be purchasedPCB 33.00To be purchasedMounting Mats 100.00As neededE-Matrix, Hopper, Nitrogen tank. 150.00DonatedPaintballs 29.99DonatedExtra Electrical Components 30.00As neededTotal retail cost 530.87Total cost to group 336.58Servo MotorsMSP-EXP430 LaunchPadTurbo series TMC-86-530-W Alarm

STILL TO DO PCB Construct base and gun mounting. Create formula for converting pixel coordinates to angles. Interface laptop with embedded hardware. Creating targets, and threat level Facial recognition log-in

ISSUES AND MILESTONES Converting pixel coordinates to angles. Communication from PC to I2C bus. Construction of the base and gun mount.MilestoneDatesOrder PCBFebruary 24thCompletion of SoftwareMarch 9thCompletion of BaseMarch 9thIntegrationMarch 16thSystem TestingMarch 23rdComplete working prototypeApril 1st

QUESTIONS?

Turbo series TMC-86-530-W Alarm 9.64 To be purchased HK – E03358 Laser Pointer 14.99 To be purchased PCB 33.00 To be purchased Mounting Mats 100.00 As needed E-Matrix, Hopper, Nitrogen tank. 150.00 Donated Paintballs 29.99 Donated Extra Electrical Components 3