A Thermal Camera Based Continuous Body Temperature .
A Thermal Camera based Continuous Body Temperature Measurement SystemJia-Wei Lin, Ming-Hung Lu, Yuan-Hsiang LinDepartment of Electronic and Computer Engineering, Taiwan Building Technology CenterNational Taiwan University of Science and Technology, Taipei, Taiwan{m10702109, m10602107, linyh}@mail.ntust.edu.twAbstract At present, most of the contact methods are used to measurebody temperature (e.g., medical thermometers andthermistors), but this method not only requires labor, timeand consumables, but also increases the workload of thecare workers and causes a waste of resources. In addition,for the accuracy in the measurement process, the subject'sactivities will be limited, and the process may also causediscomfort. In contrast to the contact methods, non-contactbody temperature measurement uses infrared (IR)techniques that do not require any contact with skin tissueduring the procedure. Two types of infrared systems aremainly used, IR thermometers and thermal cameras. In thiswork, we chose the thermal camera to visualize the bodytemperature as an image, and measure the temperature atmany points over a specific area.This paper is organized as follows. In Section 2 presentsthe related work. In Section 3, we describe the proposedframework and our methods. In Section 4, the experimentand results are presented. Discussions and possibleimprovements for this work are drawn in Section 5. Finally,the conclusion is given in Section 6.2. Related Work1. IntroductionBody temperature is one of the most important vital signsin human bodies. By monitoring the subject's bodytemperature, it is most directly known whether the patienthas a fever or not, and even further can speculate on theeffect of the treatment on the patient. It is especiallyimportant for users who need long-term monitoring (e.g.,infants and the elderly). So now in hospitals, the bodytemperature is an indicator most commonly used byphysicians to judge the subject's physiological conditions.Therefore, automated CBTM system has become aworldwide research issue, and the main goal is to measurebody temperature accurately and for a long time.As [1] mentioned, today's continuous body temperaturemeasurement methods can be mainly divided into 2 types:the direct contact type and the non-contact type. The contactmeasurement method has the advantages of beingunconstrained by the environment and having a widetemperature measurement range (about -90 C to 300 C).Many papers on CBTM have been proposed today, whilethe main measurement methods are using non-invasivecontact sensors. As W. Chenintroduced in [2], theyuse thermistors to monitor the baby's body temperature.However, when the user moves, the measured resistancewill shift and cause temperature changes and errors. Also,it takes a long time to wait for the stable measurementreaction time, so the baby's body temperature cannot beknown immediately. To improve the reaction time, thepaper [3] proposed a multiple Artificial Neural Networks(ANNs) based wearable CBTM method to correct thethermistor, but the temperature will still be interfered by theuser's movement. In 2018, NA. Livanosproposed ahandheld device using passive microwave radiometer(MWR) technology to measure the internal bodytemperature of a human body [4]. The device uses a noncontact measurement method to avoid user’s movementartifacts. However, this device requires the user to aim atthe part that he wants to measure, so the automaticmeasurement cannot be achieved.
Figure 1. The proposed thermal camera-based continuous body temperature measurement framework.Face detection is a very popular research topic in imageprocessing, which has also been developed very maturelyin the field of general full-color RGB cameras. On the otherhand, to the best of our knowledge, there are currently threemain methods for face detection in thermal images:1) Image Projection method firstly convert thermalimage to gray-scale image, and binarize the image by usingthe Ostu’s method. After that, the vertical and horizontalprojection of the binarized image is calculated, facial partcan be defined from projection curve finally. In [5][6], thismethod has achieved good results and short processing time.However, it can only detect one face at a time.2) Haar-Cascade method also known as Viola-Jones (VJ)method [7]. It can detect objects based on Haar-like features,Integral image, Adaptive Boosting (AdaBoost), andCascade Classifier. This method has been successfullyapplied in several object detection applications such as face,animal, and vehicle detection. Authors of [8][9] used thisalgorithm to detect thermal face; however, the presentedresults indicated that the method was sensitive to thesubject’s head movement.3) Machine Learning-based method was proposed in[10][11], the authors introduced a high-resolution thermalfacial image database, which can be used to adapt methodsfrom the visual domain for IR images. But unfortunately,the training images data of this database is too differentfrom our camera specifications (our image has a poorresolution while their database were recorded with1024x768 pixel-sized). The application way are alsodissimilar, so it is not suitable for this paper.To sum up of the advantages and disadvantages of abovepapers, it can be observed that the current CBTM system ismainly limited by reaction time, movement noise andwhether it can achieve automation. In this paper, weproposed a novel system according to these limitations. Weused a thermal imaging sensor to build a non-contactCBTM system, and added the neural network-based facedetection method and object tracking algorithm. After thethermal face detection is performed, the face isautomatically tracked and the surface temperature in theregion of interest (ROI) will be measured.3. MethodsAs the proposed framework illustrated in Figure 1, weadopt a FLIR Lepton 2.5 with breakout board to fetchsequential thermal images of subjects. FLIR Lepton 2.5 is along-wave infrared (LWIR) camera sensor. It can captureinfrared radiation input and output a uniform thermal image,where image size is 80x60 pixels, and thermal sensitivity isless than 50 mK (0.05 C). The sensor’s export compliantframe rate is around 9 Hz, video data can be accessedserially via the Serial Peripheral Interface (SPI) protocol.Each frame is analyzed with the proposed framework.First of all, we use thermal face detection and tracking tolocate the ROI in the facial part. Next, the raw value of bodysurface temperature is extracted from the determined ROI,and then pass through the calibrated formula to get theresult. Our methods are constructed on the NVIDIA JetsonTX2 and written with C language and OpenCV library.3.1. Thermal Face DetectionIn this work, a good face detection is an important stepfor further processing. To achieve this goal, we decided totrain a new model based on deep-learning method. Ourmethod adopted the Single-Shot-Multibox Detector (SSD)[12] with MobileNet [13]. The MobileNet-SSD architectureis shown in Figure 2. One issue with deep-learning is theheavy demand for training data. To tackle this problem, wetransferred the learned weights from pre-trained model [14],and fine-tuned it to our thermal images. This transferredlearning process helped our detection model capture thefeatures of thermal face with only a small dataset. The dataused in our work came from a series of real-time execution,
Conv 15 23x3x256Conv 16 22x2x256Conv 17 classes)300x300Input ImageConv 14 110x10x256MobileNet v1-300Non-Maximum SuppressionConv 11 Conv 12/13 Conv 14 219x19x512 10x10x1024 5x5x512Conv 15 1 Conv 16 1 Conv 17 15x5x128 3x3x1282x2x64Figure 2. The overview of detection network architecture. Here, classes indicates the number of classes, which is 2 (thermal face andbackground) in this paper.Calculate the HOG featureswithin the tracking targetTrain a Correlation Filter (CF)with HOG featuresCalculate the correlationbetween CF and the next frameFigure 3. The result of deep-learning based thermal facedetection.Get the most relevant placewhich we sampled 1000 images in total. Among theseimages, 80% are used for training, 10% for validation, and10% for testing. After 20000 iterations, our model had aloss rate of 0.169352. If any face is found, it returns thepositions of detected faces as (1): ݁ܿܽܨ ൌ ܴ݁ܿ ݐ ሺ ݔ ǡ ݕ ǡ ݓ ǡ ݄ሻFigure 4. General flow of KCF algorithm.(1)where ݔ and ݕ represent the coordinate of the top-leftcorner, ݓ means width of the rectangle and ݄ for height.Once we get these locations, we can easily draw a greenrectangle for the face. Figure 3 shows the correspondingresults of our thermal face detection.3.2. Face TrackingAlthough our deep-learning based face detection canaccurately determine the location of the face in each frame,there is a disadvantage that the processing time is too long,leading to a decrease in frames per second (FPS). Therefore,we have added face tracking algorithm to improve the FPSin this system. Once the face is detected, kernel correlationfilter (KCF) tracker [15] will be used to track the face regionacross each images. KCF is a highly accurate trackingalgorithm. The general flow of the algorithm is as seen inFigure 4.Figure 5. The illustration diagram of ROI assignment.3.3. ROI LocatingAfter the face detection and tracking, we can determinethe position of the subject’s face in consecutively frames.For the subsequent measurement of body temperature, weuse the face facial features to define the forehead area asour ROI (see Figure 5). The formula is defined as (2) and(3) below.
ܴܱ ܫ ൌ ܴ݁ܿ ݐ ൫ܴܱ ܫ ௫ ǡ ܴܱ ܫ ௬ ǡ ܴܱ ܫ ௪ ǡ ܴܱ ܫ ൯ ݁ܿܽܨ ௪ ܫܱܴۓ ௫ ൌ ݁ܿܽܨ ௫ ʹ ൈ ۖ ݁ܿܽܨ ۖ ܴܱ ܫ ௬ ൌ ݁ܿܽܨ ௬ ʹ ൈͻ ܫܱܴ ۔ ൌ ʹ ൈ ݁ܿܽܨ ௪ ௪ۖ ݁ܿܽܨ ۖ ܫܱܴ ە ൌ ͻ (2)(3)thermal imaging camera with temperature-variety water tocorrect the ܴ, ܤ , ܨ , and ܱ parameters used in equation (6).Ideally, the heat source should be located at the distancesimilar to targets of interest in the applications. In this study,our system is measuring subjects at 40-80 cm to the camera.For the purpose of collecting data for curve-fitting, it isrecommended to average the values in a ROI of the image.Typical expected ranges for ܴ, ܤ , ܨ , and ܱ parameters arerevealed in [16], which are shown below:ܴ ൌ ሾͳͲͲͲͲǡ ͳͲͲͲͲͲͲሿ ܤ ൌ ሾͳʹͲͲǡ ͳ ͲͲሿ ܨ ൌ ሾͲǤͷǡ ሿ ܱ ൌ ሾെͳ ͺͶǡ ͳ ͺ ሿ 3.4. Body Temperature MeasurementBy referring the document [16], when radiometric modeis enabled, the 14-bit pixel value from Lepton is stabilizedand normalized. A scene will correspond to a particularvalue in the video stream. The signal from the camera iscalled flux-linear because it is linear to the radiometric fluxwithin Lepton’s spectral band. The flux-linear signal isrelated to scene temperature by the Planck curve:ఒమܵൌ නఒభͳʹߨ݄ܿ ଶܴሺߣሻ ή ߜߣߣହ ሺ ݄ܿ ሻ െ ͳߣ݇ܶ (4)where ܵ is the output signal. ߣଵ and ߣଶ define the spectralband. ݄ is Planck’s constant. ܿ is the velocity of light. ݇ isBoltzmann’s constant. ܴሺߣሻ is the camera responsivity. ܶ is absolute temperature in units of Kelvin.Since equation (4) is impractical to calculate in software,the conversion is typically approximated by equation (5):ܵൌܴ ܱ ܤ ቀ ቁ െ ܨ ܶ (5)The ܨ parameter is generally not varied from a value ofͳ except for measuring a very high scene temperatureoutside of camera’s valid range. The typical values for ܤ parameter is ͳͶʹͺ, which is given in the document. If ܨ and ܤ are fixed at the typical values, it is possible tocalibrate the ܴ (the camera responsivity) and ܱ (the offset)by fitting the regression line. Figure 6 shows the outputcurve fitting with the nonlinear least squares (NLS) method.The calibrated value of ܴ and ܱ are ʹ ͳͳͷͻǤͷ and ͲͻͶǤʹͶͺ, respectively. Finally, we can rewrite equation (6)to equation (8):ܶൌͳͶʹͺെ ʹ Ǥͳͷʹ ͳͳͷͻǤͷ ͳቁ ቀܵ െ ͲͻͶǤʹͶͺ ܤ ܴ ܨ ቁ ቀܵെܱ(6)3.5. Parameters UpdatingIn this paper, we can use the FLIR Lepton 2.5 camerasensor to get the thermal scene. However, the output of thiscamera is the unstabilized 14-bit pixel value instead of theactual temperature value. Therefore, in order to get accuratehuman body temperature, we must first perform theradiometry calibration on this sensor. According to thespecification manual [16], we used the Keysight U5855A(8)where ܶ is our measured skin temperature in units ofCelsius. ܵ denotes the output signal from the thermalcamera. Through equation (8), we can get the foreheadtemperature ܶ (in units of Celsius) in each frame.where ܵ denotes the output signal from the camera. ܴ, ܤ , ܨ ,and ܱ are parameters generated during calibration. ܶ is thetarget’s absolute temperature in units of Kelvin. Theconversion from flux to temperature is performed using theinverse of equation (5). So we can express equation (5) as:ܶ ൌ(7)Figure 6. The fitting result of measured output.
ே40-80 cmͳ ൌ ȁܶ െ ܴ ܨܧ ȁܰKeysight U5855AFLIR Lepton 2.5Jetson TX2(9) ୀଵேͳ ൌ ඩ ሺܶ െ ܴ ܨܧ ሻଶܰ(10) ୀଵwhere ܶ is the average temperature of all pixel points inROI. ܴ ܨܧ denotes the average temperature of similar areaacquired from the GT device. These values are comparedevery 10 second. ܰ refers to the number of obtainedtemperature within 10 second.Finally, Figure 9 presents an example of the temperatureestimated from our methods as well as the temperaturecorresponding to the GT. These representative signals arefrom subject 6.Figure 7. Illustration of the study setup.(Phase A)02Apply thehot bagMeasurement(Phase B)49Measurement(Phase C)1116Time (Minutes)Figure 8. Experimental protocol.4. Experiment and Results4.1. Experimental Setup and ProtocolThe experiment was conducted with 6 healthy subjects(4 males and 2 females). During the experiment, the FLIRthermal camera was set up 40-80 cm in front of the subjectsto capture the sequential images at the speed of 26 FPS. Allsubjects were instructed to sit and maintain a stationarystate. Furthermore, the Keysight U5855A camera was alsoplaced to synchronize the ground-truth (GT) temperature atthe maximum speed of 8 FPS. Figure 7 illustrates themeasurement scenario.In addition, the study protocol was carried out in a periodof 16 minutes, which is represented in Figure 8. First, wemeasured the stable body temperature for two minutes(Phase A). Then, in order to simulate the change of bodysurface temperature, we put an ice bag and hot bag on thesubject's forehead for two minutes, and then measured thebody temperature change trend for five minutes,respectively (Phase B and C).Table 1. The results of the proposed CBTM system.Phase APhase BPhase 66䯲The unit of the values are Celsius ( C).Phase ATemperature (ƱC)Apply theice bagMeasurementPhase BPhase C4.2. Experiment ResultsIn this paper, we adopted mean absolute error (MAE) androot-mean-squared-error (RMSE) metrics to express thedifference between the proposed framework and theverified device. The operations are shown in (9) and (10).Table 1 shows the evaluation results provided by eachsubject.Time (10 seconds)Figure 9. Representative example of the temperature of subject6. The blue line indicates the temperature obtained from oursystem and the red line represents the ground-truth (GT).
5. Discussion5.1. Face DetectionWe have described a deep-learning approach to achievethis function. Although deep-learning based methodrequires more computing power and time than traditionalimage processing methods, it can overcome uncontrolledand complex environments in practical applications (i.e., inbedtime condition or some heat sources in background). Onthe issue of computing speed, we added a tracking methodto improve the FPS of our system (from 10 FPS to 26 FPS),the deep-learning based detection method will only bereused in the first frame or lost tracking. Therefore, we canfulfill the good detection accuracy and low execution timeat the same time.5.2. Experiment MethodsOur experimental results are compared with an industrialinstrument. However, this limits our ability to only requirethe subject to remain stationary state during the experimentbecause it cannot track a specific area like our proposedsystem. For further experimentation, it is recommended touse an additional contact sensor such as medicalthermometers or thermistors. In doing so, there may be adeviation between our non-contact method and contactmethod needs to be adjusted.5.3. Future WorkIn addition to body surface temperature, it is alsopossible to measure the core temperature of the human body.The scholars of the paper [17][18] used the body surfacetemperature (ܶ௦ ), heat flow ( )ܨܪ , heart rate ( )ܴܪ and otherparameters to estimate the change of core temperature,which is also one of the main researches in the future.Since thermal images have many advantages, such asunaffected by illumination variation, working well indarkness, and hard to fake than visible images, they arewidely used in biometric applications. In the future, we cancombine thermal face recognition and vital signs (i.e., pulserate and respiration rate) monitoring under daily life.6. ConclusionsIn this paper, we developed a real-time, and contactlessCBTM system with a low-cost and poor-resolution LWIRcamera. To locate the subject’s face, we trained a newmodel based on deep-learning method with our own dataset.We also applied a tracking algorithm to track detected face.Further, the immediate forehead temperature is obtained bythe calibrated conversion formula. The experiment for theproposed framework shows that our system’s overall meanabsolute error (MAE) and root-mean-squared-error (RMSE)are 0.375 C and 0.439 C.AcknowledgmentThis work was supported in part by a research grant fromthe Ministry of Science and Technology under GrantMOST 107-2221-E-011-107- and financially supported bythe Taiwan Building Technology Center from The FeaturedAreas Research Center Program within the framework ofthe Higher Education Sprout Project by the Ministry ofEducation in Taiwan.References[1] N. Sellier, E. Guettier, and C. Staub. A review of methods tomeasure animal body temperature in precision farming.American Journal of Agricultural Science and Technology,2(2):74-99, 2014.[2] W. Chen, S. Dols, S. B. Oetomo, and L. Feijs. Monitoringbody temperature of newborn infants at neonatal intensivecare units using wearable sensors. In Proceedings of theIEEE 5th International Conference on Body Area Networks(BodyNets), pages 188-194, 2010.[3] C. Song, P. Zeng, Z. Wang, H. Zhao, and H. Yu. Wearablecontinuous body temperature measurement using multipleartificial neural networks. IEEE Transactions on IndustrialInformatics, 14(10):4395-4406, 2018.[4] NA. Livanos et al. Design and interdisciplinary simulationsof a hand-held device for internal-body temperature sensingusing microwave radiometry. IEEE Sensors Journal,18(6):2421-2433, 2018.[5] J. Mekyska, V. Espinosa-Duró, and M. Faundez-Zanuy. Facesegmentation: a comparison between visible and thermalimages. In IEEE International Carnahan Conference onSecurity Technology (ICCST), pages 185-189, 2010.[6] Y. K. Cheong, V. V. Yap, and H. Nisar. A novel facedetection algorithm using thermal imaging. In IEEESymposium on Computer Applications and IndustrialElectronics (ISCAIE), pages 208-213, 2014.[7] P. Viola, and M. Jones. Rapid object detection using aboosted cascade of simple features. In Proceedings of the2001 IEEE Computer Society Conference on ComputerVision and Pattern Recognition (CVPR), 2001.[8] A. KwaĞniewska, and J. Ruminski. Face detection in imagesequences using a portable thermal camera. In Proceedingsof the 13th Quantitative Infrared Thermography Conference,pages 4-8, 2016.[9] A. KwaĞniewska, and J. Ruminski. Real-time facial featuretracking in poor quality thermal imagery. In IEEE 9thInternational Conference on Human System Interaction(HSI), pages 504-510, 2016.[10] M. Kopaczka, R. Kolk, and D. Merhof. A fully annotatedthermal face database and its application for thermal trumentation and Measurement Technology Conference(I2MTC), 2018.[11] M. Kopaczka et al. A combined modular system for facedetection, head pose estimation, face tracking and emotionrecognition in thermal infrared images. In IEEE InternationalConference on Imaging Systems and Techniques (IST), 2018.[12] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, and S. Reed.SSD: single shot multibox detector. arXiv: 1512.02325,2015.
[13] AG. Howard et al. MobileNets: efficient convolutionalneural networks for mobile vision applications. arXiv:1704.04861, 2017.[14] MobileNet-SSD. GitHub repository, [Online]. SSD/.[15] J. F. Henriques, R. Caseiro, P. Martins, and J. Batista. Highspeed tracking with kernelized correlation filters. arXiv:1404.7584, 2014.[16] FLIR. Lepton radiometry application note. Datasheet, 2014.[17] X. Xu, AJ. Karis, MJ. Buller, and WR. Santee. Relationshipbetween core temperature, skin temperature, and heat fluxduring exercise in heat. Eur J Appl Physiol, 113:2381-2389,2013.[18] AP. Welles et al. Estimation of core body temperature fromskin temperature, heat flux, and heart rate using a kalmanfilter. Computers in Biology and Medicine, 99:1-6, 2018.
adopt a FLIR Lepton 2.5 with breakout board to fetch sequential thermal images of subjects. FLIR Lepton 2.5 is a long-wave infrared (LWIR) camera sensor. It can capture infrared radiation input and output a uniform thermal image, where image size is 80x60 pixels, and thermal sensitivity is
2-9V in unit & 2 AA in camera. Match polarities ( ) and ( ). Set camera date back, close camera lens and connect plug to camera port. 2 3 Secure camera, open camera shutter, and slide unit power switch to (ON) and back to (OFF), then push camera test button. Close camera Shutter, remove camera & load film, connect plug to camera, close cover. 4
User Manual Replace a Pro 3 Camera battery You can leave the camera housing in place so the camera position stays the same. 1. Release the camera from the camera housing. Press the button on the charging port underneath the camera. The camera clicks as it disengages from the camera housing. 2. Pull the camera all the way out of the camera .
Camera CCH-01G Jeep Grand Cherokee Camera CMB-16G Mercedes Benz GLK Trunk Handle Camera CCH-01S Jeep Wrangler Spare Tire Mount Camera CVW-07L VW Beetle License Plate Light Camera (LED) CVW-07G VW Beetle License Plate Light Camera CFD-03F Ford Tailgate Handle Camera CCH-01W Jeep Wrangler License Plate Light Camera CBM-01T BMW 5 Series Trunk .
1. Place the outdoor mount and secure it with screws. For drywall, use the wall anchors we provide. 2. Release the camera from the camera housing. Press the button on the charging port underneath the camera. The camera clicks as it disengages from the camera housing. 3. Pull the camera all the way out of the camera housing. 4.
On power-up, the camera interface board will read the SW1 DIP switches and set the camera video mode accordingly. If the DIP switches are set for 'Default Camera Mode' then the camera (and camera interface . Serial Camera Control The camera may be controlled by serial VISCA commands (J3). The VISCA serial signal is routed through the .
3 ICI 7640 Infrared Camera: User Manual v 1.0 Introduction ICI 7640 is a high resolution thermal imaging camera produced by Infrared Cameras Inc. (ICI). This camera is a high resolution and high sensitive infrared camera equipped by microbolometer detector. A microbolometer is a specific type of bolometer used as a detector in a thermal camera
Energies 2018, 11, 1879 3 of 14 R3 Thermal resistance of the air space between a panel and the roof surface. R4 Thermal resistance of roof material (tiles or metal sheet). R5 Thermal resistance of the air gap between the roof material and a sarking sheet. R6 Thermal resistance of a gabled roof space. R7 Thermal resistance of the insulation above the ceiling. R8 Thermal resistance of ceiling .
2 Installation Guide Introduction Product overview This is the installation guide for TruVision Series 4 IP camera models: TVC-5401 (2MPX IP box camera) TVC-5402 (3MPX IP box camera) TVC-5403 (5MPX IP box camera) TVB-5401 (2MPX IP bullet camera, 2.8 to 12 mm) TVB-5402 (2MPX IP bullet camera, 8 to 32 mm)
