Artificial Intelligence Techniques For Smart City Applications

4This paper essentially constitutes a preliminary step towards adapting XAI approaches for smart monitoring. By reviewing and categorizing AI/ML algorithms forsmart monitoring and discussing general XAI concepts, an overview of which AI/MLalgorithms used in SHM may be modified towards adopting XAI in smart monitoringis shown. Subsequently, the review presented herein is summarized, and a conciseoutlook on potential future work is provided.2Machine learning algorithms in civil engineeringBecause of the ability to recognize and to classify patterns in large data sets, ML algorithms are of increasing interest in civil engineering. In the following subsections, acategorization of ML algorithms is provided, and ML algorithms of particular relevance to smart monitoring applications are reviewed and categorized.2.1Categorization of machine learning algorithmsThe term “intelligence” in “artificial intelligence” denotes the ability of an entity tocapture, to process, and to respond to input of different kind (Legg & Hutter, 2007).Extending the definition of intelligence, the term “artificial intelligence” describes theability of an artificial entity (e.g., a software or computer system) to achieve specificgoals under a variety of environmental conditions. However, to qualify as “intelligent”, a system needs to possess the ability to respond to previously unknown (environmental) conditions through learning and adaption (Hutter, 2005). In a broadersense, AI is the ability of a computer system to approximate the intellectuality of human beings. To mimic human behavior, Russel & Norvig (2016) have defined sixcategories of AI: machine learning, robotics, computer vision, natural language processing, knowledge representation, and automated reasoning.In intelligent systems, ML helps adapt a system to new circumstances through processing and analyzing data, extrapolating patterns, and making predictions. By combining concepts of computer science with optimization and statistical concepts(Mohri, et al., 2018), ML essentially represents the learning processes of AI, oftendescribed as converting experience into expertise or knowledge (Shalev-Shwartz &Ben-David, 2014). In summary, ML algorithms show two distinct advantages, ascompared to traditional algorithms (Russel & Norvig, 2016; Shalev-Shwartz & BenDavid, 2014):1. ML algorithms operate with previously unknown (i.e., newly derived) data onwhich the system has not been trained, and2. ML algorithms are adaptable to changes in the data.However, ML algorithms need to learn from experience or knowledge of domainexperts (Shalev-Shwartz & Ben-David, 2014). Depending on the type of learning, MLalgorithms may be categorized into

5i. supervised learning,ii. unsupervised learning, andiii. reinforcement learning,as shown in Figure 1. In supervised learning, ML algorithms use labeled inputoutput pairs as training data, and the system learns based on given examples. Typicallearning problems in supervised learning are classification and regression. In classification, data is sorted into predefined categories, while in regression, the outputs tocorresponding input data are calculated. In contrast to supervised learning, the training data in unsupervised learning is not labeled. A typical problem of unsupervisedlearning is clustering, where data is grouped according to commonalities. In reinforcement learning, no training data is provided. Instead, the system develops a strategy to maximize a predefined cumulative reward. Figure 1 shows the categorizationof AI and ML algorithms as well as the subcategories mentioned above. In addition,examples of ML algorithms, corresponding to the subcategories, are illustrativelyprovided in Figure 1.Fig. 1. Categorization of machine learning algorithms.Depending on the category of ML algorithms, mixed forms of (i), (ii), and (iii) arelikely to be used (Burkov, 2019; Salehi & Burgueno, 2018). For example, artificialneural networks are trained with different specifications and, depending on the purpose and structure of the artificial neural network (ANN), may fit into any of the threecategories. Therefore, the categorization presented in Figure 1 is regarded as a startingpoint to approach the basic concepts of machine learning but cannot be consideredgenerally valid for any ML specification.

62.2Machine learning algorithms for smart monitoringThe application areas of machine learning in smart monitoring are manifold. Thispaper focuses on ML algorithms applied to data analysis, which, according to Bisby& Briglio (2005), may pursue the following goals,i.ii.iii.iv.v.damage detection,damage classification,damage localization,condition assessment, andlife-time prediction,to be achieved primarily by supervised and unsupervised ML algorithms as well asalgorithms that combine both categories. In the remainder of this subsection, MLalgorithms addressing the above goals are reviewed, distinguishing between supervised and unsupervised/hybrid ML algorithms.Supervised machine learning algorithms for smart monitoringFor damage detection and damage classification, support vector machines (SVM) arecommon. For example, Li et al. (2019) have identified damage based on SVMs andLamb waves in smart monitoring. Gui et al. (2017) have compared different SVMbased optimization techniques for damage detection with a Gaussian radial basis function (RBF) chosen as kernel function. Gardner et al. (2016) have proposed an RBFkernel based SVM, fed by a finite element-based damage model to generate outputdata, while Pan et al. (2018) have proposed a framework for data-driven structuraldiagnosis and damage detection using SVM with wavelet transform, Hilbert-Huangtransform, and Teager-Huang transform as feature extraction methods. Ghiasi et al.(2016) have reported on a new kernel function for least square support vector machines using multidimensional orthogonal-modified Littlewood-Paley wavelets and athin plate spline radial basis function. Abdeljaber et al. (2018) have presented an approach based on a 1-D convolutional neural network (CNN) to detect damage withtwo labeled sets of data, regardless of the size of the structure. Gunawan et al. (2018)have examined k-nearest neighbors (k-NN) algorithms, stating that the accuracy ofthe algorithms strongly depends on the amount of training data, which is often notsufficiently available for solving engineering problems in smart monitoring.For damage localization, Zhao et al. (2019) have presented an algorithm based onANN regression using acoustic emission sensors for carbon fiber reinforced polymercomposite materials. The training data required for the artificial neural network hasbeen obtained from a finite element model.To advance condition assessment of smart structures, Nazarian et al. (2018) havecombined SVMs, ANNs, and Gaussian naïve Bayes techniques to assess the conditionof a masonry building with timber frames. The ML model has been trained by finiteelement model simulation data to relate the change of stiffness of different buildingcomponents to intensity and location of the damage sources.

7Aiming at life-time prediction, Sysyn et al. (2019) have addressed a railway crossing based on features extracted by principal component analysis and partial leastsquare regression. Hoang et al. (2018) have predicted the scour depth at bridges byusing support vector regression, for which several feature selection algorithms havebeen combined, with the variable neighborhood search feature selection methodproviding the best outcome.A number of studies have been reported that aim at combinations of the data analysis goals within smart monitoring, for example pursuing damage detection and damage classification together. Vitola et al. (2016) have presented a combination of principal component analysis (PCA) with k-NN and PCA with bagged trees. Vitola et al(2017a) have compared different k-NN algorithms to detect and to classify damagebased on identical data sets, linked with the research conducted by Tibaduzia et al.(2018) and Vitola et al. (2017b), who combine PCA and k-NN components to detectund to classify damage of sandwich structures and composite plates. Joshuva &Sugumaran (2018) have compared classification and regression algorithms with respect to damage detection and damage classification, including a sequential minimaloptimization classifier, a simple logistic algorithm classifier, a multilayer perceptronin terms of a feedforward artificial neural network, logistic regression, and an RBFnetwork. The authors have been able to define five different damage classes. Vashishtet al. (2018) have compared Bayesian ANNs, CNNs, and long short-term memoryANNs to identify and to localize damage in a cantilever beam with training data forthe ANNs provided by finite element simulations.Unsupervised and hybrid machine learning algorithms for smart monitoringStudies applying unsupervised /hybrid ML algorithms to achieve the goals of dataanalysis in smart monitoring are less common than supervised learning approaches,because labeled training data is usually available in smart monitoring. For example,Sierra-Perez et al. (2017) have presented a multi-layer ANN-based damage detectionmethodology for strain field pattern recognition, using a hierarchical non-linear PCAdimensionality reduction technique. Santos et al. (2016) have improved Gaussianmixture models to detect and to classify damage of bridges. Senniappan et al. (2017)have applied fuzzy cognitive maps to categorize cracks in reinforced concrete columns.Furthermore, Diez et al. (2016) have used a k-NN outlier detector for performingk-means clustering on data in an attempt to isolate and to localize damaged joints of abridge. Das et al. (2019) have used Gaussian mixture models for clustering unlabeleddata and for feature separation by an SVM-calculated hyperplane for crack modeclassification.

83Results and discussion: Towards explainable artificialintelligenceThe result of the review presented in the previous section is shown in Figure 2 interms of an overview of ML algorithms for smart monitoring. As can be seen fromFigure 2, the ML algorithms are assigned to the goals of data analysis in smart monitoring, with the thickness of the lines connecting an ML algorithm and a data analysisgoal denoting the quantity of papers found in literature. Regardless of the ML algorithm and the data analysis goal, it has been concluded that intransparency and mistrust in ML algorithms that are black-box in nature are hindering the widespreadadoption of the algorithms in civil engineering practice. Particularly following theenforcement of the European data protection regulation, which requires comprehensible decision making in AI, the incomprehensibility of ML-based decision makingfurther limits the distribution and implementation of ML algorithms.Fig. 2. Review of machine learning algorithms for smart monitoring.

9XAI has the potential to overcome implementation obstacles and provide explanations as well as additional information regarding decision-making processes, henceoffering more comprehensible ML algorithms. However, when designing XAI-basedML algorithms, different levels of explanations must be considered, ranging from“comprehensive explanation” in case of complex subsymbolic ML algorithms to “noexplanation” in case of symbolic ML algorithms, as implemented in expert systemsthat inherently explain themselves. Further distinctions must be made with respect tothe experience and expertise of human individuals that are addressed by the explanations, such as technicians using ML algorithms in engineering practice or computerscientists implementing data analysis into smart monitoring systems.In general, an explanation is considered a collection of human-interpretable features, relevant to decisions provided by ML algorithms. The explainability of MLalgorithms is often referred to as interpretability, with “interpretation” denoting amapping of abstract concepts that are comprehensible for human individuals (Montavon, et al., 2017). Different efforts towards implementing XAI approaches have beenreported. For example, LIME, a local interpretable explanation, presents a modelindependent approach towards approximating black-box models around any classifierof interest and explaining the predictions of the classifier in an interpretable manner(Ribeiro, et al., 2016). The layer-wise relevance propagation (LRP) algorithm forimage classification serves as another XAI implementation example. LRP decomposes the classifier and iterates the relevance of each layer of a network backwards, starting with the output prediction (Bach, et al., 2015). Aiming to explain autonomousdecisions made by smart monitoring systems with respect to sensor fault diagnosis,Fritz (2019) has implemented an XAI approach that extends deep learning NNs coupled with blockchain technology. In summary, representing an open research problemin smart city applications, it can be concluded that explanations must be adapted tothe goal of data analysis, to the level of explainability, and to the target audience.4Summary and conclusionsSmart infrastructure is a key component of smart cities and requires smart monitoringto achieve more reliable, durable, and cost-efficient infrastructure as compared to thepast. Smart monitoring is a combination of SHM and AI algorithms. ML algorithms, asubcategory of AI algorithms, are used to automatically analyze sensor data. However, the black-box nature of ML algorithms typically used in smart monitoring, although efficient in analyzing sensor data, causes intransparency and mistrust expressedby engineers, thus hindering the exploitation of the ML full potential in engineeringpractice.XAI is supposed to enhance the transparency, thus the confidence, in ML algorithms. Drawing from trends in current ML applications for smart monitoring, thispaper has presented a preliminary step towards adapting XAI approaches in smartmonitoring. ML algorithms commonly deployed to smart monitoring have been reviewed and XAI approaches have been presented, proposed to overcome the obstaclesof incomprehensibility of ML algorithms. For smart monitoring, ML algorithms may

10require different levels of explanations based on their purpose and the human individuals addressed. In conclusion, the overview of ML algorithms in smart monitoringprovided in this paper has demonstrated that an in-depth analysis of explainability andlevels of explanation for ML algorithms is required to advance smart monitoring andsmart city developments.AcknowledgmentsThe authors gratefully acknowledge the support offered by the German ResearchFoundation (DFG) under grants SM 281/9-1, SM 281/14-1, and SM 281/15-1. Thisresearch is also partially supported by the German Federal Ministry of Transport andDigital Infrastructure (BMVI) under grant VB18F1022A. Any opinions, findings,conclusions or recommendations expressed in this paper are those of the authors anddo not necessarily reflect the views of DFG or BMVI.References1. Acatech – National Academy of Science and Engineering (2015). Industry 4.0, Urban development and German international development cooperation (Acatech position paper),Herbert Utz Verlag, Munich, Germany, 20152. Adadi, A., & Berrada, M. (2018). Peeking inside the black-box: A survey on explainableartificial intelligence (XAI). IEEE Access, 6(2018), 52138-52160.3. Abdeljaber, O., Avci, O., Kiranyaz, S., Boashash, B., Sodano, H. & Inman, D. (2018). 1-DCNNs for structural damage detection: verification on a structural health monitoringbenchmark data. Neurocomputing, 275(2018), 1308-1317.4. Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R. & Samek, W. (2015). Onpixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS One, 10(7), e0130140.5. Bao, Y., Chen, Z., Wei, S., Xu, Y., Tang, Z. & Li, H. (2019). The state of the art of datascience and engineering in structural health monitoring. Engineering 5(2), 234-242.6. Barredo Arrieta, A., Diaz Rodriguez, N., Del Ser, J., Bennetot, A., Tabik, S., et al. (2019).Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58(2020), 82-115.7. Bilek, J., Mittrup, I., Smarsly, K. & Hartmann, D. (2003). Agent-based concepts for theholistic modeling of concurrent processes in structural engineering. In: Proceedings of the10th ISPE International Conference on Concurrent Engineering: Research and Applications, Madeira, Portugal, July 26, 2003.8. Bisby, L.A. & Briglio, M.B. (2005). ISIS Educational Module 5: An introduction to structural health monitoring. SAMCO Final Report 2006. Winnipeg, Manitoba, Canada: ISISCanada.9. Burkov, A. (2019). The hundred-page machine learning book. ISBN-13: 978-199957950010. Cha, Y.-J., Choi, W. & Büyüköztürk, O. (2017). Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering, 32(5), 361-378.

1111. Das, A., Suthar, D. & Leung, C. (2019). Machine learning based crack mode classificationfrom unlabeled acoustic emission waveform features. Cement and Concrete Research,121(2019), 42-57.12. Diez, A., Khoa, N.L.D., Makki Alamdari, M., Wang, Y., Chen, F. & Runcie, P. (2016). Aclustering approach for structural health monitoring on bridges. Journal of Civil StructuralHealth Monitoring, 6(2016), 1-17.13. Dragos, K. & Smarsly, K. (2016). Distributed adaptive diagnosis of sensor faults usingstructural response data. Smart Materials and Structures, 25(10), 105019.14. Fritz, H. (2019). An explainable artificial intelligence model coupling deep learning andblockchain technology. Bachelor thesis. Chair of Computing in Civil Engineering, Bauhaus University Weimar, Germany.15. Gardner, P., Barthorpe, R.J. & Lord, C. (2016). The development of a damage model forthe use in machine learning driven SHM and comparison with conventional SHM methods. In: Proceedings of the International Conference on Noise and Vibration Engineering2016 (ISMA 2016) and International Conference on Uncertainty in Structural Dynamics(USD 2016), Leuven, Belgium, September 13, 2016.16. Ghiasi, R., Torkzadeh, P. & Noori, M. (2016). A machine-learning approach for structuraldamage detection using least square support vector machine based on a new combinationalkernel function. Structural Health Monitoring, 15(3), 302-316.17. Gunawan, F., Soewito, B., Surantha, N. & Tuga, M. (2018). One more reason to rejectmanuscript about machine learning for structural health monitoring, In: Proceedings of the2018 Indonesian Association for Pattern Recognition (INAPR) International Conference,Jakarta, Indonesia, September 7, 2018.18. Gui, G., Pan, H., Lin, Z., Li, Y. & Yuan, Z. (2017). Data-driven support vector machinewith optimization techniques for structural health monitoring and damage detection. KSCEJournal of Civil Engineering. 21(2), 523-534.19. Gunning, D. & Aha, D.W. (2019). DARPA’s explainable artificial intelligence program.AI Magazine, 40(2), 44-58.20. Guo, X., Shen, Z., Zhang, Y. & Wu, T. (2019). Review on the application of artificial intelligence in smart homes. Smart Cities, 2(3), 402-420.21. Hartmann, D., Smarsly, K. & Law, K. H., 2011. Coupling sensor-based structural healthmonitoring with finite element model updating for probabilistic lifetime estimation of windenergy converter structures. In: Proceedings of the 8th International Workshop on Structural Health Monitoring, Stanford, CA, USA, September 13, 2011.22. Haugeland, J. (1987). Artificial intelligence. The very idea. Cambridge, MA, USA: MITPress.23. Hoang, N.-D., Liao, K.-W. & Tran, X.-L. (2018). Estimation of scour depth at bridges withcomplex pier foundations using support vector regression integrated with feature selection.Journal of Civil Structural Health Monitoring, 8(3), 431–442.24. Hutter, M. (2005). Universal artificial intelligence – sequential decisions based on algorithmic probability. Heidelberg, Germany: Springer-Verlag GmbH.25. Johnson, P., Robinson, P. & Philpot, S. (2019). Ty

2019), the term "smart city" has not been officially defined (OECD, 2019; Johnson, et al., 2019). However, several key components of smart cities have already been well-established, such as smart living, smart governance, smart citizen (people), smart mobility, smart economy, and smart infrastructure (Mohanty, et al., 2016).