Deep Learning Driven Mobile Traffic Analysis - Itu.int
Attacking Mobile Traffic Analytics and Backhaul Utility Maximisation with Deep Learning Paul Patras Work with Chaoyun Zhang, Rui Li, John Thompson (U Edinburgh), Xi Ouyang (Huazhong UST), Pan Cao (U Hertfordshire)
The Goal of Mobile Traffic Analysis Mobile traffic consumption continues to grow 4-fold increase in the next 5 years
The Goal of Mobile Traffic Analysis Mobile traffic consumption continues to grow 4-fold increase in the next 5 years Mining exabytes of data may offer valuable insights
The Goal of Mobile Traffic Analysis Mobile traffic consumption continues to grow
Forecasting future mobile traffic consumption
TheofGoal of Mobile Traffic Analysis Importance Precision Mobile Traffic Forecasting 1. Precision traffic engineering - On demand allocation of resources - Building Intelligent 5G networks 2. Energy saving – Green cellular networks 3. Mobility analysis - Movement prediction - Transportation planning
TheofGoal of Mobile Traffic Analysis Importance Precision Mobile Traffic Forecasting 1. Precision traffic engineering - On demand allocation of resources - Building Intelligent 5G networks 2. Energy saving – Green cellular networks 3. Mobility analysis - Movement prediction - Transportation planning Long-term mobile traffic forecasting is key!
Fine-Grained Traffic Measurement Continuous measurements are expensive 1. Rely on dedicated infrastructure Packet Gateway (PGW) or Radio Network Controller (RNC) probes Data storage
Fine-Grained Traffic Measurement Continuous measurements are expensive 1. Rely on dedicated infrastructure Packet Gateway (PGW) or Radio Network Controller (RNC) probes Data storage 2. Data post-processing overheads Call detail record reports transfer Spatial aggregation
Fine-Grained Traditional ApproachesTraffic Measurement Exponential Smoothing (ES) and Autoregressive Integrated Moving Average model (ARIMA): 1. Operate on individual time series, while ignoring spatial correlations. 2. Performance degenerates considerably over time. 3. Do not generalise well in different locations.
The Goal ofatMobile Traffic Analysis Solving forecasting city-scale Forecasting Mobile traffic measurements (Different color corresponds to different traffic volume)
Fine-Grained Measurement The Potential of DeepTraffic Learning Convolutional neural networks (ConvNets) work particularly well in handling spatial data.
Fine-Grained Measurement The Potential of DeepTraffic Learning Convolutional neural networks (ConvNets) work particularly well in handling spatial data. Recurrent neural networks (e.g. LSTM) can capture temporal dependencies.
Fine-Grained Measurement The Potential of DeepTraffic Learning Convolutional neural networks (ConvNets) work particularly well in handling spatial data. Recurrent neural networks (e.g. LSTM) can capture temporal dependencies. Advanced GPU computing enables to train NN architectures fast and deliver real-time inference.
Fine-Grained Neural Traffic Measurement STN: Spatio-Temporal Network
Fine-Grained Neural Traffic Measurement STN: Spatio-Temporal Network
Fine-Grained Traffic Measurement ConvLSTM Convolutional Long Short-Term Memory (ConvLSTM) Advanced version of LSTM. Replaces inner dense connections with convolution operations. Works remarkably well in modelling spatio-temporal data.
Traffic Measurement ProposedFine-Grained Solution: Spatio-Temporal Neural Network
Fine-Grained Traffic Measurement 3D-ConvNet 3D Convolutional Neural Network (3D-ConvNet)
Fine-Grained Traffic Measurement 3D-ConvNet 3D Convolutional Neural Network (3D-ConvNet)
Fine-Grained Traffic STN: Spatio-Temporal NeuralMeasurement Network
Fine-Grained Traffic STN: Spatio-Temporal NeuralMeasurement Network
Fine-Grained Traffic STN: Spatio-Temporal NeuralMeasurement Network
One-stepFine-Grained predictions Traffic Measurement Input: A local r r (s 1) tensor (s 11, 2 hours)
One-stepFine-Grained predictions Traffic Measurement Input: A local r r (s 1) tensor Output: The central point in the traffic snapshot t 1
One-stepFine-Grained predictions Traffic Measurement Use the model to scan the entire input map and obtain the predicted traffic snapshot at t 1.
One-stepFine-Grained predictions Traffic Measurement The traffic consumption at a certain location largely depends on that in its neighouring cells.
Fine-Grained Traffic Measurement Extending to Long-Term Forecasting Feeding the model with previous forecasts: Error accumulates.
Fine-Grained Traffic Measurement Extending to Long-Term Forecasting Feeding the model with previous forecasts: Error accumulates.
Fine-Grained Traffic Measurement Extending to Long-Term Forecasting Feeding the model with previous forecasts: Error accumulates. Ideal
Fine-Grained Traffic(OTS) Measurement Ouroboros Training Scheme OTS: Fine-tuning the model with earlier predictions. Fine-Tune with Earlier Prediction
Fine-Grained Traffic(OTS) Measurement Ouroboros Training Scheme OTS: Fine-tuning the model with earlier predictions. 1. Reuse predictions as input of the model. Fine-Tune with Earlier Prediction
Fine-Grained Traffic(OTS) Measurement Ouroboros Training Scheme OTS: Fine-tuning the model with earlier predictions. 1.Reuse predictions as input of the model. 2.Train the model with predictions and ground truth. Fine-Tune with Earlier Prediction
Fine-Grained Traffic(OTS) Measurement Ouroboros Training Scheme OTS: Fine-tuning the model with earlier predictions. 1.Reuse predictions as input of the model. 2.Train the model with predictions and ground truth. Fine-Tune with Earlier Prediction
Fine-Grained Traffic(OTS) Measurement Ouroboros Training Scheme OTS: Fine-tuning the model with earlier predictions. 1.Reuse predictions as input of the model. 2.Train the model with predictions and ground truth. Fine-Tune with Earlier Prediction
Fine-Grained Traffic(OTS) Measurement Ouroboros Training Scheme OTS: Fine-tuning the model with earlier predictions. 1.Reuse predictions as input of the model. 2.Train the model with predictions and ground truth. Fine-Tune with Earlier Prediction
Fine-Grained Traffic(OTS) Measurement Ouroboros Training Scheme OTS: Fine-tuning the model with earlier predictions. 1.Reuse predictions as input of the model. 2.Train the model with predictions and ground truth. Fine-Tune with Earlier Prediction
Fine-Grained Traffic Measurement Embedding Historical Statistics Problem: Uncertainty still grows with the number of prediction steps.
Fine-Grained Traffic Measurement Embedding Historical Statistics Problem: Uncertainty still grows with the number of prediction steps. Observation: Individual traffic series are relatively close to their mean values calculated over historical data.
Fine-Grained Traffic Measurement Embedding Historical Statistics Problem: Uncertainty still grows with the number of prediction steps.
Fine-Grained Traffic Measurement Embedding Historical Statistics Problem: Uncertainty still grows with the number of prediction steps. Observation: Individual traffic series are relatively close to their mean values calculated over historical data. Solution: Mix model predictions with empirical mean by a decaying weight. Forecast 𝑡 1 𝑤 𝑡 STN 𝑤 𝑡 Mean 1 𝑤 𝑡 1 𝑒 𝑎 𝑡 𝑏
Fine-Grained Traffic Measurement D-STN - Long-Term Forecasting OTS mixing data with empirical mean allows feeding earlier predictions as input, while achieving precise forecasting: Double STN (D-STN)
Dataset Dataset Telecom Italia's Big Data Challenge
Dataset Dataset Telecom Italia's Big Data Challenge Measurements of mobile traffic volume between 1 Nov 2013 and 1 Jan 2014 in the city of Milan and province of Trentino.
Dataset Dataset Telecom Italia's Big Data Challenge Measurements of mobile traffic volume between 1 Nov 2013 and 1 Jan 2014 in the city of Milan and province of Trentino. Aggregated every 10 minutes.
Dataset Dataset Telecom Italia's Big Data Challenge Measurements of mobile traffic volume between 1 Nov 2013 and 1 Jan 2014 in the city of Milan and province of Trentino. Aggregated every 10 minutes. Partitioned in 100 100 grids for Milan and 117 98 grids for Trentino.
Comparison MethodsMethods used forfor comparison 1. Traditional Forecasting Tools (Trained on both Milan and Trentino) Holt-Winters Exponential Smoothing (HW-ExpS) Autoregressive Integrated Moving Average Model (ARIMA)
Comparison MethodsMethods used forfor comparison 1. Traditional Forecasting Tools (Trained on both Milan and Tretino) Holt-Winters Exponential Smoothing (HW-ExpS) Autoregressive Integrated Moving Average Model (ARIMA) 2. Machine Learning Approaches (Trained on Milan ONLY) Support Vector Machine (SVM) Auto-Encoder Long Short-Term Memory (AE LSTM)
Comparison MethodsMethods used forfor comparison 1. Traditional Forecasting Tools (Trained on both Milan and Tretino) Holt-Winters Exponential Smoothing (HW-ExpS) Autoregressive Integrated Moving Average Model (ARIMA) 2. Machine Learning Approaches (Trained on Milan ONLY) Support Vector Machine (SVM) Auto-Encoder Long Short-Term Memory (AE LSTM) 3. Components of D-STN (Trained on Milan ONLY) STN (D-STN without OTS), ConvLSTM, 3D-ConvNet, MLP
Results Evaluation Milan 5 mins 100 mins 5h Normalised Root Mean Square Error (NRMSE): The prediction accuracy. 10h
Results Evaluation Milan 5 mins 100 mins 5h Normalised Root Mean Square Error (NRMSE): The prediction accuracy. 10h
Results Evaluation Milan 5 mins 100 mins 5h Normalised Root Mean Square Error (NRMSE): The prediction accuracy. 10h
Results Evaluation Milan 5 mins 100 mins 5h Normalised Root Mean Square Error (NRMSE): The prediction accuracy. 10h
Results Evaluation Milan 5 mins 100 mins 5h Normalised Root Mean Square Error (NRMSE): The prediction accuracy. 10h
Results Evaluation Milan 5 mins 100 mins 5h Normalised Root Mean Square Error (NRMSE): The prediction accuracy. 10h
Results Evaluation Milan 5 mins 100 mins 5h 10h (D-)STN can deliver reliable 10-hour forecasting, given 2 hours of observation. Achieve up to 61% lower prediction error.
Example (10 hours)
Example (10 hours)
Example (10 hours)
Example (10 hours)
Results Generalisation Trentino 5 mins 100 mins 5h 10h
Results Generalisation Trentino 5 mins 100 mins 5h 10h Generalises well to a different deployment without retraining.
Summary 1 Spatio-Temporal Neural Network (STN) to perform mobile traffic forecasting 2 D-STN: OTS mixing predictions with empirical mean 3 Reliable long-term forecasting, outperforming other methods; generalise well to different deployments. C. Zhang, P. Patras, "Long-Term Mobile Traffic Forecasting Using Deep Spatio-Temporal Neural Networks", in Proceedings of the ACM International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc), Los Angeles, USA, June 2018.
Maximising the utility of virtualised backhauls
Increasingly diverse services 5G networks need to accommodate services with distinct performance requirements Bandwidth: UHD video streaming and AR/VR Delay: Autonomous vehicles and remote medical care Network slicing Partitioning physical infrastructure into logically isolated networks Network densification High speed wireless backhauling tangible (mm-wave, free space optics, etc.)
Small Cell Backhauling 4 2 5 3 1 GW
Virtually Sliced (mm-wave) Backhaul Network Slice 1 Video Slice 2 IoT 4 2 Physical Infrastructure 5 1 GW 3 Resource Allocation : Rate ri,j for flow fi,j To meet the service requirements and to maximise resource utilitsation
Utility Functions for Different Applications QoS (AR/VR) Sigmoid: Usig(r) Best-effort (IoT) 1 1 𝑒 𝛼1(𝑟 𝛽1) Logarithmic: Ulog(r) log(𝛼2𝑟 𝛽2)
Utility Functions for Different Applications Delay sensitive (Tele-Operation) Revenue (other) Polynomial: Uply(r) 𝛼3(𝑟 𝛽3 ) Linear: Ulnr(r) 𝛼4𝑟 𝛽4
Combining some of these in a “simple” scenario
General utility framework Arbitrary combinations of all know types of utilities Sigmoid: Usig(r) 1 1 𝑒 𝛼1(𝑟 𝛽1) Polynomial: Uply(r) 𝛼2(𝑟 𝛽2 ) Logarithmic: Ulog(r) log(𝛼3𝑟 𝛽3) Linear: Ulnr(r) 𝛼4𝑟 𝛽4 arg 𝑚𝑎𝑥 σ 𝑈 ri,j
Utility Maximisation High-dimensional problem, highly non-convex Global search is time consuming Heuristic methods method can cansolve solvebut butsub-optimal sub-optimal
Utility Maximisation High-dimensional problem, highly non-convex Global search is time consuming Heuristic methods method can cansolve solvebut butsub-optimal sub-optimal Deep Learning Approach: Learn the correlation between flow demands and optimal allocations
Deep Learning approach to Maximising network Utility (DELMU)
Dataset 4 multi-hop topologies Flows i with all types of utilities over each path j A range of Flow demands (d𝑖,𝑗 ) and minimum service rates (δ𝑖,𝑗 ) Optimal solutions to utility maximization problem obtained using Global Search (GS) algorithm* 10,000 (“ground truth”) data points in total 4/5 used for training, 1/5 for testing * Optimality of GS proven in Z. Ugray et al. Scatter search and local NLP solvers: A multistart framework for global optimization. Journal on Computing, 19(3), 2007.
Training & Benchmarks NN training performed using GPU (minutes) and SGD algorithm; inference using CPU “Sanity check” routine to ensure flows do not violate capacity constraints after allocation
Training & Benchmarks NN training performed using GPU (minutes) and SGD algorithm; inference using CPU “Sanity check” routine to ensure flows do not violate capacity constraints after allocation Benchmarks Global Search (optimal but slow) Greedy – recursively increase flow rates so as to improve utility (should work fast)
Evaluation: topologies Topology 1 Topology 2 Path 1 1 2772 Mbps 2 1732 Mbps 3 693 Mbps 4 Path 1 3465 Mbps 5 4158 Mbps 5197 Mbps 6 7 6756 Mbps 8 9 1 4 Topology 4 693 Mbps 0 2079 Mbps Path 3 Path 2 2 2772 Mbps 3 Path 3 Topology 3 Path 1 2 2772 Mbps Path 2 6756 Mbps Path 2 693Mbps 0 866 Mbps 0 1 1386 Mbps Path 1 2 1 3465 Mbps 5197 Mbps 3 3465 Mbps Path 2 Path 3 0 Path 3 4 3 866 Mbps 866 Mbps
Results: Total Utility Distribution
Results: Total Utility Distribution 62%
Results: Total Utility Distribution 5% 62%
Results: Utility per Traffic Type
Results: Utility per Traffic Type
Results: Utility per Traffic Type
Results: Utility per Traffic Type
Computation time Dell workstation Average computation time over 2k instances
Computation time Dell workstation Average computation time over 2k instances
Computation time Dell workstation Average computation time over 2k instances Inference up to 2300x faster than GS runtime At least 42x faster than Greedy
Summary A general utility framework that encompasses all known types of utility functions Delmu achieves close-to-optimal utility solutions, and makes rapid inferences Suitable for 5G backhauls with real-time and dynamic requirements R. Li, C. Zhang, P. Cao, P. Patras, J. S. Thompson, "DELMU: A Deep Learning Approach to Maximising the Utility of Virtualised Millimetre-Wave Backhauls", in Proceedings International Conference on Machine Learning for Networking (MLN), Paris, France, November 2018.
The Goal of Mobile Traffic Analysis 1. Precision traffic engineering - On demand allocation of resources - Building Intelligent 5G networks 2. Energy saving -Green cellular networks 3. Mobility analysis - Movement prediction - Transportation planning Importance of Precision Mobile Traffic Forecasting Long-term mobile traffic forecasting is key!
Deep Learning: Top 7 Ways to Get Started with MATLAB Deep Learning with MATLAB: Quick-Start Videos Start Deep Learning Faster Using Transfer Learning Transfer Learning Using AlexNet Introduction to Convolutional Neural Networks Create a Simple Deep Learning Network for Classification Deep Learning for Computer Vision with MATLAB
nature, deep learning algorithms can represent traffic features without prior knowledge, which has good performance for traffic flow prediction. In this paper, we propose a deep-learning-based traffic flow del is used to learn generic traffic flow features, and it is trained
2.3 Deep Reinforcement Learning: Deep Q-Network 7 that the output computed is consistent with the training labels in the training set for a given image. [1] 2.3 Deep Reinforcement Learning: Deep Q-Network Deep Reinforcement Learning are implementations of Reinforcement Learning methods that use Deep Neural Networks to calculate the optimal policy.
SA Learner Driver Manual Road Traffic Signs Version: Draft Page 1 of 56 2. ROAD TRAFFIC SIGNS, SIGNALS AND MARKINGS The purpose of road traffic signs is to regulate traffic in such a way that traffic flow and road traffic safety are promoted. 1. SIGNS IN GENERAL Road traffic signs can be divided into the following six main groups:
Deep Learning Models for Traffic Accident Prediction:Some recent work have attempted to tackle the traffic accident analysis problem using deep learning models. Chen et al. [11] utilized mobile phone data and historical accident records to build a model for real-time accident risk assessment. Najjar et al. [19] used Convolutional
-The Past, Present, and Future of Deep Learning -What are Deep Neural Networks? -Diverse Applications of Deep Learning -Deep Learning Frameworks Overview of Execution Environments Parallel and Distributed DNN Training Latest Trends in HPC Technologies Challenges in Exploiting HPC Technologies for Deep Learning
Deep Learning Personal assistant Personalised learning Recommendations Réponse automatique Deep learning and Big data for cardiology. 4 2017 Deep Learning. 5 2017 Overview Machine Learning Deep Learning DeLTA. 6 2017 AI The science and engineering of making intelligent machines.
ADVANCED ENGINEERING MATHEMATICS By ERWIN KREYSZIG 9TH EDITION This is Downloaded From www.mechanical.tk Visit www.mechanical.tk For More Solution Manuals Hand Books And Much Much More. INSTRUCTOR’S MANUAL FOR ADVANCED ENGINEERING MATHEMATICS imfm.qxd 9/15/05 12:06 PM Page i. imfm.qxd 9/15/05 12:06 PM Page ii. INSTRUCTOR’S MANUAL FOR ADVANCED ENGINEERING MATHEMATICS NINTH EDITION ERWIN .
