DLoc:Deep Learning Based Wireless Localization For Indoor Navigation
Mobicom 2020 Deep Learning based Wireless Localization for Indoor Navigation Roshan Ayyalasomayajula, Aditya Arun, Chenfeng Wu, Sanatan Sharma, Abhishek Sethi, Deepak Vasisht and Dinesh Bharadia https://wcsng.ucsd.edu/dloc/ 1
Outdoor Localization 2
Outdoor Localization Navigation https://play.google.com/ Delivery Bots Autonomous Driving g-is-no-threat-autonomousnavigation/ Autonomous Drones ast-mile-solutionaebebe557ad4 2
Outdoor Localization Navigation https://play.google.com/ Delivery Bots Autonomous Driving g-is-no-threat-autonomousnavigation/ Autonomous Drones ast-mile-solutionaebebe557ad4 2
Outdoor Localization Navigation https://play.google.com/ Delivery Bots Autonomous Driving g-is-no-threat-autonomousnavigation/ Autonomous Drones ast-mile-solutionaebebe557ad4 https://www.nasa.gov/sites/default/files/gps constellation 0.jp g 2
Indoor Localization 3
Indoor Localization https://www.clipartkey.com/upic/4785/ 3
Indoor Localization https://www.clipartkey.com/upic/4785/ https://medium.com/@brian.moran hopping-mallsbe6376bf3d2 3
Indoor Localization https://www.clipartkey.com/upic/4785/ https://medium.com/@brian.moran hopping-mallsbe6376bf3d2 ll-e-were-probably-better-off-without-him/ 3
Lack of Context 4
Lack of Context Y-axis (m) 19, 13 X-axis (m) 4
Lack of Context Y-axis (m) 19, 13 X-axis (m) 4
Lack of Context Y-axis (m) 19, 13 X-axis (m) 4
Lack of Context Y-axis (m) 19, 13 X-axis (m) 4
WiFi based localization 5
WiFi based localization MonoLoco MobiSys'18 Chronos NSDI'16 ToneTrack Mobicom'15 SpotFi Sigcomm'15 ArrayTrack NSDI'13 5
WiFi based localization MonoLoco MobiSys'18 Chronos NSDI'16 ToneTrack Mobicom'15 SpotFi Sigcomm'15 ArrayTrack NSDI'13 Median: few decimeters 5
WiFi based localization MonoLoco MobiSys'18 Chronos NSDI'16 Median: few decimeters ToneTrack Mobicom'15 SpotFi Sigcomm'15 ArrayTrack NSDI'13 10% of cases: few meters 5
Deep Learning based Wireless Localization for Indoor Navigation DLoc and MapFind 6
Deep Learning based Wireless Localization 7
Deep Learning based Wireless Localization Localization: Novel learning based approach to solve for the environment dependent localization. 7
Deep Learning based Wireless Localization Localization: Novel learning based approach to solve for the environment dependent localization. Table Context: Bot that collects both Visual and WiFi data. Tables Desk 7
Deep Learning based Wireless Localization Localization: Novel learning based approach to solve for the environment dependent localization. Table Context: Bot that collects both Visual and WiFi data. Dataset: Deployed it in 8 different in a Simple and Complex Environment Tables Desk 7
Deep Learning based Wireless Localization Localization: Novel learning based approach to solve for the environment dependent localization. Table Context: Bot that collects both Visual and WiFi data. Dataset: Deployed it in 8 different in a Simple and Complex Environment Tables Desk Results: Shown a 85% improvement compared to state of the art at 90th percentile. 7
Challenge: Multipath, Non-Line of Sight 8
Challenge: Multipath, Non-Line of Sight Smartphon e r ΞΈ A P 8
Challenge: Multipath, Non-Line of Sight Reflecto r Smartphon e r ΞΈ A P 8
Challenge: Multipath, Non-Line of Sight Smartphon e Reflecto r Obstacl e A P 8
Challenge: Multipath, Non-Line of Sight Smartphon e Reflecto r Obstacl e Need KnowledgeAof Environment P 8
Requirements to design the neural network 9
Requirements to design the neural network Input Representation 9
Requirements to design the neural network Input Representation Output/Target Representation 9
Requirements to design the neural network Input Representation Network Output/Target Representation 9
Requirements to design the neural network Input Representation Network Output/Target Representation Objective/Loss Function 9
Requirements to design the neural network Input Representation Network Gradient Output/Target Representation Objective/Loss Function 9
Input Representation: Raw CSI data 10
Input Representation: Raw CSI data Maximillian Arnold et. al., SCC 2019 Michal Nowicki et. al., ICA, 2017 Xuyu Wang, et al., IEEE Access 5, 2017 Xialong Zheng, et al., IEEE/ACM Transactions on Networking, 2017 10
Input Representation: Raw CSI data Maximillian Arnold et. al., SCC 2019 Michal Nowicki et. al., ICA, 2017 Xuyu Wang, et al., IEEE Access 5, 2017 Xialong Zheng, et al., IEEE/ACM Transactions on Networking, 2017 Complex Channel Values and AWG noise 10
Input Representation: Raw CSI data Maximillian Arnold et. al., SCC 2019 Michal Nowicki et. al., ICA, 2017 Xuyu Wang, et al., IEEE Access 5, 2017 Xialong Zheng, et al., IEEE/ACM Transactions on Networking, 2017 Complex Channel Values and AWG noise Can we represent them as images? 10
Input Representation: AoA-ToF images 11
Input Representation: AoA-ToF images 11
Input Representation: AoA-ToF images Does not have context of Space and AP locations 11
Input Representation: XY images Angle of Arrival, π· AoA-ToF (Polar) to XY (cartesian) Time of Flight(m) 12
Input Representation: XY images Angle of Arrival, π· AoA-ToF (Polar) to XY (cartesian) Y axis (m) Polar to Cartesian Time of Flight(m) X axis (m) 12
Input Representation: XY images Angle of Arrival, π· AoA-ToF (Polar) to XY (cartesian) Y axis (m) Polar to Cartesian Time of Context Flight(m) embedded Input X axis Representation (m) 12
Location Decoder Output targets 13
Location Decoder Output targets 13
Location Decoder Output targets 13
Location Decoder Output targets Image-to-Image translation problem 13
Network Architecture 14
Network Architecture Location Decoder Encoder 3 Resnet Blocks 6 Resnet Blocks 128 4 256 256 256 Output Location 12 8 64 64 1 Input Images 1. 2. 3. Conv2d 1. [7,7,1,3] Instance norm 2. Tanh 3. Conv2d [7,7,1,3] Instance norm ReLU 1. 2. 3. Conv2d [3,3,2,1] Instance norm ReLU Resnet Block* 1. 2. 3. ConvTranspose2d [3,3,2,1] Instance norm ReLU 1. 2. 3. Conv2d [7,7,1,3] Instance norm Sigmoid 14
Network Architecture Location Decoder Encoder 3 Resnet Blocks 6 Resnet Blocks 128 4 256 256 256 Output Location 12 8 64 64 1 Input Images 1. 2. 3. Conv2d 1. [7,7,1,3] Instance norm 2. Tanh 3. Conv2d [7,7,1,3] Instance norm ReLU 1. 2. 3. Conv2d [3,3,2,1] Instance norm ReLU Resnet Block* 1. 2. 3. ConvTranspose2d [3,3,2,1] Instance norm ReLU 1. 2. 3. Llocation Conv2d [7,7,1,3] Instance norm Sigmoid 14
Location Loss Closeness in MSE sense πΏπππππ‘πππ πΏ2[π·πππππ‘πππ πΈ π» π] 15
Location Loss Closeness in MSE sense πΏπππππ‘πππ πΏ2[π·πππππ‘πππ πΈ π» π] Penalize multiple peaks 15
Location Loss Closeness in MSE sense πΏπππππ‘πππ πΏ2[π·πππππ‘πππ πΈ π» π] Penalize multiple peaks πΏπππππ‘πππ πΏ2 π·πππππ‘πππ πΈ π» π Ξ» πΏ1[π·πππππ‘πππ πΈ π» ] 15
th 90 High Clocks percentile errors: Asynchronous 16
th 90 High Clocks percentile errors: Asynchronous Smartphon e r AP 16
th 90 High Clocks percentile errors: Asynchronous Smartphone r Ξr AP 16
ToF offset 17
ToF offset compensation 18
DLoc: Network Architecture Location Decoder Encoder 6 Resnet Blocks 128 4 64 256 Offset Corrected Images 256 3 Resnet Blocks 256 12 8 64 1 Output Location Input Images 19
DLoc: Network Architecture Offset Corrected Images Consistency Decoder (π·ππππ ππ π‘ππππ¦ ) 6 Resnet Blocks 256 12 8 6 Resnet Blocks 128 4 64 4 Location Decoder Encoder 256 64 256 3 Resnet Blocks 256 12 8 64 1 Output Location Input Images 19
Insight: Single source 20
DLoc: Network Architecture Offset Corrected Images Consistency Decoder (π·ππππ ππ π‘ππππ¦ ) 6 Resnet Blocks 256 12 8 3 Resnet Blocks 6 Resnet Blocks 128 4 Input Images 64 4 Location Decoder Encoder 256 64 256 256 12 8 64 1 Output Location πΏπππππ‘πππ 21
DLoc: Network Architecture Offset Corrected Images Consistency Decoder (π·ππππ ππ π‘ππππ¦ ) 6 Resnet Blocks 256 12 8 64 4 πΏππππ ππ π‘ππππ¦ Location Decoder Encoder 3 Resnet Blocks 6 Resnet Blocks 128 4 Input Images 64 256 256 256 12 8 64 1 Output Location πΏπππππ‘πππ 21
Offset Compensation Loss Defines consistency across images ππ΄π πΏππππ ππ π‘ππππ¦ 1 π¦ (πΈ(π»)) πππππ ππ π‘ππππ¦ ]π ππ΄π π 1 22
Offset Compensation Loss Defines consistency across images ππ΄π πΏππππ ππ π‘ππππ¦ 1 π¦ (πΈ(π»)) πππππ ππ π‘ππππ¦ ]π ππ΄π π 1 22
Context: MapFind 23
Context: MapFind WiFi Client LiDAR RGB-D Server Kobuki 23
Context: MapFind Table Tables Desk 23
Context: MapFind Table Tables Desk How much data is needed? 23
Path Planning 24
Path Planning Maximize coverage Minimize traversal length 24
Path Planning Maximize coverage Minimize traversal length Context Enabled Accurate Indoor Localization 24
Results 25
Setup 26
Setup Plasma Screens AP 3 AP 2 AP4 NLOS Plasma Screens AP 1 Complex High-multipath and NLOS environment (1500 sq. ft.) 26
Setup Plasma Screens AP 3 AP 2 Blocka ge AP4 NLOS Plasma Screens AP 1 Complex High-multipath and NLOS environment (1500 sq. ft.) 26
Setup Plasma Screens AP 3 AP 2 Blocka ge AP4 NLOS Plasma Screens AP 3 AP 1 Complex High-multipath and NLOS environment (1500 sq. ft.) Reflect or AP 2 AP 1 Simple LOS based environment (500 sq. ft.) 26
CDF DLoc Result β Complex Environment 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0.4 0.8 1.2 1.6 2 2.4 2.8 3.2 3.6 Localization Error (m) 4 4.4 4.8 27
CDF DLoc Result β Complex Environment 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 DLoc Baseline DL Model SpotFi 0 0.4 0.8 1.2 1.6 2 2.4 2.8 3.2 3.6 Localization Error (m) 4 4.4 4.8 27
CDF DLoc Result β Complex Environment 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 DLoc Baseline DL Model SpotFi 0 0.4 0.8 1.2 1.6 2 2.4 2.8 3.2 3.6 Localization Error (m) 4 4.4 4.8 27
CDF DLoc Result β Complex Environment 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 DLoc Baseline DL Model SpotFi 0 0.4 0.8 1.2 1.6 2 2.4 2.8 3.2 3.6 Localization Error (m) 4 4.4 4.8 27
CDF DLoc Result β Complex Environment 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 DLoc Baseline DL Model SpotFi 0 0.4 0.8 1.2 1.6 2 2.4 2.8 3.2 3.6 Localization Error (m) 4 4.4 4.8 27
CDF DLoc Result β Simple Environment 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.2 0.4 0.6 0.8 1 1.2 1.4 Localization Error (m) 1.6 1.8 2 28
CDF DLoc Result β Simple Environment 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 DLoc Baseline DL Model SpotFi 0 0.2 0.4 0.6 0.8 1 1.2 1.4 Localization Error (m) 1.6 1.8 2 28
CDF DLoc Result β Simple Environment 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 DLoc Baseline DL Model SpotFi 0 0.2 0.4 0.6 0.8 1 1.2 1.4 Localization Error (m) 1.6 1.8 2 28
CDF DLoc Result β Simple Environment 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 DLoc Baseline DL Model SpotFi 0 0.2 0.4 0.6 0.8 1 Localization 1.2 1.4 1.6 Accurate Indoor Localization Error (m) 1.8 2 28
Generalization across multiple setups 29
Generalization across multiple setups Setup-1 Setup-3 Setup-2 Setup-4 29
Generalization across multiple setups Trained Tested on on Setup Setup Setup-1 Setup-3 Setup-2 1,3,4 2 1,2,4 3 1,2,3 4 Median Error (cm) 90th Percentile Error (cm) DLoc DLoc SpotFi SpotFi Setup-4 29
Generalization across multiple setups Trained Tested on on Setup Setup Setup-1 Setup-3 Setup-2 Median Error (cm) 90th Percentile Error (cm) DLoc DLoc SpotFi SpotFi 1,3,4 2 198 420 1,2,4 3 154 380 1,2,3 4 161 455 Setup-4 29
Generalization across multiple setups Trained Tested on on Setup Setup Setup-1 Setup-3 Setup-2 Median Error (cm) 90th Percentile Error (cm) DLoc SpotFi DLoc SpotFi 1,3,4 2 71 198 171 420 1,2,4 3 82 154 252 380 1,2,3 4 105 161 277 455 Setup-4 29
Open-Sourced Dataset 30
Open-Sourced Dataset Enabling Baseline comparison for all algorithms 30
Open-Sourced Dataset Enabling Baseline comparison for all algorithms Pushing Indoor Localization to realization 30
Open-Sourced Dataset Enabling Baseline comparison for all algorithms Pushing Indoor Localization to realization Pushing towards a competition similar to ImageNet program 30
Open-Sourced Dataset https://wcsng.ucsd.edu/wild/ Enabling Baseline comparison for all algorithms Pushing Indoor Localization to realization Pushing towards a competition similar to ImageNet program Labelled WiFi CSI data (WILD-v1) 8 different setups 4 different days 108K datapoints 2 different environments 30
Open-Sourced Dataset https://wcsng.ucsd.edu/wild/ Enabling Baseline comparison for all algorithms Pushing Indoor Localization to realization Pushing towards a competition similar to ImageNet program Labelled WiFi CSI data (WILD-v1) 8 different setups 4 different days 108K datapoints 2 different environments WILD-v2 Coming Soon 20 different setups 10 different days 1 million datapoints 8 different environments 20 different AP locations 30
Conclusion and Future Work Novel Deep Learning based algorithm with 85% incremental performance compared to state-of-the-art. MapFind we have collected over 108k datapoints (and expanding) that is opensourced. Enabling large scale and autonomous indoor navigation https://wcsng.ucsd.edu/dloc/ 31
Deep Learning based Wireless Localization Localization: Novel learning based approach to solve for the environment dependent localization. Context: Bot that collects both Visual and WiFi data. Dataset: Deployed it in 8 different in a Simple and Complex Environment Results: Shown a 85% improvement compared to state of the art at 90th percentile .
and maintained by the University of Florida Libraries by two programmers at a cost of 150,000 USD annually, on more than 60,000 USD of computer and network hardware. This cost is extremely affordable compared to the open marketplace because UF leverages the same resources to power dLOC, the Florida Digital Newspaper Library, and other large .
Deep Learning: Top 7 Ways to Get Started with MATLAB Deep Learning with MATLAB: Quick-Start Videos Start Deep Learning Faster Using Transfer Learning Transfer Learning Using AlexNet Introduction to Convolutional Neural Networks Create a Simple Deep Learning Network for Classification Deep Learning for Computer Vision with MATLAB
2.3 Deep Reinforcement Learning: Deep Q-Network 7 that the output computed is consistent with the training labels in the training set for a given image. [1] 2.3 Deep Reinforcement Learning: Deep Q-Network Deep Reinforcement Learning are implementations of Reinforcement Learning methods that use Deep Neural Networks to calculate the optimal policy.
-The Past, Present, and Future of Deep Learning -What are Deep Neural Networks? -Diverse Applications of Deep Learning -Deep Learning Frameworks Overview of Execution Environments Parallel and Distributed DNN Training Latest Trends in HPC Technologies Challenges in Exploiting HPC Technologies for Deep Learning
Deep Learning Personal assistant Personalised learning Recommendations RΓ©ponse automatique Deep learning and Big data for cardiology. 4 2017 Deep Learning. 5 2017 Overview Machine Learning Deep Learning DeLTA. 6 2017 AI The science and engineering of making intelligent machines.
English teaching and Learning in Senior High, hoping to provide some fresh thoughts of deep learning in English of Senior High. 2. Deep learning . 2.1 The concept of deep learning . Deep learning was put forward in a paper namedon Qualitative Differences in Learning: I -
TRENDnetβs AC1750 Dual Band Wireless Router, model TEW-812DRU, produces the ultimate wireless experience with gigabit wireless speeds. Manage two wireless networksβthe 1300 Mbps Wireless AC band for the fastest wireless available and the 450 Mbps Wireless N ba
This textbook is designed for use on ten- or twelve-week introductory courses on English phonology of the sort taught in the ο¬rst year of many English Language and Linguistics degrees, in British and American universities. Students on such courses can struggle with phonetics and phonology; it is sometimes difο¬cult to see past the new .
