AI On The Edge: Characterizing AI-based IoT Applications Using . - UMass
AI on the Edge: Characterizing AI-based IoTApplications using Specialized Edge ArchitecturesQianlin LiangPrashant ShenoyDavid cs.umass.edu
EDGEComputing infrastructure that is positioned between endpoint device and cloud.IoT DevicesEdgeCloud
EDGE-BASED AI WORKLOADSAn emerging class of edge workloads:§ Running deep learning inference on edgeDeep Neural Network§ Computationally intensive“cat”
COMPUTING PARADIGMS FOR IOT APPLICATIONSCloud ServerCloud AcceleratorEdge ServerEdge AcceleratorIoT DevicesDevice Accelerator
TRADITIONAL EDGE VS. ACCELERATORSVS.1) What are the price, performance, and energy benefits offered by edge hardware accelerators?2) How should modern IoT applications exploit the distributed processing capabilities of specializededge nodes and the cloud by using various types of split processing?3) How suitable are edge accelerators for supporting concurrent edge applications from multipletenants?
SPECIALIZED EDGE ACCELERARTORSIntel NCS2Google EdgeTPUNvidia Jetson NanoNvidia Jetson TX2Power:1-2 WPower:0.5-2 WPower:5-10 WPower:7.5-15 WMemory:512 MBMemory:8 MBMemory:4 GBMemory:8 GBPrice: 99Price: 75Price: 99Price: 399Accelerate computervision workloadsAccelerate 8-bitquantized modelsAccelerate any GPUworkloadsAccelerate any GPUworkloads
METHODOLOGYTo ensure a fair comparison across hardware platforms, we run the same model on all platforms andsubject it to the same inference workload.WorkloadsPlatforms MobileNet V2 (Image classification) AWS p3.2xlarge (Server-class) Inception V4 (Image classification) Nvidia Tesla V100 GPU (Server-class) SSD MobileNet V1 (Object detection) Raspberry Pi 3 B (Edge-class) SSD MobileNet V2 (Object detection) Intel NCS 2 (Accelerator) cnn-trad-fpool3 (Keyword spotting) Google EdgeTPU (Accelerator) Nvidia Jetson Nano/TX2 (Accelerator)
PERFORMANCE AND ENERGY MICROBENCHMARKSEdge Accelerators can achieve cloudCPU level throughput. Some of thencan even outperform cloud CPUEdge accelerators exhibit very lowpower consumption compare to cloudCPU and cloud GPU, which consume131.26W and 111.66W respectively.
PERFORMANCE AND ENERGY MICROBENCHMARKSEdge accelerators have lowernormalized power consumption thancloud CPU.Edge accelerators have 10-100Xhigher throughput per dollar thancloud CPU and GPU.
SPLIT PROCESSING ACROSS APPLICATION TIERSModel splittingHow should IoT applicationsexploit distributed and splitprocessing capabilities offeredat various tiers?Model compression
MODEL SPLITTINGMobileNet V2Splitting the model at layer 10yields nearly 8x network savingover using lossless compressionfor a not-split model.Inception V4We cannot achieve any networkbandwidth saving withoutsplitting at the last 4 layers in thiscase.
MODEL COMPRESSIONModel compression yielddifferent level of networkbandwidth saving depending onthe thresholdModel compression can alsoimprove inference latency whenthe threshold is small
MODEL COMPRESSION – SKEWED WORKLOADConsider a scenario where the inputs are not random but skewed towards the common case (e.g. surveillancecamera). The compressed model is well-trained for frequently occurring inputs.3x – 4x latency reductionMore latency reduction when networklatency is high
CONCURRENCY AND MULTI-TENANCY For VPN, Nano and TX2, the degree ofconcurrency is bounded by device memory For Nano and TX2, memory are sharedbetween host RAM and GPU. More RAMused by host process, less memory can beallocated by GPU For EdgeTPU, the degree of concurrency isunbounded as it automatically performsmodel swapping on-demand. However, thisalso result in switch overhead at run time ifmultiple models are loaded
CONCLUSIONS1. Edge accelerators show promising performance§ Higher throughput per watt§ Higher throughput per dollar2. Spiting processing paradigm with specialized edge accelerators can achieveconsiderable benefit§ Model splitting for bandwidth saving and running large model§ Model compression for both bandwidth saving and latency deduction3. The degree of concurrency depends on the device memory, model size,framework software overheads, and system optimizations.
Thank you!!UNIVERSITY OF MASSACHUSETTS AMHERST 18
AI on the Edge: Characterizing AI-based IoT Applications using Specialized Edge Architectures Prashant Shenoy shenoy@cs.umass.edu David Irwin irwin@ecs.umass.edu. EDGE Computing infrastructure that is positioned between endpoint device and cloud. IoT Devices Edge Cloud. EDGE-BASED AI WORKLOADS
May 02, 2018 · D. Program Evaluation ͟The organization has provided a description of the framework for how each program will be evaluated. The framework should include all the elements below: ͟The evaluation methods are cost-effective for the organization ͟Quantitative and qualitative data is being collected (at Basics tier, data collection must have begun)
Silat is a combative art of self-defense and survival rooted from Matay archipelago. It was traced at thé early of Langkasuka Kingdom (2nd century CE) till thé reign of Melaka (Malaysia) Sultanate era (13th century). Silat has now evolved to become part of social culture and tradition with thé appearance of a fine physical and spiritual .
On an exceptional basis, Member States may request UNESCO to provide thé candidates with access to thé platform so they can complète thé form by themselves. Thèse requests must be addressed to esd rize unesco. or by 15 A ril 2021 UNESCO will provide thé nomineewith accessto thé platform via their émail address.
̶The leading indicator of employee engagement is based on the quality of the relationship between employee and supervisor Empower your managers! ̶Help them understand the impact on the organization ̶Share important changes, plan options, tasks, and deadlines ̶Provide key messages and talking points ̶Prepare them to answer employee questions
Dr. Sunita Bharatwal** Dr. Pawan Garga*** Abstract Customer satisfaction is derived from thè functionalities and values, a product or Service can provide. The current study aims to segregate thè dimensions of ordine Service quality and gather insights on its impact on web shopping. The trends of purchases have
Chính Văn.- Còn đức Thế tôn thì tuệ giác cực kỳ trong sạch 8: hiện hành bất nhị 9, đạt đến vô tướng 10, đứng vào chỗ đứng của các đức Thế tôn 11, thể hiện tính bình đẳng của các Ngài, đến chỗ không còn chướng ngại 12, giáo pháp không thể khuynh đảo, tâm thức không bị cản trở, cái được
Le genou de Lucy. Odile Jacob. 1999. Coppens Y. Pré-textes. L’homme préhistorique en morceaux. Eds Odile Jacob. 2011. Costentin J., Delaveau P. Café, thé, chocolat, les bons effets sur le cerveau et pour le corps. Editions Odile Jacob. 2010. Crawford M., Marsh D. The driving force : food in human evolution and the future.
The level of management responsible for developing strategic goals is: A. line B. supervisor C. functional D. senior . D. other organisations that do business with the 14 Where a stakeholder is identified as having high interest and low power, an organisation should: keep them satisfied monitor their interests and power C. manage them closely keep them informed 15. Highfield Aeet 12 .
