OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS
OPTIMIZING BIG DATA ANALYTICSON HETEROGENEOUS PROCESSORSMAYANK DAGA, MAURICIO BRETERNITZ, JUNLI GUAMD RESEARCH
HETEROGENEOUS PROCESSORS - EVERYWHERESMARTPHONES TO SUPER-COMPUTERSSuper computerDense ServerTabletPhoneWorkstationNotebookFrom Phil Rogers APU13 Keynote2 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
35 YEARS OF MICROPROCESSOR TREND DATAHomogeneous processors3 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
IMPORTANT SHIFTS2004 - 20052007 - 20082010 - 2011GPUCPUThe Era of Heterogeneous Computing Is Here !4 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
CENTRAL PROCESSING UNIT (CPU) Few BIG cores Ideal for scalar & control-intensive parts of application5 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
GRAPHICS PROCESSING UNIT (GPU) Lots of small cores Resides over the PCIe bus Ideal for data-parallel parts of the application6 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
ACCELERATED PROCESSING UNIT (APU)Data-parallelGPUDualx86 CPUModuleSerial A heterogeneous platform with CPU GPU on the same silicon die Ideal for both serial and data-parallel parts of application TDP of less than 100 Watts7 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
HOW IS AN APU DIFFERENTSys. Mem300-500 GB/sCPUPCIe16 GB/sGPU Manage data-movement across PCIe Use local scratchpad memory (cache)LDSSys. Mem No data-movement overhead Programming is similar to discrete GPUbut simplerAccelerated Processing Unit (APU)8 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
MODERN WORKLOADS ARE HETEROGENEOUSSCALAR CONTENT WITH A GROWING MIX OF PARALLEL CONTENT Video is expected to represent two thirds of mobile data traffic by 2017‒ Video processing is inherently parallel and can be accelerated Big data growing exponentially with exabytes of data crawled monthly‒ Map reduce is a heterogeneous workload Rapid growth of Sensor Networks‒ Drives exponential increase in data Internet of Things (IoT) results in explosion of data sources‒ Another exponential growth in data at local and cloud level9 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
HETEROGENEOUS SYSTEM ARCHITECTURE (HSA)CPU1CPUCPU 2NCU1CU2CU3 CUM-1CUMUnified Coherent MemoryAll processors use same memory addressesPower efficientFull access to virtual and physical memoryEasy to program10 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
LARGE-SCALE DATA ANALYTICS & HETEROGENEOUSCOMPUTE (HC)Large amounts of data stored in cloudHC makes the cloud energy-efficient47% GPUNeeds to be analyzed quicklyGPU provides the computationalhorsepowerDifferent types of structuredand unstructured dataA heterogeneous platform for theheterogeneous data11 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
ProgrammingHeterogeneousProcessors
PROGRAMMING LANGUAGES PROLIFERATING ON APUOpenCL AppOpenMPAppC RuntimesVariousRuntimesHSAILHSAHelperLibrariesHSA CoreRuntimeHSAFinalizerHSA KernelDriver13 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
OpenCLPerformanceC AMPProductivity14 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
WHAT IS OPENCLOpenArchitecture OPENCL ComputingLanguage– Platform Model‒ Open standard managed by the Khronos Group Platform Agnostic --- CPUs, GPUs, FPGAs, DSPsHostCompute DeviceProcessingElementCompute Unit15 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
OPENCL: EXECUTION MODEL Host Program‒ Executes on the host (usually a CPU)‒ Sends commands to the compute devices using a queue Kernel‒ Basic unit of executable code which runs on compute devices‒ A grid of parallel threads execute the kernel on the compute deviceGroup of threads executingon the same GPU coreAMDIndividual threads16 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
CODE EXAMPLEfor loop {// do work;}1. OpenCL Initialization2. Allocate memory3. Data copy GPU4. Launch GPU Kernel5. Data copy HostHost-side codekernel {// do work;}Device-side code17 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
GETTING STARTED RESOURCES (OPENCL) AMD APP Programming SDK‒ e/amd-accelerated-parallelprocessing-app-sdk/ AMD APP Programming Guide‒ dia/2013/12/AMD OpenCL Programming User Guide2.pdf‒ dia/2013/12/AMD OpenCL Programming Optimization Guide2.pdf Works for both Windows and Linux18 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
C AMP C , not C Mainstream: programmed by millions Minimal: just one language extension Portable: mix and match hardware from any vendor General and Future Proof: designed to cover full-range of heterogeneity19 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
CODE EXAMPLEfor loop {parallel for each(num threads,[ ] (t idx) {// do work;// do work;}} ); Combination of library and extensions to C standard Single-source Substantially boosts programmer productivity-No asynchronous data-transfers20 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
GETTING STARTED RESOURCES (C AMP) Compiler and runtime‒ ng/wiki/Home‒ Microsoft Visual Studio Programming Guide‒ aspx Works for both Windows and Linux21 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
PerformancePERFORMANCE2xApp1App2App3OpenCLApp4Geo MeanC AMPDaga et al. IISWC 201522 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
o MeanC AMPDaga et al. IISWC 201523 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
SUPPORT FOR POPULAR LIBRARIES Computer Vision‒ OpenCV Data Science‒ SciPy‒ NumPy Image Processing‒ ImageMagick Parallel Standard Template Library‒ Bolt Linear Algebra Library‒ AMD clBLAS24 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
OUR THREE-PRONGED APPROACH TO DATA ANALYTICS Enhancing Programming Model‒ HadoopCL and Apache Spark: flexibility, reliability, and programmability of Hadoopaccelerated by OpenCL Enhancing Data Operations‒ Deep Neural Networks: achieved 2x energy efficiency on the APU than discrete GPUs‒ Breadth-first Search: fastest single-GPU Graph500 implementation (June 2014)‒ SpMV: state-of-the-art CSR-based SpMV (13x faster than prior CSR-SpMV and 2x fasterthan other storage formats Enhancing Data Organization‒ In-Memory B Trees: efficient memory reorganization to achieve 3x speedup on theAPU over a multicore implementation25 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
NESTED PROCESSING OFMACHINE LEARNINGMAP-REDUCE, BIG DATAON APUSMAURICIO BRETERNITZ, PH.D.AMD RESEARCHTECHNOLOGY & ENGINEERING GROUPAMDthanks: Max Grossman, Vivek Sarkar
MapReduce in 30 seconds: Estimating PI1- pick random points in unit rectangle2- count fraction inside circleArea: π/ 4Map-Reduce:Map: random point inside? Issue k 1, v 1 else k 0,v 1Reduce: count 0 keys and count 1 keysProgrammer: writes { map, reduce } methods, system doesrest27 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
HADOOP Open source implementation of MapReduceprogramming modelbarrierV1MapMapMapV2V1V2ReduceV3ReduceV3 Runs on distributed network in several Java VMs Distributed file system, reliability guarantees,speculative execution, Java programming language andlibraries, implicit parallelism28 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
TARGET ARCHITECTURE Target: CLUSTER of APUs Two-Level Parallelism:‒ Across nodes in cluster‒ Within Node (APU)‒ Multicore (CPU)‒ Data parallel(GPU)29 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
NESTED PROCESSING APPROACHPARTITIONWORKLOCALLYCOMBINERESULTSnest: CPU GPU30 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
WHY APUS – MAP REDUCE-REDUCESPLIT: enough work for one GPUNO MEMORY LIMITCPU GPU executionNetworkbarrierV1MapMapMapV2V1 V2ReduceV1 V2ReduceV3ReduceV3ReduceV1 V2ReduceV1 apV2V1 V2ReduceV1 V2ReduceV3ReduceV3ReduceV3CPU GPU execution on each nodeFinal reduction in clusterAggregate node’s results31 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
HADOOPCLImplemented with AMD’s APARAPI: Java methods - GPUCollaboration with Rice University:HadoopCLMapReduce on Distributed Heterogeneous Platforms ThroughSeamless Integration of Hadoop and OpenCLMax Grossman1, Mauricio Breternitz2, Vivek Sarkar11RiceUniversity, 2AMD Research2013 International Workshop on High Performance Data Intensive Computing.May 2013.M. Grossman, M. Breternitz, V. Sarkar. “HadoopCL2: Motivating the Design of a Distributed,Heterogeneous Programming System With Machine-Learning Applications.”IEEE Transactions on Parallel and Distributed Systems, 201432 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
RUNNINGHADOOPCLclass PiMapper extendsDoubleDoubleBoolIntHadoopCLMapper {public void map(double x,double y) {if(x * x y * y 0.25) {write(false, 1);} else {write(true, 1);}}} HadoopCL supports‒Java syntax & MapReduceabstractions‒Dynamic memory allocation‒A variety of data types(primitives, sparse vectors,tuples, etc.) HadoopCL does not support‒Arbitrary inputs, outputs‒Object referencesjob.waitForCompletion(true); javac.class hadoop jar Pi.jar input output33 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
HADOOPCL CLUSTER ARCHITECTURE hadoop jar Pi.jarinput outputHadoop DataNodeTaskNameNode JobTrackerDataNodeDataNodeMap orReduceHadoopCL ChildTaskTrackerHadoopCL MLDevice SchedulerHadoopCL ChildHadoopCL ChildHadoopCL ChildAPU: CPU, GPU share work34 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
HADOOPCL NODE ARCHITECTUREHadoopCL ChildBuffer RunnerTaskMap orReduceLaunchOpenCLDeviceInputBufferRetry KernelStoreOutputExecutorOutputBufferStoreHDFSEach Child JVM enclosesa data-driven pipeline ofInputAggregatorcommunication andcomputation tasksInputBufferManagerOutputBufferManagerKernel Executor handles:Auto-generation and optimization ofOpenCL kernels from JVM bytecodeTransfer of inputs, outputs to deviceAsynch launch of OpenCL kernels35 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
HCL2 EXECUTION FLOWcompileJVMruntime36 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
EVALUATION Mahout Clustering‒ Mahout provides Hadoop MapReduceimplementations of a variety of ML algorithms‒ KMeans iteratively searches for K clusters Evaluated on 1 NameNode and 3 DataNodes in an AMD APU cluster Dataset built from the ASF e-mail archives‒ 1.4GB‒ 1 iteration of searching for 64 clusters‒ Recordedoverall execution time,time spent on compute,time spent on I/O in each mapper and reducer37 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
MAHOUT EXAMPLES KMEANS‒ Finds clusters FUZZY KMEANS‒ Probabilistic “soft clusters” PAIRWISE SIMILARITY‒ Recommender pipeline NAÏVE BAYES Probabilistic classifier DIRICHLET‒ Finds document topics, cluster via probability distribution over ‘topics’38 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
EVALUATIONSpeedup on AMD A10-7300 95w APU for 5 MahoutBenchmarksSpeedup over Mahout-CPU252015Speedup over Mahout-CPU1050KmeansFuzzyCPU boundDirichletPairwiseBayesI/O bound39 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
wisebayes10.5Number of Nodes001234540 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
FUTURE/ONGOING WORK Evaluation on more Mahout applications, more data sets, more platforms‒ Xiangyu Li, Prof David Kaeli /Northeastern University :Mahout Recommenders Evaluate potential power savings In-depth analysis of effectiveness of machine learning on performance Target HSA instead of OpenCL, via Sumatra/APARAPI Various performance improvements41 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
APACHE SPARK Fast, MapReduce-like engine‒ In-memory storage abstraction for iterative/interactive queries‒ General execution graphs‒ Up to 100x faster than Hadoop MR Compatible with Hadoop’s storage APIs‒ Can access HDFS, HBase, S3, SequenceFiles, etc. Great example of ML/Systems/DB collaboration http://spark.apache.org42 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
SWATval rdd CLWrapper.cl(sc.objectFile(inputPath))val nextRdd rdd.map(.).43 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
SWATSWAT RDDAPARAPI-SWAT (Code Generation)SWAT SerializationJVMNative/OpenCLSWAT-OpenCL BridgeMemory Allocation And CachingDevice ManagementGPU 0GPU 1GPU 244 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
SWATBig wins over HadoopCL: Built on Spark. And Scala.No HadoopCL-specific data structures required for representing complex data types User-defined classes w/ restrictions, MLlib DenseVector, Mllib SparseVector, ScalaTuple2 for (key, value) pairs, PrimitivesSimpler semantics for some Spark parallel operations e.g. Spark map() forces one output per input, MapReduce allows arbitrary # ofoutputs (though Spark has flatMap())Better locality, caching of on-device data based on broadcast, RDD IDsSimplified dynamic memory allocator on GPUMore stable, nearing production-ready implementation.Max Grossman / Rice University45 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
SWATCurrent benchmarks: Fuzzy CMeans, KMeans, Neural Net, Pagerank, Connected ComponentsMajor Challenges: Architected as a third-party JAR, internal Spark state is hidden (unlikeHadoopCL)Garbage collection Allocation patterns of a SWAT program are very differentfrom those of an equivalent Spark versionCan we do better auto-scheduling than HadoopCL? Offline? Experiment withother classification algorithms based on IBM work?46 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
CONCLUSION HadoopCL offers the flexibility, reliability, and programmability ofHadoop accelerated by native, heterogeneous OpenCL threads Using HadoopCL is a tradeoff: lose parts of the Java language butgain improved performance Evaluation of KMeans with real-world data sets shows thatHadoopCL is flexible and efficient enough to improve performanceof real-world applicationsThanks: Max Grossman, max.grossman@rice.edu47 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
e/aparapi/HADOOPCL achments/4425835/hpdic.pdf?version 1&modificationDate 1366561784922&api v2HADOOP on star-hadoop-gpuHADOOPCL presentationhttps://www.youtube.com/watch?v KMpjFsOO4nw48 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
DEEP NEURAL NETWORK(DNN)ACCELERATION ON AMDACCELERATORSJUNLI GU, AMD RESEARCH
MACHINE LEARNING BIG DATA INDUSTRY TREND Why machine learning for Big Data?‒ Original human defined algorithms don’t work well for Big Data‒ Competing in machine learning to understand Big Data DNN (deep neural networks) is breaking through & leading direction‒ Large scale of image classification/recognition/search‒ Face recognition, Online recommendation, Ads‒ Documentation retrieval, Optical Character Recognition (OCR) Long list of companies looking for DNN solutionsDNN Big Data is believed to bethe evolutionary trend for apps &HPC systems.50 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
DEEP LEARNING BRINGS CHALLENGES TO SYSTEM DESIGN Typical scale of data set Image search: 1M OCR: 100M Speech: 10B, CTR: 100BBig Data inputDNN modelAnswer Projected data to grow 10X per year DNN model training time Weeks to months on GPU clusters Trained DNNs then deployed on cloudDNN compute & memory intensive, thus clusters System is the final enabler Current platform runs into bottleneck CPU clusters CPU GPU clusters Looking at dGPUs, APUs, FPGAs, ASIC, etc.51 OPTIMIZING BIG DATA ANALYTICS ON HETEROGENEOUS PROCESSORS OCTOBER 23, 2015 2015 IEEE BIG DATA SANTA CLARA, CA, USA
WHAT HAVE WE DONEBASED ON TODAY’S INDUSTRY Focused on three major industry DNN algorithms CNN: Convolutional Neural Network (image/video classification)‒ Reference Source: Univ. of Toronto CUDA version (http://github.com/bvlc/caffe)‒ AMD implementation open sourced at https://github.com/amd/OpenCL-caffe Multi-layer Perceptron (Voice Recognition)‒ Source: publications, interaction with industry experts and ISVs‒ AMD implementation in C , OpenCL Auto-encoder L-BFGS training image and document retrieval)‒ Reference Source: Stanford Univ. Matlab code(http://ai.stanford.edu
3 Reduce Reduce V 1 V 2 V 3 barrier Map Map V Map V 1 V 2 V 3 Reduce Reduce V 1 V 2 3 Map Map Reduce Map V 1 V 2 V 3 V 1 V 2 Reduce V 3 V 1 V 2 Reduce 3 Reduce Reduce V 1 V 2 V 3 V 1 V 2 Reduce 3 SPLIT: enough work for one GPU NO MEMORY LIMIT CPU GPU execution CPU GPU execution on each node Final reduction in cluster Aggregate nodes results Network
tdwi.org 5 Introduction 1 See the TDWI Best Practices Report Next Generation Data Warehouse Platforms (Q4 2009), available on tdwi.org. Introduction to Big Data Analytics Big data analytics is where advanced analytic techniques operate on big data sets. Hence, big data analytics is really about two things—big data and analytics—plus how the two have teamed up to
big data analytics" To discuss the in-depth analysis of hardware and software platforms for big data analytics The study only focused on the hardware and software platform for big data analytics. The review is centered on the impact of parameters such as scalability, data sizes, resources availability on big data analytics. However, the
India has the second largest unmet demand for AI and Big Data/Analytics, driven primarily by large service providers, GCCs and the start-up ecosystem NCR Others Hyderabad Pune Mumbai Bangalore Chennai Top Skills Talent Big Data/ Analytics 5,800 AI 1,200 Top Skills Talent Big Data/ Analytics 19,100 AI 7.400 Top Skills Talent Big Data/ Analytics .
Q) Define Big Data Analytics. What are the various types of analytics? Big Data Analytics is the process of examining big data to uncover patterns, unearth trends, and find unknown correlations and other useful information to make faster and better decisions. Few Top Analytics tools are: MS Excel, SAS, IBM SPSS Modeler, R analytics,
example, Netflix uses Big Data Analytics to prescribe favourite song/movie based on customer‟s interests, behaviour, day and time analysis. 3. Python For Big Data Analytics 3.1 . Advantages. of . Python for Big Data Analytics Python. is. the most popular language amongst Data Scientists for Data Analytics not only because of its ease in
The Rise of Big Data Options 25 Beyond Hadoop 27 With Choice Come Decisions 28 ftoc 23 October 2012; 12:36:54 v. . Gauging Success 35 Chapter 5 Big Data Sources.37 Hunting for Data 38 Setting the Goal 39 Big Data Sources Growing 40 Diving Deeper into Big Data Sources 42 A Wealth of Public Information 43 Getting Started with Big Data .
Retail. Big data use cases 4-8. Healthcare . Big data use cases 9-12. Oil and gas. Big data use cases 13-15. Telecommunications . Big data use cases 16-18. Financial services. Big data use cases 19-22. 3 Top Big Data Analytics use cases. Manufacturing Manufacturing. The digital revolution has transformed the manufacturing industry. Manufacturers
The process of analyzing big data to extract useful information and insights is usually referred to as big data analytics or big data valu e chain [6], which is considered as one of the key enabling technologies of smart cities [7, 8, 9]. However, big data complexities comprise non-trivial challenges for the processes of big data analytics [3].
