IBM Accelerators For Big Data - TDWI

IBM SoftwareData SheetIBM acceleratorsfor big dataSpeeding the development and implementation of bigdata solutionsHighlights Analytic accelerators provide advancedanalytics for various data types Application accelerators address specificbig data use cases Included with IBM Big Data Platformcomponents IBM InfoSphere BigInsightsand IBM InfoSphere Streams, at noadditional chargeWith information streaming in from more sources than ever before—social media, outdoor sensors, cameras, videos, check-out transactionsand so on—organizations face the daunting challenge of gaining insightsfrom new data sources and types. Many business leaders know thatbig data is an important resource for creating new opportunities andbecoming more agile. However, wanting to get started and knowing howto start are two different notions.Building applications for handling big data typically requires a certainskill set: understanding numerous data sources and types, familiarity withparticular programming languages, knowledge of advanced analytics andmore. Unfortunately, companies often lack the time or money to invest inbuilding these areas of expertise. Enter IBM accelerators for big data.IBM accelerators for big data are packaged software componentsthat speed the development and implementation of specific big datasolutions. They are included with two components of the IBM BigData Platform—IBM InfoSphere BigInsights and IBM InfoSphereStreams—at no additional cost. The accelerators offer organizationsbusiness logic, data processing and UI/visualization capabilities thatcan be tailored for a given use case or industry need. By leveragingIBM experience and information management best practices, they helpeliminate the complexity of building big data applications and reduce thetime-to-value for big data deployments.IBM provides two types of accelerators for big data (see Figure 1):1. Analytic accelerators address specific data types or operations withadvanced analytics, such as text analytics and geospatial data.2. Application accelerators address specific use cases, such as loganalysis and social media insights, and include both industry-specificand cross-industry features.

IBM SoftwareData SheetSolutionsAnalytics and decision managementIBM Big Data Platform{Analytic acceleratorsVisualization and discoveryApplication developmentSystems managementAcceleratorsHadoop System TextGeospatialTime seriesData miningApplication acceleratorsStream computingData warehouseInformation integration and governance FinanceMachine dataSocial dataTelco event dataBig data infrastructureFigure 1: IBM accelerators for big data and the IBM Big Data Platform.Analytic acceleratorsmeaning. For example, the text analytics accelerator candifferentiate between a first and last name, or indicate whetheran address is for a street or apartment. The annotators arecontext-sensitive and can determine the relationship betweenterms even if they are separated by other text.The IBM portfolio of analytic accelerators for big data includescapabilities for text, geospatial and time series analysis, as wellas data mining.Text analytics: Gain broad insights with highlyaccurate analysis of textual contextThe IBM text analytics engine is a powerful informationextraction system, designed for use in big data andMapReduce parallel processing environments withexceptional performance. Developed by IBM Research,this analytics engine can identify meaning within text usingtechnology similar to that demonstrated in IBM Watson for natural language processing. It includes more than 100pre-built rules (called annotators) that understand textualCase in point: Text analyticsIn the healthcare industry, text analytics can pull informationout of patient notes taken during routine clinical rounds,understand the context and detect meaning. This leads tomore accurate records and analysis, better diagnoses andimproved patient care.2

IBM SoftwareData SheetGeospatial analytics: Do more with location dataThe geospatial accelerator calculates distances and directionsbetween points on the globe using flat earth, spherical andspheroid models. Key functions of this accelerator include: Case in point: Time series analyticsEnergy utilities depend on data to measure demand forresources, manage distribution and react to outages or otherdisruptions. Utilities can use the time series accelerator toanalyze huge volumes of environmental measurements todetect anomalies in weather patterns, or tap into constantstreams of data from smart meters to help predict energyusage based on price and temperature.Real-time processing of location dataEfficient search for entities in or around an area of interestGPS location data mapped to line—for example, roadnetwork segmentsCalculators to determine the spatial relationship betweendifferent features on the EarthData mining: Score data against predictive modelsThe IBM accelerator for data mining provides a set of IBMStreams Processing Language (SPL) operators for scoring realtime data against predictive models represented in PredictiveModel Markup Language (PMML). The PMML standard issupported by multiple analytics platforms such as IBM SPSS ,SAS and open-source R. Analytics operators provided by thisaccelerator include:Case in point: Geospatial analyticsIn an urban environment, geospatial analytics can improvethe efficiency of public transportation by enabling city bussystems to detect traffic patterns, calculate the fastestroutes and optimize directions in real time.Time series analytics: Predict the future The time series accelerator facilitates real-time predictiveanalysis of regularly generated data, including: Physiology/nature data: Audio, ECG, weather data andseismic dataSystems data: Response times and performance measuresMarket/finance data: Stocks, sales and other metricsSensor data: Smart meters, temperature sensors, pressuresensors and water salinity sensors Classification: Decision trees, Naïve Bayes andlogistic regressionClustering: Demographic clustering and Kohonen clusteringRegression: Linear regression, polynomial regression andtransform regressionAssociations: Association rulesWith this accelerator, operators can take a PMML filedescribing a predictive model as an input stream and updatethe model dynamically for rapid insight. For seamlessintegration, this accelerator embeds libraries directly fromIBM InfoSphere Warehouse.The time series accelerator also includes digital filteringcapabilities to remove noise or unwanted artifacts, smoothout data and decompose in low-varying and high-varyingpatterns. It can perform pattern and correlation analysis toidentify patterns in the data and expose correlations in variousdata streams, as well as decomposition to estimate spectralcomponents of the data stream and transform and project theminto new data representations. To help “predict the future,” theaccelerator includes forecasting capabilities to detect generaltrends and estimate the value of future data streams before theyarrive (also known as look-ahead stream processing).Case in point: Data miningA bank can use the data mining accelerator to detect fraudin real time, based on analysis of historic records thatinclude fraudulent purchases. With the insights gleaned,banks can greatly improve their decision making andanalysis of customer purchase patterns over time for betterretention and relationships.3

IBM SoftwareData SheetApplication acceleratorsMachine data analytics: Create new possibilities foroperational data applicationsThe IBM portfolio of application accelerators for big dataincludes capabilities for finance, machine data, social data andtelecommunications (telco) event data analytics.The IBM accelerator for machine data analytics ingests andprocesses large volumes of machine data to provide in-depthbusiness insights. Machine data comprises information that wasautomatically created by a computer process, application or otherdevice; sources include machines ranging from IT equipmentand sensors to meters and network devices. The acceleratorprovides a range of data-intensive capabilities, including:Finance analytics: Incorporate critical real-timeinsights for financial marketsThe finance analytics accelerator provides automated optionsand equity trading analytics including: Real-time market data ingestion and managementReal-time decision support for equities, derivatives,commodity and foreign exchange (Forex) tradingAbility to incorporate additional contextual awareness (news,weather and so on) into trading decisionsReal-time, cross-asset pricing and real-time, continuousenterprise risk-level monitoring and liquidity managementacross trading desks and geographiesContinuous real-time trade monitoring to identifyfraudulent tradingSPL building blocks to develop InfoSphere Streams–basedfinancial applicationsOut-of-the-box adapters and operators that saveorganizations time by implementing functions that don’tneed to be built from scratch:–– Out-of-the-box adapters: Financial Information Exchange (FIX) IBM WebSphere Front Office (WFO) Low-latency messaging for InfiniBand support–– Operators: European style for options (11 methods) American style for options (11 methods) Greeks (for example, Delta and Theta) Data ingestion, metadata validation and writing to ApacheHadoop Distributed File System (HDFS)Data parsing and extraction, available out of the box and asextendable rulesData indexing for complete re-indexing (ingest new batches)or batch-incremental indexing updates (update alreadyindexed batches) using InfoSphere BigIndex, a component ofInfoSphere BigInsights that delivers low-latency, full-textsearch capabilities for big data, with user-configurable fieldsand fact hierarchiesConfigurable faceted search leveraging Lucene wildcardsyntax support for text searchData transformation to link records and create sessions basedon time windows, a set of “seed” records and transitive linkageStatistical modeling to model patterns and performcorrelation analysisCase in point: Machine data analyticsIn the energy and utilities industry, data streams in from the field24x7—distribution grid monitors, sensors on drilling platforms,pipeline gauges, local and regional transmission stationsand so on. The IBM accelerator for machine data analyticscan ingest data from multiple equipment sources; analyze,index and parse it; and turn it into accessible insights thathelp companies manage peak demand, address constrainedresources and improve system reliability and efficiency.Case in point: Finance analyticsA financial services company can take the out-of-the boxcapabilities of the finance analytics accelerator, includingsample trading options algorithms, and plug them into itsown decision-making algorithms. This helps facilitateautomated options trading.4