Graph Databases And Analytics: How To Use Them - Oracle
Graph Databases andAnalytics: How to Use ThemMelli Annamalai and Jean IhmGraph TechnologiesOracle1Copyright 2019 Oracle and/or its affiliates.
Safe HarborThe following is intended to outline our general product direction. It is intended for information purposesonly, and may not be incorporated into any contract. It is not a commitment to deliver any material, code,or functionality, and should not be relied upon in making purchasing decisions. The development,release, timing, and pricing of any features or functionality described for Oracle’s products may changeand remains at the sole discretion of Oracle Corporation.Statements in this presentation relating to Oracle’s future plans, expectations, beliefs, intentions andprospects are “forward-looking statements” and are subject to material risks and uncertainties. A detaileddiscussion of these factors and other risks that affect our business is contained in Oracle’s Securities andExchange Commission (SEC) filings, including our most recent reports on Form 10-K and Form 10-Qunder the heading “Risk Factors.” These filings are available on the SEC’s website or on Oracle’s websiteat http://www.oracle.com/investor. All information in this presentation is current as of September 2019and Oracle undertakes no duty to update any statement in light of new information or future events.Copyright 2019 Oracle and/or its affiliates.
What is a Graph?A collection of points (vertices) andlines between those points (edges)Mellicollaborates withJeancollaborates withfeuds withJohnCopyright 2019 Oracle and/or its affiliates.
What is a Graph?A collection of points (vertices) andlines between those points (edges)Mellicollaborates withLucycollaborates withcollaborates withfeuds withfeuds withSophieCopyright 2019 Oracle and/or its affiliates.JeanJohn
A More Complex Graph:Find popular nodes in a social networkIdentify influencersCopyright 2019 Oracle and/or its affiliates.
Graph Analytics Community detection and influencer analysis Clustering Identify fraudPath analysis and reachability Product recommendationAnomaly detection Churn risk analysis/targeted marketing, HR Turnover analysisManage Bill of Materials, Outage analysis in utilities networks, vulnerability analysis inIP networksPattern matching Tax fraud detection, data extraction6
nalytics-technology-trends-123479234/Copyright 2019 Oracle and/or its affiliates.
Gartner MQ Data Management Solutions for AnalyticsCopyright 2019 Oracle and/or its
Graph Data ModelsProperty GraphModel Path Analytics Graph Analytics Detect patterns andanomalies FinancialRetail, MarketingSocial MediaSmart ManufacturingShipping for 3 yearsRDF GraphModelGraph ModelCopyright 2019 Oracle and/or its affiliates. Data federation Knowledge representation Semantic WebUse Cases Life SciencesHealth CarePublishingFinanceIndustry DomainShipping for 12 years
Customer Success StoriesCopyright 2019 Oracle and/or its affiliates.
Graph Technology Applications Oracle’s graph technology has been applied to solve many real problems forinternal/external customers
Paysafe: Money Transfer Fraud DetectionMultiple paths going to the samedestinationCopyright 2019 Oracle and/or its affiliates.Limited number of source and destinationvertices
Banco De Galicia: Which Nodes areTransferring money to other Banks?Copyright 2019 Oracle and/or its affiliates.
Circular Payments and Tax FraudTax authorityCopyright 2019 Oracle and/or its affiliates.Tax authorityTax authorityBORDER 0% VAT BORDER 0% VAT BORDER 0% VAT BORDERExportercompanyGettingVAT refundBufferResellerPaying VATImporter„fake” companyNot paying VATCompany inother EU membercountryBORDER 0% VAT BORDER 0% VAT BORDER 0% VAT BORDER Company inother EU membercountry
Find nodes that are in most shortest pathsbetween other nodesIdentify nodes that can cause vulnerabilities in a grid
Manufacturing: What is the Impact ofChanging this Part? AxleA car has 30,000 partsRodWheelDiscScrewCopyright 2019 Oracle and/or its affiliates.
Southern European Police ForceAnalyzing suspicious travel patternsDetecting potential threats earlyData sourcesIntegrated Operations Management System (SIGO,built by Accenture)Advance Passenger Information System (APIS, datafrom Non-Schengen Airport and Ports)Combining legal requests and co-travelerseg. determining „hot flights“ with at least one knowncriminal on boardLooking at passenger relationshipseg. Betweenness centrality to determine recurringtravelers17
Property GraphsCopyright 2019 Oracle and/or its affiliates.
Property Graph Product Overview Store, manage, query and analyze graphs Highly scalable in-memory analytics 10s of billions of edges and vertices50 pre-built graph analysis algorithmsDetecting components and communitiesTarjan’s, Kosaraju’s, Weakly connected components, label propagation, etc.Ranking and walkingPagerank, personalized pagerank, betweenness centrality, etc.Evaluating community structuresConductance, modularity, triangle counting, Adamic-Adar, etc.Path-findingPath distance, Dijkstra’s, Bellman-Ford’s, etc.Copyright 2019 Oracle and/or its affiliates.
Property Graph Product Overview PGQL: Powerful graph query languageSQL-like language for specifying graph patternsWorking on graph additions to the SQL standard PGQL example:PATH any edge as ()-[]-()SELECT n, m MATCH(n) -/: any edge/ - mWHERE n.name ‘Lucy’ and m.name ‘Jean’Java API for analyticsRich user interfaceNotebookShell UIGraph Visualization Enterprise capabilities – built on Oracle infrastructureManageability, fine-grained security, high availability, integration, and moreCopyright 2019 Oracle and/or its affiliates.
Architecture ApplicationProduct:Oracle DatabaseSpatial and Graph In-memory parallelgraph analytics server (PGX) Load graph into memoryfor analysis Automate graph refreshShell,ZeppelinClient LibraryVizGraph AnalyticsApplicationIn-Memory GraphGraph StoreOracle DatabaseShell,ZeppelinClient Library 21Client libraries Java API to develop applications Command-line submission of graph queries Graph visualization tool APIs to update graph storeIn-database parallelgraph traversal Run PGQL queries(converted to SQL)in the databaseVizGraph QueriesGraph StoreOracle Database
Using Graphs on aBig Data PlatformApplicationShell,ZeppelinClient LibraryVizProduct: Oracle Big Data Spatial and GraphRuns on the Hadoop platform In-memory parallel graph analytics server (PGX) Load graph into memory for analysis Automate graph refresh Client libraries Java API to develop applications Command-line submission of graph queries Graph visualization tool APIs to update graph storeGraph AnalyticsIn-Memory GraphGraph StoreBig Data: HDFS, HBase orOracle NoSQL Database22
From Tables to a GraphOrganizationsorg id namePeopletype religiongenrecountryperson name com- occup- .pany ationidcountryTVNetw5PopeFrancisPopeVaticanAma- yActorUSACBSRelationshipsrelati- from from to idon id idtypetotyperelationtype1person5person admires1person7person collaboratesCopyright 2019 Oracle and/or its rates with7Occupation:ActorName:KeeganMichael Key
Query GraphsCopyright 2019 Oracle and/or its affiliates.
PGQL Graph Query Language Graph pattern matching Basic patterns and reachability patterns (person) –[:collaborates] - (person)Can we reach from A to B with an arbitrary number of hops?Familiarity for SQL users Similar language constructs and syntaxSELECT WHERE .GROUP BY ORDER BY“Result set” (table) as outputCopyright 2019 Oracle and/or its affiliates.
PGQL ExamplesSELECT eMATCH ()-[e]- ()Copyright 2019 Oracle and/or its affiliates.
PGQLSELECT eMATCH (n)-[e]- (m)WHERE n.name ‘Barack Obama’Copyright 2019 Oracle and/or its affiliates.
PGQLPATH any edge as ()-[]-()SELECT n, m MATCH(n) -/: any edge*/ - (m)WHERE n.name 'Barack Obama'Copyright 2019 Oracle and/or its affiliates.
PGQLSELECT label(e), count(*)MATCH () -[e]- ()GROUP BY label(e) ------------------------- label(e) count(*) ------------------------- admires 28 leads 9 feuds 45 collaborates 82 ------------------------- Copyright 2019 Oracle and/or its affiliates.
PGQLSELECT n, e, mMATCH (n)–[e:collaborates]- (m)Copyright 2019 Oracle and/or its affiliates.
Analyze GraphsCopyright 2019 Oracle and/or its affiliates.
Analyzing the Marvel Graphg lity(g)g.publish(VertexProperty.ALL, EdgeProperty.ALL)Copyright 2019 Oracle and/or its affiliates.
Displaying Query Results with PagerankCopyright 2019 Oracle and/or its affiliates.
Copyright 2019 Oracle and/or its
Compute Shortest PathFind shortest path between “WARR” and “MEREE”w g.createEdgeProperty(PropertyType.DOUBLE, "weight")w.fill((Double) 1.0)src g.getVertex("WARR")dst g.getVertex("MERREE")path analyst.shortestPathDijkstra(g,src,dst,w)// Creating boolean properties to highlight vertices and edges in the path easierin path g.createEdgeProperty(PropertyType.BOOLEAN, "in path")for (PgxEdge e: path.edges) {in path.set(e, true);}v in path g.createVertexProperty(PropertyType.BOOLEAN, "v in path")for (PgxVertex v: path.vertices) {v in path.set(v, true);}// Shortest PathSELECT n, e, m, e2, m2, e3, m3, n.pagerank, m.pagerank, m2.pagerank, m3.pagerank, e.in path, e2.in path, e3.in path,n.v in path, m.v in path, m2.v in path, m3.v in pathMATCH (n)-[e]-(m), (n)-[e2]-(m2), (m2)-[e3]-(m3)WHERE e.in path AND e2 ! e AND e3 ! e AND e2 ! e3 AND NOT m2.v in path AND OUT DEGREE(m2) 10 AND NOT m3.v in pathORDER BY n.pagerank ASC, m2.pagerank ASC LIMIT 500Copyright 2019 Oracle and/or its affiliates.
Compute Shortest PathFind shortest pathbetween “WARR”and “MEREE”Copyright 2019 Oracle and/or its affiliates.
Compute Shortest PathFind shortest pathbetween “WARR”and “MEREE”Copyright 2019 Oracle and/or its affiliates.
Performance and ScaleCopyright 2019 Oracle and/or its affiliates.
LDBC benchmarkScale factor 1Number of vertices: 3,181,724Number of edges: 17,256,038Memory footprint: 1.3GBScale factor 3Number of vertices: 9,281,922Number of edges: 52,695,735Memory footprint: 4.0GBScale factor 10Number of vertices: 29,987,835Number of edges: 176,623,445Memory footprint: 13.3 GBOraclePostgres SQLSparksee
Property Graph Sizing RecommendationsCopyright 2019 Oracle and/or its affiliates.
Future PlansCopyright 2019 Oracle and/or its affiliates.
Graph Cloud ServiceCopyright 2019 Oracle and/or its affiliates.
Graph Cloud ServiceAutomated, end-to-end analyticserviceFully managed Create graphs Automated modeler creates graphsfrom database tables and file formats Explore connections Using visualization tools, notebooks,and query languages Analyze relationships With pre-built analytics, visualizationtools, and query languages
Why use Graph Cloud?Quick to DeployGraph Studio, a new paradigm for data scientists andanalysts to build graph applicationsQuick to Get StartedIndustry-specific analytic workflowsSimplified Graph Application DevelopmentAutomatic graph model creation from relationaltablesEasy to Share ResultsPublish and share analysis in notebooks or RESTendpoints
Developer and Data Scientist Friendly
Graph StudioGraph NotebookApache Zeppelin-compatibleinterpretersUser Interface built with Oracle JETConsistent look-and-feelPowerful and interactive visualizationCustom graph visualization46
Extensions to the SQLStandardCopyright 2019 Oracle and/or its affiliates.
SQL/PGQ SQL extensions to query property graphs Our team is working with ISO and ANSI committeesTarget: Next version of SQL ( 2020/21)Create a Property Graph using SQL Data DefinitionCREATE PROPERTY GRAPH myGraphVERTEX TABLES (Person, Message)EDGE TABLES (Created SOURCE Person DESTINATION Message,Commented SOURCE Person DESTINATION Message )Query a Property Graph in SQLSELECT GT.creationDate, GT.contentFROM myGraph GRAPH TABLE (MATCH(Creator IS Person WHERE Creator.email :email1)-[ IS Created ]- (M IS Message) -[ IS Commented ](Commenter IS Person WHERE Commenter.email :email2)WHERE ALL DIFFERENT (Creator, Commenter)ONE ROW PER MATCHCOLUMNS (M.creationDate,M.content )) AS GT
SQL/PGQ: Implementation PlanTransition graph support to align with proposed SQL/PGQ standardExternal graph analytics enginenot requiredQuery and analysis within databasePerformance trade-offLarge data: use existing SQL engineIn-memory columnar graph indexfor performanceExternal engine still usefulFor alternative deployment modelsBig Data audience: Spark, Hadoop,HBase Scalable distributed in-memoryexecution
What’s AheadTuesdayWednesday11:15-12:0010:00-10:45New Tools to Fight Against Financial Crime [CON6222]Moscone South – Room 152CMoscone South – Room 20412:30-1:1512:30-1:15Using Graph Analysis and Fraud Detection in the Fintech Industry (Paysafe1:30-2:15Traversing and Querying Graphs with PGQL and Gremlin with Oracle Spatial andcustomer session) Moscone South – Room 152CGraph [DEV4084] Moscone South – Room 202Blazing-Fast Distributed Graph Query Processing: 100x as Fast as SparkMeet the Experts[DEV3712] Moscone South – Room 3073:15-4:00Graph Database and Analytics: How To Use Them [TRN 4755]Introducing Oracle Graph Cloud: Automating Graph Analysis [TRN4754]Moscone South – Room 159BCopyright 2019 Oracle and/or its affiliates.1:30-2:20Graph Database and Analysis2:30-3:20Graph Cloud Service: Automating Graph AnalysisLounge C, Code One Groundbreakers Hub, Moscone South Level 1
DemogroundsDate/TimeTitleLocationMonday 10:00 am – 4:00 pmTuesday 10:30 am – 5:30 pmWednesday 10:00 am– 4:30 pmSpatial and Graph:Database, Analyticsand CloudMoscone South Exhibit Hall (‘The Exchange’) Oracle Demogrounds Data Managementarea Kiosk # ODB-017Spatial & GraphDemosCopyright 2019 Oracle and/or its affiliates.
The Spatial & Graph SIG User CommunityNow part of BIWA User GroupWe are a vibrant community ofcustomers and partners that connectsand exchanges knowledge online,and at conferences and events.Meet us at OpenWorld! Monday-WednesdayMoscone West, Level 3, User Group areaat the BIWA/Analytics Community tableJoin us raspatialsigoraclespatialsig@gmail.com
analyticsanddatasummit.orgSeeking customer use cases and technology sessionsDedicated Spatial & Graph track with 20 sessionsCopyright 2019 Oracle and/or its affiliates.
Thank YouMelli Annamalai(Melliyal.Annamalai@oracle.com)
Session SurveyHelp us make the contenteven better. Please completethe session survey in theMobile App.Copyright 2019 Oracle and/or its affiliates.
Oracle Database Spatial and Graph In-memory parallel graph analytics server (PGX) Load graph into memory for analysis . Command-line submission of graph queries Graph visualization tool APIs to update graph store : Graph Store In-Memory Graph Graph Analytics : Oracle Database Application : Shell, Zeppelin : Viz .
The totality of these behaviors is the graph schema. Drawing a graph schema . The best way to represent a graph schema is, of course, a graph. This is how the graph schema looks for the classic Tinkerpop graph. Figure 2: Example graph schema shown as a property graph . The graph schema is pretty much a property-graph.
a graph database framework. The proposed approach targets two kinds of graph models: - IFC Meta Graph (IMG) based on the IFC EXPRESS schema - IFC Objects Graph (IOG) based on STEP Physical File (SPF) format. Beside the automatic conversation of IFC models into graph database this paper aims to demonstrate the potential of using graph databases .
4 SPATIAL AND GRAPH ANALYTICS WITH ORACLE DATABASE 12C RELEASE2 Graph analytics Graph analytics are powered by the in-memory analyst (PGX) with 40 built-in, powerful, parallel, in-memory analytics, including ranking, centrality, recommendation, community detection, and path finding for social network analysis. Example property graph use cases
Graph querying is the most primitive operation for infor-mation access, retrieval, and analytics over graph data that enables applications including knowledge graph search, and cyber-network security. We consider the problem of query-ing a graph database, where the input is a data graph and a graph query, and the goal is to find the answers to the
Computational Graph Analytics Graph Pattern Matching 17 Graph Analytics workloads Pagerank Modularity Clustering Coefficient Shortest Path Connected Components Conductance Centrality . Spatial and Graph Approaches -Reads snapshot of graph data from database (or file) -Support delta-update from
Graph Algorithms: The Core of Graph Analytics Melli Annamalai and Ryota Yamanaka, Product Management, Oracle August 27, 2020. 2 AskTOM Office Hours: Graph Database and Analytics Welcome to our AskTOM Graph Office Hours series! We’re back with
of the graph database is illustrated below, courtesy of Neo4j, a leader in graph database technology. Figure 1: Graph Database Concept Graph databases hold the relationships between data as a priority. Querying relationships within a graph database is fast because the data relationships themselves are perpetually stored within the database.
Aronson, E., Wilson, T. D., Akert, R. M., & Sommers, S. R. (2016). Social psychology (9th ed.). Upper Saddle River, NJ: Pearson Education. American Psychological Association (2010). Publication manual of the American Psychological Association (6th ed.). Washington, D.C.: American Psychological Association. Course Learning Outcomes: The primary objective of the course is to provide you with a .
