Analyzing A Social Network Using Big Data Spatial And Graph Property Graph

Analyzing a social network using Big Data Spatialand Graph Property GraphOskar van RestPrincipal Member of Technical StaffGabriela Montiel-MorenoPrincipal Member of Technical StaffCopyright 2017, Oracle and/or its affiliates. All rights reserved.

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended forinformation purposes only, and may not be incorporated into any contract. It is not acommitment to deliver any material, code, or functionality, and should not be relied uponin making purchasing decisions. The development, release, and timing of any features orfunctionality described for Oracle’s products remains at the sole discretion of Oracle.Copyright 2017, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Internal/Restricted/Highly Restricted2

Program Agenda1Introduction2Property Graph Data Model & BDSG Architecture3Oracle Big Data Spatial and Graph Core Features4Graph Analytics using PGX Graph Analytics Engine5HoL: Analyzing a social network using Property Graphs3Copyright 2017, Oracle and/or its affiliates. All rights reserved.

Graph Database DefinitionGraph database is a database that uses graph structureswith nodes, edges, and properties to represent and storedata.1BCDAFE1. http://en.wikipedia.org/wiki/Graph databaseCopyright 2017, Oracle and/or its affiliates. All rights reserved. 4

Why do we care?Graphs are intuitive and flexible Easy to navigate, easy to form a path, natural tovisualizeEnables views and queries that would beexpensive on other databasesGraphs are everywhere Road networks, power grids, biological networks Social Networks Knowledge graphs (RDF, OWL)Copyright 2017, Oracle and/or its affiliates. All rights reserved. 5

Graph Use Case Scenarios Fraud detection– Find parties in insurance data who are on both sides of multiple claims, who live near each other Internet of Things– Manage graph of interconnected devices and predict the effect of an disruptions across network Cyber Security– Find entry points and affected machines Border Control– Analyze flight histories of a suspicious passenger. Indentify his co-travelers, co-traveler’s co-travelers, Copyright 2017, Oracle and/or its affiliates. All rights reserved.

Program Agenda1Introduction2Property Graph Data Model & BDSG Architecture3Oracle Big Data Spatial and Graph Core Features4Graph Analytics using PGX Graph Analytics Engine5HoL: Analyzing a social network using Property Graphs7Copyright 2017, Oracle and/or its affiliates. All rights reserved.

Property Graph Data Modelname “lop”lang “java”weight 0.4name “marko”age 29weight 0.29createdweight 1.07weight 0.4knowsknows2– each vertex has a set of outgoing/incoming edgescreated1weight 0.5– each vertex has a unique identifier3created8 A set of vertices11– each vertex has a collection of key-value properties126name “peter”age 35name “josh”age 32410weight 1.0name “vadas”age 27created5name “ripple”lang “java”(example from -a-Property-Graph) A set of edges– each edge has a unique identifier– each edge has a head/tail vertex– each edge has a label that denotes the type ofrelationship between two vertices– each edge has a collection of key-value properties Blueprints Java APIs Implementations Neo4j, Titan, InfiniteGraph, Dex, Sail, MongoDB Copyright 2017, Oracle and/or its affiliates. All rights reserved. 8

Oracle Big Data Spatial and Graph ArchitectureGraph AnalyticsGraph Data Access Layer (DAL)Apache Blueprints & Lucene/SolrCloudPython, Perl, PHP, Ruby,Javascript, Java APIsREST/Web ServiceParallel In-Memory Graph Analytics (PGX)Java APIsScalable and Persistent Storage ManagementApache HBaseProperty GraphformatsGraphMLRDF (RDF/XML,GMLNTriples, N-Quads,Graph-SONTriG,N3,JSON)Flat FilesCSVRelationalOracle NoSQLDatabaseCopyright 2017, Oracle and/or its affiliates. All rights reserved. 9

Program Agenda1Introduction2Property Graph Data Model & BDSG Architecture3Oracle Big Data Spatial and Graph Core Features4Graph Analytics using PGX Graph Analytics Engine5HoL: Analyzing a social network using Property Graphs10Copyright 2017, Oracle and/or its affiliates. All rights reserved.

Data Access (APIs) Blueprints 2.3.0, Gremlin 2.3.0, Rexster 2.3.0 Groovy shell for accessing property graph data REST APIs (through Rexster integration) PGQL (Property Graph Query Languge)2/28/2017Copyright 2017, Oracle and/or its affiliates. All rights reserved. 11

Text Search through Apache Lucene/Solr Use text indexing to access vertices or edges– Eg. find person with given name as starting point for reachability analysis– oraclePropertyGraph.createKeyIndex(“name”, Vertex.class);– oraclePropertyGraph.getVertices(“name”, “*Obama*”, true); Based on Apache Solr/Solr Cloud– Highly scaleable through sharding and replication Uses Apache Lucene under the covers– open source text search engine library– inverted index, ranked searching, fuzzy matching Supports manual and auto indexing of Graph elementsCopyright 2017, Oracle and/or its affiliates. All rights reserved. 12

Support for Cytoscape Open Source VisualizationCopyright 2017, Oracle and/or its affiliates. All rights reserved.

Integration with Tom Sawyer Perspectivesvia property graph REST APIsCopyright 2017, Oracle and/or its affiliates. All rights reserved.

Python Interface Installation– property graph/pyopg/README Usage– cd {ORACLE HOME}/md/property graph/pyopg./pyopg.shipython notebookCopyright 2017, Oracle and/or its affiliates. All rights reserved. 15

Program Agenda1Introduction2Property Graph Data Model & BDSG Architecture3Oracle Big Data Spatial and Graph Core Features4Graph Analytics using PGX Graph Analytics Engine5HoL: Analyzing a social network using Property Graphs16Copyright 2017, Oracle and/or its affiliates. All rights reserved.

Graph Analytics workloadsComputational Graph AnalyticsConnected ComponentsModularityGraph Pattern MatchingCompute certain values onnodes and edgesConductanceShortest PathSpanning TreePagerankColoringGiven a description of apatternfriendfriendWhile (repeatedly) traversingor iterating on the graphFind every sub-graphthat matches itClustering CoefficientCentralityexfriendIn certain procedural waysTypical graph analysis systems do not support bothCopyright 2017, Oracle and/or its affiliates. All rights reserved. 17

In-Memory Analyst (PGX) PGX is the in-memory, parallel graphanalytics engine of Oracle Big DataSpatial and GraphOracle Big Data Spatial and GraphAnalyticAnalyticRequestRequest Approaches– Reads snapshot of graph data fromdatabase (or file)– Support delta-update fromtransactional changes in database– Processes analytic requests efficientlyin-memory Supports remote clients via RESTREST PGX)Bulk ReadFileCopyright 2017, Oracle and/or its affiliates. All rights reserved. DatabaseConfidential – Oracle Internal18

Computational Analytics: Built-in PackageRich set of built-in parallel graph algorithmsDetecting Components andCommunitiesTarjan’s, Kosaraju’s,Weakly ConnectedComponents, LabelPropagation (w/ variants),Soman and Narang’sSpacificationEvaluating Community Structures Conductance,ModularityClustering Coefficient(Triangle Counting)Adamic-AdarLink Prediction SALSA(Twitter’s Who-to-follow) and parallel graph mutation operationsaPagerank, Personalized Pagerank,Betwenness Centrality (w/ variants),Closeness Centrality, DegreeCentrality,Eigenvector Centrality, HITS,Random walking and sampling (w/variants)Hop-Distance (BFS)Dijkstra’s,Bi-directional Dijkstra’sBellman-Ford’sfddaLeft Set: “a,b,e”gfgbbdeehgibhCreate UndirectedGraphicicecCreate BipartiteGraphPath-FindingaThe original graphRanking and WalkingadSort-By-Degree (Renumbering)fdgbegbiOther Classics Vertex CoverMinimum hCopyright 2017, Oracle and/or its affiliates. All rights reserved. iSimplify Graph19

Pattern matching using PGQL SQL-like syntax but with graph patterndescription and property access– Interactive (real-time) analysis– Supporting aggregates, comparison, such asmax, min, order by, group by Finding a given pattern in graph–––– Proposed for standardization byOracle– Specification available on-line– Open-sourced front-end (i.e. parser)https://github.com/oracle/pgql-langFraud detectionAnomaly detectionSubgraph extraction. Recursive path queryingCopyright 2017, Oracle and/or its affiliates. All rights reserved. 20

PGQL Example query Find all instances of a given pattern/template in data graph Fast, scaleable query mechanismSELECT v3.name, v3.ageFROM ‘myGraph’WHERE(v1:Person WITH name ‘Amber’) –[:friendOf]- (v2:Person) –[:knows]- (v3:Person)query:worksAt{1831}startDate ’09/01/2015’:Person{100}name ‘Amber’age 25:friendOf{1173}:Company{777}name ‘Oracle’location ‘Redwood City’data graph‘myGraph’Query: Find all people who are knownto friends of ‘Amber’.:friendOf {2513}since ’08/01/2014’:Person{200}name ‘Paul’age 30:knows{2200}:Person{300}name ‘Heather’age lang.org/Copyright 2017, Oracle and/or its affiliates. All rights reserved. 21

In-Memory Analyst (PGX) in Apache ZeppelinCreate notebooks with paragraphs that run graph queries or graph algorithmsCopyright 2017, Oracle and/or its affiliates. All rights reserved. 22

Apache Zeppelin Integration Apache Zeppelin is a multi-purposenotebook for data analysis andvisualization similar toiPython/Jupyter Lots of language bindings andinterpreters built in - JVM based Very active developmentcommunity Easy extensibleCopyright 2017, Oracle and/or its affiliates. All rights reserved. 23

In-Memory Analyst (PGX) in Apache ZeppelinWeb BrowserNotebookHTTPSIn-MemoryAnalyst (PGX)ServerZeppelin ServerIn-MemoryAnalyst (PGX)InterpreterHTTPSCopyright 2017, Oracle and/or its affiliates. All rights reserved. 24

Program Agenda1Introduction2Property Graph Data Model & BDSG Architecture3Oracle Big Data Spatial and Graph Core Features4Graph Analytics using PGX Graph Analytics Engine5HoL: Analyzing a social network using Property Graphs25Copyright 2017, Oracle and/or its affiliates. All rights reserved.

Computational Graph Analytics Graph Pattern Matching 17 Graph Analytics workloads Pagerank Modularity Clustering Coefficient Shortest Path Connected Components Conductance Centrality . Spatial and Graph Approaches -Reads snapshot of graph data from database (or file) -Support delta-update from