Apache’Phoenix’

2y ago
14 Views
3 Downloads
2.83 MB
58 Pages
Last View : 22d ago
Last Download : 2m ago
Upload by : Rosemary Rios
Transcription

se@ApachePhoenixh p://phoenix.apache.org/V5

Agenda WhatisApachePhoenix(andApacheHBase)? ADeeperLook––––ViewsandMul8- ‐TenancySecondaryIndexesJoinsQueryOp8miza8ons Roadmap Q&A

WhatisApachePhoenix(andApacheHBase)?

WhatisApachePhoenix? Arela8onaldatabaselayerforApacheHBase– Queryengine TransformsSQLqueriesintona8veHBaseAPIcalls lexecu8on– Metadatarepository TypedaccesstodatastoredinHBasetables– AJDBCdriver AnewApacheSoXwareFounda8onproject– OriginallydevelopedatSalesforce– Nowatop- ‐levelprojectattheASF– Agrowingcommunitywithmomentum

veIn- lMachineHadoopCommonJNI

WhatisApacheHBase? i8caldata

WhatisApacheHBase? Anemergingplacormforscaleoutrela8onaldatastores

TheHBaseDataModel Tablespaces ted,mul8- ‐dimensional,sortedmap”

nmentsRegionServersRegions

HowisHBaseDifferentfromaRDBMS?RDBMSHBaseData layoutRow orientedColumn orientedTransactionsMulti-row ACIDSingle row or adjacent rowgroups onlyQuery languageSQLNone (API access)JoinsYesNoIndexesOn arbitrary columnsSingle row index onlyMax data sizeTerabytesPetabytes*R/W throughput limits1000s of operations per secondMillions of operations persecond**- hroughput

SQL:InandOutOfFashion HadoopandNoSQLwithSQL ,StrataNYC2014

SQL:InandOutOfFashion Implemen8ngstructuredquerieswellishard– Systemscannotjust“runthequery”aswri en– yplanner,anop8mizer,metadata,sta8s8cs,etc. buttheresultisveryusefultonon- ‐technicalusers– mance– Addingnewalgorithms(e.g.abe a8onsfromonedatastoretoanotheraretransparent thing,InMemory”byJulianHyde,StrataNYC2014

PhoenixPutstheSQLBackinNoSQL Acompleterela8onalsystem a(DDL,DML,etc.)withaddi8onalbenefits– Readonlyviewsonexis8ngHBasedata– Dynamiccolumnsextendschemaatrun8me– hbackqueriesusingpriorversionsofmetadata thit Secondaryindexes,queryop8miza8on,sta8s8cs, anotherJDBCdatasource

PhoenixPutstheSQLBackinNoSQLSupported?CREATE / DROP / ALTER TABLEYesUPSERT / DELETEYesSELECTYesWHERE / HAVINGYesGROUP BY / ORDER BYYesLIMITYesJOINYes, with limitationsViewsYesSecondary IndexesYesTransactionsNot yet**- ‐PHOENIX- ‐400Transac8onsupporth ps://issues.apache.org/jira/browse/PHOENIX- ‐400

PhoenixPutstheSQLBackinNoSQL erthandirectHBaseAPIuseSELECT*FROMfooWHEREbar 30VersusYourBItoolprobablycan’tdoHTablet newHTable(“foo”);thisRegionScanners t.getScanner(newScan( tor(30)), ));while((Resultr s.next())! ludeerrorhandling )s.close();t.close();

PhoenixPutstheSQLBackinNoSQL erthandirectHBaseAPIuse– �� Phoenixpushesprocessingtotheserver- ‐most"byhand”APIaccessesdonotusecoprocessors Ahugedifferenceforaggrega8onqueries– Phoenixsupportsandusessecondaryindexes– smallscans,skipscans,etc.

WhoUsesApachePhoenix?Andmore .

SalesforcePhoenixUseCases En8tyhistory SystemofRecord– Highscale,secure,non- ‐transac8onalstorefornewusecases "CustomObjects"SQL- ‐likepublicfacingAPIs– Customobjectsaretheuser’sbusinessobjects– nix HBasebackendforbigdatausecases

ADeeperLook

Phoenix HBaseArchitecture

Phoenix HBaseArchitecturePhoenix spath

Phoenix HBaseArchitecturePhoenixPhoenix spath ThePhoenixJDBCdriverisinstalledontheclient

Phoenix HBaseArchitecturePhoenixPhoenix spath ThePhoenixJDBCdriverisinstalledontheclient Theapplica8onspeaksSQLtoHBase

PhoenixDataModel se TableHBaseTableHBaseColumn FamilyA TableRow Key 1Row Key 1Row Key 1Row Key 2Row Key 2Row Key 2Row Key 3Row Key 3Row Key 3Column Family AColumn Family AQualifier 1Qualifier 2Qualifier 1Qualifier 2Qualifier1Qualifier ValueColumn Family BColumn Family BColumn Family BQualifier 3Qualifier 3Qualifier se,consistent,distributed,mul - ‐dimensional,sortedmap”

PhoenixDataModelPhoenixtableHBase TableHBaseTableHBaseColumn FamilyA TableRow Key 1Row Key 1Row Key 1Row Key 2Row Key 2Row Key 2Row Key 3Row Key 3Row Key 3Column Family AColumn Family AQualifier 1Qualifier 2Qualifier 1Qualifier 2Qualifier1Qualifier ValueColumn Family BColumn Family BColumn Family BQualifier 3Qualifier 3Qualifier 3KeyValueValueValue

PhoenixDataModelPhoenixtableHBase TableHBaseTableHBaseColumn FamilyA TableRow Key 1Row Key 1Row Key 1Row Key 2Row Key 2Row Key 2Row Key 3Row Key 3Row Key 3Column Family AColumn Family AQualifier 1Qualifier 2Qualifier 1Qualifier 2Qualifier1Qualifier ValueColumnsColumn Family BColumn Family BColumn Family BQualifier 3Qualifier 3Qualifier 3KeyValueValueValue

PhoenixDataModelPhoenixtableHBase TableHBaseTableHBaseColumn FamilyA TableRow Key 1Row Key 1Row Key 1Row Key 2Row Key 2Row Key 2Row Key 3Row Key 3Row Key 3Column Family AColumn Family AQualifier 1Qualifier 2Qualifier 1Qualifier 2Qualifier1Qualifier straintKeyValueValueValueColumnsColumn Family BColumn Family BColumn Family BQualifier 3Qualifier 3Qualifier 3KeyValueValueValue

DataModel- ‐Example schemalikethis:SERVER METRICSHOSTDATERESPONSE TIMEVARCHARDATEINTEGERGC TIMECPU TIMEIO TIMEINTEGERINTEGERINTEGERRow KeyColumns

DataModel- ‐Example TheDDLcommandwouldlooklike:CREATE TABLE SERVER METRICS (HOSTDATERESPONSE TIMEVARCHARDATEINTEGER,NOT NULL,NOT NULL,GC TIMEINTEGER,CPU TIMEINTEGER,IO TIMEINTEGER,CONSTRAINT pk PRIMARY KEY (HOST, DATE))

DataModel- ‐Example AndintheHBasetablethedatawouldbelaidoutlike:SERVER METRICSHOST DATERESPONSE TIMESF113967435891234SF11396743589GC TIME8012 21234 RowKeyColumns

Views Updatableviews– eWHEREclauseareupdatable Read- ‐onlyViews– lresultinreadonlyviews– Na8veHBasetablescanbemappedwithread- ‐onlyviews Mul8- ‐TenantViews– Tenant- ‐specificviewsmayonlybecreatedusingtenant- ‐specificconnec8ons(moreonthislater)

Views supported(yet) dun8lallviewsaredropped Crea8ngindexesoverviewsissupported Anyindexdataforaviewwillbedeletedifitisdropped

MappingExis8ngHBaseTables �� CreateaPhoenixtableusingCREATETABLE– OrcreateaviewusingCREATEVIEW– columnnames tabletoenforceprimarykeyconstraints Phoenixtypeserializa8ons– Seeh p://phoenix.apache.org/language/datatypes.html

DynamicColumns Extendschemaduringquery whiletheremaindercanbeop8onallysurfacedatquery8me CREATE TABLE “t” (K“f1”.”col1”VARCHARVARCHAR);SELECT * FROM “t” (“f1”.”col2” VARCHAR);PRIMARY KEY,

Mul8- ‐Tenancy Phoenixcanprovidemul8- ‐tenantisola8onviaacombina8onofmul8- ‐tenanttablesandtenant- ‐specificconnec8ons– Tenant- hetenant– Tenantscancreatetenant- ‐specificviewsandaddtheirowncolumns Mul8- ‐TenantTables– DeclaretheseusingtheMULTI TENANT true DDLproperty Tenant- ‐SpecificConnec8ons– antIdpropertyintheJDBCconnec8onstring– Atenant- ‐specificconnec8onmayonlyquery: Theirowndatainmul8- ‐tenanttables Alldatainnon- ‐mul8- ‐tenant(global)tables Theirownschema(tenant- ‐specificviews)

Mul8- ‐TenancyCREATE TABLE event (tenant id VARCHAR,FirstPKcolumniden8fiestenantIDtype CHAR(1),Completedevent id BIGINT,created date DATE,created by VARCHAR,CONSTRAINT pk PRIMARY KEY (tenant id, type, event id))MULTI TENANT true;Tenant- :phoenix:localhost;tenantId me”);

SecondaryIndexes DDLprovidesseveralindextypes Candefinecoveredcolumns Guarantees– edtoallinterestedindexesandtheprimarytable– Updatesarefirstmadetotheindextables table Failurehandling– ocessedaspartoflogrecovery– ecomesunavailable,un8litcomesback

SecondaryIndexes Mutableindexes– Globalmutableindexes e Forreadheavy,lowwriteusescases– Localmutableindexes ers ionloca8onofindexdatacannotbepredetermined) Forwriteheavy,spaceconstrainedusecases Immutableindexes– Manageden8relybytheclient(writesscalemore)– Contractis:Oncewri enprimaryrowsareneverupdated– Forusecasesthatarewriteonce,appendonly

SecondaryIndexes (Mutable)indexupdateflow

SecondaryIndexes Crea8ngaglobalindexwithcoveredcolumnsCREATE TABLE t (k VARCHAR PRIMARY KEY,v1 VARCHAR, v2 INTEGER);CREATE INDEX i ON t (v1) INCLUDE (v2);Coveredcolumn– Dataincoveredcolumnswillbecopiedintotheindex– queryarecontainedbyit

Joins StandardJOINsyntaxsupported,withlimita8ons

Joins JOINlimita8ons– FULLOUTERJOINandCROSSJOINarenotsupported– Onlyequality( )issupportedinjoiningcondi8ons Norestric8ononotherpredicatesintheONclause Enhancementsinlatestreleases– Derivedtablesaresupportedasof4.1– Sub- anonlybeanamedtable– Semi- ‐andan8- ywithequality( )constraints

Joins Onlyhashjoinphysicalplansareavailable– lserversandheldinmemoryduringqueryexecu8on ngjoinqueriesifavailable Server- ‐sidecachesareusedtoholdthehashedsub- ‐queryresults– ounterInsufficientMemoryExcep8ons Shouldbeconsideredaworkinprogress

SaltedTables ithmonotonicallyincreasingrowkeys– soflexiographicallysortedrows– r fanoutN- ARCHARPRIMARYKEY, )SALT BUCKETS 32; ServersintheHBasecluster

QueryOp8miza8on Examplequeryplanfora32regiontable

QueryOp8miza8on WithasecondaryindexonRESPONSE TIME

QueryOp8miza8on dexesifavailableRewritesformul8tenanttables Sta8s8cs– Useguidepoststoincreaseintra- ‐regionparallelism ar8alaggrega8onTopNHashjoins

QueryOp8miza8on �–h n ,8les,etc.

Sta8s8cs – creasedparalleliza8onofqueriesovertargetregions– Helpsavoidregionscannerlease8meouts splits

SkipScans Theop8mizeriden8fiessub- 1scan1scan2R2R3scan3R4

SkipScans ��lterthatusesSEEK NEXT HINTtoquicklyskipthroughdataINCLUDEINCLUDEINCLUDE

Par8alAggrega8on hedatalives– GROUPBYand/oraggregatefunc8ons– RVER METRICSSERVER METRICSHOST DATEKV1 KV2KV3HOSTAGGREGATE 3SF3SF3SF3SF3SF3SF3 239674234341SF1SF2SF3342121459823Jun 2 10:10:10.234Jun 3 23:05:44.975Jun 9 08:10:32.147Jun 9 08:10:32.147Jun 1 11:18:28.456Jun 3 22:03:22.142Jun 3 22:03:22.142Jun 1 10:29:58.950Jun 2 14:55:34.104Jun 3 12:46:19.123Jun 3 12:46:19.123Jun 8 08:23:23.456Jun 1 10:31:10.234Jun 1 10:31:10.234Jun 3 10:31:10.234Jun 8 10:31:10.234Jun 1 10:31:10.234Jun 1 10:31:10.234Jun 8 10:31:10.234Jun 9 10:31:10.234Jun 3 10:31:10.234 25623523443223314256876234432314256235876234 234432 2331425623314235876234235876234432276

TopN postsifavailable) TopNcoprocessorholdsontotopNrowsbychunk ClientdoesafinalmergesortR1scan1scan2R2R3scan3R4

HashJoins SeparatequeryintoLHSandRHS HSRS1CompletedRS2

HashJoins n3R3scan4R4RHSCompletedRHS

Roadmap Transac8ons,probablyviaTephra* Many- ‐to- ‐manyjoins Cost- ‐basedqueryop8mizer– Enhancedsta8s8cscollec8on,histograms– ApacheCalciteintegra8on Queryserver– LikeApacheHive’sHiveServer2– ver- ‐sideresources OLAPextensions– WINDOW,PARTITIONOVER,RANK,andotherSQL- ‐92extensions*- ‐h p://blog.cask.co/2014/07/meet- ‐tephra- ‐an- ‐open- ‐source- ‐transac8on- ‐engine- ‐2/

Roadmap versioningSecurity– GRANTandREVOKEusingtheHBaseAccessController– Percelllabelsandvisibilityexpressions– Transparentencryp8on

End(Ques8ons?)

Oct 04, 2014 · How’is’HBase’Differentfrom’aRDBMS?’ RDBMS HBase Data layout Row oriented Column oriented Transactions Multi-row ACID Single row or adjacent row groups only Query language SQL None (API access) Joins Yes No Indexes On arbitrary columns Single row index only Max data size Terabytes Petaby

Related Documents:

Getting Started with the Cloud . Apache Bigtop Apache Kudu Apache Spark Apache Crunch Apache Lucene Apache Sqoop Apache Druid Apache Mahout Apache Storm Apache Flink Apache NiFi Apache Tez Apache Flume Apache Oozie Apache Tika Apache Hadoop Apache ORC Apache Zeppelin

CDH: Cloudera’s Distribution Including Apache Hadoop Coordination Data Integration Fast Read/Write Access Languages / Compilers Workflow Scheduling Metadata APACHE ZOOKEEPER APACHE FLUME, APACHE SQOOP APACHE HBASE APACHE PIG, APACHE HIVE APACHE OOZIE APACHE OOZIE APACHE HIVE File System Mount UI

1.2 VICTRON PHOENIX INVERTER The coding of the Phoenix inverter model is composed as follows: For example Phoenix 12/500: '12' 12 VDC battery voltage '500' 500 W continuous electrical load The Victron Phoenix inverter is designed for 12 or 24 VDC battery input voltages and produces a sinusoïdal output voltage of 230 VAC, 50 Hz

APACHE III VS. APACHE II S COR EIN OUT OM PR DIC TON OF OL TR AUM Z D. 103 bidities, and location prior to ICU admission. The range of APACHE III score is from 0 to 299 points6. Goal: the aim of this study was to investigate the ability of APACHE II and APACHE III in predicting mortality rate of multiple trauma patients. Methods

1 October 21, 2020 Rating Affirmation Phoenix Insurance Ltd. The Phoenix Holdings Ltd. Phoenix Capital Raising (2009) Ltd. . of unit linked life insurance policies premiums, as a result of the regulatory change to unit-linked life . provides additional distribution benefits to Phoenix Insurance. The Company's

the "Licensing of Phoenix software" section of the Getting Started Guide.pdf that was downloaded with the Phoenix installer. Phoenix Product Interoperability Phoenix 8.3.4 is compatible with the Integral 21.10.1 repository, which utilizes the 21.5.1, 21.8.1, and 21.10.1 versions of the Phoenix Integral Plugin. Phoenix 8.3.4 is also compatible

various Big Data tools like Apache Hadoop, Apache Spark, Apache Flume, Apache Impala, Apache Kudu and Apache HBase needed by data scientists. In 2011, Hortonworks was founded by a group of engineers from Yahoo! Hortonworks released HDP (Hortonworks Data Platform), a competitor to CDH. In 2019, Cloudera and Hortonworks merged, and the two .

Phoenix Sky Harbor Airport Comfort Suites Phoenix Airport DoubleTree by Hilton Phoenix Tempe Embassy Suites by Hilton Phoenix Tempe Fairfield Inn & Suites Phoenix Tempe/Airport Hampton Inn & Suites Tempe/Phoenix Airport 83 385 0 4.5 92 850 1 3.4 270 30,000 19 4.2 224 10,000 12 7.6 110 540 1 4.6 104 850 2 3.3