se@ApachePhoenixh p://phoenix.apache.org/V5
Agenda WhatisApachePhoenix(andApacheHBase)? ADeeperLook––––ViewsandMul8- ‐TenancySecondaryIndexesJoinsQueryOp8miza8ons Roadmap Q&A
WhatisApachePhoenix(andApacheHBase)?
WhatisApachePhoenix? Arela8onaldatabaselayerforApacheHBase– Queryengine TransformsSQLqueriesintona8veHBaseAPIcalls lexecu8on– Metadatarepository TypedaccesstodatastoredinHBasetables– AJDBCdriver AnewApacheSoXwareFounda8onproject– OriginallydevelopedatSalesforce– Nowatop- ‐levelprojectattheASF– Agrowingcommunitywithmomentum
veIn- lMachineHadoopCommonJNI
WhatisApacheHBase? i8caldata
WhatisApacheHBase? Anemergingplacormforscaleoutrela8onaldatastores
TheHBaseDataModel Tablespaces ted,mul8- ‐dimensional,sortedmap”
nmentsRegionServersRegions
HowisHBaseDifferentfromaRDBMS?RDBMSHBaseData layoutRow orientedColumn orientedTransactionsMulti-row ACIDSingle row or adjacent rowgroups onlyQuery languageSQLNone (API access)JoinsYesNoIndexesOn arbitrary columnsSingle row index onlyMax data sizeTerabytesPetabytes*R/W throughput limits1000s of operations per secondMillions of operations persecond**- hroughput
SQL:InandOutOfFashion HadoopandNoSQLwithSQL ,StrataNYC2014
SQL:InandOutOfFashion Implemen8ngstructuredquerieswellishard– Systemscannotjust“runthequery”aswri en– yplanner,anop8mizer,metadata,sta8s8cs,etc. buttheresultisveryusefultonon- ‐technicalusers– mance– Addingnewalgorithms(e.g.abe a8onsfromonedatastoretoanotheraretransparent thing,InMemory”byJulianHyde,StrataNYC2014
PhoenixPutstheSQLBackinNoSQL Acompleterela8onalsystem a(DDL,DML,etc.)withaddi8onalbenefits– Readonlyviewsonexis8ngHBasedata– Dynamiccolumnsextendschemaatrun8me– hbackqueriesusingpriorversionsofmetadata thit Secondaryindexes,queryop8miza8on,sta8s8cs, anotherJDBCdatasource
PhoenixPutstheSQLBackinNoSQLSupported?CREATE / DROP / ALTER TABLEYesUPSERT / DELETEYesSELECTYesWHERE / HAVINGYesGROUP BY / ORDER BYYesLIMITYesJOINYes, with limitationsViewsYesSecondary IndexesYesTransactionsNot yet**- ‐PHOENIX- ‐400Transac8onsupporth ps://issues.apache.org/jira/browse/PHOENIX- ‐400
PhoenixPutstheSQLBackinNoSQL erthandirectHBaseAPIuseSELECT*FROMfooWHEREbar 30VersusYourBItoolprobablycan’tdoHTablet newHTable(“foo”);thisRegionScanners t.getScanner(newScan( tor(30)), ));while((Resultr s.next())! ludeerrorhandling )s.close();t.close();
PhoenixPutstheSQLBackinNoSQL erthandirectHBaseAPIuse– �� Phoenixpushesprocessingtotheserver- ‐most"byhand”APIaccessesdonotusecoprocessors Ahugedifferenceforaggrega8onqueries– Phoenixsupportsandusessecondaryindexes– smallscans,skipscans,etc.
WhoUsesApachePhoenix?Andmore .
SalesforcePhoenixUseCases En8tyhistory SystemofRecord– Highscale,secure,non- ‐transac8onalstorefornewusecases "CustomObjects"SQL- ‐likepublicfacingAPIs– Customobjectsaretheuser’sbusinessobjects– nix HBasebackendforbigdatausecases
ADeeperLook
Phoenix HBaseArchitecture
Phoenix HBaseArchitecturePhoenix spath
Phoenix HBaseArchitecturePhoenixPhoenix spath ThePhoenixJDBCdriverisinstalledontheclient
Phoenix HBaseArchitecturePhoenixPhoenix spath ThePhoenixJDBCdriverisinstalledontheclient Theapplica8onspeaksSQLtoHBase
PhoenixDataModel se TableHBaseTableHBaseColumn FamilyA TableRow Key 1Row Key 1Row Key 1Row Key 2Row Key 2Row Key 2Row Key 3Row Key 3Row Key 3Column Family AColumn Family AQualifier 1Qualifier 2Qualifier 1Qualifier 2Qualifier1Qualifier ValueColumn Family BColumn Family BColumn Family BQualifier 3Qualifier 3Qualifier se,consistent,distributed,mul - ‐dimensional,sortedmap”
PhoenixDataModelPhoenixtableHBase TableHBaseTableHBaseColumn FamilyA TableRow Key 1Row Key 1Row Key 1Row Key 2Row Key 2Row Key 2Row Key 3Row Key 3Row Key 3Column Family AColumn Family AQualifier 1Qualifier 2Qualifier 1Qualifier 2Qualifier1Qualifier ValueColumn Family BColumn Family BColumn Family BQualifier 3Qualifier 3Qualifier 3KeyValueValueValue
PhoenixDataModelPhoenixtableHBase TableHBaseTableHBaseColumn FamilyA TableRow Key 1Row Key 1Row Key 1Row Key 2Row Key 2Row Key 2Row Key 3Row Key 3Row Key 3Column Family AColumn Family AQualifier 1Qualifier 2Qualifier 1Qualifier 2Qualifier1Qualifier ValueColumnsColumn Family BColumn Family BColumn Family BQualifier 3Qualifier 3Qualifier 3KeyValueValueValue
PhoenixDataModelPhoenixtableHBase TableHBaseTableHBaseColumn FamilyA TableRow Key 1Row Key 1Row Key 1Row Key 2Row Key 2Row Key 2Row Key 3Row Key 3Row Key 3Column Family AColumn Family AQualifier 1Qualifier 2Qualifier 1Qualifier 2Qualifier1Qualifier straintKeyValueValueValueColumnsColumn Family BColumn Family BColumn Family BQualifier 3Qualifier 3Qualifier 3KeyValueValueValue
DataModel- ‐Example schemalikethis:SERVER METRICSHOSTDATERESPONSE TIMEVARCHARDATEINTEGERGC TIMECPU TIMEIO TIMEINTEGERINTEGERINTEGERRow KeyColumns
DataModel- ‐Example TheDDLcommandwouldlooklike:CREATE TABLE SERVER METRICS (HOSTDATERESPONSE TIMEVARCHARDATEINTEGER,NOT NULL,NOT NULL,GC TIMEINTEGER,CPU TIMEINTEGER,IO TIMEINTEGER,CONSTRAINT pk PRIMARY KEY (HOST, DATE))
DataModel- ‐Example AndintheHBasetablethedatawouldbelaidoutlike:SERVER METRICSHOST DATERESPONSE TIMESF113967435891234SF11396743589GC TIME8012 21234 RowKeyColumns
Views Updatableviews– eWHEREclauseareupdatable Read- ‐onlyViews– lresultinreadonlyviews– Na8veHBasetablescanbemappedwithread- ‐onlyviews Mul8- ‐TenantViews– Tenant- ‐specificviewsmayonlybecreatedusingtenant- ‐specificconnec8ons(moreonthislater)
Views supported(yet) dun8lallviewsaredropped Crea8ngindexesoverviewsissupported Anyindexdataforaviewwillbedeletedifitisdropped
MappingExis8ngHBaseTables �� CreateaPhoenixtableusingCREATETABLE– OrcreateaviewusingCREATEVIEW– columnnames tabletoenforceprimarykeyconstraints Phoenixtypeserializa8ons– Seeh p://phoenix.apache.org/language/datatypes.html
DynamicColumns Extendschemaduringquery whiletheremaindercanbeop8onallysurfacedatquery8me CREATE TABLE “t” (K“f1”.”col1”VARCHARVARCHAR);SELECT * FROM “t” (“f1”.”col2” VARCHAR);PRIMARY KEY,
Mul8- ‐Tenancy Phoenixcanprovidemul8- ‐tenantisola8onviaacombina8onofmul8- ‐tenanttablesandtenant- ‐specificconnec8ons– Tenant- hetenant– Tenantscancreatetenant- ‐specificviewsandaddtheirowncolumns Mul8- ‐TenantTables– DeclaretheseusingtheMULTI TENANT true DDLproperty Tenant- ‐SpecificConnec8ons– antIdpropertyintheJDBCconnec8onstring– Atenant- ‐specificconnec8onmayonlyquery: Theirowndatainmul8- ‐tenanttables Alldatainnon- ‐mul8- ‐tenant(global)tables Theirownschema(tenant- ‐specificviews)
Mul8- ‐TenancyCREATE TABLE event (tenant id VARCHAR,FirstPKcolumniden8fiestenantIDtype CHAR(1),Completedevent id BIGINT,created date DATE,created by VARCHAR,CONSTRAINT pk PRIMARY KEY (tenant id, type, event id))MULTI TENANT true;Tenant- :phoenix:localhost;tenantId me”);
SecondaryIndexes DDLprovidesseveralindextypes Candefinecoveredcolumns Guarantees– edtoallinterestedindexesandtheprimarytable– Updatesarefirstmadetotheindextables table Failurehandling– ocessedaspartoflogrecovery– ecomesunavailable,un8litcomesback
SecondaryIndexes Mutableindexes– Globalmutableindexes e Forreadheavy,lowwriteusescases– Localmutableindexes ers ionloca8onofindexdatacannotbepredetermined) Forwriteheavy,spaceconstrainedusecases Immutableindexes– Manageden8relybytheclient(writesscalemore)– Contractis:Oncewri enprimaryrowsareneverupdated– Forusecasesthatarewriteonce,appendonly
SecondaryIndexes (Mutable)indexupdateflow
SecondaryIndexes Crea8ngaglobalindexwithcoveredcolumnsCREATE TABLE t (k VARCHAR PRIMARY KEY,v1 VARCHAR, v2 INTEGER);CREATE INDEX i ON t (v1) INCLUDE (v2);Coveredcolumn– Dataincoveredcolumnswillbecopiedintotheindex– queryarecontainedbyit
Joins StandardJOINsyntaxsupported,withlimita8ons
Joins JOINlimita8ons– FULLOUTERJOINandCROSSJOINarenotsupported– Onlyequality( )issupportedinjoiningcondi8ons Norestric8ononotherpredicatesintheONclause Enhancementsinlatestreleases– Derivedtablesaresupportedasof4.1– Sub- anonlybeanamedtable– Semi- ‐andan8- ywithequality( )constraints
Joins Onlyhashjoinphysicalplansareavailable– lserversandheldinmemoryduringqueryexecu8on ngjoinqueriesifavailable Server- ‐sidecachesareusedtoholdthehashedsub- ‐queryresults– ounterInsufficientMemoryExcep8ons Shouldbeconsideredaworkinprogress
SaltedTables ithmonotonicallyincreasingrowkeys– soflexiographicallysortedrows– r fanoutN- ARCHARPRIMARYKEY, )SALT BUCKETS 32; ServersintheHBasecluster
QueryOp8miza8on Examplequeryplanfora32regiontable
QueryOp8miza8on WithasecondaryindexonRESPONSE TIME
QueryOp8miza8on dexesifavailableRewritesformul8tenanttables Sta8s8cs– Useguidepoststoincreaseintra- ‐regionparallelism ar8alaggrega8onTopNHashjoins
QueryOp8miza8on �–h n ,8les,etc.
Sta8s8cs – creasedparalleliza8onofqueriesovertargetregions– Helpsavoidregionscannerlease8meouts splits
SkipScans Theop8mizeriden8fiessub- 1scan1scan2R2R3scan3R4
SkipScans ��lterthatusesSEEK NEXT HINTtoquicklyskipthroughdataINCLUDEINCLUDEINCLUDE
Par8alAggrega8on hedatalives– GROUPBYand/oraggregatefunc8ons– RVER METRICSSERVER METRICSHOST DATEKV1 KV2KV3HOSTAGGREGATE 3SF3SF3SF3SF3SF3SF3 239674234341SF1SF2SF3342121459823Jun 2 10:10:10.234Jun 3 23:05:44.975Jun 9 08:10:32.147Jun 9 08:10:32.147Jun 1 11:18:28.456Jun 3 22:03:22.142Jun 3 22:03:22.142Jun 1 10:29:58.950Jun 2 14:55:34.104Jun 3 12:46:19.123Jun 3 12:46:19.123Jun 8 08:23:23.456Jun 1 10:31:10.234Jun 1 10:31:10.234Jun 3 10:31:10.234Jun 8 10:31:10.234Jun 1 10:31:10.234Jun 1 10:31:10.234Jun 8 10:31:10.234Jun 9 10:31:10.234Jun 3 10:31:10.234 25623523443223314256876234432314256235876234 234432 2331425623314235876234235876234432276
TopN postsifavailable) TopNcoprocessorholdsontotopNrowsbychunk ClientdoesafinalmergesortR1scan1scan2R2R3scan3R4
HashJoins SeparatequeryintoLHSandRHS HSRS1CompletedRS2
HashJoins n3R3scan4R4RHSCompletedRHS
Roadmap Transac8ons,probablyviaTephra* Many- ‐to- ‐manyjoins Cost- ‐basedqueryop8mizer– Enhancedsta8s8cscollec8on,histograms– ApacheCalciteintegra8on Queryserver– LikeApacheHive’sHiveServer2– ver- ‐sideresources OLAPextensions– WINDOW,PARTITIONOVER,RANK,andotherSQL- ‐92extensions*- ‐h p://blog.cask.co/2014/07/meet- ‐tephra- ‐an- ‐open- ‐source- ‐transac8on- ‐engine- ‐2/
Roadmap versioningSecurity– GRANTandREVOKEusingtheHBaseAccessController– Percelllabelsandvisibilityexpressions– Transparentencryp8on
End(Ques8ons?)
Oct 04, 2014 · How’is’HBase’Differentfrom’aRDBMS?’ RDBMS HBase Data layout Row oriented Column oriented Transactions Multi-row ACID Single row or adjacent row groups only Query language SQL None (API access) Joins Yes No Indexes On arbitrary columns Single row index only Max data size Terabytes Petaby
Getting Started with the Cloud . Apache Bigtop Apache Kudu Apache Spark Apache Crunch Apache Lucene Apache Sqoop Apache Druid Apache Mahout Apache Storm Apache Flink Apache NiFi Apache Tez Apache Flume Apache Oozie Apache Tika Apache Hadoop Apache ORC Apache Zeppelin
CDH: Cloudera’s Distribution Including Apache Hadoop Coordination Data Integration Fast Read/Write Access Languages / Compilers Workflow Scheduling Metadata APACHE ZOOKEEPER APACHE FLUME, APACHE SQOOP APACHE HBASE APACHE PIG, APACHE HIVE APACHE OOZIE APACHE OOZIE APACHE HIVE File System Mount UI
1.2 VICTRON PHOENIX INVERTER The coding of the Phoenix inverter model is composed as follows: For example Phoenix 12/500: '12' 12 VDC battery voltage '500' 500 W continuous electrical load The Victron Phoenix inverter is designed for 12 or 24 VDC battery input voltages and produces a sinusoïdal output voltage of 230 VAC, 50 Hz
APACHE III VS. APACHE II S COR EIN OUT OM PR DIC TON OF OL TR AUM Z D. 103 bidities, and location prior to ICU admission. The range of APACHE III score is from 0 to 299 points6. Goal: the aim of this study was to investigate the ability of APACHE II and APACHE III in predicting mortality rate of multiple trauma patients. Methods
1 October 21, 2020 Rating Affirmation Phoenix Insurance Ltd. The Phoenix Holdings Ltd. Phoenix Capital Raising (2009) Ltd. . of unit linked life insurance policies premiums, as a result of the regulatory change to unit-linked life . provides additional distribution benefits to Phoenix Insurance. The Company's
the "Licensing of Phoenix software" section of the Getting Started Guide.pdf that was downloaded with the Phoenix installer. Phoenix Product Interoperability Phoenix 8.3.4 is compatible with the Integral 21.10.1 repository, which utilizes the 21.5.1, 21.8.1, and 21.10.1 versions of the Phoenix Integral Plugin. Phoenix 8.3.4 is also compatible
various Big Data tools like Apache Hadoop, Apache Spark, Apache Flume, Apache Impala, Apache Kudu and Apache HBase needed by data scientists. In 2011, Hortonworks was founded by a group of engineers from Yahoo! Hortonworks released HDP (Hortonworks Data Platform), a competitor to CDH. In 2019, Cloudera and Hortonworks merged, and the two .
Phoenix Sky Harbor Airport Comfort Suites Phoenix Airport DoubleTree by Hilton Phoenix Tempe Embassy Suites by Hilton Phoenix Tempe Fairfield Inn & Suites Phoenix Tempe/Airport Hampton Inn & Suites Tempe/Phoenix Airport 83 385 0 4.5 92 850 1 3.4 270 30,000 19 4.2 224 10,000 12 7.6 110 540 1 4.6 104 850 2 3.3