Data Streaming With Kafka

Data Streamingwith KafkaNovember 11, 2020

BUSINESSTECHNICALThree initiatives to turn your Siebel into the best non-SaaS CRMTACTICALSTRATEGICALKAFKACLOUD NATIVEAdoptReal-Time Data Streamingwith KafkaModernizeyour Siebel Platform withCloud Native ArchitectureSIEBEL UXSIEBEL UXImproveyour Siebel UX withNexus Face and Nexus AppsTransformyour Siebel UX withNexus Face and Nexus Apps

KAFKAAdopt Real-TimeData Streamingwith Kafka

Siebel CRM Cloud Native Sessions: https://bit.ly/SiebelCloudNative

Kafka comes with the Siebel Cloud Native Architecture

Siebel Kafka integration: background and demohttp://bit.ly/SiebelKafka35'20'' - 45'50''

01TECHNICAL CHARACTERISTICS Delivers messages with very low latency Scales to a thousand brokers and trillions of messages per day Stores streams of data safely in fault-tolerant cluster Processes streams of events in real timeKAFKA02SIEBEL RELATED NOTES Expected in 2021; update to the latest version is required Publish and subscribe to events declaratively and using scripting Siebel Cloud Native Architecture uses Kafka internally Now Siebel can work with Kafka using CDC or customConnectors03USE CASES Audit of the customers’ data access events Speed up queries: replace read calls with data syncs Google-like search: push Siebel data into Elasticsearch Identify and act on risks and opportunities in Siebeldata updates using Streams API

Data streaming with Kafka: Real-time performance and reliabilityProducers/Source SystemsSocial mers/Target SystemsData Lakes / DWHClicks StreamsSearchSensor DataAuditMobile AppsMobile AppsMicroservicesMicroservicesSaaS AppsSaaS AppsEnterprise AppsStream Processors(Kafka Streams API,ksqlDB, Spark)Enterprise Apps

Common Kafka Usage Scenario for Siebel applicationsAudit: recorduser actionsand customers’data accesseventsQueryperformance:replace readcalls with datasyncsGoogle-likesearch: pushSiebel datainto ElasticsearchStreams API:identify and acton risks &opportunities inSiebel dataupdates

Speeding up queries with CQRS consumerqueriesqueryhandlerreadstorage

Speeding up queries in the Siebel CRM ecosystemAzureOn-premSiebel RESTAPImobile appAPI GatewaySiebel DB

Speeding up queries in the Siebel CRM ecosystemRead Performance GaincommandsSiebel RESTAPI Record count in DB, BC, CosmosDB Accounts 600k 550k 500kSiebel DB Assets: 85m 2.5m 1mOn-prem Flat by System ID – 1.2-1.7x Hierachy by System ID – 2.5-3.8xAzure Large Response Size – 5.5-13.5xWhen to applykafkamobile appAPI Gateway Pushing Siebel data into«Digital» applications Replacing Siebel VBC by datastreaming from source systemsqueriesREST APICosmosDBCurrent Kafka integrationoptions for Siebel Source connector – CDC (Debezium,GoldenGate, Striim, etc) Sync connector – REST API based

Implementing Google-like search in the Siebel UISiebel PM viaNexus BridgeSiebel ConfigSiebel DBNexus Facebuilt UIvisibilityreplicationsearchREST APIData AccessContolkafkaElasticsearch

Kafka is emerging as a middleware platform123SUPPORT OF VARIOSPATTERNSSINGLE SET OFTOOLSRELIABLE & SCALABLEINFRASTRUCTUREWhile Kafka’s coreintegration pattern is eventbased, it also supports Fireand-Forget, Publish /Subscribe, RequestResponse / RPC, Batch andother patterns.Kafka provides all requiredmiddleware components,e.g., messaging, storage,connectors, processing.How many products do youcurrently run in yourmiddleware stack?Kafka offers extreme scaleand throughput while beinghighly available. With thedecoupling of clients, itsolved the problems ofbackpressure or omparison/

Organizational implications of Kafka’s “Dumb pipes smart endpoints” approachESBTopicsEvent ProducerorSource SystemSinkConnector(if needed)SourceConnector(if needed)1st System Dev TeamESB Dev TeamEvent ConsumerorTarget System2nd System Dev TeamKAFKATopicsEvent ProducerorSource SystemSinkConnector(if needed)SourceConnector(if needed)1st System Dev TeamOperations teamEvent ConsumerorTarget System2nd System Dev Team

Stream processingPresent Kafka topics asProducers/SourceSystemsConsumers/TargetSystems Streams TablesTwo Kafka’s technologies ksqlDB (SQL like) Streaming API (Java, Scala)Capabilities Transformation of messages Filtering AggregationStream Processors(Kafka Streams API,ksqlDB, Spark) Joining topics Time windowingConsumers/TargetSystemsMain use-cases: real time action on Detected anomalies Prediction / / Analytics

Adoption of Kafka in the aming-benefits-increase-with-greater-maturity/

Your next stepsFind out if yourorganization is alreadyrunning Kafka1Watch recordingsfrom Siebel CRMVirtual Summit2Deploy and run Kafka inKubernetes cluster3Determine youruse-cases for Kafka45Adopt Event DrivenFramework and CloudNative Architecture

Siebel DB Siebel REST API mobile app API Gateway Azure On-prem. Speeding up queries in the Siebel CRM ecosystem Siebel DB Siebel REST API API Gateway REST API CosmosDB commands queries kafka mobile app On-prem Azure Read Performance Gain Record count in DB, BC, CosmosDB Accounts 600k 550k 500k Assets: 85m 2.5m 1m Flat by .