Best Practices For Implementing Cloud Data Governance And . - Informatica

1m ago
37 Views
0 Downloads
2.86 MB
45 Pages
Last View : 3d ago
Last Download : n/a
Upload by : Carlos Cepeda
Transcription

June 20th, 2023 Best Practices for Implementing Cloud Data Governance and Catalog Kristin Feeback, Senior Consultant, IPS Steven Fleishman, Solution Architect, IPS

Housekeeping Tips 2 Today’s Webinar is scheduled for 1 hour The session will include a webcast and then your questions will be answered live at the end of the presentation All dial-in participants will be muted to enable the speakers to present without interruption Questions can be submitted to “All Panelists" via the Q&A option and we will respond at the end of the presentation The webinar is being recorded and will be available on our INFASupport YouTube channel and Success Portal - where you can download the slide deck for the presentation. The link to the recording will be emailed as well. Please take time to complete the post-webinar survey and provide your feedback and suggestions for upcoming topics. Informatica. Proprietary and Confidential.

Feature Rich Success Portal Bootstrap trial and POC Customers 3 Enriched Customer Onboarding experience Informatica. Proprietary and Confidential. Product Learning Paths and Weekly Expert Sessions Informatica Concierge Tailored training and content recommendations

More Information 4 Success Portal Communities & Support Documentation University https://success.informatica.com https://network.informatica.com https://docs.informatica.com https://www.informatica.com/in/ser vices-and-training/informaticauniversity.html Informatica. Proprietary and Confidential.

Safe Harbor The information being provided today is for informational purposes only. The development, release, and timing of any Informatica product or functionality described today remain at the sole discretion of Informatica and should not be relied upon in making a purchasing decision. Statements made today are based on currently available information, which is subject to change. Such statements should not be relied upon as a representation, warranty or commitment to deliver specific products or functionality in the future.

Agenda 6 1 Introduction 2 Technical Considerations 3 Roles and Responsibilities 4 Foundation Design 5 Data Quality Capabilities within CDGC 6 Question and Answer Informatica. Proprietary and Confidential.

Technical Considerations

Account Verification and Services Screen Welcome Email Services Available 8 Informatica. Proprietary and Confidential.

Secure Agent Sizing Requirements for CDGC If agent status is inactive, communication Informatica Cloud Servers can be blocked by: Windows Firewall Virus Scanner Content Filter Recommended Prerequisites - 16 CPU cores 64 GB RAM 200 GB HDD SSD disks are strongly recommended High Availability Load Balancing Job isolation Management API On-Premise/Cloud Easy to deploy Agent Group - Download and install the secure agent - Register agent using generated token Scale with IDMC agent grouping capabilities Secure Agent Group Agent 1 Agent 2 - 2 or more SA for sources more than 100k assets All agents can execute all the capabilities (Sizing Guidelines) - Metadata extraction - Profiling How To: Plan Secure Agent(s) with Best Practices 35 Informatica. Proprietary and Confidential. Container Scaling By Job Spec Task Group 1 T11 T12 Container Scaling By Job Spec T21 Task Group 2 T22

Cloud Apps Public IDMC Services Firewall Multi-tenant Metadata (TLS 1.2) ERP Cloud Secure Agent Group CRM Metadata Databases The Secure Agent makes outbound communication to the Informatica Cloud server. (TLS 1.2) Documents Data Warehouse Hadoop Secure Agent Group (on-prem) 23 Only metadata and profile results are sent back to Informatica Cloud For metadata in IDMC, Administrator can restrict to view Sensitive Data & Profile Statistics Informatica. Proprietary and Confidential. User HTTPS: 443 (Outbound) On-Premises Mainframe Application Servers Web Client Design Admin Other SaaS Apps AES Encryption (256 bit) Network Architecture Customer VNET On-Prem

Secure Agent Connectivity Open up the firewall of the Secure Agent to communicate with: 1. The Data Source (ADLS, Redshift, Snowflake, Oracle .) 2. The Informatica Cloud This ensures that the Informatica Cloud Secure related services can connect to IICS Job Agent doesn’tand initialize servers to perform all necessary tasks. KB articles for firewall requirements Base IICS Services Customer Name Data Governance and Catalog Cloud Data Quality Cloud Data Profiling 11 Informatica. Proprietary and Confidential. “failed to deploy task”

Secure Agent Configuration Concurrent Jobs Update maxDTMProcesses to increase number of concurrent mapping tasks Connection Type Formula to set value Example on 16 CPU Server File-based or ODBC connections 0.75 * Number of logical CPUs 0.75 * 16 12 Cloud data warehouse or cloud data lake connectors 0.33 * Number of logical CPUs 0.33 * 16 5

Secure Agent Configuration Memory Additional memory for metadata scanning Increase maxHeap size Agent-wide virtual memory for all IICS Services INFA MEMORY Out of the box it’s 512m Must be set at least to 2048M '-Xms512m -Xmx8192m -XX:MaxPermSize 384m' 40 Informatica. Proprietary and Confidential. Memory on the Mapping level JVMOption1 Improve performance Reduce writing to disk '-Xmx8192m'

Don’t Forget to Import Pre-Defined Content

Import Data Quality Bundles with Pre-Built Content 15 Informatica. Proprietary and Confidential.

Check Command Center Source to see if DataMetadata Governance & Catalog Scanners Scanners in Red denote no connection in Administrator Service Connection is required inrequired Administrator Data Integration Azure Azure SQL DB Azure Synapse Azure ADLS Gen 2 Azure Blob Azure Data Factory On-premises Oracle SQL Server IBM Db2 MySQL Teradata Postgres JDBC Netezza MongoDB Kafka Local/Shared Filesystem Cloud Data Integration PowerCenter Microsoft SSIS Talend Cloud Data Catalog Google & OCI Google BigQuery GCS Oracle ADB 16 BI & Analytics Tableau PowerBI SSRS QlikView Qlik Sense Databricks Notebooks Cognos Microstrategy Looker AWS AWS S3 AWS Redshift AWS RDS (Oracle, MS SQL Server, PostgreSQL and MySQL) AWS Athena DynamoDBl Applications CDL/DW Snowflake DW Databricks Delta Tables Informatica. Proprietary and Confidential. *Current as of December 2022 – Check with Informatica for latest updates. SAP BW SAP BW/4HANA SAP ECC SAP S/4HANA SAP BO SAP HANA DB Salesforce Marketo Dynamics CRM Workday Informatica MDM B360

Roles and Responsibilities

Why are you using CDGC? What was the deciding factor for wanting to start/bring data governance to the next level Need to think about how the program wants users to interact with the solution and in what ways they may want to operationalize it. Will there be a heavy data privacy, data quality, or master data management focus? Will priorities be split between multiple areas?

Preliminary Considerations What needs to be done for the program to be functional? What operations need to be completed? What is your current team’s strengths and weaknesses? 19 Informatica. Proprietary and Confidential. Does your company already have a data first culture?

Preliminary Considerations Who will be the primary stewards and owners? What is their capacity for work? What do they care about? What do they need to do to be successful? 20 Informatica. Proprietary and Confidential. What is the minimal level of interaction do they need to be successful?

User interactions 21 Informatica. Proprietary and Confidential.

User Interactions Functions within the Roles 22 Informatica. Proprietary and Confidential.

Catalog Data Steward Communicates technical workings of data, fosters the goal of data re-use, and designs policies and processes to improve the underlying integrity of information to the organization. Responsible for uploading and curating catalog assets. Asset Permissions MCC Permissions Workflow Inbox 23 Informatica. Proprietary and Confidential. CDGC Permissions

Governance Data Steward Responsible for working with business and technical stakeholders to ensure high levels of data quality, integrity, availability, trustworthiness and security. Creates and curates business assets. Asset Permissions 24 Informatica. Proprietary and Confidential. CDGC Permissions Workflow Inbox

Role Interactions Product Data Data Quality Stewards Product Data Quality Stewards 25 Informatica. Proprietary and Confidential.

Role interactions Governance Data Owner Governance Data Steward 26 Informatica. Proprietary and Confidential.

Foundation Design

Foundation for the Future CDGC’s strength for the glossary lies in depth of information and its relationships. CDGC should be kept in a standard model to maximize use and growth. Keep in mind that the use case is just giving you the first set of information to input information. Harder to change the model once the use case information is built out. Where do things belong? 28 Informatica. Proprietary and Confidential.

Finding the start 29 Informatica. Proprietary and Confidential.

Starting concept Critical Data Element Standard Definition Definition Variation 1 30 Informatica. Proprietary and Confidential. Definition Variation 2 Definition Variation 3

Collection of assets Customer Profit SQL Server Business Term Pricing Price Invoice Catalog CDGC Negotiation Glossary Oracle Order Process Debt Backs Policy Metric 31 Informatica. Proprietary and Confidential. Purchase Order Execution Price Sheet Generation Cost Snowflake Purchase Order Business Rules Data Standards Technical Rule Customer Agreement State must not be null Pricing Hierarchy Pricing Quote

Data Quality within CDGC

Data Governance and Catalog After Enrichment Profiling, Classification, Glossary Association, Relationship Discovery 33 Informatica. Proprietary and Confidential.

Informatica Cloud Data Governance and Catalog Data Quality Process Metadata Command Center Cloud Data Quality Metadata Command Center Run Metadata Extraction and enrichment, including Data Quality 34 Cloud Data Governance and Catalog Connect data to a business term Data Profiling Collect additional information on the data Informatica. Proprietary and Confidential. Identify a data Ruleset Create Rule Specification Data Profiling Validate rule specification Ensure Data Quality is configured Cloud Data Governance and Catalog Create Data Rule Template with Automation enabled Cloud Data Governance and Catalog (If adhoc) Start Rule Occurrence Cloud Data Governance and Catalog Review DQ scores Job won’t run unless DQ is enabled There are multiple ways to create DQ Rule Specifications Data Profiling - Generated from Data Quality Insights Data Governance and Catalog – Leveraging Natural Language Processing Data Quality – Create a new Rule Specification Administrator – Data Quality Bundles

Metadata Extraction - Data Quality/DQ Enabled Apply on data elements linked with business data set Apply on all data elements Sampling options based on source All Rows Limit N Rows Random N Rows Random N Percentage Custom Query 35 Informatica. Proprietary and Confidential.

Associate Technical Asset to Business Term 36 Informatica. Proprietary and Confidential.

Business Term Associated to Four Columns 37 Informatica. Proprietary and Confidential.

Data Profiling – Create Connection and Run Profile

Collect Information & Create Rule Set

Create the Business Rule Open the Data Quality Service In the Data Quality Service, this appears under “Pick an existing rule” Create, Test, and Save a Rule Specification

Associate Rule Template to Glossary & Rule Specification 41 Informatica. Proprietary and Confidential.

Data Quality Rule Template and Natural Language Processing Data Quality Rule Template

Automate Rule Occurrence Generation 43 Informatica. Proprietary and Confidential.

Automate Rule Occurrence Generation

Managing the Data Quality Automation Process Driven by Metadata Command Center and Data Governance and Catalog 45 isAutomated option on rule templates Data quality option in Metadata in Data Governance and Catalog Command Center Data quality automation option in Metadata Command Center Result Yes Yes Yes Yes Yes No Yes No Not applicable No Yes Yes Create rule occurrences for all data elements that are associated with glossary business assets. Does not create new rule occurrences for data elements or update an existing rule occurrence on data elements. Does not affect the execution of the existing rule occurrences in Data Governance and Catalog Does not create any rule occurrences for data elements. Data quality execution stops for existing rule occurrences that are associated with assets of a particular catalog source. Does not create rule occurrences for data elements. Does not affect the execution of the existing rule occurrences in Data Governance and Catalog Informatica. Proprietary and Confidential.

16 CPU cores. 64 GB RAM. 200 GB HDD. SSD disks are strongly recommended. Easy to deploy. Download and install the secure agent. Register agent using generated token. Scale with IDMC agent grouping capabilities. 2 or more SA for sources more than 100k assets.

Related Documents:

Bruksanvisning för bilstereo . Bruksanvisning for bilstereo . Instrukcja obsługi samochodowego odtwarzacza stereo . Operating Instructions for Car Stereo . 610-104 . SV . Bruksanvisning i original

10 tips och tricks för att lyckas med ert sap-projekt 20 SAPSANYTT 2/2015 De flesta projektledare känner säkert till Cobb’s paradox. Martin Cobb verkade som CIO för sekretariatet för Treasury Board of Canada 1995 då han ställde frågan

service i Norge och Finland drivs inom ramen för ett enskilt företag (NRK. 1 och Yleisradio), fin ns det i Sverige tre: Ett för tv (Sveriges Television , SVT ), ett för radio (Sveriges Radio , SR ) och ett för utbildnings program (Sveriges Utbildningsradio, UR, vilket till följd av sin begränsade storlek inte återfinns bland de 25 största

Hotell För hotell anges de tre klasserna A/B, C och D. Det betyder att den "normala" standarden C är acceptabel men att motiven för en högre standard är starka. Ljudklass C motsvarar de tidigare normkraven för hotell, ljudklass A/B motsvarar kraven för moderna hotell med hög standard och ljudklass D kan användas vid

LÄS NOGGRANT FÖLJANDE VILLKOR FÖR APPLE DEVELOPER PROGRAM LICENCE . Apple Developer Program License Agreement Syfte Du vill använda Apple-mjukvara (enligt definitionen nedan) för att utveckla en eller flera Applikationer (enligt definitionen nedan) för Apple-märkta produkter. . Applikationer som utvecklas för iOS-produkter, Apple .

sites cloud mobile cloud social network iot cloud developer cloud java cloud node.js cloud app builder cloud cloud ng cloud cs oud database cloudinfrastructureexadata cloud database backup cloud block storage object storage compute nosql

Switch and Zoning Best Practices 28-30 2. IP SAN Best Practices 30-32 3. RAID Group Best Practices 32-34 4. HBA Tuning 34-38 5. Hot Sparing Best Practices 38-39 6. Optimizing Cache 39 7. Vault Drive Best Practices 40 8. Virtual Provisioning Best Practices 40-43 9. Drive

och krav. Maskinerna skriver ut upp till fyra tum breda etiketter med direkt termoteknik och termotransferteknik och är lämpliga för en lång rad användningsområden på vertikala marknader. TD-seriens professionella etikettskrivare för . skrivbordet. Brothers nya avancerade 4-tums etikettskrivare för skrivbordet är effektiva och enkla att