Secret Management With Hashicorps Vault

Secret Managementwith Hashicorp'sVaultDaniel Bornkessel

Secret Management with Hashicorp's VaultX

Secret Managementwith Hashicorp'sVaultDaniel Bornkessel

Secret Management with Hashicorp's VaultFocus of this talk what is secret management why do you need it what is Vault and how can it help you with secret management some Vault internalsQuelle / Max Mustermann3

Secret Management with Hashicorp's VaultGoal of this talk think about best practices with secrets that your company couldimprove on go and play with VaultQuelle / Max Mustermann4

Secret Management with Hashicorp's VaultWhy focus on Vault unmatched (afaik) feature set not vendor or framework specific open source (mostly some closed sourced enterprise features)Quelle / Max Mustermann5

Secret Management with Hashicorp's VaultOther solutions* KeyWiz from Square: not as many features, no dynamic secrets,HSM in open source version Cloud Foundry CredHub: tailored and specific to Cloud Foundry AWS Secrets Manager: AWS specific, promising, dynamic’esquesecrets for certain AWS services, automatic rotation (forsupported services extendable via Lambda functions) self made: a lot of complexity and work* I have not personally used those solutionsQuelle / Max Mustermann6

Secret Management with Hashicorp's VaultSecret Managemant7

Secret Management with Hashicorp's VaultSecrets sensitive data ! secrets but: secrets sensitive data tokens passwords certificates API keys etc.Quelle / Max MustermannX

Secret Management with Hashicorp's VaultSecret Management part of your security concept one focus: on internal threads like rogue employees unauthorized access to secrets long living secrets audit log: who requested credentials for which system at whatpoint of time high automation for changing / revoking / rolling secrets high entropy passwordsQuelle / Max Mustermann8

Secret Management with Hashicorp's Vaulttodo: extreme exampleQuelle / Max Mustermann9

Secret Management with Hashicorp's Vault10

Secret Management with Hashicorp's VaultSecret Management: current situation best practices are widely known is usually seen as (very) important implementation is hard solutions are rare apps and frameworks not ready for modern secret management high automation still an exception (as opposed to external threadmitigation measures) often neglected in favour of business critical featuresQuelle / Max Mustermann11

Secret Management with Hashicorp's VaultQuestionWho here has production credentials on their laptop at this verymoment (e.g. AWS credentials file, DB credentials, passwordless sshprivate keys to access machines or git repos, API-keys, etc.)?Who thinks this is a good idea?Quelle / Max Mustermann12

Secret Management with Hashicorp's VaultWhy am I talking about secretmanagement13

Secret Management with Hashicorp's VaultAbout meDaniel Bornkessel / @kesselborn Senior Consultant at INNOQ (part time) Focus on DevOps & Continuous DeliveryINNOQ Consulting, reviews and development Quelle / Max Mustermann14

Secret Management with Hashicorp's VaultTypical project Monolith - Micro Services / Self Contained Systems Language: set (mostly Java) Framework: set (often Spring Boot) Data center: set (mostly AWS or on premise) Container Management: set (mostly Kubernetes) CI: set (whatever they used before egod’susesake:use Gitlab Logging / Monitoring: set (ELK & ?)prometheus) Secret Management: sure eh wat?Quelle / Max Mustermann15

Secret Management with Hashicorp's VaultTypical project: Secret Management we pass secrets in via env vars we read the values from Kubernetes secrets we have role based access control all figured out changing and updating passwords is a manual process for now yeah: audit log is something we are looking into no, we can not confidently say who has the password for DB xy no, we do not change all passwords if an employee leaves thecompany revoking credentials is not something we currently supportQuelle / Max Mustermann16

Secret Management with Hashicorp's VaultIntroducing VaultQuelle / Max MustermannX

Secret Management with Hashicorp's VaultVault — executive summary17

Secret Management with Hashicorp's VaultVault — executive summary“A Tool for Managing Secrets”Quelle / Max MustermannX

Secret Management with Hashicorp's VaultVault — executive summary not comparable to password managers like 1Password, LastPass,etc. Vault is designed for the system side of things — passwordmanagers “just” encrypt your static secrets and provide a niceway use themQuelle / Max MustermannX

Secret Management with Hashicorp's VaultVault — executive summary secures, stores and tightly controls tokens passwords certificates API keys and other secretsQuelle / Max Mustermann18

Secret Management with Hashicorp's VaultVault — executive summary handles leasing key revocation key rolling auditing provides an API for all operations is not meant as a service or token provider which gets embeddedin your request / response cycleQuelle / Max Mustermann19

Secret Management with Hashicorp's VaultVaultTokensLDAPAWSKubernetesGoogle Cloudauth-n auth-zAppRoleGitHubMFAOktaRADIUSTLS CertificatesQuelle / Max Mustermann aws creds admin developerdatabase creds clients contractspki issue example-comsecret team app1 api-keys google-analytics paypal app2 foossh creds erika ernatransit decrypt team-1-key encrypt secretsNomadPKI (Certificates)RabbitMQSSHTOTPTransit20

Secret Management with Hashicorp's VaultVault auth backends TokensLDAPAWSKubernetesGoogle CloudUsername & PasswordAppRoleGitHubMFAOktaRADIUSTLS CertificatesQuelle / Max Mustermann aws creds admin developerdatabase creds clients contractspki issue example-comsecret team app1 api-keys google-analytics paypal app2 foossh creds erika ernatransit decrypt team-1-key encrypt team-1-keyVault AWSConsulCubbyholeDatabasesIdentityStatic secrets (Key /Value)NomadPKI (Certificates)RabbitMQSSHTOTPTransit21

Secret Management with Hashicorp's VaultVault secret backends TokensLDAPAWSKubernetesGoogle CloudUsername & PasswordAppRoleGitHubMFAOktaRADIUSTLS CertificatesQuelle / Max Mustermann aws creds admin developerdatabase creds clients contractspki issue example-comsecret team app1 api-keys google-analytics paypal app2 foossh creds erika ernatransit decrypt team-1-key encrypt team-1-keyVault AWSConsulCubbyholeDatabasesIdentityStatic secrets (Key /Value)NomadPKI (Certificates)RabbitMQSSHTOTPTransit22

Secret Management with Hashicorp's VaultVault — secret backends23

Secret Management with Hashicorp's VaultVault secret backends — static secretsX

Secret Management with Hashicorp's VaultVault secret backends — static secrets12vault write secret/app1/api-key1234foobar vault read secret/app1/api-keyapp1Quelle / Max Mustermann1234foobaraws creds admin developerdatabase creds clients contractspki issue example-comsecret team app1 api-keys google-analytics paypal app2 foossh creds erika ernatransit decrypt team-1-key encrypt team-1-keyVault3X

Secret Management with Hashicorp's VaultVault secret backends — dynamic secrets24

Secret Management with Hashicorp's VaultVault secret backends — dynamic secretsWhat they are on-the-fly created credentials (hence dynamic) for each instanceof an app / user who wants a secret usually short to medium long ttl fully auditedQuelle / Max Mustermann25

Secret Management with Hashicorp's VaultVault secret backends — dynamic secretsHow they work (in a Nutshell)1. provide Vault credentials for a user that has rights to create usersor tokens in a remote system (e.g. db)2. configure Vault with settings on how to create credentials3. configure Vault with settings on how to invalidate credentials inthe remote systemQuelle / Max Mustermann26

Secret Management with Hashicorp's VaultVault secret backends — Databases27

Secret Management with Hashicorp's VaultVault secret backends — Databases Idea: get access to databases Vault gets configured with credentials for a database user thathas necessary permissions on the database Vault gets a policy that maps users and roles to users withconfigured permissions in the database when user requests credentials, Vault creates a new databaseuser on the fly when configured (usually the case), all created users have a ttlassigned — when the ttl is reached, Vault deletes the user fromthe databaseQuelle / Max MustermannX

Secret Management with Hashicorp's VaultVault secret backends — Databases0Quelle / Max Mustermanndb adminconfig aws creds admin developerdatabase creds clients contractspki issue example-comsecret team app1 api-keys google-analytics paypal app2 foossh creds erika ernatransit decrypt team-1-key encrypt team-1-keyVault28

Secret Management with Hashicorp's VaultVault secret backends — Databases0Quelle / Max Mustermanndb adminconfigvault secrets enable -path db database aws creds admin developerdatabase creds clients contractspki issue example-comsecret team app1 api-keys google-analytics paypal app2 foossh creds erika ernatransit decrypt team-1-key encrypt team-1-keyVaultvault write db/config/clients \plugin name mysql-database-plugin \connection url "admin:pw@tcp(db.example.com)/" \allowed roles “clients-ro,clients-rw”vault write database/roles/clients-ro \db name clients \creation statements “\CREATE USER '{{name}}'@'%' IDENTIFIED BY \'{{password}}'; \GRANT SELECT ON clients.* TO '{{name}}'@'%';" \default ttl "1h" \max ttl "240h"29

Secret Management with Hashicorp's VaultVault secret backends — Databases0Quelle / Max Mustermanndb adminconfigvault secrets enable -path db database aws creds admin developerdatabase creds clients contractspki issue example-comsecret team app1 api-keys google-analytics paypal app2 foossh creds erika ernatransit decrypt team-1-key encrypt team-1-keyVaultvault write db/config/clients \plugin name mysql-database-plugin \connection url "admin:pw@tcp(db.example.com)/" \allowed roles “clients-ro, clients-rw”vault write database/roles/clients-ro \db name clients \creation statements “\CREATE USER '{{name}}'@'%' IDENTIFIED BY \'{{password}}'; \GRANT SELECT ON clients.* TO '{{name}}'@'%';" \default ttl "1h" \max ttl “240h"30

Secret Management with Hashicorp's VaultVault secret backends — Databases0db adminconfig1 read}configuredroledb/creds/clients-roApp4Quelle / Max Mustermanndb login /db password aws creds admin developerdatabase creds clients contractspki issue example-comsecret team app1 api-keys google-analytics paypal app2 foossh creds erika ernatransit decrypt team-1-key encrypt team-1-keyVault2 create user ttl5delete user 3OK31

Secret Management with Hashicorp's VaultVault secret backends — DatabasesAvailable Plugins: Cassandra HanaDB MongoDB MSSQL MySQL/MariaDB PostgreSQL OracleQuelle / Max Mustermann32

Secret Management with Hashicorp's VaultVault secret backends — Google CloudX

Secret Management with Hashicorp's VaultVault secret backends — Google Cloud0 credentials.json or service accountQuelle / Max Mustermann aws creds admin developerdatabase creds clients contractspki issue example-comsecret team app1 api-keys google-analytics paypal app2 foossh creds erika ernatransit decrypt team-1-key encrypt team-1-keydefine rolesets to generateoauth2 access tokens (preferred)or Service AccountsVaultX

Secret Management with Hashicorp's VaultVault secret backends — Google Cloud0 credentials.json or service lle / Max Mustermannoauth2 token/service accountkey aws creds admin developerdatabase creds clients contractspki issue example-comsecret team app1 api-keys google-analytics paypal app2 foossh creds erika ernatransit decrypt team-1-key encrypt team-1-key2create token /service accountGCPApiVault3oauth2 token /service accountkeyttlX

Secret Management with Hashicorp's VaultVault secret backends — AWS33