Best Practice: From Spring Cloud To Istio

Best practice:from spring cloud to Istiozhangchaomeng@huawei.com#IstioCon

About meChaomeng Zhang is the Chief Architect of HuaweiCloud ASM (Application Service Mesh) service,which is based on Istio and Kubernetes.Chaomeng has been working on cloud nativetechnologies for more than 6 years, includingKubernetes, microservices, service catalog, APM,devops and service mesh for now. He is an Istiocommunity member, author of one bestsellingChinese Istio book "Cloud Native Service MeshIstio". He is also an experienced speaker of manycloud native and open source conferences,including KubeCon, Cloud Native Days, Servicemesh meetup, k8smeetup.#IstioCon

Agenda Concepts Problems Solutions Practice#IstioCon

Complexities of micro odulemodulesvcmodulemodule

Microservice SDKNode 1Node 2Consumer svcProducer svcappappSDKSDKServicemanagementService discoveryLoad balancingCircuit breakerDynamic routing.Network#IstioConServicemanagement

Service MeshNode 1Node 2Consumer svcProducer svcappappProxyService managementProxyService discoveryLoad balancingCircuit breakerDynamic routing.Network#IstioConService management

Spring cloud#IstioCon Spring Spring Spring Spring SpringBroker Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring Spring SpringCloudCloudCloudCloudCloudBusCircuit BreakerCLIfor Cloud Foundry- Cloud Foundry ubernetesNetflixOpen Service BrokerOpenFeignPipelinesSchema RegistrySecuritySkipperSleuthStreamStream App StartersStream ApplicationsTaskTask App StartersVaultZookeeperApp Broker

IstioAutomatic load balancing for HTTP,gRPC, WebSocket, and TCP traffic.Fine-grained control of trafficbehavior with rich routing rules,retries, failovers, and fault injection.A pluggable policy layer andconfiguration API supporting accesscontrols, rate limits and quotas.Automatic metrics, logs, and tracesfor all traffic within a cluster,including cluster ingress and egress.Secure service-to-servicecommunication in a cluster withstrong identity-based authenticationand authorization.#IstioCon

Service Discovery & Load chRegisterDiscoverConsumer(Eureka client& Ribbon)DiscoverLoadbalanceProducer(Eureka client)(Spring cloud)Spring cloudIstioServiceRegistryService register to EurekaNot needServicediscoveryConsumer calls eureka getinstance listPilot listwatch kubeapiserver service andendpointsLoad balanceRibbon in SDK selet instanceProxy select instanceLocationIn processOut of process#IstioConKubeletKubeletPodLoad balancePodEnvoyEnvoySVCSVC(Istio)

Circuit breakerreset(timeout)Half OpenOpensuccessClosedfail(not match threshold)#IstioConIstioMethodActionsWhite boxCoding fallbackWraped withHystrixCommandBlack boxOnly configurationCircuit breaker,threadpoolMicro service call,Protectand otherfunctionalitypotentially riskyfunctionalityFunctionsuccessfail(match Non-invasive, by proxyOutlier detection,threadpoolService call over thenetwork with fault andlatency tolerance

Agenda Concepts Problems Solutions Practice#IstioCon

Problem 1: Multi languageControl planeSpring cloudSpring cloudSpring cloudSpring cloudSpring cloudC PythonGolangNode.js#IstioCon

Problem 2: Discovery latency when Spring cloud running inside PodEurekaKubelet#IstioConProducer(Springcloud)pod ngcloud)

Problem 3: Upgrade all application in case of service management changingsvc101svc1Spring cloud SDK(v1)Spring cloud SDK(v1)Spring cloud SDK(v1)svc3Spring cloud SDK(v1)#IstioConSDK Bugfix, newfeaturesvc102svc2 Spring cloud SDK(v1)svc103Spring cloud SDK(v1)svc101svc1Spring cloud SDK(v2)Spring cloud SDK(v2)svc102svc2All apprebuild,reimage,upgradeSpring cloud SDK(v2)svc3Spring cloud SDK(v2) Spring cloud SDK(v2)svc103Spring cloud SDK(v2)

Problem 4: Gradually migrate from a monolith to micro servicesMicro ServiceMonolithic Servicesvc11SDKsvc1SDKRewrite withunified SDKsvc13SDKsvc2svc22SDK#IstioConsvc12The only thing a Big Bang rewrite guarantees is a Big Bang-- Martin Fowlersvc21SDKsvc23SDK

Agenda Concepts Problems Solutions Practice#IstioCon

Solution 1: Multi languageIstioDSpring cloudProxySpring cloudProxySpring cloudProxySpring cloudProxySpring cloudProxyC ProxyPythonProxyGolangProxyNode.jsProxy#IstioCon

Solution 2: Native Kubernetes service oxyKubeletProducer(Springcloud)Podpod :Pod1Pod2Pod2ProxyProducer(Springcloud)#IstioCon

Solution 3: Application NOT affected in case of Service management 01Proxy(v2)Svc102Proxy(v2)Svc103Proxy(v2)Istio upgradesvc2Proxy(v1)svc3Proxy(v1)#IstioCon svc102Proxy(v1)svc103Proxy(v1)ApplicationNot affected

Solution 4: Gradually migrate from a monolith to micro vc22svc23ProxyProxy#IstioConOld Monolith also equally managed by Istio as well as new micro service

Agenda Concepts Problems Solutions Practice#IstioCon

Make SDK to be SDKDevelopment ManagementApplicationSpring ngCloud native Infrastructure: Kubernetes: Flexible application deployment, management andscaling Istio: Non-intrusive traffic management, security and observability#IstioConDevelopmentRunning ManagementApplication Development:Spring Boot: Create production-grade applications thatyou can "just run".

Detailed migrating er①Application callApplication call③migrateSDKEnvoyService discoveryService discoveryLoad balance②Load balance ③ #IstioCon②ProducerProducerDiscard SDK serviceregistryBypass sdk servicediscovery andLoadbalancer， callproducer directly byKubernetes servicenameReplace SDK’sservice managementlogic with meshgradually.

Bypass SDK by changing configurationapplication.yaml# disable eureka discovery#eureka:# client:# /eureka/# instance:# leaseRenewalIntervalInSeconds:10# ribbon static instance set tokubernetes service name and portproducer:ribbon:listOfServers: producer:7111#IstioCon(Similar modification to annotation based config)

PASSTHROUGH: Offload CLOUD in Spring cloudSpring cloudSpring#IstioCon

Micro service Gateway ewayGateway(or zuul)(or ion 1：for specifiedgateway scenario(specifiedapplicationrelated filters)SpringcloudGatewaytoAppfacadeSolution 2:for 1IngressGatewaysvc2svc3

Multi language, Multi framework, Multi Env. Control plane: The unified control plane of Istio isused for service discovery and policy management.IstioD Data plane: Service discovery, load balancing, andmanagement are performed on the Envoy of theIstio data plane. The application can be code in c, cpp,python normal java ,or springcloud framework .TheSpring Cloud serves as the development frameworkand provides the service development frameworkand service protocols.VMVMEnvoyEnvoySDKC dEnvoyEnvoySDKJavaRestPythonEnvoygRPCC

Example: Istio canary for SpringCloud appkubectl logs helloclient-6fcc9cb8c9-qz5ng -c istio-proxy -nspringcloud-passthrough -f#IstioCon

Example: Istio circuit breaker help isolate unhealthy Springcloud instanceThe traffic on the unhealthy instance is gradually reduced until it is completely isolated. As awhole, only healthy instances of the service receive traffic.① Unhealthy instancetriggers circuit breaker#IstioCon② Traffic of the unhealthyinstance decreases③ The unhealthy instance isisolated

Example: Istio circuit breaker help isolate unhealthy Springcloud instanceWhen the unhealthy instance is normal back, under circuit breaker policy, traffic will beautomatically distributed to it .① There is no traffic on therecovered failed instance#IstioCon② A small amount of trafficstarts to be received on therecovered failed instance③ The recovered failed instancereceives the same traffic asother instances

Summary: Micro service, Container, Kubernetes，IstioApplication①Micro hestrateIntegrate②Kubernetes③① Containers and microservices share the same lightweight and agile features.② The use of Kubernetes for container orchestration is already the current standard.③ Istio and Kubernetes are closely combined to provide an end-to-end microservice running &management platform.④ Istio becomes the trend of microservice management.#IstioCon

Thank you!@idouba#IstioCon

Complexities of micro service module module ule module module ule module svc svc c svc svc c. #IstioCon Microservice SDK Node 1 Consumer svc app SDK Service management Node 2 Producer svc app SDK . Spring Cloud - Cloud Foundry Service Broker Spring Cloud Cluster Spring Cloud Commons Spring Cloud Config Spring Cloud Connectors .