BGP Multihoming Techniques - North American Network .

BGP Multihoming TechniquesPhilip Smith pfs@cisco.co m NANOG 32Reston, VirginiaNANOG 321

Presentation Slides Available Multihoming.pdfNANOG 322

Preliminaries Presentation has many configuration examplesUses Cisco IOS CLI Aimed at Service ProvidersTechniques can be used by many enterprises too Feel free to ask questionsNANOG 323

BGP Multihoming Techniques Why Multihome? Definition & Options Preparing the Network Basic Multihoming Service Provider Multihoming Using Communities Case StudyNANOG 324

Why Multihome?It’s all about redundancy, diversity and reliabilityNANOG 325

Why Multihome? RedundancyOne connection to internet means the networkis dependent on:Local router (configuration, software,hardware)WAN media (physical failure, carrier failure)Upstream Service Provider (configuration,software, hardware)NANOG 326

Why Multihome? ReliabilityBusiness critical applications demandcontinuous availabilityLack of redundancy implies lack of reliabilityimplies loss of revenueNANOG 327

Why Multihome? Supplier DiversityMany businesses demand supplier diversity as a matterof courseInternet connection from two or more suppliersWith two or more diverse WAN pathsWith two or more exit pointsWith two or more international connectionsTwo of everythingNANOG 328

Why Multihome? Not really a reason, but oft quoted Leverage:Playing one ISP off against the other for:Service QualityService OfferingsAvailabilityNANOG 329

Why Multihome? Summary:Multihoming is easy to demand as requirement for anyservice provider or end-site networkBut what does it really mean:In real life?For the network?For the Internet?And how do we do it?NANOG 3210

BGP Multihoming Techniques Why Multihome? Definition & Options Preparing the Network Basic Multihoming Service Provider Multihoming Using Communities Case StudyNANOG 3211

Multihoming Definition &OptionsWhat does it mean, what do we need, and how do we do it?NANOG 3212

Multihoming Definition More than one link external to the localnetworktwo or more links to the same ISPtwo or more links to different ISPs Usually two external facing routersone router gives link and provider redundancyonlyNANOG 3213

AS Numbers An Autonomous System Number is required byBGP Obtained from upstream ISP or RegionalRegistry (RIR)APNIC, ARIN, LACNIC, RIPE NCC Necessary when you have links to more than oneISP or to an exchange point 16 bit integer, ranging from 1 to 65534Zero and 65535 are reserved64512 through 65534 are called Private ASNsNANOG 3214

Private-AS – Application Applications65001193.0.32.0/24An ISP with customersmultihomed on theirbackbone (RFC2270)-orA corporate networkwith several regionsbut connections to theInternet only in 003193.2.35.0/24A193.1.32.0/22 1880Within a BGPConfederationNANOG 3215

Private-AS – Removal Private ASNs MUST be removed from all prefixesannounced to the public InternetInclude configuration to remove private ASNs in theeBGP template As with RFC1918 address space, private ASNsare intended for internal useThey should not be leaked to the public Internet Cisco IOSneighbor x.x.x.x remove-private-ASNANOG 3216

Configuring Policy Three BASIC Principles for IOSconfiguration examples throughoutpresentation:prefix-lists to filter prefixesfilter-lists to filter ASNsroute-maps to apply policy Route-maps can be used for filtering, butthis is more “advanced” configurationNANOG 3217

Policy Tools Local preferenceoutbound traffic flows Metric (MED)inbound traffic flows (local scope) AS-PATH prependinbound traffic flows (Internet scope) Communitiesspecific inter-provider peeringNANOG 3218

Originating Prefixes: Assumptions MUST announce assigned address block toInternet MAY also announce subprefixes – reachability isnot guaranteed Current RIR minimum allocation is /21Several ISPs filter RIR blocks on this boundarySeveral ISPs filter the rest of address space accordingto the IANA assignmentsThis activity is called “Net Police” by someNANOG 3219

Originating Prefixes RIRs publish their minimum allocation nRIPE ml IANA publishes the address space it has assigned to end-sitesand allocated to the RIRs:www.iana.org/assignments/ipv4-address-space Several ISPs use this published information to filter prefixeson:What should be routed (from IANA)The minimum allocation size from the RIRsNANOG 3220

“Net Police” prefix list issues meant to “punish” ISPs who pollute the routing tablewith specifics rather than announcing aggregates impacts legitimate multihoming especially at theInternet’s edge impacts regions where domestic backbone isunavailable or costs compared with internationalbandwidth hard to maintain – requires updating when RIRs startallocating from new address blocks don’t do it unless consequences understood and youare prepared to keep the list currentNANOG 3221

Multihoming Scenarios Stub network Multi-homed stub network Multi-homed network Load-balancingNANOG 3222

Stub NetworkAS101AS100 No need for BGP Point static default to upstream ISP Router will load share on the two parallel circuits Upstream ISP advertises stub network Policy confined within upstream ISP’s policyNANOG 3223

Multi-homed Stub NetworkAS65530AS100 Use BGP (not IGP or static) to loadshare Use private AS (ASN 64511) Upstream ISP advertises stub network Policy confined within upstream ISP’s policyNANOG 3224

Multi-Homed NetworkGlobal InternetAS200AS300AS100 Many situations possiblemultiple sessions to same ISPsecondary for backup onlyload-share between primary and secondaryselectively use different ISPsNANOG 3225

Multiple Sessions to an ISP– Example One Use eBGP multihopeBGP to loopback addresseseBGP prefixes learned with loopbackaddress as next hopAS 2001.1.1.1 Cisco IOSrouter bgp 65534neighbor 1.1.1.1 remote-as 200neighbor 1.1.1.1 ebgp-multihop 2!ip route 1.1.1.1 255.255.255.255 serial 1/0ip route 1.1.1.1 255.255.255.255 serial 1/1ip route 1.1.1.1 255.255.255.255 serial 1/2NANOG 32AS 6553426

Multiple Sessions to an ISP– Example One Try and avoid use of ebgp-multihop unless:It’s absolutely necessary –or–Loadsharing across multiple links Many ISPs discourage its use, for example:We will run eBGP multihop, but do not support it as a standard offeringbecause customers generally have a hard time managing it due to: routing loops failure to realise that BGP session stability problems are usually dueconnectivity problems between their CPE and their BGP speakerNANOG 3227

Multiple Sessions to an ISP– Example Two BGP multi-path Limit to number of parallelpaths depending onimplementation For this example, threeBGP sessions required Cisco IOS Configurationrouter bgp 201neighbor 1.1.2.1 remote-as 200neighbor 1.1.2.5 remote-as 200neighbor 1.1.2.9 remote-as 200maximum-paths 3NANOG 32AS 200AS 20128

Multiple Sessions to an ISP Simplest scheme is to usedefaultsISP Learn/advertise prefixes forbetter control Planning and some workrequired to achieveloadsharingCDABPoint default towards one ISPLearn selected prefixes fromsecond ISPModify the number of prefixeslearnt to achieve acceptableload sharingAS 201 No magic solutionNANOG 3229

BGP Multihoming Techniques Why Multihome? Definition & Options Preparing the Network Basic Multihoming Service Provider Multihoming Using Communities Case StudyNANOG 3230

Preparing the Network We will deploy BGP across the network beforewe try and multihome BGP will be used therefore an ASN is required If multihoming to different ISPs, public ASNneeded:Either go to upstream ISP who is a registry member, orApply to the RIR yourself for a one off assignment, orAsk an ISP who is a registry member, orJoin the RIR and get your own IP address allocation too(this option strongly recommended)!NANOG 3231

Preparing the Network Will look at two examples of BGPdeployment:Example One: network is only static routesExample Two: network is currently running anIGPNANOG 3232

Preparing the NetworkExample One The network is not running any BGP at themomentsingle statically routed connection to upstreamISP The network is not running any IGP at allStatic default and routes through the networkto do “routing”NANOG 3233

Preparing the NetworkIGP Decide on IGP: OSPF or ISIS J Assign loopback interfaces and /32 addresses toeach router which will run the IGPLoopback is used for OSPF and BGP router id anchorUsed for iBGP and route origination Deploy IGP (e.g. OSPF)IGP can be deployed with NO IMPACT on the existingstatic routingOSPF distance is 110, static distance is 1Smallest distance winsNANOG 3234

Preparing the NetworkIGP (cont) Be prudent deploying IGP – keep the Link StateDatabase Lean!Router loopbacks go in IGPWAN point to point links go in IGP(In fact, any link where IGP dynamic routing will be runshould go into IGP)Summarise on area/level boundaries (if possible) – i.e.think about your IGP address planNANOG 3235

Preparing the NetworkIGP (cont) Routes which don’t go into the IGP include:Dynamic assignment pools (DSL/Cable/Dial)Customer point to point link addressing(using next-hop-self in iBGP ensures that these do NOTneed to be in IGP)Static/Hosting LANsCustomer assigned address spaceAnything else not listed in the previous slideNANOG 3236

Preparing the NetworkiBGP Second step is toconfigure the localnetwork to use iBGPBAD iBGP can run onall routers, ora subset of routers, orFCEAS200just on the upstream edge iBGP must run on allrouters which are in thetransit path betweenexternal connectionsNANOG 3237

Preparing the NetworkiBGP (Transit Path) iBGP must run on allrouters which are in thetransit path betweenexternal connections Routers C, E and F are notin the transit pathStatic routes or IGP willsufficeBADFCEAS200 Router D is in the transitpathWill need to be in iBGPmesh, otherwise routingloops will resultNANOG 3238

Preparing the NetworkLayers Typical SP networks have three layers:Core – the backbone, usually the transit pathDistribution – the middle, PoP aggregationlayerAggregation – the edge, the devicesconnecting customersNANOG 3239

Preparing the NetworkAggregation Layer iBGP is optionalMany ISPs run iBGP here, either partial routing (morecommon) or full routing (less common)Full routing is not needed unless customers want full tablePartial routing is cheaper/easier, might usually consist ofinternal prefixes and, optionally, external prefixes to aidexternal load balancingCommunities and peer-groups make this administratively easy Many aggregation devices can’t run iBGPStatic routes from distribution devices for address poolsIGP for best exitNANOG 3240

Preparing the NetworkDistribution Layer Usually runs iBGPPartial or full routing (as with aggregation layer) But does not have to run iBGPIGP is then used to carry customer prefixes (does notscale)IGP is used to determine nearest exit Networks which plan to grow large shoulddeploy iBGP from day oneMigration at a later date is extra workNo extra overhead in deploying iBGP, indeed IGPbenefitsNANOG 3241

Preparing the NetworkCore Layer Core of network is usually the transit path iBGP necessary between core devicesFull routes or partial routes:Transit ISPs carry full routes in coreEdge ISPs carry partial routes only Core layer includes AS border routersNANOG 3242

Preparing the NetworkiBGP ImplementationDecide on: Best iBGP policyWill it be full routes everywhere, or partial, orsome mix? iBGP scaling techniqueCommunity policy?Route-reflectors?Techniques such as peer groups and peertemplates?NANOG 3243

Preparing the NetworkiBGP Implementation Then deploy iBGP:Step 1: Introduce iBGP mesh on chosen routersmake sure that iBGP distance is greater than IGP distance (itusually is)Step 2: Install “customer” prefixes into iBGPCheck! Does the network still work?Step 3: Carefully remove the static routing for theprefixes now in IGP and iBGPCheck! Does the network still work?Step 4: Deployment of eBGP followsNANOG 3244