ISP & IXP Design - MENOG
ISP & IXP Design30thPhilip SmithMENOG 11AmmanSeptember – 9th October 20121
ISP & IXP Network DesignPoP Topologies and Designp Backbone Designp Upstream Connectivity & Peeringp Addressingp Routing Protocolsp Out of Band Managementp Operational Considerationsp Internet Exchange Pointsp 2
Point of PresenceTopologies3
PoP TopologiesCore routers – high speed trunkconnectionsp Distribution routers and Access routers –high port densityp Border routers – connections to otherprovidersp Service routers – hosting and serversp Some functions might be handled by asingle routerp 4
PoP DesignModular Designp Aggregation Services separated accordingtop n n n n connection speedcustomer servicecontention ratiosecurity considerations5
Modular PoP DesignOther ISPsISP Services(DNS, Mail, News,FTP, WWW)Web CacheHosted Services &DatacentreBackbone linkto another PoPBackbone linkto another PoPNetworkCoreConsumer cable,xDSL andwireless AccessConsumerDial AccessBusiness customeraggregation layerMetroE customeraggregation layerNetworkOperationsCentreChannelised circuitsfor leased line circuit deliveryGigE fibre trunksfor MetroE circuit delivery6
Modular Routing Protocol Designp Modular IGP implementationn n n p IGP “area” per PoPCore routers in backbone area (Area 0/L2)Aggregation/summarisation where possibleinto the coreModular iBGP implementationn n n BGP route reflector clusterCore routers are the route-reflectorsRemaining routers are clients & peer withroute-reflectors only7
Point of Presence Design8
PoP Modulesp Low Speed customer connectionsn n n p PSTN/ISDN dialupLow bandwidth needsLow revenue, large numbersLeased line customer connectionsn n n n E1/T1 speed rangeDelivery over channelised mediaMedium bandwidth needsMedium revenue, medium numbers9
PoP Modulesp Broad Band customer connectionsn n n p xDSL, Cable and WirelessHigh bandwidth needsLow revenue, large numbersMetroE & Highband customer connectionsn n n n Trunk onto GigE or 10GigE of 10Mbps andhigherChannelised OC3/12 delivery of E3/T3 andhigherHigh bandwidth needsHigh revenue, low numbers10
PoP Modulesp PoP Coren n n n p Two dedicated routersHigh Speed interconnectBackbone Links ONLYDo not touch them!Border Networkn n n n Dedicated border router to other ISPsThe ISP’s “front” doorTransparent web caching?Two in backbone is minimum guarantee forredundancy11
PoP Modulesp ISP Servicesn n n n p DNS (cache, secondary)News (still relevant?)Mail (POP3, Relay, Anti-virus/anti-spam)WWW (server, proxy, cache)Hosted Services/DataCentresn n n Virtual Web, WWW (server, proxy, cache)Information/Content ServicesElectronic Commerce12
PoP Modulesp Network Operations Centren n n n p Consider primary and backup locationsNetwork monitoringStatistics and log gatheringDirect but secure accessOut of Band Management Networkn The ISP Network “Safety Belt”13
Low Speed Access ModuleWeb CacheAccess NetworkGateway RoutersPrimary RateT1/E1Access ServersPSTN lines tomodem bankTo Core RoutersPSTN lines tobuilt-in modemsTACACS /Radiusproxy, DNS resolver,Content14
Medium Speed Access ModuleAggregation EdgeChannelised T1/E164K and nx64KcircuitsTo Core RoutersMixture of channelisedT1/E1, 56/64K andnx64K circuits15
High Speed Access ModuleAggregation EdgeMetro EthernetChannelised T3/E3To Core RoutersChannelised OC3/OC1216
Broadband Access ModuleWeb CacheTelephone NetworkDSLAMBRASIP, ATMCable RASAccess NetworkGateway RoutersTo Core RoutersThe cable systemSSG, DHCP, TACACS or Radius Servers/Proxies,DNS resolver, Content17
ISP Services ModuleTo core routersService NetworkGateway he18
Hosted Services ModuleTo core routersHosted NetworkGateway Routersvlan11vlan12 vlan13vlan14vlan15vlan16vlan17Customer 1Customer 3Customer 5Customer 7Customer 2Customer 4Customer 619
Border ModuleTo local IXPNB: router has no default route local AS routing table onlyISP1ISP2NetworkBorder RoutersTo core routers20
NOC ModuleCritical ServicesModuleTo core routersCorporate LANOut of BandHosted NetworkGateway RoutersManagement NetworkFirewall2811/32asyncNetFlow TACACS SYSLOG Primary DNSAnalyserserverserverNetwork Operations Centre StaffBilling, Databaseand AccountingSystems21
Out of Band NetworkOut of BandManagement NetworkRouterconsolesTerminal serverTo the NOCNetFlowenabledroutersNetFlowCollectorOut of Band Ethernet22
Backbone NetworkDesign23
Backbone DesignRouted Backbonep Switched Backbonep n n p Point-to-point circuitsn p ATM/Frame Relay core networkNow obsoletenx64K, T1/E1, T3/E3, OC3, OC12, GigE, OC48,10GigE, OC192, OC768ATM/Frame Relay service from telcon n n T3, OC3, OC12, deliveryEasily upgradeable bandwidth (CIR)Almost vanished in availability now24
Distributed Network Designp PoP design “standardised”n operational scalability and simplicityISP essential services distributed aroundbackbonep NOC and “backup” NOCp Redundant backbone linksp 25
Distributed Network DesignCustomerconnectionsISP ServicesBackupOperations CentrePOP TwoCustomerconnectionsCustomerconnectionsISP ServicesPOP OnePOP ThreeISP ServicesExternalconnectionsOperations CentreExternalconnections26
Backbone Linksp ATM/Frame Relayn n p Virtually disappeared due to overhead, extraequipment, and shared with other customersof the telcoMPLS has replaced ATM & FR as the telcofavouriteLeased Line/Circuitn n Most popular with backbone providersIP over Optics and Metro Ethernet verycommon in many parts of the world27
Long Distance Backbone LinksThese usually cost morep Important to plan for the futurep n n n This means at least two years aheadStay in budget, stay realisticUnplanned “emergency” upgrades will bedisruptive without redundancy in the networkinfrastructure28
Long Distance Backbone Linksp Allow sufficient capacity on alternativepaths for failure situationsn n n n Sufficient can depend on the business strategySufficient can be as little as 20%Sufficient is usually over 50% as this offers“business continuity” for customers in the caseof link failureSome businesses choose 0%p Very short sighted, meaning they have no sparecapacity at all!!29
Long Distance LinksPOP TwoLong distance linkPOP OnePOP ThreeAlternative/Backup Path30
Metropolitan Area Backbone Linksp Tend to be cheapern n p Circuit concentrationChoose from multiple suppliersThink bign n n More redundancyLess impact of upgradesLess impact of failures31
Metropolitan Area Backbone LinksPOP TwoMetropolitan LinksPOP OnePOP ThreeMetropolitan LinksTraditional Point to Point Links32
Upstream Connectivityand Peering33
Transitsp Transit provider is another autonomous systemwhich is used to provide the local network withaccess to other networksn n p Might be local or regional onlyBut more usually the whole InternetTransit providers need to be chosen wisely:n Only onep n Too manyp p p p no redundancymore difficult to load balanceno economy of scale (costs more per Mbps)hard to provide service qualityRecommendation: at least two, no morethan three
Common Mistakesp ISPs sign up with too many transit providersn n n p Lots of small circuits (cost more per Mbps than largerones)Transit rates per Mbps reduce with increasing transitbandwidth purchasedHard to implement reliable traffic engineering thatdoesn’t need daily fine tuning depending on customeractivitiesNo diversityn n Chosen transit providers all reached over same satelliteor same submarine cableChosen transit providers have poor onward transit andpeering
Peersp p A peer is another autonomous system with whichthe local network has agreed to exchange locallysourced routes and trafficPrivate peern p Public peern p Private link between two providers for the purpose ofinterconnectingInternet Exchange Point, where providers meet andfreely decide who they will interconnect withRecommendation: peer as much as possible!
Common MistakesMistaking a transit provider’s “Exchange”business for a no-cost public peering pointp Not working hard to get as much peeringas possiblep n n p Physically near a peering point (IXP) but notpresent at it(Transit is rarely cheaper than peering!!)Ignoring/avoiding competitors becausethey are competitionn Even though potentially valuable peeringpartner to give customers a better experience
Private Interconnectionp Two service providers agree tointerconnect their networksn n They exchange prefixes they originate into therouting system (usually their aggregatedaddress blocks)They share the cost of the infrastructure tointerconnectp p n Typically each paying half the cost of the link (be itcircuit, satellite, microwave, fibre, )Connected to their respective peering routersPeering routers only carry domestic prefixes38
Private InterconnectionUpstreamUpstreamISP2PRPRISP1p PR peering routern n n n p Runs iBGP (internal) and eBGP (with peer)No default routeNo “full BGP table”Domestic prefixes onlyPeering router used for all private interconnects39
Public Interconnectionp Service provider participates in anInternet Exchange Pointn n It exchanges prefixes it originates into therouting system with the participants of the IXPIt chooses who to peer with at the IXPp p n n Bi-lateral peering (like private interconnect)Multi-lateral peering (via IXP’s route server)It provides the router at the IXP and providesthe connectivity from their PoP to the IXPThe IXP router carries only domestic prefixes40
Public 3-PRp ISP1ISP2-PRISP1-PR peering router of our ISPn n n n p ISP1-PRRuns iBGP (internal) and eBGP (with IXP peers)No default routeNo “full BGP table”Domestic prefixes onlyPhysically located at the IXP41
Public Interconnectionp The ISP’s router IXP peering router needs carefulconfiguration:n n n n It is remote from the domestic backboneShould not originate any domestic prefixes(As well as no default route, no full BGP table)Filtering of BGP announcements from IXP peers (in andout)42
Upstream/Transit Connectionp Two scenarios:n Transit provider is in the localityp n Transit provider is a long distance awayp p Which means bandwidth is cheap, plentiful, easy toprovision, and easily upgradedOver undersea cable, satellite, long-haul crosscountry fibre, etcBoth scenarios have differentrequirements which need to be considered43
Local Transit ProviderISP1ARBRTransitp BR ISP’s Border Routern n n n Runs iBGP (internal) and eBGP (with transit)Either receives default route or the full BGP table fromupstreamBGP policies are implemented here (depending onconnectivity)Packet filtering is implemented here (as required)44
Distant Transit ProviderAR1TransitBRISP1AR2p BR ISP’s Border Routern n n n n Co-located in a co-lo centre (typical) or in the upstream provider’spremisesRuns iBGP with rest of ISP1 backboneRuns eBGP with transit provider router(s)Implements BGP policies, packet filtering, etcDoes not originate any domestic prefixes45
Distant Transit Providerp Positioning a router close to the TransitProvider’s infrastructure is stronglyencouraged:n n n Long haul circuits are expensive, so the routerallows the ISP to implement appropriatefiltering firstMoves the buffering problem away from theTransit providerRemote co-lo allows the ISP to choose anothertransit provider and migrate connections withminimum downtime46
Distant Transit Providerp Other points to consider:n n n n Does require remote hands support(Remote hands would plug or unplug cables,power cycle equipment, replace equipment, etcas instructed)Appropriate support contract from equipmentvendor(s)Sensible to consider two routers and two longhaul links for redundancy47
Summaryp Design considerations for:n Private interconnectsp n Public interconnectsp n Router co-lo at an IXPLocal transit providerp n Simple private peeringSimple upstream interconnectLong distance transit providerp Router remote co-lo at datacentre or Transitpremises48
Addressing49
Getting IPv4 & IPv6 address spacep Take part of upstream ISP’s PA spaceorp Become a member of your Regional InternetRegistry and get your own allocationn n p Require a plan for a year aheadGeneral policies are outlined in RFC2050, morespecific details are on the individual RIR websiteThere is no more IPv4 address space at IANAn n n APNIC & RIPE NCC are now in their “final /8” IPv4delegation policy phaseLimited IPv4 availableIPv6 allocations are simple to get in most RIR regions50
What about RFC1918 addressing?p RFC1918 defines IPv4 addresses reserved forprivate Internetsn n p Commonly used within end-user networksn n p Not to be used on Internet backboneshttp://www.ietf.org/rfc/rfc1918.txtNAT used to translate from private internal to publicexternal addressingAllows the end-user network to migrate ISPs without amajor internal renumbering exerciseISPs must filter RFC1918 addressing at theirnetwork edgen http://www.cymru.com/Documents/bogonlist.html51
What about RFC1918 addressing?p There is a long list of well known problems:n p :n n n n n n n n n False belief it conserves address spaceAdverse effects on TracerouteEffects on Path MTU DiscoveryUnexpected interactions with some NAT implementationsInteractions with edge anti-spoofing techniquesPeering using loopbacksAdverse DNS InteractionSerious Operational and Troubleshooting issuesSecurity Issuesp false sense of security, defeating existing securitytechniques52
What about RFC1918 addressing?p Infrastructure Security: not improved by usingprivate addressingn p Troubleshooting: made an order of magnitudehardern n p Still can be attacked from inside, or from customers, orby reflection techniques from the outsideNo Internet view from routersOther ISPs cannot distinguish between down and brokenSummary:n ALWAYS use globally routable IP addressing for ISPInfrastructure53
Addressing Plans – ISPInfrastructurep p Address block for router loop-back interfacesAddress block for infrastructuren n n p Per PoP or whole backboneSummarise between sites if it makes senseAllocate according to genuine requirements, not historicclassful boundariesSimilar allocation policies should be used for IPv6as welln ISPs just get a substantially larger block (relatively) soassignments within the backbone are easier to make54
Addressing Plans – CustomerCustomers are assigned address spaceaccording to needp Should not be reserved or assigned on aper PoP basisp n n ISP iBGP carries customer netsAggregation not required and usually notdesirable55
Addressing Plans (contd)p Document infrastructure allocationn p Eases operation, debugging and managementDocument customer allocationn n n Contained in iBGPEases operation, debugging and managementSubmit network object to RIR Database56
Routing Protocols57
Routing Protocolsp IGP – Interior Gateway Protocoln n p EGP – Exterior Gateway Protocoln n p Carries infrastructure addresses, point-to-pointlinksExamples are OSPF, ISIS,.Carries customer prefixes and Internet routesCurrent EGP is BGP version 4No connection between IGP and EGP58
Why Do We Need an IGP?p ISP backbone scalingn n n n HierarchyModular infrastructure constructionLimiting scope of failureHealing of infrastructure faults using dynamicrouting with fast convergence59
Why Do We Need an EGP?p Scaling to large networkn n p HierarchyLimit scope of failurePolicyn n n Control reachability to prefixesMerge separate organizationsConnect multiple IGPs60
Interior versus Exterior RoutingProtocolsp Interiorn n n n Automatic neighbourdiscoveryGenerally trust your IGProutersPrefixes go to all IGProutersBinds routers in one AStogetherp Exteriorn n n n Specifically configuredpeersConnecting with outsidenetworksSet administrativeboundariesBinds AS’s together61
Interior versus Exterior RoutingProtocolsp Interiorn n Carries ISPinfrastructure addressesonlyISPs aim to keep theIGP small for efficiencyand scalabilityp Exteriorn n n Carries customerprefixesCarries InternetprefixesEGPs are independentof ISP network topology62
Hierarchy of Routing ProtocolsOther ISPsBGP4BGP4and OSPF/ISISBGP4IXPStatic/BGP4Customers63
Routing Protocols:Choosing an IGPp p OSPF and ISIS have very similar propertiesWhich to choose?n n n n n n Choose which is appropriate for your operators’experienceIn most vendor releases, both OSPF and ISIS havesufficient “nerd knobs” to tweak the IGP’s behaviourOSPF runs on IPISIS runs on infrastructure, alongside IPISIS supports both IPv4 and IPv6OSPFv2 (IPv4) plus OSPFv3 (IPv6)64
Routing Protocols:IGP Recommendationsp Keep the IGP routing table as small as possiblen p If you can count the routers and the point-to-point linksin the backbone, that total is the number of IGP entriesyou should seeIGP details:n n n Should only have router loopbacks, backbone WANpoint-to-point link addresses, and network addresses ofany LANs having an IGP running on themStrongly recommended to use inter-routerauthenticationUse inter-area summarisation if possible65
Routing Protocols:More IGP recommendationsp To fine tune IGP table size more, consider:n Using “ip unnumbered” on customer point-topoint links – saves carrying that /30 in IGPp n n n (If customer point-to-point /30 is required formonitoring purposes, then put this in iBGP)Use contiguous addresses for backbone WANlinks in each area – then summarise intobackbone areaDon’t summarise router loopback addresses –as iBGP needs those (for next-hop)Use iBGP for carrying anything which does notcontribute to the IGP Routing process66
Routing Protocols:iBGP Recommendationsp iBGP should carry everything whichdoesn’t contribute to the IGP routingprocessn n n n Internet routing tableCustomer assigned addressesCustomer point-to-point linksAccess network dynamic address pools,passive LANs, etc67
Routing Protocols:More iBGP Recommendationsp Scalable iBGP features:n n n n Use neighbour authenticationUse peer-groups to speed update process andfor configuration efficiencyUse communities for ease of filteringUse route-reflector hierarchyp Route reflector pair per PoP (overlaid clusters)68
Security69
Securityp p p p ISP Infrastructure securityISP Network securitySecurity is not optional!ISPs need to:n n n p Protect themselvesHelp protect their customers from the InternetProtect the Internet from their customersThe following slides are general recommendationsn Do more research on security before deploying anynetwork70
ISP Infrastructure Securityp Router & Switch Securityn Use Secure Shell (SSH) for device access &managementp n Device management access filters should onlyallow NOC and device-to-device accessp n Do NOT use TelnetDo NOT allow external accessUse TACACS for user authentication andauthorisationp Do NOT create user accounts on routers/switches71
ISP Infrastructure Securityp Remote accessn n For Operations Engineers who need accesswhile not in the NOCCreate an SSH server host (this is all it does)p n Or a Secure VPN access serverOps Engineers connect here, and then they canaccess the NOC and network devices72
ISP Infrastructure Securityp Other network devices?n n p Servers and Services?n n p These probably do not have sophisticated securitytechniques like routers or switches doProtect them at the LAN or point-to-point ingress (onrouter)Protect servers on the LAN interface on the routerConsider using iptables &c on the servers tooSNMPn n Apply access-list to the SNMP portsShould only be accessible by management system, notthe world73
ISP Infrastructure Securityp General Advice:n n Routers, Switches and other network devicesshould not be contactable from outside the ASAchieved by blocking typical managementaccess protocols for the infrastructure addressblock at the network perimeterp n Use the ICSI Netalyser to check access levels:p n E.g. ssh, telnet, http, snmp, http://netalyzr.icsi.berkeley.eduDon’t block everything: BGP, tracerouteand ICMP still need to work!74
ISP Network Securityp Effective filteringn Protect network borders from “traffic whichshould not be on the public Internet”, forexample:p p p n LAN protocols (eg netbios)Well known exploit ports (used by worms andviruses)Drop traffic arriving and going to private and nonroutable address space (IPv4 and IPv6)Achieved by packet filters on border routersp Remote trigger blackhole filtering75
ISP Network Security – RTBFp Remote trigger blackhole filteringn n n n p ISP NOC injects prefixes which should not be accessibleacross the AS into the iBGPPrefixes have next hop pointing to a blackhole addressAll iBGP speaking backbone routers configured to pointthe blackhole address to the null interfaceTraffic destined to these blackhole prefixes are droppedby the first router they reachApplication:n Any prefixes (including RFC1918) which should not haveroutability across the ISP backbone76
ISP Network Security – RTBFp Remote trigger blackhole filtering example:n Origin router:router bgp 64509redistribute static route-map black-hole-trigger!ip route 10.5.1.3 255.255.255.255 Null0 tag 66!route-map black-hole-trigger permit 10match tag 66set local-preference 1000set community no-exportset ip next-hop 192.0.2.1!n iBGP speaking backbone router:ip route 192.0.2.1 255.255.255.255 null077
ISP Network Security – RTBFp Resulting routing table entries:gw1#sh ip bgp 10.5.1.3BGP routing table entry for 10.5.1.3/32, version 64572219Paths: (1 available, best #1, table Default-IP-Routing-Table)Not advertised to any peerLocal192.0.2.1 from 1.1.10.10 (1.1.10.10)Origin IGP, metric 0, localpref 1000, valid, internal, bestCommunity: no-exportgw1#sh ip route 10.5.1.3Routing entry for 10.5.1.3/32Known via "bgp 64509", distance 200, metric 0, type internalLast update from 192.0.2.1 00:04:52 agoRouting Descriptor Blocks:* 192.0.2.1, from 1.1.10.10, 00:04:52 agoRoute metric is 0, traffic share count is 1AS Hops 078
ISP Network Security – uRPFp p Unicast Reverse Path ForwardingStrongly recommended to be used on allcustomer facing static interfacesn n n p BCP 38 (tools.ietf.org/html/bcp38)Blocks all unroutable source addresses thecustomer may be usingInexpensive way of filtering customer’s connection(when compared with packet filters)Can be used for multihomed connections too, butextreme care required79
What is uRPF?FIB:172.16.1.0/24 fa0/0192.168.1.0/24 se0/1src 172.16.1.1src 192.168.1.1p fa0/0se0/1routerRouter compares source address of incomingpacket with FIB entryn n If FIB entry interface matches incoming interface, thepacket is forwardedIf FIB entry interface does not match incoming interface,80the packet is dropped
Security Summaryp Implement RTBFn n Inside ISP backboneMake it available to BGP customers toop p p Implement uRPFn p p They can send you the prefix you need to block with aspecial community attachedYou match on that community, and set the next-hop to thenull addressFor all static customersUse SSH for device accessUse TACACS for authentication81
Out of Band Management82
Out of Band ManagementNot optional!p Allows access to network equipment intimes of failurep Ensures quality of service to customersp n n n Minimises downtimeMinimises repair timeEases diagnostics and debugging83
Out of Band Managementp OoB Example – Access server:n n n p modem attached to allow NOC dial inconsole ports of all network equipmentconnected to serial portsLAN and/or WAN link connects to networkcore, or via separate management link to NOCFull remote control access under allcircumstances84
Out of Band NetworkEquipment RackEquipment RackRouter, switchand ISP serverconsoles(Optional) Out of bandWAN link to other PoPsModem – accessto PSTN for out ofband dialinEthernet85to the NOC
Out of Band Managementp OoB Example – Statistics gathering:n n n p Routers are NetFlow and syslog enabledManagement data is congestion/failuresensitiveEnsures management data integrity in case offailureFull remote information under allcircumstances86
Test Laboratory87
Test Laboratoryp Designed to look like a typical PoPn Operated like a typical PoPUsed to trial new services or new softwareunder realistic conditionsp Allows discovery and fixing of potentialproblems before they are introduced tothe networkp 88
Test LaboratorySome ISPs dedicate equipment to the labp Other ISPs “purchase ahead” so thattoday’s lab equipment becomestomorrow’s PoP equipmentp Other ISPs use lab equipment for “hotspares” in the event of hardware failurep 89
Test Laboratoryp Can’t afford a test lab?n n p Set aside one spare router and server to trialnew servicesNever ever try out new hardware, software orservices on the live networkEvery major ISP in the US and Europe hasa test labn It’s a serious consideration90
OperationalConsiderations91
Operational ConsiderationsWhy design the world’s bestnetwork when you have notthought about what operationalgood practices should beimplemented?92
Operational ConsiderationsMaintenancep Never work on the live network, no matter howtrivial the modification may seemn Establish maintenance periods which your customers areaware ofp p Never do maintenance on the last working daybefore the weekendn p e.g. Tuesday 4-7am, Thursday 4-7amUnless you want to work all weekend cleaning upNever do maintenance on the first working dayafter the weekendn Unless you want to work all weekend preparing93
Operational ConsiderationsSupportp Differentiate between customer supportand the Network Operations Centren n p Customer support fixes customer problemsNOC deals with and fixes backbone andInternet related problemsNetwork Engineering team is last resortn n They design the next generation network,improve the routing design, implement newservices, etcThey do not and should not be doing support!94
Operational ConsiderationsNOC Communicationsp NOC should know contact details forequivalent NOCs in upstream providersand peers95
ISP Network DesignSummary96
ISP Design SummaryKEEP IT SIMPLE & STUPID ! (KISS)p Simple is elegant is scalablep Use Redundancy, Security, andTechnology to make life easier for yourselfp Above all, ensure quality of service foryour customersp 97
Why an InternetExchange Point?Saving money, improving QoS,encouraging a local Interneteconomy98
Internet Exchange PointWhy peer?p Consider a region with one ISPn n p Internet grows, another ISP sets up incompetitionn n p They provide internet connectivity to their customersThey have one or two international connectionsThey provide internet connectivity to their customersThey have one or two international connectionsHow does traffic from customer of one ISP get tocustomer of the other ISP?n Via the international connections99
Internet Exchange PointWhy peer?p Yes, International Connections n n p If satellite, RTT is around 550ms per hopSo local traffic takes over 1s round tripInternational bandwidthn n n Costs significantly more than domesticbandwidthCongested with local trafficWastes money, harms performance100
Internet Exchange PointWhy peer?p Solution:n p Two competing ISPs peer with each otherResult:n n n n n Both save moneyLocal traffic stays localBetter network performance, better QoS, More international bandwidth for expensiveinternational trafficEveryone is happy101
Internet Exchange PointWhy peer?p A third ISP enters the equationn n p Becomes a significant player in the regionLocal and international traffic goes over theirinternational connectionsThey agree to peer with the two otherISPsn n n To save moneyTo keep local traffic localTo improve network performance, QoS, 102
Internet Exchange PointWhy peer?p Peering means that the three ISPs have tobuy circuits between each othern p Works for three ISPs, but adding a fourth or afifth means this does not scaleSolution:n Internet Exchange Point103
Internet Exchange Pointp Every participant has to buy just onewhole circuitn p From their premises to the IXPRather than N-1 half circuits to connect tothe N-1 other ISPsn 5 ISPs have to buy 4 half circuits 2 wholecircuits already twice the cost of the IXPconnection104
Internet Exchange Pointp Solutionn n n p Every ISP participates in the IXPCost is minimal – one local circuit covers all domestictrafficInternational circuits are used for just internationaltraffic – and backing up domestic links in case the IXPfailsResult:n n n n n Local traffic stays localQoS considerations for local traffic is not an issueRTTs are typically sub 10msCustomers enjoy the Internet experienceLocal Internet economy grows rapidly105
Exchange Point Design106
IXP Designp Very simple concept:n Ethernet switch is the interconnection mediap n n p IXP is one LANEach ISP brings a router, connects it to theethernet switch provided at the IXPEach ISP peers with other participants at theIXP using BGPScaling this simple concept is thechallenge for the larger IXPs107
Layer 2 ExchangeISP 6ISP 5ISP 4IXP Services:Root & TLD DNS,Routing RegistryEthernet SwitchLooking Glass, etcISP 1ISP 2IXPManagementNetworkISP 3108
Layer 2 ExchangeISP 6ISP 5ISP 4IXP Services:IXPManagementNetworkRoot & TLD DNS,Routing RegistryEthernet SwitchesLooking Glass, etcISP 1ISP 2ISP 3109
Layer 2 ExchangeTwo switches for redundancyp ISPs use dual routers for redundancy orloadsharingp Offer services for the “common good”p n n n Internet portals and search enginesDNS Root & TLD, NTP serversRouting Registry and Looking Glass110
Layer 2 Exchangep Requires neutral IXP managementn n Usually funded equally by IXP participants24x7 cover, support, value add servicesSecure and neutral locationp Configurationp n n IPv4 /24 and IPv6 /64 for IXP LANISPs require AS, basic IXP does not111
Layer 2 Exchangep Network Security Considerationsn n n LAN switch needs to be securely configuredManagement routers require TACACS authentication, vty securityIXP services must be behind router(s) wi
ISP & IXP Design Philip Smith MENOG 11 Amman 30th September - 9th October 2012 1 . ISP & IXP Network Design ! PoP Topologies and Design ! . Distributed Network Design 26 POP One POP Two POP Three Customer connections Customer connections Customer connections External connections External connections Operations Centre
MENOG13-IXP-Network-Design.pdf " And on the MENOG 13 website ! Feel free to ask questions any time . IXP Design ! Background ! Why set up an IXP? ! Layer 2 Exchange Point . Network ISP 6 ISP 5 ISP 4 Ethernet Switch IXP Services: Root & TLD DNS, Routing Registry Looking Glass, etc . Layer 2 Exchange 22 ISP 1 ISP 2 ISP 3 IXP
ISP border routers at the IXP must NOT be configured with a default route or carry the full Internet routing table " Carrying default or full table means that this router and the ISP network is open to abuse by non-peering IXP members " Correct configuration is only to carry routes offered to IXP peers on the IXP peering router !
Internet Exchange Point Design IXP Technical Design, Technical Resources and Value Added Services 1. IXP Design Layer 2 Exchange Point . -Carrying default or full table means that this router and the ISP network is open to abuse by non-peering IXP members -Correct configuration is only to carry routes offered to IXP peers on
PSI AP Physics 1 Name_ Multiple Choice 1. Two&sound&sources&S 1∧&S p;Hz&and250&Hz.&Whenwe& esult&is:& (A) great&&&&&(C)&The&same&&&&&
ISP & IXP Network Design ! PoP Topologies and Design ! Backbone Design ! Upstream Connectivity & Peering ! Addressing ! Routing Protocols ! Out of Band Management ! Operational Considerations ! Internet Exchange Points 2 . Point of Presence Topologies 3 . PoP Topologies ! .
Argilla Almond&David Arrivederci&ragazzi Malle&L. Artemis&Fowl ColferD. Ascoltail&mio&cuore Pitzorno&B. ASSASSINATION Sgardoli&G. Auschwitzero&il&numero&220545 AveyD. di&mare Salgari&E. Avventurain&Egitto Pederiali&G. Avventure&di&storie AA.&VV. Baby&sitter&blues Murail&Marie]Aude Bambini&di&farina FineAnna
other providers around the world. He concentrates specifically on network strategies, design, technology, and operations, as well as configuration, scaling, and training. He plays or has played a major role in training ISP engineers, co-founding the Cisco ISP/IXP Workshop programme, and providing ISP training and tutorials at many
Perfusionists certified by the American Board of Cardiovascular Perfusion through December 31, 2021. LAST FIRST CITY STATE COUNTRY Al-Marhoun Sarah New Orleans LA Alouidor Benjamin Los Angeles CA Alpert Bettina P. Marlborough MA Alpha Debra Reynolds Zionville IN Alshi Hanin Nooraldin H. Jeddah MA Saudi Arabia
