[prev in list] [next in list] [prev in thread] [next in thread] 

List:       outages-discussion
Subject:    Re: [Outages-discussion] [outages] CenturyLink peering issues?
From:       Charles Sprickman <spork () bway ! net>
Date:       2020-08-30 17:06:27
Message-ID: 01613869-3A51-4EEA-BA50-9F01336B4E86 () bway ! net
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]


(on -discuss)

Two things:

My apologies for adding to the noise this morning, I'm pretty sure nether.net \
<http://nether.net/> was someone still trying to reach us via our L3 connection even \
after I turned it down due to what Stephen mentions here.

Second thing, someone better versed in the intricacies of BGP better than I am, can \
you explain a bit about how we end up in a situation where L3 continues advertising \
withdrawn routes for more than an hour?

Thanks,

Charles

> On Aug 30, 2020, at 7:25 AM, Stephen Flynn via Outages <outages@outages.org> wrote:
> 
> Confirmed Level3 carrier issue from the Orlando region --- possibly my other data \
> centers as well (haven't checked them yet, been focusing on my ORL-FL outage). 
> Odd part --- I disconnected my Level3 circuit at the ORL-FL facility so that I \
> could fully failover to my other carrier link. Level3 is still advertising my \
> routes, even though my link and BGP session is down. 
> er-02.orl-fl> show interfaces descriptions | match Level3 
> xe-2/0/0        down  down LEVEL3 |
> 
> 
> er-02.orl-fl> show bgp summary | match 3356 
> 67.30.142.81           3356          0          0       0       0       27:43 \
> Active 2001:1900:2100::26ad        3356          0          0       0       0       \
> 27:43 Idle 
> 
> BGP routing table entry for 209.208.0.0/17
> Versions:
> Process           bRIB/RIB  SendTblVer
> Speaker           86180864    86180864
> Last Modified: May 16 05:47:52.095 for 15w1d
> Paths: (25 available, best #25)
> Advertised IPv4 Unicast paths to update-groups (with more than one peer):
> 0.1 0.2 0.8 0.9 0.10 
> Path #1: Received by speaker 0
> Not advertised to any peer
> 3356 6364
> 129.250.8.174 (metric 9340) from 129.250.0.2 (129.250.0.2)
> Origin IGP, metric 4294967294, localpref 100, valid, confed-internal
> Received Path ID 0, Local Path ID 0, version 0
> Community: 2914:390 2914:1003 2914:2000 2914:3000 65504:3356
> Path #2: Received by speaker 0
> Not advertised to any peer
> 3356 6364
> 129.250.9.50 (metric 24142) from 129.250.0.6 (129.250.0.6)
> Origin IGP, metric 4294967294, localpref 100, valid, confed-internal
> Received Path ID 0, Local Path ID 0, version 0
> Community: 2914:390 2914:1214 2914:2213 2914:3200 65504:3356
> Path #3: Received by speaker 0
> Not advertised to any peer
> 174 6364
> 130.117.15.89 (metric 24174) from 129.250.0.13 (129.250.0.13)
> Origin IGP, metric 4294967294, localpref 100, valid, confed-internal
> Received Path ID 0, Local Path ID 0, version 0
> Community: 2914:390 2914:1204 2914:2205 2914:3200 65504:174
> Path #4: Received by speaker 0
> 
> 
> 
> Regards,
> 
> Stephen Flynn
> Atlantic.Net
> Direct: (321) 206-1390
> sflynn@staff.atlantic.net
> www.atlantic.net
> 
> 
> -----Original Message-----
> From: Outages <outages-bounces@outages.org> On Behalf Of randal k via Outages
> Sent: Sunday, August 30, 2020 7:10 AM
> To: outages@outages.org
> Subject: [outages] CenturyLink peering issues?
> 
> 
> CAUTION: This email originated from outside of the organization. Do not click links \
> or open attachments unless you recognize the sender and know the content is safe. 
> 
> We have been seeing serious issues with Level3/CenturyLink services over the past \
> hour or so. Everything on-net seems OK, but transit traffic looks to have problems. \
> Thoughts? _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages


[Attachment #5 (unknown)]

<html><head><meta http-equiv="Content-Type" content="text/html; \
charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; \
line-break: after-white-space;" class="">(on -discuss)<div class=""><br \
class=""></div><div class="">Two things:</div><div class=""><br class=""></div><div \
class="">My apologies for adding to the noise this morning, I'm pretty sure <a \
href="http://nether.net" class="">nether.net</a>&nbsp;was someone still trying to \
reach us via our L3 connection even after I turned it down due to what Stephen \
mentions here.</div><div class=""><br class=""></div><div class="">Second thing, \
someone better versed in the intricacies of BGP better than I am, can you explain a \
bit about how we end up in a situation where L3 continues advertising withdrawn \
routes for more than an hour?</div><div class=""><br class=""></div><div \
class="">Thanks,</div><div class=""><br class=""></div><div class="">Charles<br \
class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Aug 30, \
2020, at 7:25 AM, Stephen Flynn via Outages &lt;<a href="mailto:outages@outages.org" \
class="">outages@outages.org</a>&gt; wrote:</div><br \
class="Apple-interchange-newline"><div class=""><div class="">Confirmed Level3 \
carrier issue from the Orlando region --- possibly my other data centers as well \
(haven't checked them yet, been focusing on my ORL-FL outage).<br class=""><br \
class="">Odd part --- I disconnected my Level3 circuit at the ORL-FL facility so that \
I could fully failover to my other carrier link.<br class="">Level3 is still \
advertising my routes, even though my link and BGP session is down.<br class=""><br \
class="">er-02.orl-fl&gt; show interfaces descriptions | match Level3 <br \
class="">xe-2/0/0 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;down &nbsp;down LEVEL3 \
|<br class=""><br class=""><br class="">er-02.orl-fl&gt; show bgp summary | match \
3356 <br class="">67.30.142.81 \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3356 \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0 \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0 \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0 \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;27:43 Active<br class="">2001:1900:2100::26ad \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3356 \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0 \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0 \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0 \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;27:43 Idle<br class=""><br class=""><br \
class="">BGP routing table entry for 209.208.0.0/17<br class="">Versions:<br \
class=""> &nbsp;Process \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;bRIB/RIB \
&nbsp;SendTblVer<br class=""> &nbsp;Speaker \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;86180864 \
&nbsp;&nbsp;&nbsp;86180864<br class="">Last Modified: May 16 05:47:52.095 for \
15w1d<br class="">Paths: (25 available, best #25)<br class=""> &nbsp;Advertised IPv4 \
Unicast paths to update-groups (with more than one peer):<br class=""> \
&nbsp;&nbsp;&nbsp;0.1 0.2 0.8 0.9 0.10 <br class=""> &nbsp;Path #1: Received by \
speaker 0<br class=""> &nbsp;Not advertised to any peer<br class=""> &nbsp;3356 \
6364<br class=""> &nbsp;&nbsp;&nbsp;129.250.8.174 (metric 9340) from 129.250.0.2 \
(129.250.0.2)<br class=""> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Origin IGP, metric \
4294967294, localpref 100, valid, confed-internal<br class=""> \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Received Path ID 0, Local Path ID 0, version 0<br \
class=""> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Community: 2914:390 2914:1003 2914:2000 \
2914:3000 65504:3356<br class=""> &nbsp;Path #2: Received by speaker 0<br class=""> \
&nbsp;Not advertised to any peer<br class=""> &nbsp;3356 6364<br class=""> \
&nbsp;&nbsp;&nbsp;129.250.9.50 (metric 24142) from 129.250.0.6 (129.250.0.6)<br \
class=""> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Origin IGP, metric 4294967294, localpref 100, \
valid, confed-internal<br class=""> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Received Path ID 0, \
Local Path ID 0, version 0<br class=""> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Community: \
2914:390 2914:1214 2914:2213 2914:3200 65504:3356<br class=""> &nbsp;Path #3: \
Received by speaker 0<br class=""> &nbsp;Not advertised to any peer<br class=""> \
&nbsp;174 6364<br class=""> &nbsp;&nbsp;&nbsp;130.117.15.89 (metric 24174) from \
129.250.0.13 (129.250.0.13)<br class=""> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Origin IGP, \
metric 4294967294, localpref 100, valid, confed-internal<br class=""> \
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Received Path ID 0, Local Path ID 0, version 0<br \
class=""> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Community: 2914:390 2914:1204 2914:2205 \
2914:3200 65504:174<br class=""> &nbsp;Path #4: Received by speaker 0<br class=""><br \
class=""><br class=""><br class="">Regards,<br class=""><br class="">Stephen Flynn<br \
class=""><a href="http://Atlantic.Net" class="">Atlantic.Net</a><br class="">Direct: \
(321) 206-1390<br class="">sflynn@staff.atlantic.net<br class="">www.atlantic.net<br \
class=""><br class=""><br class="">-----Original Message-----<br class="">From: \
Outages &lt;outages-bounces@outages.org&gt; On Behalf Of randal k via Outages<br \
class="">Sent: Sunday, August 30, 2020 7:10 AM<br class="">To: outages@outages.org<br \
class="">Subject: [outages] CenturyLink peering issues?<br class=""><br class=""><br \
class="">CAUTION: This email originated from outside of the organization. Do not \
click links or open attachments unless you recognize the sender and know the content \
is safe.<br class=""><br class=""><br class="">We have been seeing serious issues \
with Level3/CenturyLink services over the past hour or so. Everything on-net seems \
OK, but transit traffic looks to have problems. Thoughts?<br \
class="">_______________________________________________<br class="">Outages mailing \
list<br class="">Outages@outages.org<br \
class="">https://puck.nether.net/mailman/listinfo/outages<br \
class="">_______________________________________________<br class="">Outages mailing \
list<br class="">Outages@outages.org<br \
class="">https://puck.nether.net/mailman/listinfo/outages<br \
class=""></div></div></blockquote></div><br class=""></div></body></html>



_______________________________________________
Outages-discussion mailing list
Outages-discussion@outages.org
https://puck.nether.net/mailman/listinfo/outages-discussion


[prev in list] [next in list] [prev in thread] [next in thread] 

Configure | About | News | Add a list | Sponsored by KoreLogic