[prev in list] [next in list] [prev in thread] [next in thread]
List: outages-discussion
Subject: Re: [Outages-discussion] [outages] Lightower/Sidera - NYC
From: "Frank Bulk" <frnkblk () iname ! com>
Date: 2013-11-08 13:37:00
Message-ID: 003e01cedc87$9a659f90$cf30deb0$ () iname ! com
[Download RAW message or body]
This is a multipart message in MIME format.
[Attachment #2 (multipart/alternative)]
This is a multipart message in MIME format.
This quote caught my attention: "Unknown to Coulson, Lightower had bought
Sidera Networks, another high-speed network provider, in April. Sidera was
another of OTC's providers, and when one network went down the other had
issues as well."
This is where organizations with critical services need to write into their
contract the requirement for notification of all business
mergers/acquisitions, etc, and also require annual certifications that the
primary and secondary links are physically and logical diverse, along with
credits or penalties if it's discovered otherwise.
Frank
From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Jared Geiger
Sent: Thursday, November 07, 2013 10:40 PM
Cc: outages@outages.org
Subject: Re: [outages] Lightower/Sidera - NYC
Looks like it stopped OTC market trading for a bit.
http://www.cnbc.com/id/101181346
On Thu, Nov 7, 2013 at 2:54 PM, <michael.mccoy@bt.com
<mailto:michael.mccoy@bt.com> > wrote:
Thanks James. I was told of some sort of packet storm causing the outage on
their MPLS network. Maybe these BPDUs have spanning tree going haywire.
Maybe they had to isolate some PEs from the network and all of us are on
those PEs?
-----Original Message-----
From: Outages [mailto:outages-bounces@outages.org
<mailto:outages-bounces@outages.org> ] On Behalf Of James Cornman
Sent: Thursday, November 07, 2013 5:35 PM
To: outages@outages.org <mailto:outages@outages.org>
Subject: Re: [outages] Lightower/Sidera - NYC
This outage is still on-going. I spoke with an executive at LT earlier today
and the word then was that some downstream customer of the Sidera/ConEd/RCN
legacy network is leaking PVST BPDUs and its wreaking havoc on their
network. I'm not sure how legitimate that is anymore since its been several
hours. Apparently though, since our circuits are fluctuating up and down
every few minutes, it would seem they're not very close to resolution. We're
approaching the 12 hour mark for certain circuits.
They also don't seem to have any tools to notify/communicate with their
customers about the on-going efforts..and they don't tweet anything but
marketing crap. The management of this incident is asinine.
-James
On Thu, Nov 7, 2013 at 4:54 PM, Tim Durack <tdurack@gmail.com
<mailto:tdurack@gmail.com> > wrote:
> As another datapoint, the traditional HV/Datanet/Lightower side of the
> network has been stable during this reported outage.
>
>
> On Thu, Nov 7, 2013 at 4:45 PM, Charles Sprickman <spork@bway.net
<mailto:spork@bway.net> > wrote:
>>
>> On Nov 7, 2013, at 4:29 PM, Vincent Aniello wrote:
>>
>> Our circuits stabilized mid-afternoon, but shortly after 4pm EST we
>> started seeing issues again.
>>
>>
>> Confirmed here as well, 7 out of 7 L2 tail circuits not passing
>> traffic after a good hour of instability.
>>
>> FWIW, I don't recall any recent maintenance announcements from them,
>> so this is a bit troubling (I'd expect this if we saw a big rollout
>> of new code the night previous) since it came out of the blue. I can
>> also confirm we saw our first outage much earlier than they
>> acknowledged, one circuit lost connectivity at 5:23 a.m. I can also
>> confirm that we did not see the physical layer go down once on any of
>> our circuits, this seems to be occurring in the core, not the rings that
actually service the buildings.
>>
>> We're legacy ConEd/RCN/Sidera, have no idea if this is happening with
>> the LT network or not.
>>
>> We got what appears to be a bogus "all restored" email at 2:50 p.m.
>>
>> Charles
>>
>> --Vincent
>>
>>
>>
>> From: Karl Clapp <kclapp@staff.gwi.net
<mailto:kclapp@staff.gwi.net> >
>> To: "outages@outages.org <mailto:outages@outages.org> "
<outages@outages.org <mailto:outages@outages.org> >
>> Date: 11/07/2013 03:23 PM
>> Subject: Re: [outages] Lightower/Sidera - NYC
>> Sent by: "Outages" <outages-bounces@outages.org
<mailto:outages-bounces@outages.org> >
>> ________________________________
>>
>>
>>
>> I contacted their NOC about 45min ago. They have stated they are
>> still working on resolution. They have reports about some services
>> being functional, while other still experiencing packet-loss and
>> latency. It does not sound like they have been able to identify the
>> root cause and are still troubleshooting.
>>
>> Here is the number I used to get through to their NOC: 888-583-4237
<tel:888-583-4237> ,
>> option 1. From there, just follow the prompts.
>>
>>
>> Fwiw, my customer circuit from NYC to NJ is still affected.
>>
>>
>> On Thu, Nov 7, 2013 at 10:25 AM, John Kinsella <jlk@thrashyour.com
<mailto:jlk@thrashyour.com> > wrote:
>> Just as a data point, LT for me is routing into Equinix NY5 over HE,
>> so at least there's connectivity...
>>
>> My traceroute [v0.82]
>> icmp (0.0.0.0) Thu Nov 7
>> 07:24:09 2013
>> Keys: Help Display mode Restart statistics Order of fields quit
>> Packets Pings
>> Host Loss% Snt Last Avg
Best
>> Wrst StDev
>> 1. 10.0.1.1 0.0% 29 6.7 3.2
0.8
>> 12.2 2.7
>> 2. 10.0.1.49 0.0% 28 8.0 4.3
1.5
>> 13.1 3.0
>> 3. 10.0.24.73 0.0% 28 6.7 4.5
1.9
>> 14.7 3.4
>> 4. 25.66.208.web-pass.com <http://25.66.208.web-pass.com>
0.0% 28 1.9 11.2 1.9
>> 48.0 12.3
>> 5. 2-1.760.core.fre.web-pass.com <http://2-1.760.core.fre.web-pass.com>
0.0% 28 9.2 6.4 3.1
>> 19.7 4.0
>> 6. gige-g6-2.core1.fmt1.he.net <http://gige-g6-2.core1.fmt1.he.net>
0.0% 28 9.6 12.4 3.5
>> 26.4 6.2
>> 7. 10gigabitethernet1-2.core1.sjc2.he.net
<http://10gigabitethernet1-2.core1.sjc2.he.net> 0.0% 28 11.9 9.5
3.5
>> 18.7 4.3
>> 8. 10gigabitethernet9-1.core1.nyc4.he.net
<http://10gigabitethernet9-1.core1.nyc4.he.net> 0.0% 28 87.9 79.5
73.3
>> 92.7 5.5
>> 9. 10gigabitethernet4-1.core1.nyc5.he.net
<http://10gigabitethernet4-1.core1.nyc5.he.net> 0.0% 28 76.6 80.5
73.6
>> 106.7 7.0
>> 10. lightower-fiber-networks.10gigabitether 0.0% 28 78.0 82.3
74.4
>> 108.0 9.4
>> 11. xe-8-3-0.nycmnyzrj91.lightower.net
<http://xe-8-3-0.nycmnyzrj91.lightower.net> 0.0% 28 89.1 83.6
74.2
>> 144.7 15.8
>> 12. ae2-jrcynj67j41.lightower.net <http://ae2-jrcynj67j41.lightower.net>
0.0% 28 85.4 88.6 80.1
>> 131.5 12.1
>> 13. 192.154.116.14.lightower.net <http://192.154.116.14.lightower.net>
0.0% 28 85.6 83.9 80.2
>> 95.6 3.5
>> 14. ???
>>
>> John
>>
>> On Nov 7, 2013, at 7:19 AM, Karl Clapp <kclapp@staff.gwi.net
<mailto:kclapp@staff.gwi.net> > wrote:
>>
>> LightTower/Sidera master ticker for this is LT1323725.
>>
>> I was able to get through their support line now.
>>
>> The ETR is 1-2 hours from now.
>>
>> ---
>> Karl Clapp
>> Network Engineering Manager
>> Biddeford Internet Corp. d.b.a. GWI (AS5760)
>> Office: 207.602.1144 <tel:207.602.1144>
>> Mobile: 207.641.3907 <tel:207.641.3907>
>> Fax: 207.286.2061 <tel:207.286.2061>
>>
>>
>> On Thu, Nov 7, 2013 at 9:39 AM, <SRonan@eexchange.com
<mailto:SRonan@eexchange.com> > wrote:
>> Likely related to the fiber cut they experienced on their network
>> earlier this week which they routed around.
>>
>> Shane Ronan, Vice President - Technology Architect State Street
>> Global Exchange | 1230 Avenue of the Americas, 18th Fl | New York, NY
>> 10020
>> P (212) 259-3023 <tel:%28212%29%20259-3023> | M (347) 413-4503
<tel:%28347%29%20413-4503>
>> sronan@eexchange.com <mailto:sronan@eexchange.com>
>>
>> From: Jayson Navitsky <jnavitsky@gmail.com <mailto:jnavitsky@gmail.com> >
>> Date: Thursday, November 7, 2013 at 6:55 AM
>> To: "outages@outages.org <mailto:outages@outages.org> "
<outages@outages.org <mailto:outages@outages.org> >
>> Subject: [outages] Lightower/Sidera - NYC
>>
>> Anyone else seeing any issues in NYC with Lightower/Sidera? Starting
>> around 6:05AM (EST) we saw heavy packet loss across their network
>> (from Sidera legacy) and some of our point to points bounce. The P to
>> P's have some what stabilized (though we've seen a couple more
>> bounces since then) the IP side of things have been pretty much
>> unusable (we shut our session
>> down.) It also looks like their phone system is having a problem -
>> calls into them seem to drop after trying to connect to the NOC -
>> which I'm sure is pretty busy.
>>
>> -Jayson
>>
>> ________________________________
>>
>> http://www.statestreet.com/emaildisclaimer/
>>
>> _______________________________________________
>> Outages mailing list
>> Outages@outages.org <mailto:Outages@outages.org>
>> https://puck.nether.net/mailman/listinfo/outages
>>
>>
>> _______________________________________________
>> Outages mailing list
>> Outages@outages.org <mailto:Outages@outages.org>
>> https://puck.nether.net/mailman/listinfo/outages
>>
>>
>> _______________________________________________
>> Outages mailing list
>> Outages@outages.org <mailto:Outages@outages.org>
>> https://puck.nether.net/mailman/listinfo/outages
>>
>> _______________________________________________
>> Outages mailing list
>> Outages@outages.org <mailto:Outages@outages.org>
>> https://puck.nether.net/mailman/listinfo/outages
>>
>> _______________________________________________
>> Outages mailing list
>> Outages@outages.org <mailto:Outages@outages.org>
>> https://puck.nether.net/mailman/listinfo/outages
>>
>>
>>
>> _______________________________________________
>> Outages mailing list
>> Outages@outages.org <mailto:Outages@outages.org>
>> https://puck.nether.net/mailman/listinfo/outages
>>
>
>
>
> --
> Tim:>
>
> _______________________________________________
> Outages mailing list
> Outages@outages.org <mailto:Outages@outages.org>
> https://puck.nether.net/mailman/listinfo/outages
>
--
James Cornman
Chief Technology Officer
jcornman@atlanticmetro.net <mailto:jcornman@atlanticmetro.net>
212.792.9950 <tel:212.792.9950> - ext 101
Atlantic Metro Communications, Inc.
4 Century Drive, Parsippany NJ 07054
Data Centers * Managed Hosting * Internet Access * Cloud Computing Follow
@atlanticmetro on Twitter to learn more about daily specials.
www.atlanticmetro.net <http://www.atlanticmetro.net>
_______________________________________________
Outages mailing list
Outages@outages.org <mailto:Outages@outages.org>
https://puck.nether.net/mailman/listinfo/outages
_______________________________________________
Outages mailing list
Outages@outages.org <mailto:Outages@outages.org>
https://puck.nether.net/mailman/listinfo/outages
[Attachment #5 (text/html)]
<html xmlns:v="urn:schemas-microsoft-com:vml" \
xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:x="urn:schemas-microsoft-com:office:excel" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type \
content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 15 \
(filtered medium)"><style><!-- /* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div \
class=WordSection1><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>This quote \
caught my attention: “Unknown to Coulson, Lightower had bought Sidera Networks, \
another high-speed network provider, in April. Sidera was another of OTC's providers, \
and when one network went down the other had issues as \
well.”<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>This is \
where organizations with critical services need to write into their contract the \
requirement for notification of all business mergers/acquisitions, etc, and also \
require annual certifications that the primary and secondary links are physically and \
logical diverse, along with credits or penalties if it’s discovered \
otherwise.<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Frank<o:p></o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><b><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>From:</span></b><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> Outages \
[mailto:outages-bounces@outages.org] <b>On Behalf Of </b>Jared Geiger<br><b>Sent:</b> \
Thursday, November 07, 2013 10:40 PM<br><b>Cc:</b> \
outages@outages.org<br><b>Subject:</b> Re: [outages] Lightower/Sidera - \
NYC<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><div><p \
class=MsoNormal>Looks like it stopped OTC market trading for a bit. <a \
href="http://www.cnbc.com/id/101181346">http://www.cnbc.com/id/101181346</a><o:p></o:p></p></div><div><p \
class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p><div><p \
class=MsoNormal>On Thu, Nov 7, 2013 at 2:54 PM, <<a \
href="mailto:michael.mccoy@bt.com" target="_blank">michael.mccoy@bt.com</a>> \
wrote:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC \
1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><p \
class=MsoNormal>Thanks James. I was told of some sort of packet storm causing the \
outage on their MPLS network. Maybe these BPDUs have spanning tree going haywire. \
Maybe they had to isolate some PEs from the network and all of us are on those \
PEs?<o:p></o:p></p><div><div><p class=MsoNormal><br>-----Original \
Message-----<br>From: Outages [mailto:<a \
href="mailto:outages-bounces@outages.org">outages-bounces@outages.org</a>] On Behalf \
Of James Cornman<br>Sent: Thursday, November 07, 2013 5:35 PM<br>To: <a \
href="mailto:outages@outages.org">outages@outages.org</a><br>Subject: Re: [outages] \
Lightower/Sidera - NYC<br><br>This outage is still on-going. I spoke with an \
executive at LT earlier today and the word then was that some downstream customer of \
the Sidera/ConEd/RCN legacy network is leaking PVST BPDUs and its wreaking havoc on \
their network. I'm not sure how legitimate that is anymore since its been several \
hours. Apparently though, since our circuits are fluctuating up and down every few \
minutes, it would seem they're not very close to resolution. We're approaching the 12 \
hour mark for certain circuits.<br><br>They also don't seem to have any tools to \
notify/communicate with their customers about the on-going efforts..and they don't \
tweet anything but marketing crap. The management of this incident is \
asinine.<br><br>-James<br><br>On Thu, Nov 7, 2013 at 4:54 PM, Tim Durack <<a \
href="mailto:tdurack@gmail.com">tdurack@gmail.com</a>> wrote:<br>> As another \
datapoint, the traditional HV/Datanet/Lightower side of the<br>> network has been \
stable during this reported outage.<br>><br>><br>> On Thu, Nov 7, 2013 at \
4:45 PM, Charles Sprickman <<a href="mailto:spork@bway.net">spork@bway.net</a>> \
wrote:<br>>><br>>> On Nov 7, 2013, at 4:29 PM, Vincent Aniello \
wrote:<br>>><br>>> Our circuits stabilized mid-afternoon, but shortly \
after 4pm EST we<br>>> started seeing issues \
again.<br>>><br>>><br>>> Confirmed here as well, 7 out of 7 L2 tail \
circuits not passing<br>>> traffic after a good hour of \
instability.<br>>><br>>> FWIW, I don't recall any recent maintenance \
announcements from them,<br>>> so this is a bit troubling (I'd expect this if \
we saw a big rollout<br>>> of new code the night previous) since it came out of \
the blue. I can<br>>> also confirm we saw our first outage much earlier \
than they<br>>> acknowledged, one circuit lost connectivity at 5:23 a.m. \
I can also<br>>> confirm that we did not see the physical layer go down \
once on any of<br>>> our circuits, this seems to be occurring in the core, not \
the rings that actually service the buildings.<br>>><br>>> We're legacy \
ConEd/RCN/Sidera, have no idea if this is happening with<br>>> the LT network \
or not.<br>>><br>>> We got what appears to be a bogus "all \
restored" email at 2:50 p.m.<br>>><br>>> \
Charles<br>>><br>>> \
--Vincent<br>>><br>>><br>>><br>>> From: \
Karl Clapp <<a \
href="mailto:kclapp@staff.gwi.net">kclapp@staff.gwi.net</a>><br>>> To: \
"<a \
href="mailto:outages@outages.org">outages@outages.org</a>" <<a \
href="mailto:outages@outages.org">outages@outages.org</a>><br>>> Date: \
11/07/2013 03:23 PM<br>>> Subject: \
Re: [outages] Lightower/Sidera - NYC<br>>> Sent by: \
"Outages" <<a \
href="mailto:outages-bounces@outages.org">outages-bounces@outages.org</a>><br>>> \
________________________________<br>>><br>>><br>>><br>>> I \
contacted their NOC about 45min ago. They have stated they are<br>>> still \
working on resolution. They have reports about some services<br>>> being \
functional, while other still experiencing packet-loss and<br>>> latency. It \
does not sound like they have been able to identify the<br>>> root cause and \
are still troubleshooting.<br>>><br>>> Here is the number I used to get \
through to their NOC: <a href="tel:888-583-4237">888-583-4237</a>,<br>>> option \
1. From there, just follow the prompts.<br>>><br>>><br>>> Fwiw, my \
customer circuit from NYC to NJ is still \
affected.<br>>><br>>><br>>> On Thu, Nov 7, 2013 at 10:25 AM, John \
Kinsella <<a href="mailto:jlk@thrashyour.com">jlk@thrashyour.com</a>> \
wrote:<br>>> Just as a data point, LT for me is routing into Equinix NY5 over \
HE,<br>>> so at least there's connectivity...<br>>><br>>> \
\
My traceroute [v0.82]<br>>> icmp (0.0.0.0) \
\
Thu Nov \
7<br>>> 07:24:09 2013<br>>> Keys: Help Display mode \
Restart statistics Order of fields quit<br>>> \
\
Packets \
Pings<br>>> Host \
\
Loss% Snt Last Avg \
Best<br>>> Wrst StDev<br>>> 1. 10.0.1.1 \
\
0.0% 29 6.7 3.2 0.8<br>>> 12.2 \
2.7<br>>> 2. 10.0.1.49 \
0.0% \
28 8.0 4.3 1.5<br>>> 13.1 \
3.0<br>>> 3. 10.0.24.73 \
0.0% 28 \
6.7 4.5 1.9<br>>> 14.7 3.4<br>>> 4. <a \
href="http://25.66.208.web-pass.com" target="_blank">25.66.208.web-pass.com</a> \
0.0% 28 \
1.9 11.2 1.9<br>>> 48.0 12.3<br>>> \
5. <a href="http://2-1.760.core.fre.web-pass.com" \
target="_blank">2-1.760.core.fre.web-pass.com</a> \
0.0% 28 9.2 6.4 3.1<br>>> 19.7 \
4.0<br>>> 6. <a href="http://gige-g6-2.core1.fmt1.he.net" \
target="_blank">gige-g6-2.core1.fmt1.he.net</a> \
0.0% 28 9.6 12.4 3.5<br>>> \
26.4 6.2<br>>> 7. <a \
href="http://10gigabitethernet1-2.core1.sjc2.he.net" \
target="_blank">10gigabitethernet1-2.core1.sjc2.he.net</a> 0.0% \
28 11.9 9.5 3.5<br>>> 18.7 4.3<br>>> \
8. <a href="http://10gigabitethernet9-1.core1.nyc4.he.net" \
target="_blank">10gigabitethernet9-1.core1.nyc4.he.net</a> 0.0% \
28 87.9 79.5 73.3<br>>> 92.7 5.5<br>>> \
9. <a href="http://10gigabitethernet4-1.core1.nyc5.he.net" \
target="_blank">10gigabitethernet4-1.core1.nyc5.he.net</a> 0.0% \
28 76.6 80.5 73.6<br>>> 106.7 7.0<br>>> \
10. lightower-fiber-networks.10gigabitether 0.0% 28 78.0 \
82.3 74.4<br>>> 108.0 9.4<br>>> 11. <a \
href="http://xe-8-3-0.nycmnyzrj91.lightower.net" \
target="_blank">xe-8-3-0.nycmnyzrj91.lightower.net</a> 0.0% \
28 89.1 83.6 74.2<br>>> 144.7 \
15.8<br>>> 12. <a href="http://ae2-jrcynj67j41.lightower.net" \
target="_blank">ae2-jrcynj67j41.lightower.net</a> \
0.0% 28 85.4 88.6 80.1<br>>> 131.5 \
12.1<br>>> 13. <a href="http://192.154.116.14.lightower.net" \
target="_blank">192.154.116.14.lightower.net</a> \
0.0% 28 85.6 83.9 80.2<br>>> 95.6 \
3.5<br>>> 14. ???<br>>><br>>> John<br>>><br>>> On Nov \
7, 2013, at 7:19 AM, Karl Clapp <<a \
href="mailto:kclapp@staff.gwi.net">kclapp@staff.gwi.net</a>> \
wrote:<br>>><br>>> LightTower/Sidera master ticker for this is \
LT1323725.<br>>><br>>> I was able to get through their support line \
now.<br>>><br>>> The ETR is 1-2 hours from now.<br>>><br>>> \
---<br>>> Karl Clapp<br>>> Network Engineering Manager<br>>> \
Biddeford Internet Corp. d.b.a. GWI (AS5760)<br>>> Office: <a \
href="tel:207.602.1144">207.602.1144</a><br>>> Mobile: <a \
href="tel:207.641.3907">207.641.3907</a><br>>> Fax: <a \
href="tel:207.286.2061">207.286.2061</a><br>>><br>>><br>>> On Thu, \
Nov 7, 2013 at 9:39 AM, <<a \
href="mailto:SRonan@eexchange.com">SRonan@eexchange.com</a>> wrote:<br>>> \
Likely related to the fiber cut they experienced on their network<br>>> earlier \
this week which they routed around.<br>>><br>>> Shane Ronan, Vice \
President - Technology Architect State Street<br>>> Global Exchange | 1230 \
Avenue of the Americas, 18th Fl | New York, NY<br>>> 10020<br>>> P \
<a href="tel:%28212%29%20259-3023">(212) 259-3023</a> | M <a \
href="tel:%28347%29%20413-4503">(347) 413-4503</a><br>>> <a \
href="mailto:sronan@eexchange.com">sronan@eexchange.com</a><br>>><br>>> \
From: Jayson Navitsky <<a \
href="mailto:jnavitsky@gmail.com">jnavitsky@gmail.com</a>><br>>> Date: \
Thursday, November 7, 2013 at 6:55 AM<br>>> To: "<a \
href="mailto:outages@outages.org">outages@outages.org</a>" <<a \
href="mailto:outages@outages.org">outages@outages.org</a>><br>>> Subject: \
[outages] Lightower/Sidera - NYC<br>>><br>>> Anyone else seeing any \
issues in NYC with Lightower/Sidera? Starting<br>>> around 6:05AM (EST) \
we saw heavy packet loss across their network<br>>> (from Sidera legacy) and \
some of our point to points bounce. The P to<br>>> P's have some what \
stabilized (though we've seen a couple more<br>>> bounces since then) the IP \
side of things have been pretty much<br>>> unusable (we shut our \
session<br>>> down.) It also looks like their phone system is having a \
problem -<br>>> calls into them seem to drop after trying to connect to the NOC \
-<br>>> which I'm sure is pretty busy.<br>>><br>>> \
-Jayson<br>>><br>>> \
________________________________<br>>><br>>> <a \
href="http://www.statestreet.com/emaildisclaimer/" \
target="_blank">http://www.statestreet.com/emaildisclaimer/</a><br>>><br>>> \
_______________________________________________<br>>> Outages mailing \
list<br>>> <a href="mailto:Outages@outages.org">Outages@outages.org</a><br>>> \
<a href="https://puck.nether.net/mailman/listinfo/outages" \
target="_blank">https://puck.nether.net/mailman/listinfo/outages</a><br>>><br>>><br>>> \
_______________________________________________<br>>> Outages mailing \
list<br>>> <a href="mailto:Outages@outages.org">Outages@outages.org</a><br>>> \
<a href="https://puck.nether.net/mailman/listinfo/outages" \
target="_blank">https://puck.nether.net/mailman/listinfo/outages</a><br>>><br>>><br>>> \
_______________________________________________<br>>> Outages mailing \
list<br>>> <a href="mailto:Outages@outages.org">Outages@outages.org</a><br>>> \
<a href="https://puck.nether.net/mailman/listinfo/outages" \
target="_blank">https://puck.nether.net/mailman/listinfo/outages</a><br>>><br>>> \
_______________________________________________<br>>> Outages mailing \
list<br>>> <a href="mailto:Outages@outages.org">Outages@outages.org</a><br>>> \
<a href="https://puck.nether.net/mailman/listinfo/outages" \
target="_blank">https://puck.nether.net/mailman/listinfo/outages</a><br>>><br>>> \
_______________________________________________<br>>> Outages mailing \
list<br>>> <a href="mailto:Outages@outages.org">Outages@outages.org</a><br>>> \
_______________________________________________
Outages-discussion mailing list
Outages-discussion@outages.org
https://puck.nether.net/mailman/listinfo/outages-discussion
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic