News
|
Jul 22 |
[Core Network] Maintenance window 4th-15th August 2014
Posted by David Croft on 22 July 2014 11:09 AM |
|
Please be advised of the following maintenance window scheduled at our Telecity HEX6-7 (LNPOP02) site: Start date: Monday, 4th August 2014 Start time: 08:00 BST (UTC+0100) End date: Friday, 15th August 2014 End time: 17:00 BST (UTC+0100) Purpose: The data centre will be performing maintenance on their UPS systems. Services affected: All services hosted in or single-homed out of Telecity HEX6-7 (LNPOP02). Impact: Low. TelecityGroup will be carrying out maintenance works on the uninterruptible power supply (UPS) systems at 6/7 Harbour Exchange from 4th to 15th August 2014 between 08:00 and 17:00 (BST) as part of their planned preventative maintenance programme. The works will involve isolating the UPS units one at a time to allow the servicing and testing of all UPS modules and associated batteries. Each unit will be reinstated before work on the next unit commences. During this maintenance window there will be a reduced level of resilience, however we do not anticipate any impact to your services. Our engineering team will be monitoring the power supply systems at all times throughout these works. Comtec NOC is tracking this issue under ticket [#AXH-878-96039]. Regards, David Croft Read more » | |
|
Jul 17 |
[Core Network] Service Alert 17th July 2014 - Update 2
Posted by David Croft on 17 July 2014 02:39 PM |
|
This is an incident notification regarding our core network at Interxion LON1 (Site CLBIS02). Date: Monday, 9th June 2014 Start time: 13:54 End time: 14:18 Services affected: Core Network Services single-homed out of Interxion LON1 (CLBIS02). Report: We observed a large increase in temperature over a short time window. The data centre dispatched engineers to investigate and the temperature quickly dropped again. A service alert was issued at 14:08 to warn customers of the possibility of a short-notice shutdown to prevent equipment damage. However, no shutdown was necessary and no services were impacted. The probable cause appears to have been other customers working nearby placing equipment in front of our rack and blocking airflow. We have asked the data centre to install additional ventilation floor tiles in front of our rack to prevent this issue happening again and will follow up to ensure that this work is completed. This is the final update. Comtec NOC was tracking this issue under ticket [#BHC-445-92607]. Best regards, David Croft Read more » | |
|
Jul 17 |
[Core Network] Service Alert 17th July 2014 - Update 1
Posted by David Croft on 17 July 2014 02:21 PM |
|
This is an incident notification regarding our core network at Interxion LON1 (Site CLBIS02). Date: Monday, 9th June 2014 Start time: 13:54 Services affected: Core Network Services single-homed out of Interxion LON1 (CLBIS02). Report: We observed an increase in temperature of six degrees C in a five minute window. The data centre dispatched engineers to investigate. The engineer has arrived and the temperature is dropping again. We will hear more when he returns to his desk. The next update will be issued in 15 minutes. Comtec NOC is tracking this issue under ticket [#BHC-445-92607]. Best regards, David Croft Read more » | |
|
Jul 17 |
[Core Network] Service Alert 17th July 2014
Posted by David Croft on 17 July 2014 02:08 PM |
|
This is an incident notification regarding our core network at Interxion LON1 (Site CLBIS02). Date: Monday, 9th June 2014 Start time: 13:54 Services affected: Core Network Services single-homed out of Interxion LON1 (CLBIS02). Report: We have observed an increase in temperature of six degrees C in a five minute window. The data centre has dispatched engineers to investigate. If the temperature continues to rise at the current rate, we may find it necessary to shut down equipment at short notice to prevent damage. If this becomes necessary, services hosted in Interxion LON1 CLBIS02 (including customers with Cloud Ethernet circuits that are single-homed to CLBIS02) will be interrupted. The rest of the network will route around this site and operations will not be impacted. The next update will be issued in 15 minutes. Comtec NOC is tracking this issue under ticket [#BHC-445-92607]. Best regards, David Croft Read more » | |
|
Jul 13 |
[Core Network] Maintenance window 13th July 2014 - Update 1
Posted by David Croft on 13 July 2014 12:53 PM |
|
Good afternoon, Our maintenance has been completed successfully. If you experience any issues following this work, please contact noc at comtec.com. Regards, David Croft On 27 June 2014 22:51, David Croft <david at sargasso.net> wrote: > Please be advised of the following maintenance window scheduled on our network: > > Date: Sunday, 13th July 2014 > Start time: 10:00 BST (UTC+0100) > End time: 14:00 BST (UTC+0100) > > Purpose: > > We will be performing software upgrades on our distribution switches at Reigate. > > Services affected: > > All services hosted in Comtec House, Reigate (LSREI01). > > Impact: > > Low. > > The redundant pair of distribution switches will be upgraded one at a > time. Customers who have connected to both distribution switches per > our recommendations will observe no operational impact as they will > fail over to the redundant switch while the other is being upgraded. > > Customers who are single-homed to our network will observe a period of > downtime of 5-15 minutes during this window. In the event that > roll-back is required, there will be a further interruption while the > original software is restored. > > No outage is anticipated due to the resilient nature of our network > and transport out of Reigate, however services are "at-risk" during > this window. IP voice services will be unaffected. > > Comtec NOC is tracking this issue under ticket [#GZO-670-71256]. > > Regards, > > David Croft Read more » | |
|
Jul 7 |
[IP Voice Services] Service Alert 30th June 2014 - Update 5
Posted by David Croft on 07 July 2014 09:37 AM |
|
This is an incident notification regarding IP Voice Services. Date: Monday, 30th June 2014 Start time: 18:00 BST (UTC+0100) End time: 18:49 BST (UTC+0100) Services affected: IP Voice Services routing to/from PSTN. Report: The engineering team was investigating an issue affecting all off-net call routing. On-net calls remained unaffected. Root Cause: The carrier SBC cluster was unable to process calls due to the license expiring. The engineering team should have received notification from the carrier SBC network management application from 4 weeks prior to the license expiry date. Engineering and TAC use this application daily and in this instance the system did not provide the notification banner as seen previously when licenses were due to expire. The carrier SBC network management system should also have provided an email alert which was not generated. Symptoms: Unable to route off-net calls. Once off-net call routing was restored, subsequent issues of excessive post dial delay and intermittent delays with media establishing on answer were experienced. Resolution: The vendor issued and installed a perpetual license to provide a permanent fix to prevent the license expiry and associated notification failure being an issue in the future. Timeline: 18:05 - Network Monitoring system alerted the Engineering team that the carrier SBC cluster was not processing calls. 18:07 - Engineering identified that the SBCs had stopped processing calls due to the license they were using expiring. 18:08 - The Engineering team contacted the license vendor to request immediate replacements. 18:46 - Replacement license were provided and installed by the Vendor. At this point off-net service was restored. However excessive PDD (Post Dial Delay) and delays with media establishing were still being observed. The Engineering team worked in conjunction with the Vendor to resolve the ongoing problems. It was identified that the heartbeat between the two devices was not establishing correctly causing the secondary to periodically try and come into service. 19:47 - Both of the SBCs were shut down so that they could be restarted cleanly in turn to resolve the heartbeat problems. At this point off-net calls were unavailable. 19:51 - One of the SBCs was returned to service and it was confirmed that call processing to and from off-net destinations was restored and there were no longer any issues being experienced. 20:00 - The second SBC was restored to service and heartbeat re-established cleanly with no affect on service. 20:25 - Service was rolled from one SBC to the other to confirm stability. During this process there was no effect on call processing; all existing calls were sustained and new calls established normally. The Engineering Team and the Vendor closely monitored the service until 22:00 to confirm all service was restored and no residual issues remained. We apologise for the inconvenience this has caused. This is the final update. Comtec NOC was tracking this issue under ticket [#OEE-382-69090]. Best regards, David Croft Read more » | |