News
|
Jul 13 |
[Core Network] Maintenance window 13th July 2014 - Update 1
Posted by David Croft on 13 July 2014 12:53 PM |
|
Good afternoon, Our maintenance has been completed successfully. If you experience any issues following this work, please contact noc at comtec.com. Regards, David Croft On 27 June 2014 22:51, David Croft <david at sargasso.net> wrote: > Please be advised of the following maintenance window scheduled on our network: > > Date: Sunday, 13th July 2014 > Start time: 10:00 BST (UTC+0100) > End time: 14:00 BST (UTC+0100) > > Purpose: > > We will be performing software upgrades on our distribution switches at Reigate. > > Services affected: > > All services hosted in Comtec House, Reigate (LSREI01). > > Impact: > > Low. > > The redundant pair of distribution switches will be upgraded one at a > time. Customers who have connected to both distribution switches per > our recommendations will observe no operational impact as they will > fail over to the redundant switch while the other is being upgraded. > > Customers who are single-homed to our network will observe a period of > downtime of 5-15 minutes during this window. In the event that > roll-back is required, there will be a further interruption while the > original software is restored. > > No outage is anticipated due to the resilient nature of our network > and transport out of Reigate, however services are "at-risk" during > this window. IP voice services will be unaffected. > > Comtec NOC is tracking this issue under ticket [#GZO-670-71256]. > > Regards, > > David Croft Read more » | |
|
Jul 7 |
[IP Voice Services] Service Alert 30th June 2014 - Update 5
Posted by David Croft on 07 July 2014 09:37 AM |
|
This is an incident notification regarding IP Voice Services. Date: Monday, 30th June 2014 Start time: 18:00 BST (UTC+0100) End time: 18:49 BST (UTC+0100) Services affected: IP Voice Services routing to/from PSTN. Report: The engineering team was investigating an issue affecting all off-net call routing. On-net calls remained unaffected. Root Cause: The carrier SBC cluster was unable to process calls due to the license expiring. The engineering team should have received notification from the carrier SBC network management application from 4 weeks prior to the license expiry date. Engineering and TAC use this application daily and in this instance the system did not provide the notification banner as seen previously when licenses were due to expire. The carrier SBC network management system should also have provided an email alert which was not generated. Symptoms: Unable to route off-net calls. Once off-net call routing was restored, subsequent issues of excessive post dial delay and intermittent delays with media establishing on answer were experienced. Resolution: The vendor issued and installed a perpetual license to provide a permanent fix to prevent the license expiry and associated notification failure being an issue in the future. Timeline: 18:05 - Network Monitoring system alerted the Engineering team that the carrier SBC cluster was not processing calls. 18:07 - Engineering identified that the SBCs had stopped processing calls due to the license they were using expiring. 18:08 - The Engineering team contacted the license vendor to request immediate replacements. 18:46 - Replacement license were provided and installed by the Vendor. At this point off-net service was restored. However excessive PDD (Post Dial Delay) and delays with media establishing were still being observed. The Engineering team worked in conjunction with the Vendor to resolve the ongoing problems. It was identified that the heartbeat between the two devices was not establishing correctly causing the secondary to periodically try and come into service. 19:47 - Both of the SBCs were shut down so that they could be restarted cleanly in turn to resolve the heartbeat problems. At this point off-net calls were unavailable. 19:51 - One of the SBCs was returned to service and it was confirmed that call processing to and from off-net destinations was restored and there were no longer any issues being experienced. 20:00 - The second SBC was restored to service and heartbeat re-established cleanly with no affect on service. 20:25 - Service was rolled from one SBC to the other to confirm stability. During this process there was no effect on call processing; all existing calls were sustained and new calls established normally. The Engineering Team and the Vendor closely monitored the service until 22:00 to confirm all service was restored and no residual issues remained. We apologise for the inconvenience this has caused. This is the final update. Comtec NOC was tracking this issue under ticket [#OEE-382-69090]. Best regards, David Croft Read more » | |
|
Jun 30 |
[IP Voice Services] Service Alert 30th June 2014 - Update 4
Posted by David Croft on 30 June 2014 08:57 PM |
|
This is an incident notification regarding IP Voice Services. Date: Monday, 30th June 2014 Start time: 18:00 BST (UTC+0100) End time: 18:49 BST (UTC+0100) Services affected: IP Voice Services routing to/from PSTN. Report: The engineering team was investigating an issue affecting all off-net call routing. On-net calls remained unaffected. Following the incident, we experienced continuing issues with excessive post-dial delay and/or a short delay before media establishing on answer. There was also some ongoing instability with calls to/from the PSTN. At this time, engineering has fully resolved all residual issues on the primary and secondary carrier-side SBCs in liaison with our vendor. Calls are now progressing normally to/from the PSTN. Further details will be supplied once all investigations have been completed. Comtec NOC is tracking this issue under ticket [#OEE-382-69090]. Best regards, David Croft Read more » | |
|
Jun 30 |
[IP Voice Services] Service Alert 30th June 2014 - Update 3
Posted by David Croft on 30 June 2014 08:25 PM |
|
This is an incident notification regarding IP Voice Services. Date: Monday, 30th June 2014 Start time: 18:00 BST (UTC+0100) End time: 18:49 BST (UTC+0100) Services affected: IP Voice Services routing to/from PSTN. Report: The engineering team was investigating an issue affecting all off-net call routing. On-net calls remained unaffected. Although off-net calling has been restored, we are experiencing some issues with excessive post-dial delay and/or a short delay before media establishing on answer. We are currently experiencing some ongoing instability with calls to/from the PSTN. However the issue with post dial-delay has been reduced. Engineering are continuing to investigate these issues. We will update you as the situation progresses. Comtec NOC is tracking this issue under ticket [#OEE-382-69090]. Best regards, David Croft Read more » | |
|
Jun 30 |
[IP Voice Services] Service Alert 30th June 2014 - Update 2
Posted by David Croft on 30 June 2014 07:35 PM |
|
This is an incident notification regarding IP Voice Services. Date: Monday, 30th June 2014 Start time: 18:00 BST (UTC+0100) End time: 18:49 BST (UTC+0100) Services affected: IP Voice Services routing to/from PSTN. Report: The engineering team was investigating an issue affecting all off-net call routing. On-net calls remained unaffected. This issue has been resolved as of 18:49. Although off-net calling has been restored, we are experiencing some issues with excessive post-dial delay and/or a short delay before media establishing on answer. Engineering are continuing to investigate these issues. We will update you as the situation progresses. Comtec NOC is tracking this issue under ticket [#OEE-382-69090]. Best regards, David Croft Read more » | |
|
Jun 30 |
[IP Voice Services] Service Alert 30th June 2014 - Update 1
Posted by David Croft on 30 June 2014 06:54 PM |
|
This is an incident notification regarding IP Voice Services. Date: Monday, 30th June 2014 Start time: 18:00 BST (UTC+0100) End time: 18:49 BST (UTC+0100) Services affected: IP Voice Services routing to/from PSTN. Report: The engineering team was investigating an issue affecting all off-net call routing. On-net calls remained unaffected. This issue has been resolved as of 18:49. Further information as to the cause of this outage will be forwarded when it is made available to us. Comtec NOC is tracking this issue under ticket [#OEE-382-69090]. Best regards, David Croft Read more » | |