EcoStruxure IT Gateway and IT Expert

Log4j is a standard logging library used by many Java applications, including the EcoStruxure IT Expert and IT Gateway. 

EcoStruxure IT Gateway

A new EcoStruxure IT Gateway version (1.13.1.5) containing log4j version 2.16 is now available. We strongly encourage all customers to upgrade.

EcoStruxure IT Gateway versions 1.5.0 to 1.13.0 contain the affected versions of the library and may be susceptible to remote code execution as described in CVE-2021-44228. It is still unclear if or how an exploit of log4j in the EcoStruxure IT Gateway is possible. Earlier EcoStruxure IT Gateway versions (1.4.3 and earlier) do not contain an impacted version of log4j.

EcoStruxure IT Expert

The cloud-based EcoStruxure IT Expert has already been updated with log4j version 2.15, which includes a fix for CVE-2021-44228. A newer and further hardened version of log4j, version 2.16, has just been released and will be implemented shortly.

We have now received a further update on this.

Equinix's overview of the Incident:

At 04:33 on the 18th August 2020 at our LD8 IBX, Equinix facilities technicians responded to a fire alarm. Once deemed safe to do so, the site team started investigations into the source of the alarm, it was noted that it had originated from the building 8/9 main power room. The building fire suppression system was not activated nor required to do so at any point.

Site technicians proceeded to the main power room to investigate the issue and discovered smoke (no fire) within the room. The cause of the smoke was identified as a result of a failure on the main static transfer switch common output cabinet of the Galaxy UPS system. The fault had caused the system to shut down, resulting in a total loss of all customer power supplies supported by busbar risers ISP8 and ISP9 both of which are connected to the same UPS output.

Following the power beginning to be restored yesterday at LD8 many providers and transit operators have been working to restore their services which has affected the internet transit at large. Please note we have not received a 'works complete notification from Equinix at this time'

We have been kept informed that some of the larger operators such as LINX (London Internet Exchange) had to replace their line cards but that work is now complete and for many recipients of service there appears to be now a stability since many providers have completed the works overnight. 

Due to the large and unexpected impact of a power outage affecting so many providers to the internet we are continuing to monitor this situation as their may still be works ongoing with some providers that has not been flagged and also residual after effects to service provided generally.

Following Equinix resolving the issue with the outage, providers within the datacentre have equipment which is starting to come up and internet routes are starting to be seen again.