On November 20, 2024, at 10:50 UTC, we received alerts indicating intermittent hardware performance issues. These issues resulted in CloudWorks integrations running slowly in Anaplan Data Center — Netherlands.
Our initial investigation identified intermittent delayed responses between a network component and the backend infrastructure. In parallel, we detected intermittent network communication delays causing CloudWorks integrations to queue without processing. After a comprehensive assessment, we discovered several hardware resources were unresponsive to network requests. To resolve the issue, we strategically isolated the problematic server from the network. This successfully restored connectivity at 12:18 UTC.
By 12:33 UTC, our CloudWorks service functionality had returned to normal operational status. However, due to earlier hardware performance issues, a backlog of integrations remained pending. We continued monitoring the queued backlog integrations until it was fully cleared at 13:06 UTC, marking the complete restoration of our CloudWorks services.
After a thorough investigation into the network issues, we identified that an incremental data copy process had overwhelmed our network infrastructure. A sub-optimal network path between the network and infrastructure caused network link saturation, resulting in network congestion that directly impacted CloudWorks integrations. In response, we have updated the network path to significantly increase throughput and reduce congestion risks. We are also reassessing our data copy processes to minimize future network performance challenges.
We have also created a priority workstream for CloudWorks to improve the resiliency of the service. We have completed several improvements to the underlying infrastructure to enable vertical and horizontal growth. This has enabled us to increase the volume of requests that CloudWorks can process at any given time. This should also reduce recovery time on issues that impact CloudWorks and the need to increase resources allocation as part of the recovery process. We are closely monitoring the overall health of the service and initial findings are positive.
We apologize for any impact this issue may have had on your business operations. We are continuously strengthening our systems and procedures to ensure we avoid future disruptions to your business and users.
If you have any further questions or concerns, please contact Anaplan Customer Care. Thank you for your patience during this situation and thank you for being an Anaplan customer.