Salsify FTP server service disruption
Incident Report for Salsify
Postmortem

On July 31st 2020, our FTP tech partner Exavault released a new version of their API to production in parallel with the existing version. The deployment and configuration did not work as planned and there were a number of issues, the most impactful of which was a partial loss of webhooks/notification for a substantial period of time. For detailed information, please visit https://clients.exavault.com/serverstatus.php?view=resolved.

As a result, Salsify integration and custom configuration that utilizes the FTP server experienced long delays, and a small percentage failed to initiate. The issue was resolved at 7:50 pm EST. In case you believe some of your functionalities were effected by this, please re-run file transformation, file delivery rerouting, and custom integration. Our support team is available for questions or comments at https://support.salsify.com.

Posted Aug 05, 2020 - 11:38 EDT

Resolved
Our tech partner has resolved issues on their end. The backlog of webhooks and notifications is currently being processed.
If you're still experiencing issues, please reach out to Salsify Support.
Posted Jul 31, 2020 - 20:28 EDT
Update
Our tech partner is still seeing issues with delivering webhooks. This is causing further delays on Salsify custom integrations and services such as file transformation and file delivery rerouting that rely on the FTP servers. We are working with our partner to ensure all services that rely on these webhooks get processed.
Posted Jul 31, 2020 - 18:23 EDT
Update
Our tech partner has identified the software update that caused delays and are currently rolling this back. They expect webhooks and notifications to catch up within the next hour. Further updates can be seen at: https://www.exavault.com/ To confirm, all Salsify custom integrations and services such as file transformation and file delivery rerouting that rely on the FTP servers will run shortly thereafter.
Posted Jul 31, 2020 - 15:10 EDT
Identified
As part of the restoration process, our tech partner has disabled webhook and notification features. Custom integration and services such as file transformation and file delivery rerouting that rely on the FTP servers are currently unavailable.
Posted Jul 31, 2020 - 13:51 EDT
Investigating
Our tech partner is currently investigating scattered reports of in-app login issues on salsify.exavault.com. The team is working towards a resolution. Custom integration and services that rely on the FTP servers are experiencing degraded performance.
Posted Jul 31, 2020 - 13:25 EDT
This incident affected: Salsify Application (Salsify FTP).