Ransomware Attack Triggered ConnectWise Manage EU Outage

Update on ConnectWise Manage® outage in the EU.

Following a thorough investigation, we can confirm that the outage was caused by a security incident.

On Friday, 3 May, at approximately 7:30 am BST, the following series of events occurred:

• ConnectWise was alerted by our internal monitoring and security systems that some of our SQL databases in our EU-AWS cluster were not accessible.
• We quickly realized that several servers were inaccessible due to critical failures.
• Our incident response procedures were immediately enacted, and our internal teams responded within minutes to assess the situation and began to monitor the environment and analyze the alerts.
• These servers were immediately taken offline and access to the entire cloud network was restricted to a select number of colleagues.
• Our initial examination pointed toward some type of malware.
• The cloud team built and deployed new AWS clusters with known good backup restorations. This contributed to the downtime experienced by ConnectWise EU partners.
• As our investigation ensued, our teams discovered that the malware was ransomware.
• All partner access was restored by 3:16 pm BST.
• Email Connector service was enabled at 4:20 pm BST.
• Reporting services were back online by 5:15 pm BST.
• A third- party forensics firm was engaged to perform a comprehensive investigation.

The forensics firm confirmed that the ransomware variant used in the attack only encrypts files, and is not designed or capable of reading, removing, or altering data.  The only impact of the intrusion was loss of access to our hosted SaaS application. We found no indication that any personal data was destroyed, altered, disclosed to, or accessed by an unauthorized party. Accordingly, we do not believe there is a risk to the rights and freedoms of EU data subjects as a result of this outage.  We were able to identify that the intrusion came from an offsite machine that was used for cloud performance testing outside of our network. Going forward, we have immediately prohibited any such offsite systems testing.

The following actions are being taken to prevent a similar incident from happening in the future:

• We have completely rebuilt, scanned, and setup all new servers in our cloud infrastructure across North America, EMEA, and ANZ.
• All passwords in the environment were reset immediately.
• All access to the infrastructure from outside the network was blocked immediately.
• An additional layer of authentication was added to the environment for all users.
• An additional layer of security was added between the SQL clusters and the rest of the environment.
• An additional step was added to snapshot the transaction log backups each hour to reduce the recovery point in the event the transaction logs are compromised.
• We have updated our procedures for remote access and added additional monitoring and training.

Now that our investigation is complete, we will be filing a complaint with the appropriate law enforcement agencies. Over the coming weeks we will provide more thorough documentation regarding our security practices, penetration testing, SOC, and product security analysis. Our team is here to help with any questions you may have around an incident of this kind. Please direct questions to [email protected]

• May 10, 2019: CloudJumper suffered a RYUK ransomware attack on a small portion of its Workspace as a Service (WaaS) systems for MSPs.
• April 2019: Cybercriminals used TeamViewer remote access and desktop sharing software to navigate onto customer networks.
• April 2019: Hackers broke into Wirpo's network and then jumped onto end-customer systems.
• February 2019: Hackers exploited an old ConnectWise plugin for Kaseya.