Cloud Vendors Compromised
Given the number of back-channels we are a part of we get to hear horror stories where Cloud Vendors are compromised in some way or get hit by an encryption event that takes their client/customer facing systems out.
When we architect a hosting system for a hosting company looking to deploy our solutions in their hosting setup, or to set up an entirely new hosting project, there are some very important elements to our configuration that would help to prevent the above from happening.
A lot of what we have put into our design is very much a result of our experiences on the frontlines with SMB and SME clients.
One blog post that provides some insight: Protecting a Backup Repository from Malware and Ransomware.
It is absolutely critical to isolate and off-site any and all backups. We’ve also seen a number of news items of late where a company is completely hosed as a result of an encryption event or other failure only to find out the backups were either wiped by the perps or no good in the first place.
Blog Post: Backups Should Be Bare Metal and/or Virtually Test Restored Right?
The full bare metal or virtual restore is virtually impossible at hyper-scale. Though, we’ve seen that the backups being done in some hyper-scale cloud vendor’s environments have proven to be able to be restored while in others a complete failure!
However, that does not excuse the cloud customer or their cloud consultancy from making sure that any and all cloud based services are backed up _off the cloud_ and air-gapped as a just-in-case.
Now, to the specific point of this blog post.
Tenant Isolation Technique
When we set up a hosting solution we aim to provide maximum security for the tenant. That’s the aim as they are the ones that are paying the bills.
To do that, the hosting company needs to provide a series of layered protections for tenant environments.
- Hosting Company Network
- Hosting company AD
- All hosting company day-to-day operations
- All hosting company on-premises workloads specific to company operations and business
- Dedicated hosting company edges (SonicWALL ETC)
- Tenant Infrastructure Network
- Jump Point for managing via dedicated Tenant Infrastructure AD
- High Availability (HA) throughout the solution stack
- Dedicated Tenant Infrastructure HA edges
- Risk versus Reward: Could use the above edges but …
- Clusters, servers, and services providing the tenant environment
- Dedicated infrastructure switches and edges
- As mentioned, backups set up and isolated from all three!
- Tenant Environment
- Shared Tenant AD is completely autonomous
- Shared Tenant Resources such as Exchange, SQL, and more are appropriately isolated
- Dedicated Tenant AD is completely autonomous
- Dedicated Tenant Resources such as Exchange, SQL, and more are completely isolated to the tenant
- Offer a built-in off-the-cloud backup solution
With the solution architected in this manner we protect the boundaries between the Hosting Company Network and the Tenant Environment. This makes it extremely difficult for a compromise/encryption event to make the boundary traversal without some sort of Zero Day involved.
Conclusion
We’ve seen a few encryption events in our own cloud services tenants. None of them have traversed the dedicated tenant environments they were a part of. None. Nada. Zippo.
Containment is key. It’s not “if” but “when” an encryption event happens.
Thus, architecting a hosting solution with the various environment boundaries in mind is key to surviving an encryption event and looking like a hero when the tenant’s data gets restored post clean-up.