Traditional IT environments are protected by multiple levels of security. Amongst them are fine-grained access control and network security, like limiting workloads to LANs and adopting firewalls. When adopting Managed Services, the level of abstraction is higher. There, we need to adopt a different approach to maximise security and limit accessibility. Briefly said: how can we add a firewall to our Bigtable instances? I’ll explain in this blog post.
TL;DR: If you want a firewall around your BigQuery or Bigtable environment, VPC Service Controls can help.
Securing traditional workloads
When working with customers on designing their Google Cloud environment, for both them and us, security is always a main pillar. Many customers start with migrating their infrastructure as-is (“lift and shift”) and leverage some quick benefits on the side (“move and improve”), like moving to Cloud SQL instead of hosting their own PostgreSQL on a VM.
Traditional security aspects & considerations
These traditional workload migrations bring, amongst others, the traditional security aspects with them. For example, staying within private network range (the infamous RFC 1918) and managing your VPC firewalls are one of the first things that come to mind.
When we are explicitly looking at Compute Engine instances running private workloads, there are many security considerations to be made. As an example, the following two are complementary: creating instances without a public IP address and blocking ingress traffic using VPC Firewall rules.
This way, if even one of the two mitigations gets breached (i.e. an engineer adds a public IP to a VM instance), the other security layer still protects your resource (the VPC firewall will block all public access attempts to the VMs public IP).
Managed Services with Public Endpoints
Once these customers are acquainted with Google Cloud, they quickly see the benefits of the many managed services Google Cloud offers. One of these services is BigQuery, Google’s fully managed petabyte-scale warehouse. A very simple use case is exporting all Stackdriver Logging into BigQuery for log analysis.
BigQuery Managed Service
BigQuery, being a managed service, does not require its users to care about the underlying infrastructure, and provides a nice web interface on top of REST APIs to interact with the service. Customers can easily adopt the least privileges principle to limit access to a certain set of users and share specific datasets (a grouping of tables in BigQuery) to a limited set of analysts.
One of the benefits of a managed service with a cloud provider like Google, is that everything is configurable using these APIs. From anywhere in the world, you can call upon BigQuery’s HTTP REST APIs to query and manage your data. After all, BigQuery does not run in your WAN and therefore is accessible over a public http endpoint: https://bigquery.googleapis.com.
Security risk
Simply put: the only thing standing between this API and potentially massive data breaches, is credentials & IAM permissions.
If you are using a Service Account in your application to call upon BigQuery (i.e. by connecting an on-premises BI tool) and that key gets leaked, everyone in the whole wide world could have access to your BigQuery environment until that key is rotated.
Pretty sure we should do one better.
Adding the second Security Layer
As mentioned above, we would probably not want to rely only on the IAM policies to protect our Managed Services. A traditional reflex would be to add a firewall to the http endpoints, like we would want to do with a VM instance. Or prevent public routing at all. As there’s a single endpoint per service for all customers, we can’t do the last. But we can mimic the firewalling solution: enter VPC Service Controls.
VPC Service Controls
Using VPC Service Controls, we can mimic the mental model of a firewall to Google Managed Services that we don’t have control over with VPC Firewall Rules. With VPC Service Controls, you can create a perimeter around several Google Cloud projects and services. With some imagination you could even compare this to a WAN.
setting up a VPC Service Perimeter
In the above setup, we’re creating a perimeter around both the BigQuery API and Bigtable API of our projects ‘fourcast hosting’ and ‘Fourcast Anthos’, preventing outside access to both the BigQuery and Bigtable APIs. This means only resources from within these projects can access these APIs for the resources within these projects.
Extra layer of security on top of IAM permissions
This does not allow for a free flow of requests within these projects, IAM permissions still apply. No access to a BigQuery dataset? Not getting in. It’s an extra layer of security on top of IAM permissions.
A Compute Engine Instance in ‘Fourcast Anthos’ is still allowed to call the Bigtable API on the ’fourcast hosting’ project, and given it has the correct IAM permissions, it can still read data.
As per example below, the Compute Engine instance ‘fourcast anthos bastion’ runs in the ‘Fourcast Anthos’ project, and using the correct credentials we can still list the Bigtable instances in the other project:
accessing the Bigtable API from within a different project in the same perimeter
A VM running in any other project, or locally, you would not be able to access, as can be seen below when trying from my local machine:
accessing the Bigtable API from localhost: bad luck
Even the Cloud Console page for Bigtable is brutally rejecting us with a clean message:
Your access is denied by VPC Service Control. Please contact your organisation administrator for help. There was an error while loading /bigtable/instances?project= Please try again.
no access to the Bigtable overview page in the Cloud ConsoleDefining Access Level with Access Context Manager
Luckily, we can define an Access Level using Access Context Manager to fix that, in case we still want to allow controlled access. One of the features here is to filter requests based on the source IP. Simply adding my home IP (which normally would be your office IP, but hey, COVID-19, right..) to an Access Level and then adding that to the perimeter, solves this for us:
defining an access level on a single IP
adding the Access Level to the perimeter to get access from the local workstation.
With this Access Level defined and added to the perimeter, we can now access our beloved Bigtable instance again:
adding the public IP address to the perimeter’ Ingress Policy restores access
And even the console is all shiny again:
adding the public IP address to the perimeter’ Ingress Policy restores access
In Summary, many managed services are exposed through public APIs. While we don’t have access to the VPC where Google is hosting our Managed Services, preventing us from setting VPC Firewall rules, there’s more we can do to protect our resources than merely configuring IAM access.
VPC Service Controls is a next step in securing your workload and protecting it from outside access.
Have questions about building your foundation on Google Cloud?
Get in touch with us!
More information about Google Cloud
