At Venga, we host our systems using Amazon Web Services. Our account is old enough that it predates Amazon VPC so all of our systems were running in EC2 Classic. Earlier this year, we decided to migrate all of our services to VPC for two reasons:
Many new AWS features are only available in VPC
- we wanted to take advantage of the additional security offered by VPC.
When running in EC2 Classic, all hosts share the same internal network. Additionally, they need a public IP address in order to access the larger internet. Security groups allow you to limit access to only your own hosts and to restrict access from the internet, but VPC provides a better solution. When using VPC, you have control over network configuration including private IP address range, subnet configuration, and subnet and internet routing.
Once migrated to VPC, we were able to take advantage of subnets in combination with a NAT gateway in order to control access to and from our hosts. Subnets are associated with an availability zone as well as an IP address block (chosen from the larger block associated with your VPC). Subnet defaults also control whether hosts are automatically assigned public IP addresses or receive only a private address. Hosts with only a private address need access to a NAT gateway in order to access the internet while those with a public IP address can access the internet directly.
In our configuration, we used several different subnets:
- Public App: default to having a public IP address
- Backend App: default to NOT having a public IP address
- Data: default to NOT having a public IP address
As the names imply, we segregate our hosts based on their function. Only hosts in the "Public App" subnet get a public IP address (meaning only those are accessible from the internet). We still utilize security groups to limit the access to those public hosts (for example, to only allow access to ports for running services). Making just that change improved our overall security by removing the majority of our hosts from the internet.
Many of the hosts in the "Backend App" and "Data" subnets require access to the internet (to download updates, access partner APIs, etc.). In order to provide this access, we utilize an AWS NAT gateway. Each subnet can have its own route table associated (although it's common to share route tables among subnets). The route table associated with the "Public App" subnet contains a default route (0.0.0.0/0) that points to the internet gateway of the VPC. The route table associated with the other subnets contains a default route pointed to the VPC's NAT gateway instead. In this way, hosts in the "Public App" subnet use their own IP address to access the internet while those in the "Backend App" and "Data" subnets use the NAT gateway (and its associated IP address).
Route tables can also be used to segregate network traffic. For example, you could have a group of subnets used for production hosts and a separate set used for development hosts with no routes between the two. This enforces a separation between development and production and ensures that a development host can't accidentally access a production service (for instance).
Since most of our hosts are now not accessible via the internet, we need a way to connect to them for deployments, troubleshooting, maintenance, etc. One option is to run a bastion host to act as a gateway between the internet and your private subnets. Instead, we chose to use OpenVPN to create VPN connections between employees' computers and the VPC. Our OpenVPN host is in the "Public App" subnet since it needs to be accessible from the internet. The VPN server is configured to assign client IP addresses in an additional subnet reserved only for VPN users. Again, route tables can be used to control access from the VPN client subnet to resources in the VPC.
By switching all of our hosts from EC2 Classic to VPC, we were able to further harden our systems against outside influence. Removing public IP addresses, segmenting hosts by function, and implementing a VPN all help to provide a defense in depth approach to security. Additionally, we're well positioned to take advantage of new AWS service offerings that are available only to customers utilizing a VPC.