Amazon’s S3 is a cost effective way to store file but many organizations are used to mapping NFS shares to machines for file storage purposes. Amazon Storage Gateways are a good way to cache or store files on an NFS mount and then back them up to an S3 bucket. This post goes through the setup of an AWS Storage Gateway in an EC2 instance for caching files and storing them in an S3 bucket. This same solution (and a similar but different process) can be used to mount block devices through iSCSI or setup a Tape Gateway for backup products.

Prerequisites

We’ll need some basics up and running to follow along with this post. We’ll need a VPC in an AWS Account for starters. Once this is ready to go we’ll want to setup an S3 bucket so that we have a location to dump our files. I’ve created a bucket named hollows3gatewaybucket1 for this post. I also created a folder in the bucket just to have some content in it.

Next up, we want to create a security group that will be used to filter the traffic to our storage gateway. I’ll be using two rules. The first is to allow HTTP traffic to the storage gateway. This port is used to activate the gateway. The second rule is to allow NFS access to the gateway and I’ll restrict that access to only machines from within my VPC.

Deploy the Storage Gateway

Now we’re ready to get started with the storage gateway. Go to the AWS console and open the Storage Gateway service. Since this is the first time we’ve used it, click the “Get started” button.

On the wizard screen, select the gateway type. I’ve selected “File gateway” but you can also do Volume gateway for block devices, and Tape gateway for a tape library. Click Next.

On the host platform select where you’ll place the storage gateway. You can download an OVA for VMware, or spin up an EC2 instance. For this post I used the EC2 instance. Click the “Launch instance” button to take you to that wizard.

Now we walk through setting up an EC2 instance. The recommended starting size is m4.large but you can select what makes sense for your environment. Click Next.

On the next screen make sure you give the storage gateway a subnet and IP Address that is reachable. I’ve given mine a public subnet with a public IP Address but you probably wouldn’t want to do this for a production system for security reasons. When you’re done, click next.

On the storage page, add a new EBS volume for your caching. I used a 100 GB drive but this all depends on how much data you want to cache for your S3 bucket. Obviously, the larger the drive, the more data you can cache for the S3 bucket, but that comes at the cost of EBS storage.

Next, you can add any tags to your instance.

On the security group screen select an existing security group and pick the one we created earlier. You can review our rules again below.

Review your settings and launch the instance.

Select a keypair in case you need to SSH into the box later. We can’t really SSH in at this point because we didn’t open the SSH port on the security group, but maybe we’d change that later for troubleshooting purposes.

In a moment you’ll see your new instance starting up in the EC2 console.

Once the EC2 instance has had time to spin up, we can go back to our storage gateway wizard where we jumped out of to create our EC2 instance. Enter in the IP Address of the new EC2 instance. I used the public IP Address but you could use a private IP if you have connectivity from your workstation to the storage gateway. Clicking the “Connect to gateway” button just redirects your browser to activate the gateway.

Once you connect to the gateway, set the timezone and give the gateway a name. Then click “Activate gateway”.

Once you do that the gateway will be active and you need to set your cache. After a moment your spare disk should be recognized and you can allocate it to the “Cache”. Click the Save and Continue button.

Configure Shares

Now that gateway is deployed and activated, we’ll be taken to our list of storage gateways. You should see a status of running. Click the “Create file share” to add an NFS share.

The gateway should be set already, but if you have multiple gateways, select the correct one. Then enter in the name of the S3 bucket we created at the beginning of this post. Don’t make the mistake of thinking that this wizard will create an S3 bucket for you. It won’t. Then select the storage class for new objects and you can leave the “Create a new IAM role” selected so that the permissions are automatically set correctly. Click next.

Review the settings and click the “Create file share” button.

It may take a few minutes but the share status should be “Available” pretty soon. Once it is, select it and at the bottom of the screen the instructions for mounting the NFS share will be displayed for you.

On your client machine, mount the NFS share and you should be able to put your own files there after that.

Summary

The Amazon Storage Gateway is a nice utility to allow your users to leverage S3 through iSCSI, Tape or NFS mount points. It can become particularly useful when storing data close to other on-premises clients but having the data backed by S3 or a good way to migrate your data to S3 without having to think much about it.