The Download on Reading Data from Google Cloud Storage

*Check out the updated version of this post, here*

Having your data uploaded and organized into buckets is the ideal place to start thinking about what else you’d like to do with it. When serving data to your users from your Google Cloud Storage Buckets, there are a few different options you can use, and things you’ll need to consider.

Public Endpoints

Let’s start with content you need exposed through a Public endpoint — It’s important to note that by default, files in Google Cloud Storage are hidden from public view. This is adjustable, and you can update your access settings to grab a unique, public URL. To make your data public, check out the steps in this article.

Serving Publicly for a Website or Application

Hosting files in Google Cloud Storage and having them served publicly for your website or your application is a very common use case, and straightforward to manage.

To make objects in a bucket public, simply click on the three dot menu to the right, and then “add item” in the permissions pop up, following the instructions in the visual (NAME = allUsers, ACCESS = Read)

Once your object or objects are publicly readable, you’ll see a link icon in the “Public Access” column in the console.

This is the link you can click to get the URL for the object so you can fetch things via an HTTP request, and serve those assets to static websites or to your application directly.

Non- Public Endpoints

For items which aren’t exposed through a public endpoint, you’ve got a few different options to choose from. These items will have an indication in the public access column that shows they are not shared publicly.

As far as your options here, we’ve got a fairly standard list that you’ll be familiar with, so let’s just jump right in.

Google Cloud Platform Console

Coming in at the top, and among the easiest options, is the trusty Cloud Platform Console. You can use the Cloud Platform Console by simply right clicking on the object, or selecting the drop down menu (three dots) and selecting “Download.”

Apart from being just a matter of clicks, this is super helpful when you need to grab individual files while you’re debugging, or manually tricking through things.

GSUTIL

For a quick refresh, GSUTIL is a Python application that lets you access Cloud Storage from the command line, using a familiar language (more about it here).

The CP command will upload or download files from your local machine to Google Cloud. For our purposes, the command below will download from a Google Cloud Storage bucket to a local path on your device, but for other uses, you can check out this article.

gsutil cp gs://[BUCKET_NAME]/[OBJECT_NAME] [OBJECT_DESTINATION]

For added flexibility, GSUTIL also comes with a plethora of options to configure to your specific use case. For example, the -m command below allows for copy options to occur in parallel.

gsutil cp gs://[BUCKET_NAME]/[OBJECT_NAME] [OBJECT_DESTINATION] -m

As we’ve hinted at before, parallel uploads and downloads are a handy way to increase performance and avoid transactional overhead overload, but we’ll cover that in a later, dedicated blog post.

Client Libraries

If you need to download assets directly into your application, and if they don’t have publicly accessible URLs, you can use the Client libraries. As a reminder, we’ve got code samples and support for C++, C#, GO, Java, Node.JS, PHP, Python, and Ruby.

Here’s an example of downloading from your Cloud Storage bucket using Python.

def download_blob(bucket_name, source_blob_name, destination_file_name):

“””Downloads a blob from the bucket.”””

storage_client = storage.Client()

bucket = storage_client.get_bucket(bucket_name)

blob = bucket.blob(source_blob_name)

blob.download_to_filename(destination_file_name)

print(‘Blob {} downloaded to {}.’.format(

source_blob_name,

destination_file_name))

REST APIs

You can also go directly to the REST APIs using use other programs like cURL to fetch your files directly, and allow your users to log in with oauth for authentication and authorization (a bit more on that here, and in a later post).

For more information on the ways to download objects from buckets in Google Cloud Storage, you can check out this article.

Naturally, serving data comes along with questions about controlling access to that data, and luckily for you, we’ll be talking more about that in our next blog!

Google Cloud Developer Advocate, Thinker, Feeler, Adventurer, Surfer, Burner. Opinions are my own, but I’m happy to share.