GCS Bytes — Understanding and Managing Metadata

So far on Cloud Storage Bytes, we’ve talked about managing object data through renaming, moving, and deleting. But there’s another facet of managing objects that’s equally important: managing metadata. In this post, we’ll take a look at what metadata is available in a Cloud Storage object and how it’s used.

What is Cloud Storage object metadata?

Metadata identifies properties of a Cloud Storage object, and specifies how the object should be handled when it’s accessed. Metadata exists as key:value pairs. For example, Content-Type: image/png is a key:value pair that exists in GCS object metadata.

The mutability of metadata varies: some metadata you can edit at any time, some metadata you can only set at the time the object is created, and some metadata you can only view.

Editable metadata

There are two categories of metadata that are editable by users:

• Fixed-key metadata
• Custom metadata

With fixed-key metadata, keys are set, but you can specify a value. With custom metadata, you specify both a key and a value. You can edit this list of fixed-key metadata values for objects, if have sufficient permission to do so:

• Access control metadata
• Cache-Control
• Content-Disposition
• Content-Encoding
• Content-Language
• Content-Type

I know you’re curious to know what each of these metadata does. Fear not, gentle reader! Here’s a brief description of each:

Access control metadata

Cloud Storage uses Identity and Access Management and Access Control Lists to control access to objects, including access to the object’s metadata. If you’d like to know more about access control, check out our episode and related documentation!

Cache-Control

The Cache-Control metadata can specify two different aspects of how data is served from Cloud Storage: if it can be cached and if it can be transformed. I won’t say more about Cache-Control here because I’m expanding on the topic in a future video. You can also find out more in the documentation linked below.

Content-Disposition

The Content-Disposition metadata specifies presentation information about the data being transmitted. Setting Content-Disposition allows you to control presentation style of the content, for example determining whether an attachment should be automatically displayed or whether some form of action from the user should be required to open it. Check the links in the description to see the Content-Disposition specification.

Content-Encoding

The Content-Encoding metadata can be used to indicate that an object is compressed, while still maintaining the object’s underlying Content-Type. For example, a text file that is gzip compressed can have the fact that it’s a text file indicated in Content-Type and the fact that it’s gzip compressed indicated in Content-Encoding.

Content-Language

The Content-Language metadata indicates the language(s) that the object is intended for. Refer to ISO 639–1 language codes for the supported values of this metadata.

Content-Type

The most commonly set metadata is Content-Type, also known as media type. This allows browsers to render the object properly. All objects have a value specified in their Content-Type metadata, but this value does not have to match the underlying type of the object. For example, if the Content-Type isn’t specified when the object gets uploaded, and it’s not a common data type, chances are it’s going to be something generic, or just wrong in the metadata.

Custom metadata

Custom metadata are like standard metadata, except you can specify custom metadata key-value pairs. To create custom metadata, you specify both a value and a key. Once you have created a custom metadata key:value pair, you can delete the key or change the value

Editing fixed-key metadata

You can edit fixed-key and custom metadata from the REST API, gsutil, and the Google Cloud Platform Console. Here are the steps to editing in the console:

1. Open the Cloud Storage browser in the Google Cloud Platform Console.
2. In the list of buckets, click on the name of the bucket that contains the desired object, and navigate to the object.
3. Click the more options button (More actions icon.) associated with the object.
4. Click Edit metadata.
5. In the overlay window that appears, make the desired edits to the metadata.
6. For standard metadata fields, edit the Value.
7. Add your own custom metadata by clicking the + Add item button.
8. You can edit both the Key and Value of your custom metadata.
9. Delete your custom metadata by clicking the associated X.
10. Finally, click Save.

Fixed metadata

Some metadata cannot be edited directly. This metadata is set at the time of object creation or rewrite. As part of the object creation or rewrite, you can set some such metadata, such as the storage class of the object or customer-supplied encryption keys. Other metadata is automatically added and can only be viewed, such as the generation number of the object or the time of creation.

Hopefully you’ve managed to stay awake through all this metadata management talk! Thanks for joining us for this quick byte of cloud storage!