Package storage provides an easy way to work with Google Cloud Storage. Google Cloud Storage stores data in named objects, which are grouped into buckets.
More information about Google Cloud Storage is available at https://cloud.google.com/storage/docs.
See https://godoc.org/cloud.google.com/go for authentication, timeouts, connection pooling and similar aspects of this package.
All of the methods of this package use exponential backoff to retry calls that fail with certain errors, as described in https://cloud.google.com/storage/docs/exponential-backoff. Retrying continues indefinitely unless the controlling context is canceled or the client is closed. See context.WithTimeout and context.WithCancel.
Creating a Client
To start working with this package, create a client:
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: Handle error.
}
The client will use your default application credentials. Clients should be reused instead of created as needed. The methods of Client are safe for concurrent use by multiple goroutines.
If you only wish to access public data, you can create an unauthenticated client with
client, err := storage.NewClient(ctx, option.WithoutAuthentication())
Buckets
A Google Cloud Storage bucket is a collection of objects. To work with a bucket, make a bucket handle:
bkt := client.Bucket(bucketName)
A handle is a reference to a bucket. You can have a handle even if the bucket doesn't exist yet. To create a bucket in Google Cloud Storage, call Create on the handle:
if err := bkt.Create(ctx, projectID, nil); err != nil {
// TODO: Handle error.
}
Note that although buckets are associated with projects, bucket names are global across all projects.
Each bucket has associated metadata, represented in this package by BucketAttrs. The third argument to BucketHandle.Create allows you to set the initial BucketAttrs of a bucket. To retrieve a bucket's attributes, use Attrs:
attrs, err := bkt.Attrs(ctx)
if err != nil {
// TODO: Handle error.
}
fmt.Printf("bucket %s, created at %s, is located in %s with storage class %s\n",
attrs.Name, attrs.Created, attrs.Location, attrs.StorageClass)
Objects
An object holds arbitrary data as a sequence of bytes, like a file. You refer to objects using a handle, just as with buckets, but unlike buckets you don't explicitly create an object. Instead, the first time you write to an object it will be created. You can use the standard Go io.Reader and io.Writer interfaces to read and write object data:
obj := bkt.Object("data")
// Write something to obj.
// w implements io.Writer.
w := obj.NewWriter(ctx)
// Write some text to obj. This will either create the object or overwrite whatever is there already.
if _, err := fmt.Fprintf(w, "This object contains text.\n"); err != nil {
// TODO: Handle error.
}
// Close, just like writing a file.
if err := w.Close(); err != nil {
// TODO: Handle error.
}
// Read it back.
r, err := obj.NewReader(ctx)
if err != nil {
// TODO: Handle error.
}
defer r.Close()
if _, err := io.Copy(os.Stdout, r); err != nil {
// TODO: Handle error.
}
// Prints "This object contains text."
Objects also have attributes, which you can fetch with Attrs:
objAttrs, err := obj.Attrs(ctx)
if err != nil {
// TODO: Handle error.
}
fmt.Printf("object %s has size %d and can be read using %s\n",
objAttrs.Name, objAttrs.Size, objAttrs.MediaLink)
Listing objects
Listing objects in a bucket is done with the Bucket.Objects method:
query := &storage.Query{Prefix: ""}
var names []string
it := bkt.Objects(ctx, query)
for {
attrs, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
log.Fatal(err)
}
names = append(names, attrs.Name)
}
If only a subset of object attributes is needed when listing, specifying this subset using Query.SetAttrSelection may speed up the listing process:
query := &storage.Query{Prefix: ""}
query.SetAttrSelection([]string{"Name"})
// ... as before
ACLs
Both objects and buckets have ACLs (Access Control Lists). An ACL is a list of ACLRules, each of which specifies the role of a user, group or project. ACLs are suitable for fine-grained control, but you may prefer using IAM to control access at the project level (see https://cloud.google.com/storage/docs/access-control/iam).
To list the ACLs of a bucket or object, obtain an ACLHandle and call its List method:
acls, err := obj.ACL().List(ctx)
if err != nil {
// TODO: Handle error.
}
for _, rule := range acls {
fmt.Printf("%s has role %s\n", rule.Entity, rule.Role)
}
You can also set and delete ACLs.
Conditions
Every object has a generation and a metageneration. The generation changes whenever the content changes, and the metageneration changes whenever the metadata changes. Conditions let you check these values before an operation; the operation only executes if the conditions match. You can use conditions to prevent race conditions in read-modify-write operations.
For example, say you've read an object's metadata into objAttrs. Now you want to write to that object, but only if its contents haven't changed since you read it. Here is how to express that:
w = obj.If(storage.Conditions{GenerationMatch: objAttrs.Generation}).NewWriter(ctx)
// Proceed with writing as above.
Signed URLs
You can obtain a URL that lets anyone read or write an object for a limited time. You don't need to create a client to do this. See the documentation of SignedURL for details.
url, err := storage.SignedURL(bucketName, "shared-object", opts)
if err != nil {
// TODO: Handle error.
}
fmt.Println(url)
Post Policy V4 Signed Request
A type of signed request that allows uploads through HTML forms directly to Cloud Storage with temporary permission. Conditions can be applied to restrict how the HTML form is used and exercised by a user.
For more information, please see https://cloud.google.com/storage/docs/xml-api/post-object as well as the documentation of GenerateSignedPostPolicyV4.
pv4, err := storage.GenerateSignedPostPolicyV4(bucketName, objectName, opts)
if err != nil {
// TODO: Handle error.
}
fmt.Printf("URL: %s\nFields; %v\n", pv4.URL, pv4.Fields)
Errors
Errors returned by this client are often of the type googleapi.Error
.
These errors can be introspected for more information by type asserting to the richer googleapi.Error
type. For example:
if e, ok := err.(*googleapi.Error); ok {
if e.Code == 409 { ... }
}
Constants
DeleteAction, SetStorageClassAction
Declaration
const (
// DeleteAction is a lifecycle action that deletes a live and/or archived
// objects. Takes precedence over SetStorageClass actions.
DeleteAction = "Delete"
// SetStorageClassAction changes the storage class of live and/or archived
// objects.
SetStorageClassAction = "SetStorageClass"
)
NoPayload, JSONPayload
Values for Notification.PayloadFormat.
Declaration
const (
// Send no payload with notification messages.
NoPayload = "NONE"
// Send object metadata as JSON with notification messages.
JSONPayload = "JSON_API_V1"
)
ObjectFinalizeEvent, ObjectMetadataUpdateEvent, ObjectDeleteEvent, ObjectArchiveEvent
Values for Notification.EventTypes.
Declaration
const (
// Event that occurs when an object is successfully created.
ObjectFinalizeEvent = "OBJECT_FINALIZE"
// Event that occurs when the metadata of an existing object changes.
ObjectMetadataUpdateEvent = "OBJECT_METADATA_UPDATE"
// Event that occurs when an object is permanently deleted.
ObjectDeleteEvent = "OBJECT_DELETE"
// Event that occurs when the live version of an object becomes an
// archived version.
ObjectArchiveEvent = "OBJECT_ARCHIVE"
)
ScopeFullControl, ScopeReadOnly, ScopeReadWrite
Declaration
const (
// ScopeFullControl grants permissions to manage your
// data and permissions in Google Cloud Storage.
ScopeFullControl = raw.DevstorageFullControlScope
// ScopeReadOnly grants permissions to
// view your data in Google Cloud Storage.
ScopeReadOnly = raw.DevstorageReadOnlyScope
// ScopeReadWrite grants permissions to manage your
// data in Google Cloud Storage.
ScopeReadWrite = raw.DevstorageReadWriteScope
)
Variables
ErrBucketNotExist, ErrObjectNotExist
Declaration
var (
// ErrBucketNotExist indicates that the bucket does not exist.
ErrBucketNotExist = errors.New("storage: bucket doesn't exist")
// ErrObjectNotExist indicates that the object does not exist.
ErrObjectNotExist = errors.New("storage: object doesn't exist")
)
Types
ACLEntity
ACLEntity refers to a user or group. They are sometimes referred to as grantees.
It could be in the form of:
"user-
Or one of the predefined constants: AllUsers, AllAuthenticatedUsers.
Declaration
type ACLEntity string
AllUsers, AllAuthenticatedUsers
Declaration
const (
AllUsers ACLEntity = "allUsers"
AllAuthenticatedUsers ACLEntity = "allAuthenticatedUsers"
)
ACLHandle
ACLHandle provides operations on an access control list for a Google Cloud Storage bucket or object.
Declaration
type ACLHandle struct {
// contains filtered or unexported fields
}
func (*ACLHandle) Delete
Delete permanently deletes the ACL entry for the given entity.
Declaration
func (a *ACLHandle) Delete(ctx context.Context, entity ACLEntity) (err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// No longer grant access to the bucket to everyone on the Internet.
if err := client.Bucket("my-bucket").ACL().Delete(ctx, storage.AllUsers); err != nil {
// TODO: handle error.
}
}
{% endverbatim %}
func (*ACLHandle) List
List retrieves ACL entries.
Declaration
func (a *ACLHandle) List(ctx context.Context) (rules []ACLRule, err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// List the default object ACLs for my-bucket.
aclRules, err := client.Bucket("my-bucket").DefaultObjectACL().List(ctx)
if err != nil {
// TODO: handle error.
}
fmt.Println(aclRules)
}
{% endverbatim %}
func (*ACLHandle) Set
Set sets the role for the given entity.
Declaration
func (a *ACLHandle) Set(ctx context.Context, entity ACLEntity, role ACLRole) (err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Let any authenticated user read my-bucket/my-object.
obj := client.Bucket("my-bucket").Object("my-object")
if err := obj.ACL().Set(ctx, storage.AllAuthenticatedUsers, storage.RoleReader); err != nil {
// TODO: handle error.
}
}
{% endverbatim %}
ACLRole
ACLRole is the level of access to grant.
Declaration
type ACLRole string
RoleOwner, RoleReader, RoleWriter
Declaration
const (
RoleOwner ACLRole = "OWNER"
RoleReader ACLRole = "READER"
RoleWriter ACLRole = "WRITER"
)
ACLRule
ACLRule represents a grant for a role to an entity (user, group or team) for a Google Cloud Storage object or bucket.
Declaration
type ACLRule struct {
Entity ACLEntity
EntityID string
Role ACLRole
Domain string
Email string
ProjectTeam *ProjectTeam
}
BucketAttrs
BucketAttrs represents the metadata for a Google Cloud Storage bucket. Read-only fields are ignored by BucketHandle.Create.
Declaration
type BucketAttrs struct {
// Name is the name of the bucket.
// This field is read-only.
Name string
// ACL is the list of access control rules on the bucket.
ACL []ACLRule
// BucketPolicyOnly is an alias for UniformBucketLevelAccess. Use of
// UniformBucketLevelAccess is recommended above the use of this field.
// Setting BucketPolicyOnly.Enabled OR UniformBucketLevelAccess.Enabled to
// true, will enable UniformBucketLevelAccess.
BucketPolicyOnly BucketPolicyOnly
// UniformBucketLevelAccess configures access checks to use only bucket-level IAM
// policies and ignore any ACL rules for the bucket.
// See https://cloud.google.com/storage/docs/uniform-bucket-level-access
// for more information.
UniformBucketLevelAccess UniformBucketLevelAccess
// DefaultObjectACL is the list of access controls to
// apply to new objects when no object ACL is provided.
DefaultObjectACL []ACLRule
// DefaultEventBasedHold is the default value for event-based hold on
// newly created objects in this bucket. It defaults to false.
DefaultEventBasedHold bool
// If not empty, applies a predefined set of access controls. It should be set
// only when creating a bucket.
// It is always empty for BucketAttrs returned from the service.
// See https://cloud.google.com/storage/docs/json_api/v1/buckets/insert
// for valid values.
PredefinedACL string
// If not empty, applies a predefined set of default object access controls.
// It should be set only when creating a bucket.
// It is always empty for BucketAttrs returned from the service.
// See https://cloud.google.com/storage/docs/json_api/v1/buckets/insert
// for valid values.
PredefinedDefaultObjectACL string
// Location is the location of the bucket. It defaults to "US".
Location string
// MetaGeneration is the metadata generation of the bucket.
// This field is read-only.
MetaGeneration int64
// StorageClass is the default storage class of the bucket. This defines
// how objects in the bucket are stored and determines the SLA
// and the cost of storage. Typical values are "STANDARD", "NEARLINE",
// "COLDLINE" and "ARCHIVE". Defaults to "STANDARD".
// See https://cloud.google.com/storage/docs/storage-classes for all
// valid values.
StorageClass string
// Created is the creation time of the bucket.
// This field is read-only.
Created time.Time
// VersioningEnabled reports whether this bucket has versioning enabled.
VersioningEnabled bool
// Labels are the bucket's labels.
Labels map[string]string
// RequesterPays reports whether the bucket is a Requester Pays bucket.
// Clients performing operations on Requester Pays buckets must provide
// a user project (see BucketHandle.UserProject), which will be billed
// for the operations.
RequesterPays bool
// Lifecycle is the lifecycle configuration for objects in the bucket.
Lifecycle Lifecycle
// Retention policy enforces a minimum retention time for all objects
// contained in the bucket. A RetentionPolicy of nil implies the bucket
// has no minimum data retention.
//
// This feature is in private alpha release. It is not currently available to
// most customers. It might be changed in backwards-incompatible ways and is not
// subject to any SLA or deprecation policy.
RetentionPolicy *RetentionPolicy
// The bucket's Cross-Origin Resource Sharing (CORS) configuration.
CORS []CORS
// The encryption configuration used by default for newly inserted objects.
Encryption *BucketEncryption
// The logging configuration.
Logging *BucketLogging
// The website configuration.
Website *BucketWebsite
// Etag is the HTTP/1.1 Entity tag for the bucket.
// This field is read-only.
Etag string
// LocationType describes how data is stored and replicated.
// Typical values are "multi-region", "region" and "dual-region".
// This field is read-only.
LocationType string
}
BucketAttrsToUpdate
BucketAttrsToUpdate define the attributes to update during an Update call.
Declaration
type BucketAttrsToUpdate struct {
// If set, updates whether the bucket uses versioning.
VersioningEnabled optional.Bool
// If set, updates whether the bucket is a Requester Pays bucket.
RequesterPays optional.Bool
// DefaultEventBasedHold is the default value for event-based hold on
// newly created objects in this bucket.
DefaultEventBasedHold optional.Bool
// BucketPolicyOnly is an alias for UniformBucketLevelAccess. Use of
// UniformBucketLevelAccess is recommended above the use of this field.
// Setting BucketPolicyOnly.Enabled OR UniformBucketLevelAccess.Enabled to
// true, will enable UniformBucketLevelAccess. If both BucketPolicyOnly and
// UniformBucketLevelAccess are set, the value of UniformBucketLevelAccess
// will take precedence.
BucketPolicyOnly *BucketPolicyOnly
// UniformBucketLevelAccess configures access checks to use only bucket-level IAM
// policies and ignore any ACL rules for the bucket.
// See https://cloud.google.com/storage/docs/uniform-bucket-level-access
// for more information.
UniformBucketLevelAccess *UniformBucketLevelAccess
// If set, updates the retention policy of the bucket. Using
// RetentionPolicy.RetentionPeriod = 0 will delete the existing policy.
//
// This feature is in private alpha release. It is not currently available to
// most customers. It might be changed in backwards-incompatible ways and is not
// subject to any SLA or deprecation policy.
RetentionPolicy *RetentionPolicy
// If set, replaces the CORS configuration with a new configuration.
// An empty (rather than nil) slice causes all CORS policies to be removed.
CORS []CORS
// If set, replaces the encryption configuration of the bucket. Using
// BucketEncryption.DefaultKMSKeyName = "" will delete the existing
// configuration.
Encryption *BucketEncryption
// If set, replaces the lifecycle configuration of the bucket.
Lifecycle *Lifecycle
// If set, replaces the logging configuration of the bucket.
Logging *BucketLogging
// If set, replaces the website configuration of the bucket.
Website *BucketWebsite
// If not empty, applies a predefined set of access controls.
// See https://cloud.google.com/storage/docs/json_api/v1/buckets/patch.
PredefinedACL string
// If not empty, applies a predefined set of default object access controls.
// See https://cloud.google.com/storage/docs/json_api/v1/buckets/patch.
PredefinedDefaultObjectACL string
// contains filtered or unexported fields
}
func (*BucketAttrsToUpdate) DeleteLabel
DeleteLabel causes a label to be deleted when ua is used in a call to Bucket.Update.
Declaration
func (ua *BucketAttrsToUpdate) DeleteLabel(name string)
func (*BucketAttrsToUpdate) SetLabel
SetLabel causes a label to be added or modified when ua is used in a call to Bucket.Update.
Declaration
func (ua *BucketAttrsToUpdate) SetLabel(name, value string)
BucketConditions
BucketConditions constrain bucket methods to act on specific metagenerations.
The zero value is an empty set of constraints.
Declaration
type BucketConditions struct {
// MetagenerationMatch specifies that the bucket must have the given
// metageneration for the operation to occur.
// If MetagenerationMatch is zero, it has no effect.
MetagenerationMatch int64
// MetagenerationNotMatch specifies that the bucket must not have the given
// metageneration for the operation to occur.
// If MetagenerationNotMatch is zero, it has no effect.
MetagenerationNotMatch int64
}
BucketEncryption
BucketEncryption is a bucket's encryption configuration.
Declaration
type BucketEncryption struct {
// A Cloud KMS key name, in the form
// projects/P/locations/L/keyRings/R/cryptoKeys/K, that will be used to encrypt
// objects inserted into this bucket, if no encryption method is specified.
// The key's location must be the same as the bucket's.
DefaultKMSKeyName string
}
BucketHandle
BucketHandle provides operations on a Google Cloud Storage bucket. Use Client.Bucket to get a handle.
Declaration
type BucketHandle struct {
// contains filtered or unexported fields
}
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Attrs(ctx)
if err == storage.ErrBucketNotExist {
fmt.Println("The bucket does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The bucket exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
func (*BucketHandle) ACL
ACL returns an ACLHandle, which provides access to the bucket's access control list. This controls who can list, create or overwrite the objects in a bucket. This call does not perform any network operations.
Declaration
func (b *BucketHandle) ACL() *ACLHandle
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Attrs(ctx)
if err == storage.ErrBucketNotExist {
fmt.Println("The bucket does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The bucket exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
func (*BucketHandle) AddNotification
AddNotification adds a notification to b. You must set n's TopicProjectID, TopicID and PayloadFormat, and must not set its ID. The other fields are all optional. The returned Notification's ID can be used to refer to it.
Declaration
func (b *BucketHandle) AddNotification(ctx context.Context, n *Notification) (ret *Notification, err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
b := client.Bucket("my-bucket")
n, err := b.AddNotification(ctx, &storage.Notification{
TopicProjectID: "my-project",
TopicID: "my-topic",
PayloadFormat: storage.JSONPayload,
})
if err != nil {
// TODO: handle error.
}
fmt.Println(n.ID)
}
{% endverbatim %}
func (*BucketHandle) Attrs
Attrs returns the metadata for the bucket.
Declaration
func (b *BucketHandle) Attrs(ctx context.Context) (attrs *BucketAttrs, err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Attrs(ctx)
if err != nil {
// TODO: handle error.
}
fmt.Println(attrs)
}
{% endverbatim %}
func (*BucketHandle) Create
Create creates the Bucket in the project. If attrs is nil the API defaults will be used.
Declaration
func (b *BucketHandle) Create(ctx context.Context, projectID string, attrs *BucketAttrs) (err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
if err := client.Bucket("my-bucket").Create(ctx, "my-project", nil); err != nil {
// TODO: handle error.
}
}
{% endverbatim %}
func (*BucketHandle) DefaultObjectACL
DefaultObjectACL returns an ACLHandle, which provides access to the bucket's default object ACLs. These ACLs are applied to newly created objects in this bucket that do not have a defined ACL. This call does not perform any network operations.
Declaration
func (b *BucketHandle) DefaultObjectACL() *ACLHandle
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Attrs(ctx)
if err == storage.ErrBucketNotExist {
fmt.Println("The bucket does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The bucket exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
func (*BucketHandle) Delete
Delete deletes the Bucket.
Declaration
func (b *BucketHandle) Delete(ctx context.Context) (err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
if err := client.Bucket("my-bucket").Delete(ctx); err != nil {
// TODO: handle error.
}
}
{% endverbatim %}
func (*BucketHandle) DeleteNotification
DeleteNotification deletes the notification with the given ID.
Declaration
func (b *BucketHandle) DeleteNotification(ctx context.Context, id string) (err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
var notificationID string
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
b := client.Bucket("my-bucket")
// TODO: Obtain notificationID from BucketHandle.AddNotification
// or BucketHandle.Notifications.
err = b.DeleteNotification(ctx, notificationID)
if err != nil {
// TODO: handle error.
}
}
{% endverbatim %}
func (*BucketHandle) IAM
IAM provides access to IAM access control for the bucket.
Declaration
func (b *BucketHandle) IAM() *iam.Handle
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Attrs(ctx)
if err == storage.ErrBucketNotExist {
fmt.Println("The bucket does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The bucket exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
func (*BucketHandle) If
If returns a new BucketHandle that applies a set of preconditions. Preconditions already set on the BucketHandle are ignored. Operations on the new handle will return an error if the preconditions are not satisfied. The only valid preconditions for buckets are MetagenerationMatch and MetagenerationNotMatch.
Declaration
func (b *BucketHandle) If(conds BucketConditions) *BucketHandle
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Attrs(ctx)
if err == storage.ErrBucketNotExist {
fmt.Println("The bucket does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The bucket exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
func (*BucketHandle) LockRetentionPolicy
LockRetentionPolicy locks a bucket's retention policy until a previously-configured RetentionPeriod past the EffectiveTime. Note that if RetentionPeriod is set to less than a day, the retention policy is treated as a development configuration and locking will have no effect. The BucketHandle must have a metageneration condition that matches the bucket's metageneration. See BucketHandle.If.
This feature is in private alpha release. It is not currently available to most customers. It might be changed in backwards-incompatible ways and is not subject to any SLA or deprecation policy.
Declaration
func (b *BucketHandle) LockRetentionPolicy(ctx context.Context) error
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
b := client.Bucket("my-bucket")
attrs, err := b.Attrs(ctx)
if err != nil {
// TODO: handle error.
}
// Note that locking the bucket without first attaching a RetentionPolicy
// that's at least 1 day is a no-op
err = b.If(storage.BucketConditions{MetagenerationMatch: attrs.MetaGeneration}).LockRetentionPolicy(ctx)
if err != nil {
// TODO: handle err
}
}
{% endverbatim %}
func (*BucketHandle) Notifications
Notifications returns all the Notifications configured for this bucket, as a map indexed by notification ID.
Declaration
func (b *BucketHandle) Notifications(ctx context.Context) (n map[string]*Notification, err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
b := client.Bucket("my-bucket")
ns, err := b.Notifications(ctx)
if err != nil {
// TODO: handle error.
}
for id, n := range ns {
fmt.Printf("%s: %+v\n", id, n)
}
}
{% endverbatim %}
func (*BucketHandle) Object
Object returns an ObjectHandle, which provides operations on the named object. This call does not perform any network operations.
name must consist entirely of valid UTF-8-encoded runes. The full specification for valid object names can be found at: https://cloud.google.com/storage/docs/bucket-naming
Declaration
func (b *BucketHandle) Object(name string) *ObjectHandle
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Attrs(ctx)
if err == storage.ErrBucketNotExist {
fmt.Println("The bucket does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The bucket exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
func (*BucketHandle) Objects
Objects returns an iterator over the objects in the bucket that match the Query q. If q is nil, no filtering is done.
Note: The returned iterator is not safe for concurrent operations without explicit synchronization.
Declaration
func (b *BucketHandle) Objects(ctx context.Context, q *Query) *ObjectIterator
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
it := client.Bucket("my-bucket").Objects(ctx, nil)
_ = it // TODO: iterate using Next or iterator.Pager.
}
{% endverbatim %}
func (*BucketHandle) Update
Update updates a bucket's attributes.
Declaration
func (b *BucketHandle) Update(ctx context.Context, uattrs BucketAttrsToUpdate) (attrs *BucketAttrs, err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Enable versioning in the bucket, regardless of its previous value.
attrs, err := client.Bucket("my-bucket").Update(ctx,
storage.BucketAttrsToUpdate{VersioningEnabled: true})
if err != nil {
// TODO: handle error.
}
fmt.Println(attrs)
}
{% endverbatim %}
readModifyWrite
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
b := client.Bucket("my-bucket")
attrs, err := b.Attrs(ctx)
if err != nil {
// TODO: handle error.
}
var au storage.BucketAttrsToUpdate
au.SetLabel("lab", attrs.Labels["lab"]+"-more")
if attrs.Labels["delete-me"] == "yes" {
au.DeleteLabel("delete-me")
}
attrs, err = b.
If(storage.BucketConditions{MetagenerationMatch: attrs.MetaGeneration}).
Update(ctx, au)
if err != nil {
// TODO: handle error.
}
fmt.Println(attrs)
}
{% endverbatim %}
func (*BucketHandle) UserProject
UserProject returns a new BucketHandle that passes the project ID as the user project for all subsequent calls. Calls with a user project will be billed to that project rather than to the bucket's owning project.
A user project is required for all operations on Requester Pays buckets.
Declaration
func (b *BucketHandle) UserProject(projectID string) *BucketHandle
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Attrs(ctx)
if err == storage.ErrBucketNotExist {
fmt.Println("The bucket does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The bucket exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
BucketIterator
A BucketIterator is an iterator over BucketAttrs.
Note: This iterator is not safe for concurrent operations without explicit synchronization.
Declaration
type BucketIterator struct {
// Prefix restricts the iterator to buckets whose names begin with it.
Prefix string
// contains filtered or unexported fields
}
func (*BucketIterator) Next
Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns iterator.Done, all subsequent calls will return iterator.Done.
Note: This method is not safe for concurrent operations without explicit synchronization.
Declaration
func (it *BucketIterator) Next() (*BucketAttrs, error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"google.golang.org/api/iterator"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
it := client.Buckets(ctx, "my-project")
for {
bucketAttrs, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
// TODO: Handle error.
}
fmt.Println(bucketAttrs)
}
}
{% endverbatim %}
func (*BucketIterator) PageInfo
PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
Note: This method is not safe for concurrent operations without explicit synchronization.
Declaration
func (it *BucketIterator) PageInfo() *iterator.PageInfo
BucketLogging
BucketLogging holds the bucket's logging configuration, which defines the destination bucket and optional name prefix for the current bucket's logs.
Declaration
type BucketLogging struct {
// The destination bucket where the current bucket's logs
// should be placed.
LogBucket string
// A prefix for log object names.
LogObjectPrefix string
}
BucketPolicyOnly
BucketPolicyOnly is an alias for UniformBucketLevelAccess. Use of UniformBucketLevelAccess is preferred above BucketPolicyOnly.
Declaration
type BucketPolicyOnly struct {
// Enabled specifies whether access checks use only bucket-level IAM
// policies. Enabled may be disabled until the locked time.
Enabled bool
// LockedTime specifies the deadline for changing Enabled from true to
// false.
LockedTime time.Time
}
BucketWebsite
BucketWebsite holds the bucket's website configuration, controlling how the service behaves when accessing bucket contents as a web site. See https://cloud.google.com/storage/docs/static-website for more information.
Declaration
type BucketWebsite struct {
// If the requested object path is missing, the service will ensure the path has
// a trailing '/', append this suffix, and attempt to retrieve the resulting
// object. This allows the creation of index.html objects to represent directory
// pages.
MainPageSuffix string
// If the requested object path is missing, and any mainPageSuffix object is
// missing, if applicable, the service will return the named object from this
// bucket as the content for a 404 Not Found result.
NotFoundPage string
}
CORS
CORS is the bucket's Cross-Origin Resource Sharing (CORS) configuration.
Declaration
type CORS struct {
// MaxAge is the value to return in the Access-Control-Max-Age
// header used in preflight responses.
MaxAge time.Duration
// Methods is the list of HTTP methods on which to include CORS response
// headers, (GET, OPTIONS, POST, etc) Note: "*" is permitted in the list
// of methods, and means "any method".
Methods []string
// Origins is the list of Origins eligible to receive CORS response
// headers. Note: "*" is permitted in the list of origins, and means
// "any Origin".
Origins []string
// ResponseHeaders is the list of HTTP headers other than the simple
// response headers to give permission for the user-agent to share
// across domains.
ResponseHeaders []string
}
Client
Client is a client for interacting with Google Cloud Storage.
Clients should be reused instead of created as needed. The methods of Client are safe for concurrent use by multiple goroutines.
Declaration
type Client struct {
// contains filtered or unexported fields
}
func NewClient
NewClient creates a new Google Cloud Storage client. The default scope is ScopeFullControl. To use a different scope, like ScopeReadOnly, use option.WithScopes.
Clients should be reused instead of created as needed. The methods of Client are safe for concurrent use by multiple goroutines.
Declaration
func NewClient(ctx context.Context, opts ...option.ClientOption) (*Client, error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
// Use Google Application Default Credentials to authorize and authenticate the client.
// More information about Application Default Credentials and how to enable is at
// https://developers.google.com/identity/protocols/application-default-credentials.
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Use the client.
// Close the client when finished.
if err := client.Close(); err != nil {
// TODO: handle error.
}
}
{% endverbatim %}
unauthenticated
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"google.golang.org/api/option"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx, option.WithoutAuthentication())
if err != nil {
// TODO: handle error.
}
// Use the client.
// Close the client when finished.
if err := client.Close(); err != nil {
// TODO: handle error.
}
}
{% endverbatim %}
func (*Client) Bucket
Bucket returns a BucketHandle, which provides operations on the named bucket. This call does not perform any network operations.
The supplied name must contain only lowercase letters, numbers, dashes, underscores, and dots. The full specification for valid bucket names can be found at: https://cloud.google.com/storage/docs/bucket-naming
Declaration
func (c *Client) Bucket(name string) *BucketHandle
func (*Client) Buckets
Buckets returns an iterator over the buckets in the project. You may optionally set the iterator's Prefix field to restrict the list to buckets whose names begin with the prefix. By default, all buckets in the project are returned.
Note: The returned iterator is not safe for concurrent operations without explicit synchronization.
Declaration
func (c *Client) Buckets(ctx context.Context, projectID string) *BucketIterator
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
it := client.Buckets(ctx, "my-bucket")
_ = it // TODO: iterate using Next or iterator.Pager.
}
{% endverbatim %}
func (*Client) Close
Close closes the Client.
Close need not be called at program exit.
Declaration
func (c *Client) Close() error
func (*Client) CreateHMACKey
CreateHMACKey invokes an RPC for Google Cloud Storage to create a new HMACKey.
This method is EXPERIMENTAL and subject to change or removal without notice.
Declaration
func (c *Client) CreateHMACKey(ctx context.Context, projectID, serviceAccountEmail string, ...) (*HMACKey, error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
hkey, err := client.CreateHMACKey(ctx, "project-id", "service-account-email")
if err != nil {
// TODO: handle error.
}
_ = hkey // TODO: Use the HMAC Key.
}
{% endverbatim %}
func (*Client) HMACKeyHandle
HMACKeyHandle creates a handle that will be used for HMACKey operations.
This method is EXPERIMENTAL and subject to change or removal without notice.
Declaration
func (c *Client) HMACKeyHandle(projectID, accessID string) *HMACKeyHandle
func (*Client) ListHMACKeys
ListHMACKeys returns an iterator for listing HMACKeys.
Note: This iterator is not safe for concurrent operations without explicit synchronization.
This method is EXPERIMENTAL and subject to change or removal without notice.
Declaration
func (c *Client) ListHMACKeys(ctx context.Context, projectID string, opts ...HMACKeyOption) *HMACKeysIterator
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"google.golang.org/api/iterator"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
iter := client.ListHMACKeys(ctx, "project-id")
for {
key, err := iter.Next()
if err == iterator.Done {
break
}
if err != nil {
// TODO: handle error.
}
_ = key // TODO: Use the key.
}
}
{% endverbatim %}
forServiceAccountEmail
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"google.golang.org/api/iterator"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
iter := client.ListHMACKeys(ctx, "project-id", storage.ForHMACKeyServiceAccountEmail("service@account.email"))
for {
key, err := iter.Next()
if err == iterator.Done {
break
}
if err != nil {
// TODO: handle error.
}
_ = key // TODO: Use the key.
}
}
{% endverbatim %}
showDeletedKeys
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"google.golang.org/api/iterator"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
iter := client.ListHMACKeys(ctx, "project-id", storage.ShowDeletedHMACKeys())
for {
key, err := iter.Next()
if err == iterator.Done {
break
}
if err != nil {
// TODO: handle error.
}
_ = key // TODO: Use the key.
}
}
{% endverbatim %}
func (*Client) ServiceAccount
ServiceAccount fetches the email address of the given project's Google Cloud Storage service account.
Declaration
func (c *Client) ServiceAccount(ctx context.Context, projectID string) (string, error)
Composer
A Composer composes source objects into a destination object.
For Requester Pays buckets, the user project of dst is billed.
Declaration
type Composer struct {
// ObjectAttrs are optional attributes to set on the destination object.
// Any attributes must be initialized before any calls on the Composer. Nil
// or zero-valued attributes are ignored.
ObjectAttrs
// SendCRC specifies whether to transmit a CRC32C field. It should be set
// to true in addition to setting the Composer's CRC32C field, because zero
// is a valid CRC and normally a zero would not be transmitted.
// If a CRC32C is sent, and the data in the destination object does not match
// the checksum, the compose will be rejected.
SendCRC32C bool
// contains filtered or unexported fields
}
func (*Composer) Run
Run performs the compose operation.
Declaration
func (c *Composer) Run(ctx context.Context) (attrs *ObjectAttrs, err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
bkt := client.Bucket("bucketname")
src1 := bkt.Object("o1")
src2 := bkt.Object("o2")
dst := bkt.Object("o3")
// Compose and modify metadata.
c := dst.ComposerFrom(src1, src2)
c.ContentType = "text/plain"
// Set the expected checksum for the destination object to be validated by
// the backend (if desired).
c.CRC32C = 42
c.SendCRC32C = true
attrs, err := c.Run(ctx)
if err != nil {
// TODO: Handle error.
}
fmt.Println(attrs)
// Just compose.
attrs, err = dst.ComposerFrom(src1, src2).Run(ctx)
if err != nil {
// TODO: Handle error.
}
fmt.Println(attrs)
}
{% endverbatim %}
Conditions
Conditions constrain methods to act on specific generations of objects.
The zero value is an empty set of constraints. Not all conditions or combinations of conditions are applicable to all methods. See https://cloud.google.com/storage/docs/generations-preconditions for details on how these operate.
Declaration
type Conditions struct {
// GenerationMatch specifies that the object must have the given generation
// for the operation to occur.
// If GenerationMatch is zero, it has no effect.
// Use DoesNotExist to specify that the object does not exist in the bucket.
GenerationMatch int64
// GenerationNotMatch specifies that the object must not have the given
// generation for the operation to occur.
// If GenerationNotMatch is zero, it has no effect.
GenerationNotMatch int64
// DoesNotExist specifies that the object must not exist in the bucket for
// the operation to occur.
// If DoesNotExist is false, it has no effect.
DoesNotExist bool
// MetagenerationMatch specifies that the object must have the given
// metageneration for the operation to occur.
// If MetagenerationMatch is zero, it has no effect.
MetagenerationMatch int64
// MetagenerationNotMatch specifies that the object must not have the given
// metageneration for the operation to occur.
// If MetagenerationNotMatch is zero, it has no effect.
MetagenerationNotMatch int64
}
Copier
A Copier copies a source object to a destination.
Declaration
type Copier struct {
// ObjectAttrs are optional attributes to set on the destination object.
// Any attributes must be initialized before any calls on the Copier. Nil
// or zero-valued attributes are ignored.
ObjectAttrs
// RewriteToken can be set before calling Run to resume a copy
// operation. After Run returns a non-nil error, RewriteToken will
// have been updated to contain the value needed to resume the copy.
RewriteToken string
// ProgressFunc can be used to monitor the progress of a multi-RPC copy
// operation. If ProgressFunc is not nil and copying requires multiple
// calls to the underlying service (see
// https://cloud.google.com/storage/docs/json_api/v1/objects/rewrite), then
// ProgressFunc will be invoked after each call with the number of bytes of
// content copied so far and the total size in bytes of the source object.
//
// ProgressFunc is intended to make upload progress available to the
// application. For example, the implementation of ProgressFunc may update
// a progress bar in the application's UI, or log the result of
// float64(copiedBytes)/float64(totalBytes).
//
// ProgressFunc should return quickly without blocking.
ProgressFunc func(copiedBytes, totalBytes uint64)
// The Cloud KMS key, in the form projects/P/locations/L/keyRings/R/cryptoKeys/K,
// that will be used to encrypt the object. Overrides the object's KMSKeyName, if
// any.
//
// Providing both a DestinationKMSKeyName and a customer-supplied encryption key
// (via ObjectHandle.Key) on the destination object will result in an error when
// Run is called.
DestinationKMSKeyName string
// contains filtered or unexported fields
}
func (*Copier) Run
Run performs the copy.
Declaration
func (c *Copier) Run(ctx context.Context) (attrs *ObjectAttrs, err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
src := client.Bucket("bucketname").Object("file1")
dst := client.Bucket("another-bucketname").Object("file2")
// Copy content and modify metadata.
copier := dst.CopierFrom(src)
copier.ContentType = "text/plain"
attrs, err := copier.Run(ctx)
if err != nil {
// TODO: Handle error, possibly resuming with copier.RewriteToken.
}
fmt.Println(attrs)
// Just copy content.
attrs, err = dst.CopierFrom(src).Run(ctx)
if err != nil {
// TODO: Handle error. No way to resume.
}
fmt.Println(attrs)
}
{% endverbatim %}
progress
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"log"
)
func main() {
// Display progress across multiple rewrite RPCs.
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
src := client.Bucket("bucketname").Object("file1")
dst := client.Bucket("another-bucketname").Object("file2")
copier := dst.CopierFrom(src)
copier.ProgressFunc = func(copiedBytes, totalBytes uint64) {
log.Printf("copy %.1f%% done", float64(copiedBytes)/float64(totalBytes)*100)
}
if _, err := copier.Run(ctx); err != nil {
// TODO: handle error.
}
}
{% endverbatim %}
HMACKey
HMACKey is the representation of a Google Cloud Storage HMAC key.
HMAC keys are used to authenticate signed access to objects. To enable HMAC key authentication, please visit https://cloud.google.com/storage/docs/migrating.
This type is EXPERIMENTAL and subject to change or removal without notice.
Declaration
type HMACKey struct {
// The HMAC's secret key.
Secret string
// AccessID is the ID of the HMAC key.
AccessID string
// Etag is the HTTP/1.1 Entity tag.
Etag string
// ID is the ID of the HMAC key, including the ProjectID and AccessID.
ID string
// ProjectID is the ID of the project that owns the
// service account to which the key authenticates.
ProjectID string
// ServiceAccountEmail is the email address
// of the key's associated service account.
ServiceAccountEmail string
// CreatedTime is the creation time of the HMAC key.
CreatedTime time.Time
// UpdatedTime is the last modification time of the HMAC key metadata.
UpdatedTime time.Time
// State is the state of the HMAC key.
// It can be one of StateActive, StateInactive or StateDeleted.
State HMACState
}
HMACKeyAttrsToUpdate
HMACKeyAttrsToUpdate defines the attributes of an HMACKey that will be updated.
This type is EXPERIMENTAL and subject to change or removal without notice.
Declaration
type HMACKeyAttrsToUpdate struct {
// State is required and must be either StateActive or StateInactive.
State HMACState
// Etag is an optional field and it is the HTTP/1.1 Entity tag.
Etag string
}
HMACKeyHandle
HMACKeyHandle helps provide access and management for HMAC keys.
This type is EXPERIMENTAL and subject to change or removal without notice.
Declaration
type HMACKeyHandle struct {
// contains filtered or unexported fields
}
func (*HMACKeyHandle) Delete
Delete invokes an RPC to delete the key referenced by accessID, on Google Cloud Storage. Only inactive HMAC keys can be deleted. After deletion, a key cannot be used to authenticate requests.
This method is EXPERIMENTAL and subject to change or removal without notice.
Declaration
func (hkh *HMACKeyHandle) Delete(ctx context.Context, opts ...HMACKeyOption) error
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
hkh := client.HMACKeyHandle("project-id", "access-key-id")
// Make sure that the HMACKey being deleted has a status of inactive.
if err := hkh.Delete(ctx); err != nil {
// TODO: handle error.
}
}
{% endverbatim %}
func (*HMACKeyHandle) Get
Get invokes an RPC to retrieve the HMAC key referenced by the HMACKeyHandle's accessID.
Options such as UserProjectForHMACKeys can be used to set the userProject to be billed against for operations.
This method is EXPERIMENTAL and subject to change or removal without notice.
Declaration
func (hkh *HMACKeyHandle) Get(ctx context.Context, opts ...HMACKeyOption) (*HMACKey, error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
hkh := client.HMACKeyHandle("project-id", "access-key-id")
hkey, err := hkh.Get(ctx)
if err != nil {
// TODO: handle error.
}
_ = hkey // TODO: Use the HMAC Key.
}
{% endverbatim %}
func (*HMACKeyHandle) Update
Update mutates the HMACKey referred to by accessID.
This method is EXPERIMENTAL and subject to change or removal without notice.
Declaration
func (h *HMACKeyHandle) Update(ctx context.Context, au HMACKeyAttrsToUpdate, opts ...HMACKeyOption) (*HMACKey, error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
hkh := client.HMACKeyHandle("project-id", "access-key-id")
ukey, err := hkh.Update(ctx, storage.HMACKeyAttrsToUpdate{
State: storage.Inactive,
})
if err != nil {
// TODO: handle error.
}
_ = ukey // TODO: Use the HMAC Key.
}
{% endverbatim %}
HMACKeyOption
HMACKeyOption configures the behavior of HMACKey related methods and actions.
This interface is EXPERIMENTAL and subject to change or removal without notice.
Declaration
type HMACKeyOption interface {
// contains filtered or unexported methods
}
func ForHMACKeyServiceAccountEmail
ForHMACKeyServiceAccountEmail returns HMAC Keys that are associated with the email address of a service account in the project.
Only one service account email can be used as a filter, so if multiple of these options are applied, the last email to be set will be used.
This option is EXPERIMENTAL and subject to change or removal without notice.
Declaration
func ForHMACKeyServiceAccountEmail(serviceAccountEmail string) HMACKeyOption
func ShowDeletedHMACKeys
ShowDeletedHMACKeys will also list keys whose state is "DELETED".
This option is EXPERIMENTAL and subject to change or removal without notice.
Declaration
func ShowDeletedHMACKeys() HMACKeyOption
func UserProjectForHMACKeys
UserProjectForHMACKeys will bill the request against userProjectID if userProjectID is non-empty.
Note: This is a noop right now and only provided for API compatibility.
This option is EXPERIMENTAL and subject to change or removal without notice.
Declaration
func UserProjectForHMACKeys(userProjectID string) HMACKeyOption
HMACKeysIterator
An HMACKeysIterator is an iterator over HMACKeys.
Note: This iterator is not safe for concurrent operations without explicit synchronization.
This type is EXPERIMENTAL and subject to change or removal without notice.
Declaration
type HMACKeysIterator struct {
// contains filtered or unexported fields
}
func (*HMACKeysIterator) Next
Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns iterator.Done, all subsequent calls will return iterator.Done.
Note: This iterator is not safe for concurrent operations without explicit synchronization.
This method is EXPERIMENTAL and subject to change or removal without notice.
Declaration
func (it *HMACKeysIterator) Next() (*HMACKey, error)
func (*HMACKeysIterator) PageInfo
PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
Note: This iterator is not safe for concurrent operations without explicit synchronization.
This method is EXPERIMENTAL and subject to change or removal without notice.
Declaration
func (it *HMACKeysIterator) PageInfo() *iterator.PageInfo
HMACState
HMACState is the state of the HMAC key.
This type is EXPERIMENTAL and subject to change or removal without notice.
Declaration
type HMACState string
Active, Inactive, Deleted
Declaration
const (
// Active is the status for an active key that can be used to sign
// requests.
Active HMACState = "ACTIVE"
// Inactive is the status for an inactive key thus requests signed by
// this key will be denied.
Inactive HMACState = "INACTIVE"
// Deleted is the status for a key that is deleted.
// Once in this state the key cannot key cannot be recovered
// and does not count towards key limits. Deleted keys will be cleaned
// up later.
Deleted HMACState = "DELETED"
)
Lifecycle
Lifecycle is the lifecycle configuration for objects in the bucket.
Declaration
type Lifecycle struct {
Rules []LifecycleRule
}
LifecycleAction
LifecycleAction is a lifecycle configuration action.
Declaration
type LifecycleAction struct {
// Type is the type of action to take on matching objects.
//
// Acceptable values are "Delete" to delete matching objects and
// "SetStorageClass" to set the storage class defined in StorageClass on
// matching objects.
Type string
// StorageClass is the storage class to set on matching objects if the Action
// is "SetStorageClass".
StorageClass string
}
LifecycleCondition
LifecycleCondition is a set of conditions used to match objects and take an action automatically.
All configured conditions must be met for the associated action to be taken.
Declaration
type LifecycleCondition struct {
// AgeInDays is the age of the object in days.
AgeInDays int64
// CreatedBefore is the time the object was created.
//
// This condition is satisfied when an object is created before midnight of
// the specified date in UTC.
CreatedBefore time.Time
// CustomTimeBefore is the CustomTime metadata field of the object. This
// condition is satisfied when an object's CustomTime timestamp is before
// midnight of the specified date in UTC.
//
// This condition can only be satisfied if CustomTime has been set.
CustomTimeBefore time.Time
// DaysSinceCustomTime is the days elapsed since the CustomTime date of the
// object. This condition can only be satisfied if CustomTime has been set.
DaysSinceCustomTime int64
// DaysSinceNoncurrentTime is the days elapsed since the noncurrent timestamp
// of the object. This condition is relevant only for versioned objects.
DaysSinceNoncurrentTime int64
// Liveness specifies the object's liveness. Relevant only for versioned objects
Liveness Liveness
// MatchesStorageClasses is the condition matching the object's storage
// class.
//
// Values include "STANDARD", "NEARLINE", "COLDLINE" and "ARCHIVE".
MatchesStorageClasses []string
// NoncurrentTimeBefore is the noncurrent timestamp of the object. This
// condition is satisfied when an object's noncurrent timestamp is before
// midnight of the specified date in UTC.
//
// This condition is relevant only for versioned objects.
NoncurrentTimeBefore time.Time
// NumNewerVersions is the condition matching objects with a number of newer versions.
//
// If the value is N, this condition is satisfied when there are at least N
// versions (including the live version) newer than this version of the
// object.
NumNewerVersions int64
}
LifecycleRule
LifecycleRule is a lifecycle configuration rule.
When all the configured conditions are met by an object in the bucket, the configured action will automatically be taken on that object.
Declaration
type LifecycleRule struct {
// Action is the action to take when all of the associated conditions are
// met.
Action LifecycleAction
// Condition is the set of conditions that must be met for the associated
// action to be taken.
Condition LifecycleCondition
}
Liveness
Liveness specifies whether the object is live or not.
Declaration
type Liveness int
LiveAndArchived, Live, Archived
Declaration
const (
// LiveAndArchived includes both live and archived objects.
LiveAndArchived Liveness = iota
// Live specifies that the object is still live.
Live
// Archived specifies that the object is archived.
Archived
)
Notification
A Notification describes how to send Cloud PubSub messages when certain events occur in a bucket.
Declaration
type Notification struct {
//The ID of the notification.
ID string
// The ID of the topic to which this subscription publishes.
TopicID string
// The ID of the project to which the topic belongs.
TopicProjectID string
// Only send notifications about listed event types. If empty, send notifications
// for all event types.
// See https://cloud.google.com/storage/docs/pubsub-notifications#events.
EventTypes []string
// If present, only apply this notification configuration to object names that
// begin with this prefix.
ObjectNamePrefix string
// An optional list of additional attributes to attach to each Cloud PubSub
// message published for this notification subscription.
CustomAttributes map[string]string
// The contents of the message payload.
// See https://cloud.google.com/storage/docs/pubsub-notifications#payload.
PayloadFormat string
}
ObjectAttrs
ObjectAttrs represents the metadata for a Google Cloud Storage (GCS) object.
Declaration
type ObjectAttrs struct {
// Bucket is the name of the bucket containing this GCS object.
// This field is read-only.
Bucket string
// Name is the name of the object within the bucket.
// This field is read-only.
Name string
// ContentType is the MIME type of the object's content.
ContentType string
// ContentLanguage is the content language of the object's content.
ContentLanguage string
// CacheControl is the Cache-Control header to be sent in the response
// headers when serving the object data.
CacheControl string
// EventBasedHold specifies whether an object is under event-based hold. New
// objects created in a bucket whose DefaultEventBasedHold is set will
// default to that value.
EventBasedHold bool
// TemporaryHold specifies whether an object is under temporary hold. While
// this flag is set to true, the object is protected against deletion and
// overwrites.
TemporaryHold bool
// RetentionExpirationTime is a server-determined value that specifies the
// earliest time that the object's retention period expires.
// This is a read-only field.
RetentionExpirationTime time.Time
// ACL is the list of access control rules for the object.
ACL []ACLRule
// If not empty, applies a predefined set of access controls. It should be set
// only when writing, copying or composing an object. When copying or composing,
// it acts as the destinationPredefinedAcl parameter.
// PredefinedACL is always empty for ObjectAttrs returned from the service.
// See https://cloud.google.com/storage/docs/json_api/v1/objects/insert
// for valid values.
PredefinedACL string
// Owner is the owner of the object. This field is read-only.
//
// If non-zero, it is in the form of "user-<userId>".
Owner string
// Size is the length of the object's content. This field is read-only.
Size int64
// ContentEncoding is the encoding of the object's content.
ContentEncoding string
// ContentDisposition is the optional Content-Disposition header of the object
// sent in the response headers.
ContentDisposition string
// MD5 is the MD5 hash of the object's content. This field is read-only,
// except when used from a Writer. If set on a Writer, the uploaded
// data is rejected if its MD5 hash does not match this field.
MD5 []byte
// CRC32C is the CRC32 checksum of the object's content using the Castagnoli93
// polynomial. This field is read-only, except when used from a Writer or
// Composer. In those cases, if the SendCRC32C field in the Writer or Composer
// is set to is true, the uploaded data is rejected if its CRC32C hash does
// not match this field.
CRC32C uint32
// MediaLink is an URL to the object's content. This field is read-only.
MediaLink string
// Metadata represents user-provided metadata, in key/value pairs.
// It can be nil if no metadata is provided.
Metadata map[string]string
// Generation is the generation number of the object's content.
// This field is read-only.
Generation int64
// Metageneration is the version of the metadata for this
// object at this generation. This field is used for preconditions
// and for detecting changes in metadata. A metageneration number
// is only meaningful in the context of a particular generation
// of a particular object. This field is read-only.
Metageneration int64
// StorageClass is the storage class of the object. This defines
// how objects are stored and determines the SLA and the cost of storage.
// Typical values are "STANDARD", "NEARLINE", "COLDLINE" and "ARCHIVE".
// Defaults to "STANDARD".
// See https://cloud.google.com/storage/docs/storage-classes for all
// valid values.
StorageClass string
// Created is the time the object was created. This field is read-only.
Created time.Time
// Deleted is the time the object was deleted.
// If not deleted, it is the zero value. This field is read-only.
Deleted time.Time
// Updated is the creation or modification time of the object.
// For buckets with versioning enabled, changing an object's
// metadata does not change this property. This field is read-only.
Updated time.Time
// CustomerKeySHA256 is the base64-encoded SHA-256 hash of the
// customer-supplied encryption key for the object. It is empty if there is
// no customer-supplied encryption key.
// See // https://cloud.google.com/storage/docs/encryption for more about
// encryption in Google Cloud Storage.
CustomerKeySHA256 string
// Cloud KMS key name, in the form
// projects/P/locations/L/keyRings/R/cryptoKeys/K, used to encrypt this object,
// if the object is encrypted by such a key.
//
// Providing both a KMSKeyName and a customer-supplied encryption key (via
// ObjectHandle.Key) will result in an error when writing an object.
KMSKeyName string
// Prefix is set only for ObjectAttrs which represent synthetic "directory
// entries" when iterating over buckets using Query.Delimiter. See
// ObjectIterator.Next. When set, no other fields in ObjectAttrs will be
// populated.
Prefix string
// Etag is the HTTP/1.1 Entity tag for the object.
// This field is read-only.
Etag string
// A user-specified timestamp which can be applied to an object. This is
// typically set in order to use the CustomTimeBefore and DaysSinceCustomTime
// LifecycleConditions to manage object lifecycles.
//
// CustomTime cannot be removed once set on an object. It can be updated to a
// later value but not to an earlier one.
CustomTime time.Time
}
ObjectAttrsToUpdate
ObjectAttrsToUpdate is used to update the attributes of an object. Only fields set to non-nil values will be updated. Set a field to its zero value to delete it.
For example, to change ContentType and delete ContentEncoding and Metadata, use ObjectAttrsToUpdate{ ContentType: "text/html", ContentEncoding: "", Metadata: map[string]string{}, }
Declaration
type ObjectAttrsToUpdate struct {
EventBasedHold optional.Bool
TemporaryHold optional.Bool
ContentType optional.String
ContentLanguage optional.String
ContentEncoding optional.String
ContentDisposition optional.String
CacheControl optional.String
CustomTime time.Time
Metadata map[string]string // set to map[string]string{} to delete
ACL []ACLRule
// If not empty, applies a predefined set of access controls. ACL must be nil.
// See https://cloud.google.com/storage/docs/json_api/v1/objects/patch.
PredefinedACL string
}
ObjectHandle
ObjectHandle provides operations on an object in a Google Cloud Storage bucket. Use BucketHandle.Object to get a handle.
Declaration
type ObjectHandle struct {
// contains filtered or unexported fields
}
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Object("my-object").Attrs(ctx)
if err == storage.ErrObjectNotExist {
fmt.Println("The object does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The object exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
func (*ObjectHandle) ACL
ACL provides access to the object's access control list. This controls who can read and write this object. This call does not perform any network operations.
Declaration
func (o *ObjectHandle) ACL() *ACLHandle
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Object("my-object").Attrs(ctx)
if err == storage.ErrObjectNotExist {
fmt.Println("The object does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The object exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
func (*ObjectHandle) Attrs
Attrs returns meta information about the object. ErrObjectNotExist will be returned if the object is not found.
Declaration
func (o *ObjectHandle) Attrs(ctx context.Context) (attrs *ObjectAttrs, err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
objAttrs, err := client.Bucket("my-bucket").Object("my-object").Attrs(ctx)
if err != nil {
// TODO: handle error.
}
fmt.Println(objAttrs)
}
{% endverbatim %}
withConditions
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"time"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
obj := client.Bucket("my-bucket").Object("my-object")
// Read the object.
objAttrs1, err := obj.Attrs(ctx)
if err != nil {
// TODO: handle error.
}
// Do something else for a while.
time.Sleep(5 * time.Minute)
// Now read the same contents, even if the object has been written since the last read.
objAttrs2, err := obj.Generation(objAttrs1.Generation).Attrs(ctx)
if err != nil {
// TODO: handle error.
}
fmt.Println(objAttrs1, objAttrs2)
}
{% endverbatim %}
func (*ObjectHandle) BucketName
BucketName returns the name of the bucket.
Declaration
func (o *ObjectHandle) BucketName() string
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Object("my-object").Attrs(ctx)
if err == storage.ErrObjectNotExist {
fmt.Println("The object does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The object exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
func (*ObjectHandle) ComposerFrom
ComposerFrom creates a Composer that can compose srcs into dst. You can immediately call Run on the returned Composer, or you can configure it first.
The encryption key for the destination object will be used to decrypt all source objects and encrypt the destination object. It is an error to specify an encryption key for any of the source objects.
Declaration
func (dst *ObjectHandle) ComposerFrom(srcs ...*ObjectHandle) *Composer
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Object("my-object").Attrs(ctx)
if err == storage.ErrObjectNotExist {
fmt.Println("The object does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The object exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
func (*ObjectHandle) CopierFrom
CopierFrom creates a Copier that can copy src to dst. You can immediately call Run on the returned Copier, or you can configure it first.
For Requester Pays buckets, the user project of dst is billed, unless it is empty, in which case the user project of src is billed.
Declaration
func (dst *ObjectHandle) CopierFrom(src *ObjectHandle) *Copier
Examples
rotateEncryptionKeys
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
var key1, key2 []byte
func main() {
// To rotate the encryption key on an object, copy it onto itself.
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
obj := client.Bucket("bucketname").Object("obj")
// Assume obj is encrypted with key1, and we want to change to key2.
_, err = obj.Key(key2).CopierFrom(obj.Key(key1)).Run(ctx)
if err != nil {
// TODO: handle error.
}
}
{% endverbatim %}
func (*ObjectHandle) Delete
Delete deletes the single specified object.
Declaration
func (o *ObjectHandle) Delete(ctx context.Context) error
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"google.golang.org/api/iterator"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// To delete multiple objects in a bucket, list them with an
// ObjectIterator, then Delete them.
// If you are using this package on the App Engine Flex runtime,
// you can init a bucket client with your app's default bucket name.
// See http://godoc.org/google.golang.org/appengine/file#DefaultBucketName.
bucket := client.Bucket("my-bucket")
it := bucket.Objects(ctx, nil)
for {
objAttrs, err := it.Next()
if err != nil && err != iterator.Done {
// TODO: Handle error.
}
if err == iterator.Done {
break
}
if err := bucket.Object(objAttrs.Name).Delete(ctx); err != nil {
// TODO: Handle error.
}
}
fmt.Println("deleted all object items in the bucket specified.")
}
{% endverbatim %}
func (*ObjectHandle) Generation
Generation returns a new ObjectHandle that operates on a specific generation of the object. By default, the handle operates on the latest generation. Not all operations work when given a specific generation; check the API endpoints at https://cloud.google.com/storage/docs/json_api/ for details.
Declaration
func (o *ObjectHandle) Generation(gen int64) *ObjectHandle
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"io"
"os"
)
var gen int64
func main() {
// Read an object's contents from generation gen, regardless of the
// current generation of the object.
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
obj := client.Bucket("my-bucket").Object("my-object")
rc, err := obj.Generation(gen).NewReader(ctx)
if err != nil {
// TODO: handle error.
}
defer rc.Close()
if _, err := io.Copy(os.Stdout, rc); err != nil {
// TODO: handle error.
}
}
{% endverbatim %}
func (*ObjectHandle) If
If returns a new ObjectHandle that applies a set of preconditions. Preconditions already set on the ObjectHandle are ignored. Operations on the new handle will return an error if the preconditions are not satisfied. See https://cloud.google.com/storage/docs/generations-preconditions for more details.
Declaration
func (o *ObjectHandle) If(conds Conditions) *ObjectHandle
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"google.golang.org/api/googleapi"
"io"
"net/http"
"os"
)
var gen int64
func main() {
// Read from an object only if the current generation is gen.
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
obj := client.Bucket("my-bucket").Object("my-object")
rc, err := obj.If(storage.Conditions{GenerationMatch: gen}).NewReader(ctx)
if err != nil {
// TODO: handle error.
}
if _, err := io.Copy(os.Stdout, rc); err != nil {
// TODO: handle error.
}
if err := rc.Close(); err != nil {
switch ee := err.(type) {
case *googleapi.Error:
if ee.Code == http.StatusPreconditionFailed {
// The condition presented in the If failed.
// TODO: handle error.
}
// TODO: handle other status codes here.
default:
// TODO: handle error.
}
}
}
{% endverbatim %}
func (*ObjectHandle) Key
Key returns a new ObjectHandle that uses the supplied encryption key to encrypt and decrypt the object's contents.
Encryption key must be a 32-byte AES-256 key. See https://cloud.google.com/storage/docs/encryption for details.
Declaration
func (o *ObjectHandle) Key(encryptionKey []byte) *ObjectHandle
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
var secretKey []byte
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
obj := client.Bucket("my-bucket").Object("my-object")
// Encrypt the object's contents.
w := obj.Key(secretKey).NewWriter(ctx)
if _, err := w.Write([]byte("top secret")); err != nil {
// TODO: handle error.
}
if err := w.Close(); err != nil {
// TODO: handle error.
}
}
{% endverbatim %}
func (*ObjectHandle) NewRangeReader
NewRangeReader reads part of an object, reading at most length bytes starting at the given offset. If length is negative, the object is read until the end. If offset is negative, the object is read abs(offset) bytes from the end, and length must also be negative to indicate all remaining bytes will be read.
If the object's metadata property "Content-Encoding" is set to "gzip" or satisfies decompressive transcoding per https://cloud.google.com/storage/docs/transcoding that file will be served back whole, regardless of the requested range as Google Cloud Storage dictates.
Declaration
func (o *ObjectHandle) NewRangeReader(ctx context.Context, offset, length int64) (r *Reader, err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"io/ioutil"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Read only the first 64K.
rc, err := client.Bucket("bucketname").Object("filename1").NewRangeReader(ctx, 0, 64*1024)
if err != nil {
// TODO: handle error.
}
defer rc.Close()
slurp, err := ioutil.ReadAll(rc)
if err != nil {
// TODO: handle error.
}
fmt.Printf("first 64K of file contents:\n%s\n", slurp)
}
{% endverbatim %}
lastNBytes
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"io/ioutil"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Read only the last 10 bytes until the end of the file.
rc, err := client.Bucket("bucketname").Object("filename1").NewRangeReader(ctx, -10, -1)
if err != nil {
// TODO: handle error.
}
defer rc.Close()
slurp, err := ioutil.ReadAll(rc)
if err != nil {
// TODO: handle error.
}
fmt.Printf("Last 10 bytes from the end of the file:\n%s\n", slurp)
}
{% endverbatim %}
untilEnd
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"io/ioutil"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Read from the 101st byte until the end of the file.
rc, err := client.Bucket("bucketname").Object("filename1").NewRangeReader(ctx, 100, -1)
if err != nil {
// TODO: handle error.
}
defer rc.Close()
slurp, err := ioutil.ReadAll(rc)
if err != nil {
// TODO: handle error.
}
fmt.Printf("From 101st byte until the end:\n%s\n", slurp)
}
{% endverbatim %}
func (*ObjectHandle) NewReader
NewReader creates a new Reader to read the contents of the object. ErrObjectNotExist will be returned if the object is not found.
The caller must call Close on the returned Reader when done reading.
Declaration
func (o *ObjectHandle) NewReader(ctx context.Context) (*Reader, error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"io/ioutil"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
rc, err := client.Bucket("my-bucket").Object("my-object").NewReader(ctx)
if err != nil {
// TODO: handle error.
}
slurp, err := ioutil.ReadAll(rc)
rc.Close()
if err != nil {
// TODO: handle error.
}
fmt.Println("file contents:", slurp)
}
{% endverbatim %}
func (*ObjectHandle) NewWriter
NewWriter returns a storage Writer that writes to the GCS object associated with this ObjectHandle.
A new object will be created unless an object with this name already exists. Otherwise any previous object with the same name will be replaced. The object will not be available (and any previous object will remain) until Close has been called.
Attributes can be set on the object by modifying the returned Writer's ObjectAttrs field before the first call to Write. If no ContentType attribute is specified, the content type will be automatically sniffed using net/http.DetectContentType.
It is the caller's responsibility to call Close when writing is done. To stop writing without saving the data, cancel the context.
Declaration
func (o *ObjectHandle) NewWriter(ctx context.Context) *Writer
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
wc := client.Bucket("bucketname").Object("filename1").NewWriter(ctx)
_ = wc // TODO: Use the Writer.
}
{% endverbatim %}
func (*ObjectHandle) ObjectName
ObjectName returns the name of the object.
Declaration
func (o *ObjectHandle) ObjectName() string
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Object("my-object").Attrs(ctx)
if err == storage.ErrObjectNotExist {
fmt.Println("The object does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The object exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
func (*ObjectHandle) ReadCompressed
ReadCompressed when true causes the read to happen without decompressing.
Declaration
func (o *ObjectHandle) ReadCompressed(compressed bool) *ObjectHandle
Examples
exists
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
attrs, err := client.Bucket("my-bucket").Object("my-object").Attrs(ctx)
if err == storage.ErrObjectNotExist {
fmt.Println("The object does not exist")
return
}
if err != nil {
// TODO: handle error.
}
fmt.Printf("The object exists and has attributes: %#v\n", attrs)
}
{% endverbatim %}
func (*ObjectHandle) Update
Update updates an object with the provided attributes. All zero-value attributes are ignored. ErrObjectNotExist will be returned if the object is not found.
Declaration
func (o *ObjectHandle) Update(ctx context.Context, uattrs ObjectAttrsToUpdate) (oa *ObjectAttrs, err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
// Change only the content type of the object.
objAttrs, err := client.Bucket("my-bucket").Object("my-object").Update(ctx, storage.ObjectAttrsToUpdate{
ContentType: "text/html",
ContentDisposition: "", // delete ContentDisposition
})
if err != nil {
// TODO: handle error.
}
fmt.Println(objAttrs)
}
{% endverbatim %}
ObjectIterator
An ObjectIterator is an iterator over ObjectAttrs.
Note: This iterator is not safe for concurrent operations without explicit synchronization.
Declaration
type ObjectIterator struct {
// contains filtered or unexported fields
}
func (*ObjectIterator) Next
Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns iterator.Done, all subsequent calls will return iterator.Done.
If Query.Delimiter is non-empty, some of the ObjectAttrs returned by Next will have a non-empty Prefix field, and a zero value for all other fields. These represent prefixes.
Note: This method is not safe for concurrent operations without explicit synchronization.
Declaration
func (it *ObjectIterator) Next() (*ObjectAttrs, error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"google.golang.org/api/iterator"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
it := client.Bucket("my-bucket").Objects(ctx, nil)
for {
objAttrs, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
// TODO: Handle error.
}
fmt.Println(objAttrs)
}
}
{% endverbatim %}
func (*ObjectIterator) PageInfo
PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
Note: This method is not safe for concurrent operations without explicit synchronization.
Declaration
func (it *ObjectIterator) PageInfo() *iterator.PageInfo
PolicyV4Fields
PolicyV4Fields describes the attributes for a PostPolicyV4 request.
Declaration
type PolicyV4Fields struct {
// ACL specifies the access control permissions for the object.
// Optional.
ACL string
// CacheControl specifies the caching directives for the object.
// Optional.
CacheControl string
// ContentType specifies the media type of the object.
// Optional.
ContentType string
// ContentDisposition specifies how the file will be served back to requesters.
// Optional.
ContentDisposition string
// ContentEncoding specifies the decompressive transcoding that the object.
// This field is complementary to ContentType in that the file could be
// compressed but ContentType specifies the file's original media type.
// Optional.
ContentEncoding string
// Metadata specifies custom metadata for the object.
// If any key doesn't begin with "x-goog-meta-", an error will be returned.
// Optional.
Metadata map[string]string
// StatusCodeOnSuccess when set, specifies the status code that Cloud Storage
// will serve back on successful upload of the object.
// Optional.
StatusCodeOnSuccess int
// RedirectToURLOnSuccess when set, specifies the URL that Cloud Storage
// will serve back on successful upload of the object.
// Optional.
RedirectToURLOnSuccess string
}
PostPolicyV4
PostPolicyV4 describes the URL and respective form fields for a generated PostPolicyV4 request.
Declaration
type PostPolicyV4 struct {
// URL is the generated URL that the file upload will be made to.
URL string
// Fields specifies the generated key-values that the file uploader
// must include in their multipart upload form.
Fields map[string]string
}
func GenerateSignedPostPolicyV4
GenerateSignedPostPolicyV4 generates a PostPolicyV4 value from bucket, object and opts. The generated URL and fields will then allow an unauthenticated client to perform multipart uploads.
Declaration
func GenerateSignedPostPolicyV4(bucket, object string, opts *PostPolicyV4Options) (*PostPolicyV4, error)
Examples
{% verbatim %}package main
import (
"bytes"
"cloud.google.com/go/storage"
"io"
"mime/multipart"
"net/http"
"time"
)
func main() {
pv4, err := storage.GenerateSignedPostPolicyV4("my-bucket", "my-object.txt", &storage.PostPolicyV4Options{
GoogleAccessID: "my-access-id",
PrivateKey: []byte("my-private-key"),
// The upload expires in 2hours.
Expires: time.Now().Add(2 * time.Hour),
Fields: &storage.PolicyV4Fields{
StatusCodeOnSuccess: 200,
RedirectToURLOnSuccess: "https://example.org/",
// It MUST only be a text file.
ContentType: "text/plain",
},
// The conditions that the uploaded file will be expected to conform to.
Conditions: []storage.PostPolicyV4Condition{
// Make the file a maximum of 10mB.
storage.ConditionContentLengthRange(0, 10<<20),
},
})
if err != nil {
// TODO: handle error.
}
// Now you can upload your file using the generated post policy
// with a plain HTTP client or even the browser.
formBuf := new(bytes.Buffer)
mw := multipart.NewWriter(formBuf)
for fieldName, value := range pv4.Fields {
if err := mw.WriteField(fieldName, value); err != nil {
// TODO: handle error.
}
}
file := bytes.NewReader(bytes.Repeat([]byte("a"), 100))
mf, err := mw.CreateFormFile("file", "myfile.txt")
if err != nil {
// TODO: handle error.
}
if _, err := io.Copy(mf, file); err != nil {
// TODO: handle error.
}
if err := mw.Close(); err != nil {
// TODO: handle error.
}
// Compose the request.
req, err := http.NewRequest("POST", pv4.URL, formBuf)
if err != nil {
// TODO: handle error.
}
// Ensure the Content-Type is derived from the multipart writer.
req.Header.Set("Content-Type", mw.FormDataContentType())
res, err := http.DefaultClient.Do(req)
if err != nil {
// TODO: handle error.
}
_ = res
}
{% endverbatim %}
PostPolicyV4Condition
PostPolicyV4Condition describes the constraints that the subsequent object upload's multipart form fields will be expected to conform to.
Declaration
type PostPolicyV4Condition interface {
json.Marshaler
// contains filtered or unexported methods
}
func ConditionContentLengthRange
ConditionContentLengthRange constraints the limits that the multipart upload's range header will be expected to be within.
Declaration
func ConditionContentLengthRange(start, end uint64) PostPolicyV4Condition
func ConditionStartsWith
ConditionStartsWith checks that an attributes starts with value. An empty value will cause this condition to be ignored.
Declaration
func ConditionStartsWith(key, value string) PostPolicyV4Condition
PostPolicyV4Options
PostPolicyV4Options are used to construct a signed post policy. Please see https://cloud.google.com/storage/docs/xml-api/post-object for reference about the fields.
Declaration
type PostPolicyV4Options struct {
// GoogleAccessID represents the authorizer of the signed URL generation.
// It is typically the Google service account client email address from
// the Google Developers Console in the form of<EMAIL_ADDRESS>// Required.
GoogleAccessID string
// PrivateKey is the Google service account private key. It is obtainable
// from the Google Developers Console.
// At https://console.developers.google.com/project/<your-project-id>/apiui/credential,
// create a service account client ID or reuse one of your existing service account
// credentials. Click on the "Generate new P12 key" to generate and download
// a new private key. Once you download the P12 file, use the following command
// to convert it into a PEM file.
//
// $ openssl pkcs12 -in key.p12 -passin pass:notasecret -out key.pem -nodes
//
// Provide the contents of the PEM file as a byte slice.
// Exactly one of PrivateKey or SignBytes must be non-nil.
PrivateKey []byte
// SignBytes is a function for implementing custom signing. For example, if
// your application is running on Google App Engine, you can use
// appengine's internal signing function:
// ctx := appengine.NewContext(request)
// acc, _ := appengine.ServiceAccount(ctx)
// url, err := SignedURL("bucket", "object", &SignedURLOptions{
// GoogleAccessID: acc,
// SignBytes: func(b []byte) ([]byte, error) {
// _, signedBytes, err := appengine.SignBytes(ctx, b)
// return signedBytes, err
// },
// // etc.
// })
//
// Exactly one of PrivateKey or SignBytes must be non-nil.
SignBytes func(hashBytes []byte) (signature []byte, err error)
// Expires is the expiration time on the signed URL.
// It must be a time in the future.
// Required.
Expires time.Time
// Style provides options for the type of URL to use. Options are
// PathStyle (default), BucketBoundHostname, and VirtualHostedStyle. See
// https://cloud.google.com/storage/docs/request-endpoints for details.
// Optional.
Style URLStyle
// Insecure when set indicates that the generated URL's scheme
// will use "http" instead of "https" (default).
// Optional.
Insecure bool
// Fields specifies the attributes of a PostPolicyV4 request.
// When Fields is non-nil, its attributes must match those that will
// passed into field Conditions.
// Optional.
Fields *PolicyV4Fields
// The conditions that the uploaded file will be expected to conform to.
// When used, the failure of an upload to satisfy a condition will result in
// a 4XX status code, back with the message describing the problem.
// Optional.
Conditions []PostPolicyV4Condition
}
ProjectTeam
ProjectTeam is the project team associated with the entity, if any.
Declaration
type ProjectTeam struct {
ProjectNumber string
Team string
}
Query
Query represents a query to filter objects from a bucket.
Declaration
type Query struct {
// Delimiter returns results in a directory-like fashion.
// Results will contain only objects whose names, aside from the
// prefix, do not contain delimiter. Objects whose names,
// aside from the prefix, contain delimiter will have their name,
// truncated after the delimiter, returned in prefixes.
// Duplicate prefixes are omitted.
// Optional.
Delimiter string
// Prefix is the prefix filter to query objects
// whose names begin with this prefix.
// Optional.
Prefix string
// Versions indicates whether multiple versions of the same
// object will be included in the results.
Versions bool
// StartOffset is used to filter results to objects whose names are
// lexicographically equal to or after startOffset. If endOffset is also set,
// the objects listed will have names between startOffset (inclusive) and
// endOffset (exclusive).
StartOffset string
// EndOffset is used to filter results to objects whose names are
// lexicographically before endOffset. If startOffset is also set, the objects
// listed will have names between startOffset (inclusive) and endOffset (exclusive).
EndOffset string
// contains filtered or unexported fields
}
func (*Query) SetAttrSelection
SetAttrSelection makes the query populate only specific attributes of objects. When iterating over objects, if you only need each object's name and size, pass []string{"Name", "Size"} to this method. Only these fields will be fetched for each object across the network; the other fields of ObjectAttr will remain at their default values. This is a performance optimization; for more information, see https://cloud.google.com/storage/docs/json_api/v1/how-tos/performance
Declaration
func (q *Query) SetAttrSelection(attrs []string) error
Reader
Reader reads a Cloud Storage object. It implements io.Reader.
Typically, a Reader computes the CRC of the downloaded content and compares it to the stored CRC, returning an error from Read if there is a mismatch. This integrity check is skipped if transcoding occurs. See https://cloud.google.com/storage/docs/transcoding.
Declaration
type Reader struct {
Attrs ReaderObjectAttrs
// contains filtered or unexported fields
}
func (*Reader) CacheControl
CacheControl returns the cache control of the object.
Deprecated: use Reader.Attrs.CacheControl.
Declaration
func (r *Reader) CacheControl() string
func (*Reader) Close
Close closes the Reader. It must be called when done reading.
Declaration
func (r *Reader) Close() error
func (*Reader) ContentEncoding
ContentEncoding returns the content encoding of the object.
Deprecated: use Reader.Attrs.ContentEncoding.
Declaration
func (r *Reader) ContentEncoding() string
func (*Reader) ContentType
ContentType returns the content type of the object.
Deprecated: use Reader.Attrs.ContentType.
Declaration
func (r *Reader) ContentType() string
func (*Reader) LastModified
LastModified returns the value of the Last-Modified header.
Deprecated: use Reader.Attrs.LastModified.
Declaration
func (r *Reader) LastModified() (time.Time, error)
func (*Reader) Read
Declaration
func (r *Reader) Read(p []byte) (int, error)
func (*Reader) Remain
Remain returns the number of bytes left to read, or -1 if unknown.
Declaration
func (r *Reader) Remain() int64
func (*Reader) Size
Size returns the size of the object in bytes. The returned value is always the same and is not affected by calls to Read or Close.
Deprecated: use Reader.Attrs.Size.
Declaration
func (r *Reader) Size() int64
ReaderObjectAttrs
ReaderObjectAttrs are attributes about the object being read. These are populated during the New call. This struct only holds a subset of object attributes: to get the full set of attributes, use ObjectHandle.Attrs.
Each field is read-only.
Declaration
type ReaderObjectAttrs struct {
// Size is the length of the object's content.
Size int64
// StartOffset is the byte offset within the object
// from which reading begins.
// This value is only non-zero for range requests.
StartOffset int64
// ContentType is the MIME type of the object's content.
ContentType string
// ContentEncoding is the encoding of the object's content.
ContentEncoding string
// CacheControl specifies whether and for how long browser and Internet
// caches are allowed to cache your objects.
CacheControl string
// LastModified is the time that the object was last modified.
LastModified time.Time
// Generation is the generation number of the object's content.
Generation int64
// Metageneration is the version of the metadata for this object at
// this generation. This field is used for preconditions and for
// detecting changes in metadata. A metageneration number is only
// meaningful in the context of a particular generation of a
// particular object.
Metageneration int64
}
RetentionPolicy
RetentionPolicy enforces a minimum retention time for all objects contained in the bucket.
Any attempt to overwrite or delete objects younger than the retention period will result in an error. An unlocked retention policy can be modified or removed from the bucket via the Update method. A locked retention policy cannot be removed or shortened in duration for the lifetime of the bucket.
This feature is in private alpha release. It is not currently available to most customers. It might be changed in backwards-incompatible ways and is not subject to any SLA or deprecation policy.
Declaration
type RetentionPolicy struct {
// RetentionPeriod specifies the duration that objects need to be
// retained. Retention duration must be greater than zero and less than
// 100 years. Note that enforcement of retention periods less than a day
// is not guaranteed. Such periods should only be used for testing
// purposes.
RetentionPeriod time.Duration
// EffectiveTime is the time from which the policy was enforced and
// effective. This field is read-only.
EffectiveTime time.Time
// IsLocked describes whether the bucket is locked. Once locked, an
// object retention policy cannot be modified.
// This field is read-only.
IsLocked bool
}
SignedURLOptions
SignedURLOptions allows you to restrict the access to the signed URL.
Declaration
type SignedURLOptions struct {
// GoogleAccessID represents the authorizer of the signed URL generation.
// It is typically the Google service account client email address from
// the Google Developers Console in the form of<EMAIL_ADDRESS>// Required.
GoogleAccessID string
// PrivateKey is the Google service account private key. It is obtainable
// from the Google Developers Console.
// At https://console.developers.google.com/project/<your-project-id>/apiui/credential,
// create a service account client ID or reuse one of your existing service account
// credentials. Click on the "Generate new P12 key" to generate and download
// a new private key. Once you download the P12 file, use the following command
// to convert it into a PEM file.
//
// $ openssl pkcs12 -in key.p12 -passin pass:notasecret -out key.pem -nodes
//
// Provide the contents of the PEM file as a byte slice.
// Exactly one of PrivateKey or SignBytes must be non-nil.
PrivateKey []byte
// SignBytes is a function for implementing custom signing. For example, if
// your application is running on Google App Engine, you can use
// appengine's internal signing function:
// ctx := appengine.NewContext(request)
// acc, _ := appengine.ServiceAccount(ctx)
// url, err := SignedURL("bucket", "object", &SignedURLOptions{
// GoogleAccessID: acc,
// SignBytes: func(b []byte) ([]byte, error) {
// _, signedBytes, err := appengine.SignBytes(ctx, b)
// return signedBytes, err
// },
// // etc.
// })
//
// Exactly one of PrivateKey or SignBytes must be non-nil.
SignBytes func([]byte) ([]byte, error)
// Method is the HTTP method to be used with the signed URL.
// Signed URLs can be used with GET, HEAD, PUT, and DELETE requests.
// Required.
Method string
// Expires is the expiration time on the signed URL. It must be
// a datetime in the future. For SigningSchemeV4, the expiration may be no
// more than seven days in the future.
// Required.
Expires time.Time
// ContentType is the content type header the client must provide
// to use the generated signed URL.
// Optional.
ContentType string
// Headers is a list of extension headers the client must provide
// in order to use the generated signed URL. Each must be a string of the
// form "key:values", with multiple values separated by a semicolon.
// Optional.
Headers []string
// QueryParameters is a map of additional query parameters. When
// SigningScheme is V4, this is used in computing the signature, and the
// client must use the same query parameters when using the generated signed
// URL.
// Optional.
QueryParameters url.Values
// MD5 is the base64 encoded MD5 checksum of the file.
// If provided, the client should provide the exact value on the request
// header in order to use the signed URL.
// Optional.
MD5 string
// Style provides options for the type of URL to use. Options are
// PathStyle (default), BucketBoundHostname, and VirtualHostedStyle. See
// https://cloud.google.com/storage/docs/request-endpoints for details.
// Only supported for V4 signing.
// Optional.
Style URLStyle
// Insecure determines whether the signed URL should use HTTPS (default) or
// HTTP.
// Only supported for V4 signing.
// Optional.
Insecure bool
// Scheme determines the version of URL signing to use. Default is
// SigningSchemeV2.
Scheme SigningScheme
}
SigningScheme
SigningScheme determines the API version to use when signing URLs.
Declaration
type SigningScheme int
SigningSchemeDefault, SigningSchemeV2, SigningSchemeV4
Declaration
const (
// SigningSchemeDefault is presently V2 and will change to V4 in the future.
SigningSchemeDefault SigningScheme = iota
// SigningSchemeV2 uses the V2 scheme to sign URLs.
SigningSchemeV2
// SigningSchemeV4 uses the V4 scheme to sign URLs.
SigningSchemeV4
)
URLStyle
URLStyle determines the style to use for the signed URL. pathStyle is the default. All non-default options work with V4 scheme only. See https://cloud.google.com/storage/docs/request-endpoints for details.
Declaration
type URLStyle interface {
// contains filtered or unexported methods
}
func BucketBoundHostname
BucketBoundHostname generates a URL with a custom hostname tied to a
specific GCS bucket. The desired hostname should be passed in using the
hostname argument. Generated urls will be of the form
"
Declaration
func BucketBoundHostname(hostname string) URLStyle
func PathStyle
PathStyle is the default style, and will generate a URL of the form
"storage.googleapis.com/
Declaration
func PathStyle() URLStyle
func VirtualHostedStyle
VirtualHostedStyle generates a URL relative to the bucket's virtual
hostname, e.g. "
Declaration
func VirtualHostedStyle() URLStyle
UniformBucketLevelAccess
UniformBucketLevelAccess configures access checks to use only bucket-level IAM policies.
Declaration
type UniformBucketLevelAccess struct {
// Enabled specifies whether access checks use only bucket-level IAM
// policies. Enabled may be disabled until the locked time.
Enabled bool
// LockedTime specifies the deadline for changing Enabled from true to
// false.
LockedTime time.Time
}
Writer
A Writer writes a Cloud Storage object.
Declaration
type Writer struct {
// ObjectAttrs are optional attributes to set on the object. Any attributes
// must be initialized before the first Write call. Nil or zero-valued
// attributes are ignored.
ObjectAttrs
// SendCRC specifies whether to transmit a CRC32C field. It should be set
// to true in addition to setting the Writer's CRC32C field, because zero
// is a valid CRC and normally a zero would not be transmitted.
// If a CRC32C is sent, and the data written does not match the checksum,
// the write will be rejected.
SendCRC32C bool
// ChunkSize controls the maximum number of bytes of the object that the
// Writer will attempt to send to the server in a single request. Objects
// smaller than the size will be sent in a single request, while larger
// objects will be split over multiple requests. The size will be rounded up
// to the nearest multiple of 256K.
//
// ChunkSize will default to a reasonable value. If you perform many
// concurrent writes of small objects (under ~8MB), you may wish set ChunkSize
// to a value that matches your objects' sizes to avoid consuming large
// amounts of memory. See
// https://cloud.google.com/storage/docs/json_api/v1/how-tos/upload#size
// for more information about performance trade-offs related to ChunkSize.
//
// If ChunkSize is set to zero, chunking will be disabled and the object will
// be uploaded in a single request without the use of a buffer. This will
// further reduce memory used during uploads, but will also prevent the writer
// from retrying in case of a transient error from the server, since a buffer
// is required in order to retry the failed request.
//
// ChunkSize must be set before the first Write call.
ChunkSize int
// ProgressFunc can be used to monitor the progress of a large write.
// operation. If ProgressFunc is not nil and writing requires multiple
// calls to the underlying service (see
// https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload),
// then ProgressFunc will be invoked after each call with the number of bytes of
// content copied so far.
//
// ProgressFunc should return quickly without blocking.
ProgressFunc func(int64)
// contains filtered or unexported fields
}
func (*Writer) Attrs
Attrs returns metadata about a successfully-written object. It's only valid to call it after Close returns nil.
Declaration
func (w *Writer) Attrs() *ObjectAttrs
func (*Writer) Close
Close completes the write operation and flushes any buffered data. If Close doesn't return an error, metadata about the written object can be retrieved by calling Attrs.
Declaration
func (w *Writer) Close() error
func (*Writer) CloseWithError
CloseWithError aborts the write operation with the provided error. CloseWithError always returns nil.
Deprecated: cancel the context passed to NewWriter instead.
Declaration
func (w *Writer) CloseWithError(err error) error
func (*Writer) Write
Write appends to w. It implements the io.Writer interface.
Since writes happen asynchronously, Write may return a nil error even though the write failed (or will fail). Always use the error returned from Writer.Close to determine if the upload was successful.
Writes will be retried on transient errors from the server, unless Writer.ChunkSize has been set to zero.
Declaration
func (w *Writer) Write(p []byte) (n int, err error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
wc := client.Bucket("bucketname").Object("filename1").NewWriter(ctx)
wc.ContentType = "text/plain"
wc.ACL = []storage.ACLRule{{Entity: storage.AllUsers, Role: storage.RoleReader}}
if _, err := wc.Write([]byte("hello world")); err != nil {
// TODO: handle error.
// Note that Write may return nil in some error situations,
// so always check the error from Close.
}
if err := wc.Close(); err != nil {
// TODO: handle error.
}
fmt.Println("updated object:", wc.Attrs())
}
{% endverbatim %}
checksum
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"hash/crc32"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
data := []byte("verify me")
wc := client.Bucket("bucketname").Object("filename1").NewWriter(ctx)
wc.CRC32C = crc32.Checksum(data, crc32.MakeTable(crc32.Castagnoli))
wc.SendCRC32C = true
if _, err := wc.Write([]byte("hello world")); err != nil {
// TODO: handle error.
// Note that Write may return nil in some error situations,
// so always check the error from Close.
}
if err := wc.Close(); err != nil {
// TODO: handle error.
}
fmt.Println("updated object:", wc.Attrs())
}
{% endverbatim %}
timeout
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"context"
"fmt"
"time"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
tctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel() // Cancel when done, whether we time out or not.
wc := client.Bucket("bucketname").Object("filename1").NewWriter(tctx)
wc.ContentType = "text/plain"
wc.ACL = []storage.ACLRule{{Entity: storage.AllUsers, Role: storage.RoleReader}}
if _, err := wc.Write([]byte("hello world")); err != nil {
// TODO: handle error.
// Note that Write may return nil in some error situations,
// so always check the error from Close.
}
if err := wc.Close(); err != nil {
// TODO: handle error.
}
fmt.Println("updated object:", wc.Attrs())
}
{% endverbatim %}
Functions
func SignedURL
SignedURL returns a URL for the specified object. Signed URLs allow the users access to a restricted resource for a limited time without having a Google account or signing in. For more information about the signed URLs, see https://cloud.google.com/storage/docs/accesscontrol#Signed-URLs.
Declaration
func SignedURL(bucket, name string, opts *SignedURLOptions) (string, error)
Examples
{% verbatim %}package main
import (
"cloud.google.com/go/storage"
"fmt"
"io/ioutil"
"time"
)
func main() {
pkey, err := ioutil.ReadFile("my-private-key.pem")
if err != nil {
// TODO: handle error.
}
url, err := storage.SignedURL("my-bucket", "my-object", &storage.SignedURLOptions{
GoogleAccessID<EMAIL_ADDRESS>PrivateKey: pkey,
Method: "GET",
Expires: time.Now().Add(48 * time.Hour),
})
if err != nil {
// TODO: handle error.
}
fmt.Println(url)
}
{% endverbatim %}