Google Cloud Storage Rewrite: Move Conservation Scans Without Losing the Approved Generation

Rewriting is a copy operation, not proof of archival integrity

Imagine that a regional museum needs to move a large conservation scan from a working bucket to a controlled long-term bucket. Google Cloud Storage provides objects.rewrite to copy an object to a destination and optionally adjust metadata. The source object is not modified when the source and destination differ. A successful copy request alone is not an archive acceptance decision; staff still need to confirm the exact source generation, expected metadata, destination location, and controlled access. Prepare a migration register with source bucket, key, source generation, target bucket, target name, retention requirements, and owner approval. Do not delete the source merely because a request was initiated.

Pin the source generation to avoid a moving target

An object name can have several generations. If an archivist updates a master scan while transfer is in progress, a migration job must not silently copy whichever version happens to be latest at the moment of execution. The rewrite API allows a specific sourceGeneration and conditional options including ifSourceGenerationMatch. Record the intended version before starting the job, and stop when a precondition fails rather than switching to a newer object without authorization. For an illustrative scan, compare the approved master version in the catalog to the generation number actually used by the request. A matching filename is not enough to demonstrate that the correct scan was transferred.

Use destination conditions before overwriting

The destination bucket may already contain a scan under the expected name. Before writing, decide whether the migration creates a new object, copies into a versioned destination, or intentionally replaces an approved asset. Use relevant destination preconditions and IAM controls to avoid overwriting existing data by accident. Permissions for destination creation are different from the permissions needed when overwriting. If the destination has retention policies, encryption requirements, or restricted catalog metadata, verify those independently. Avoid an all-purpose service identity with both delete and write access everywhere. The operations record should say what happens if the destination key exists, and that rule should be tested in staging.

Illustrated three-stage workflow: Pin source generation; Continue rewrite token; Inspect destination copy.

Continue large rewrites using rewriteToken

The official API documents a multi-request pattern: if the response contains done:false, it also supplies rewriteToken. The client must send the token on subsequent rewrite calls until done becomes true. This avoids relying on one very long request for a large or cross-location transfer. The token is scoped to its rewrite operation and can expire; the reference specifies a one-week completion window. Never label a progress response as copied merely because the transferred-byte counter increased. Persist controlled job state securely, handle expired tokens explicitly, and avoid logging live access credentials or confidential object metadata. A completed transfer is the beginning of verification, not the end.

Understand when a single call is enough

Not every copy returns a continuation token. Google notes that under specified conditions, including no storage-class change and the same location, a rewrite can complete in one request. A portable worker should therefore branch on done rather than assuming every job requires retries or every job completes immediately. Similarly, a network timeout does not show whether the server finished the operation; inspect the destination and job state before restarting from scratch. A migration queue should use a stable task identifier, list each requested object once, and record the final destination generation only after successful completion. That keeps retries observable without inventing a transactional guarantee.

Compare pending and completed destinations honestly

A useful diagram shows three states: the source approved generation remains unchanged, the destination rewrite is in progress with done:false, and the destination exists with done:true and a known generation. The destination should not appear as an accepted preservation copy until access, metadata, and file verification are finished. The illustration is a conceptual model, not a screenshot of a live transfer or a benchmark. If a progress counter shows that most bytes were rewritten, that still does not prove the final resource is usable. Readers should distinguish transferred data, completed API operation, and archivally accepted object as separate labels in their tracking system.

Conceptual comparison of different states: Approved source intact; Rewrite still in progress; Copied; not yet accepted. No measured outcomes.

Check size, checksums, and metadata after completion

When the rewrite completes, inspect the resource returned by the API or retrieve the destination object metadata. Compare the expected size and applicable server-supplied integrity fields to an approved source record, taking care with composite objects and checksum conventions. Validate the content with an authorized sample download or stronger in-house fixity process when the organization requires one. Check the content type, retention, custom metadata, location, and encryption configuration rather than assuming they all remain identical. Do not call a copy verified merely because it has the original filename. Record source generation, destination generation, and the result of each independent QA check for future audits.

Protect access and budget across locations

Moving a large image set across storage locations can introduce transfer charges or operational constraints. Google documents that rewrite requests can involve costs beyond the Class A operation depending on storage class and location. Confirm the live pricing and region policy before moving thousands of scans. Apply destination IAM based on who should access approved masters, not on who could read working files. Verify that preview images or internal conservation notes do not become public because of inherited bucket settings. The API offers a copying mechanism; repository governance still determines which materials are authorized, how long they are retained, and who can share them.

Rehearse failures before the production move

Use harmless staging objects to test a single-request rewrite, a forced multi-call path, an expired or invalid rewrite token, a changed source generation, and an existing destination. Inspect whether retry logic follows the documented done and token fields and whether a blocked copy leaves the approved source untouched. Create a final reconciliation report that lists completed, failed, and awaiting-approval objects rather than reporting only the number of submitted requests. These are suggested tests, not a claim that this article moved real museum content. A preservation migration should end with an archivist signoff and a plan for retaining or retiring the original according to policy.

Illustrated verification checkpoints: Generation matches?; Metadata and fixity?; Access and retention?.

How We Research

We reviewed these named official developer references: Google Cloud Storage objects.rewrite reference; Google Cloud Storage object copying guide. We compared documented API behavior, supported configuration fields, and stated limits against a hypothetical museum workflow. We did not perform live customer API calls, examine restricted collection records, run a real production benchmark, or measure service speed or costs. The conceptual figures show workflow logic rather than screenshots or test results. Reconfirm documentation and available API versions before deployment; use authorized staging tests and human review for every public-facing workflow.

Frequently Asked Questions

Does objects.rewrite delete the source object?

No, when source and destination differ it copies the source; any later deletion is a separate authorized action.

When is rewriteToken required?

Use the token returned when done is false and continue until the API reports done true.

Can a rewrite token be reused indefinitely?

No. Google documents a limited completion window; expired tokens require recovery handling.

How do I avoid copying a newer, unapproved scan?

Specify the approved source generation and appropriate conditional checks.

Does done:true mean archivists have accepted the copy?

No. Check destination identity, metadata, integrity, permissions, and retention before signoff.

Scroll to Top