Manage catalogs

Learn how to:

• Enable and disable R2 Data Catalog on your buckets.
• Enable and disable table maintenance features like compaction and snapshot expiration.
• Authenticate Iceberg engines using API tokens.

Enable R2 Data Catalog on a bucket

Enabling the catalog on a bucket turns on the REST catalog interface and provides a Catalog URI and Warehouse name required by Iceberg clients. Once enabled, you can create and manage Iceberg tables in that bucket.

Dashboard

1. In the Cloudflare dashboard, go to the R2 Data Catalog page.

   Go to R2 Data Catalog

2. Select Create catalog.

3. Enter an R2 bucket name. You can select an existing bucket or enter a new bucket name to create one. If creating a new bucket, optionally select a location hint.

4. Select Next to configure table maintenance. Optionally enable compaction and snapshot expiration during setup.

5. Review your configuration and select Create catalog.

6. Once created, the catalog detail page displays your Catalog URI and Warehouse name.

Wrangler CLI

To enable the catalog on your bucket, run the r2 bucket catalog enable command:

npx wrangler r2 bucket catalog enable <BUCKET_NAME>

After enabling, Wrangler will return your catalog URI and warehouse name.

Disable R2 Data Catalog on a bucket

When you disable the catalog on a bucket, it immediately stops serving requests from the catalog interface. Any Iceberg table references stored in that catalog become inaccessible until you re-enable it.

Dashboard

1. In the Cloudflare dashboard, go to the R2 Data Catalog page.

   Go to R2 Data Catalog

2. Select the catalog you want to disable.

3. Go to the Settings tab and scroll to the Disable section.

4. Select Disable and confirm.

Wrangler CLI

To disable the catalog on your bucket, run the r2 bucket catalog disable command:

npx wrangler r2 bucket catalog disable <BUCKET_NAME>

Enable compaction

Compaction improves query performance by combining the many small files created during data ingestion into fewer, larger files according to the set target file size. For more information about compaction and why it is valuable, refer to About compaction.

API token permission requirements

Table maintenance operations such as compaction and snapshot expiration require a Cloudflare API token with both R2 storage and R2 Data Catalog read/write permissions to act as a service credential.

Refer to Authenticate your Iceberg engine for details on creating a token with the required permissions.

Dashboard

1. In the Cloudflare dashboard, go to the R2 Data Catalog page.

   Go to R2 Data Catalog

2. Select the catalog you want to enable compaction on.

3. Go to the Settings tab and scroll to the Table Maintenance section.

4. Select Enable or Edit next to compaction.

5. Toggle compaction on and optionally set a target file size. The default is 128 MB.

6. Select Save.

Note

If no service credential has been generated for this catalog, the dashboard will prompt you to generate one. Compaction requires a credential to access and rewrite files in your bucket.

Wrangler CLI

To enable the compaction on your catalog, run the r2 bucket catalog compaction enable command:

# Enable catalog-level compaction (all tables)
npx wrangler r2 bucket catalog compaction enable <BUCKET_NAME> --target-size 128 --token <API_TOKEN>

# Enable compaction for a specific table
npx wrangler r2 bucket catalog compaction enable <BUCKET_NAME> <NAMESPACE> <TABLE> --target-size 128

Table-level vs Catalog-level compaction

• Catalog-level: Applies to all tables in the bucket; requires an API token as a service credential.
• Table-level: Applies to a specific table only.

Once enabled, compaction applies retroactively to all existing tables (for catalog-level compaction) or the specified table (for table-level compaction). During open beta, we currently compact up to 2 GB worth of files once per hour for each table.

Disable compaction

Disabling compaction will prevent the process from running for all tables (catalog level) or a specific table (table level). You can re-enable it at any time.

Dashboard

1. In the Cloudflare dashboard, go to the R2 Data Catalog page.

   Go to R2 Data Catalog

2. Select the catalog you want to disable compaction on.

3. Go to the Settings tab and scroll to the Table Maintenance section.

4. Select Edit next to compaction.

5. Toggle compaction off.

6. Select Save.

Wrangler CLI

To disable the compaction on your catalog, run the r2 bucket catalog compaction disable command:

# Disable catalog-level compaction (all tables)
npx wrangler r2 bucket catalog compaction disable <BUCKET_NAME>

# Disable compaction for a specific table
npx wrangler r2 bucket catalog compaction disable <BUCKET_NAME> <NAMESPACE> <TABLE>

Enable snapshot expiration

Snapshot expiration automatically removes old table snapshots and any unreferenced data files to reduce metadata overhead and storage costs. You can configure:

• Max snapshot age - Snapshots older than this duration are expired. Specify a value followed by a unit (d for days, h for hours, m for minutes, s for seconds). For example, 7d expires snapshots older than 7 days.
• Min snapshots to keep - The minimum number of snapshots to retain, regardless of age.

Dashboard

1. In the Cloudflare dashboard, go to the R2 Data Catalog page.

   Go to R2 Data Catalog

2. Select the catalog you want to enable snapshot expiration on.

3. Go to the Settings tab and scroll to the Table Maintenance section.

4. Select Enable or Edit next to snapshot expiration.

5. Toggle snapshot expiration on.

6. Set the Max snapshot age and Min snapshots to keep values.

7. Select Save.

Wrangler CLI

Note

Snapshot expiration commands are available as of Wrangler version 4.56.0.

To enable snapshot expiration on your catalog, run the r2 bucket catalog snapshot-expiration enable command:

# Enable catalog-level snapshot expiration (all tables)
npx wrangler r2 bucket catalog snapshot-expiration enable <BUCKET_NAME> \
  --token <API_TOKEN> \
  --older-than-days 7 \
  --retain-last 10

# Enable snapshot expiration for a specific table
npx wrangler r2 bucket catalog snapshot-expiration enable <BUCKET_NAME> <NAMESPACE> <TABLE> \
  --older-than-days 2 \
  --retain-last 5

Disable snapshot expiration

Disabling snapshot expiration prevents the process from running for all tables (catalog level) or a specific table (table level). You can re-enable snapshot expiration at any time.

Dashboard

1. In the Cloudflare dashboard, go to the R2 Data Catalog page.

   Go to R2 Data Catalog

2. Select the catalog you want to disable snapshot expiration on.

3. Go to the Settings tab and scroll to the Table Maintenance section.

4. Select Edit next to snapshot expiration.

5. Toggle snapshot expiration off.

6. Select Save.

Wrangler CLI

# Disable catalog-level snapshot expiration (all tables)
npx wrangler r2 bucket catalog snapshot-expiration disable <BUCKET_NAME>

# Disable snapshot expiration for a specific table
npx wrangler r2 bucket catalog snapshot-expiration disable <BUCKET_NAME> <NAMESPACE> <TABLE>

Authenticate your Iceberg engine

To connect your Iceberg engine to R2 Data Catalog, you must provide a Cloudflare API token with both R2 Data Catalog permissions and R2 storage permissions. Iceberg engines interact with R2 Data Catalog to perform table operations. The catalog also provides engines with SigV4 credentials, which are required to access the underlying data files stored in R2.

Create API token in the dashboard

Create an R2 API token with Admin Read & Write permissions. These permissions include both:

• Access to R2 Data Catalog (read/write)
• Access to R2 storage (read/write)

Providing the resulting token value to your Iceberg engine gives it the ability to manage catalog metadata and handle data operations (reads or writes to R2).

Create API token via API

To create an API token programmatically for use with R2 Data Catalog, you need to specify both R2 Data Catalog and R2 storage permission groups in your Access Policy.

Example Access Policy

[
  {
    "id": "f267e341f3dd4697bd3b9f71dd96247f",
    "effect": "allow",
    "resources": {
      "com.cloudflare.edge.r2.bucket.4793d734c0b8e484dfc37ec392b5fa8a_default_my-bucket": "*",
      "com.cloudflare.edge.r2.bucket.4793d734c0b8e484dfc37ec392b5fa8a_eu_my-eu-bucket": "*"
    },
    "permission_groups": [
      {
        "id": "d229766a2f7f4d299f20eaa8c9b1fde9",
        "name": "Workers R2 Data Catalog Write"
      },
      {
        "id": "2efd5506f9c8494dacb1fa10a3e7d5b6",
        "name": "Workers R2 Storage Bucket Item Write"
      }
    ]
  }
]

To learn more about how to create API tokens for R2 Data Catalog using the API, including required permission groups and usage examples, refer to the Create API tokens via API documentation.

R2 Local Uploads

Local Uploads writes object data to a nearby location, then asynchronously copies it to your bucket. Data is queryable immediately and remains strongly consistent. This can significantly improve latency of writes from Apache Iceberg clients outside of the region of the respective R2 Data Catalog bucket.

To enable R2 Local Uploads, you can use the following Wrangler command:

npx wrangler r2 bucket catalog local-uploads enable <R2_Data_Catalog_BUCKET_NAME>

Limitations

• R2 Data Catalog does not currently support R2 buckets in a non-default jurisdiction.

Learn more

Get started Learn how to enable the R2 Data Catalog on your bucket, load sample data, and run your first query.

Connect to Iceberg engines Find detailed setup instructions for Apache Spark and other common query engines.