Overview#
Crunch REST APIs for resource management provide a set of endpoints that let you programmatically interact with catalogs, tables, actions, managed tables, policies, activities, users, and system configuration.
These REST APIs support:
- Catalog Discovery — List catalogs (e.g. Iceberg, Delta Lake), namespaces, and tables.
- Table Actions — List and trigger actions (expire snapshots, compact, etc.) at the catalog table level.
- Managed Table Onboarding — Create policies to onboard a catalog, a prefix, or a table (tables inherit default policy unless an explicit table-level policy exists).
- Managed Table Discovery — List and inspect managed tables (tables with at least one policy) and optimization opportunities.
- Managed Table Policies — List, get, delete, and enable/disable policies on managed tables.
- Table Activities — View and retry runs (policy- or action-triggered) for catalog tables.
- Metrics — Time-series metrics (DUM, bytes crunched, DRR, etc.).
- Users — User and role management.
- System Config — System-wide configuration.
Requirements#
To use the Crunch REST APIs you need:
- A way to send REST requests (e.g. Postman, curl, or an HTTP client in your language).
- Valid authentication credentials for the API.
Suggested tools#
- Postman
- curl
- Any HTTP client in your preferred programming language
Catalog Discovery APIs#
Discover catalogs, namespaces, and tables. Many customers use both Iceberg and Delta Lake.
List catalogs#
GET /api/v1/catalogs
List all catalogs used by customers.
Path Parameters
| Parameter | Description |
|---|
| — | This endpoint does not use path parameters. |
Query Parameters
| Parameter | Description |
|---|
| — | This endpoint does not use query parameters. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns list of catalogs. |
| 4xx / 5xx | Error response (e.g. unauthorized, server error). |
List namespaces#
GET /api/v1/catalogs/{catalog_id}/namespaces
List all namespaces in a tree structure for filtering. Nesting can have an arbitrary number of levels (e.g. bucket/gold_warehouse/risk/staging/test_data/tablename_1).
Path Parameters
| Parameter | Description |
|---|
catalog_id | Identifier for the catalog. |
Query Parameters
| Parameter | Description |
|---|
prefix | Filter namespaces by prefix. |
page_size | Maximum number of items to return per page. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns list of namespaces. |
| 4xx / 5xx | Error response. |
List tables in catalog#
GET /api/v1/catalogs/{catalog_id}/tables
List tables in a catalog. Use type and prefix to narrow the list. Use q for wildcard matching on table names.
Path Parameters
| Parameter | Description |
|---|
catalog_id | Identifier for the catalog. |
Query Parameters
| Parameter | Description |
|---|
q | Wildcard pattern to match table names. |
prefix | Filter tables by namespace prefix. |
type | Filter by table type. |
page_size | Maximum number of items to return per page. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns list of tables. |
| 4xx / 5xx | Error response. |
Get table details (catalog)#
GET /api/v1/catalogs/{catalog_id}/tables/{table_id}
Get table details and metadata from a catalog.
Path Parameters
| Parameter | Description |
|---|
catalog_id | Identifier for the catalog. |
table_id | Identifier for the table. |
Query Parameters
| Parameter | Description |
|---|
| — | This endpoint does not use query parameters. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns table details and metadata. |
| 4xx / 5xx | Error response (e.g. not found). |
Table Action APIs#
List, trigger, and manage actions for a table in a catalog. Supported actions can vary by table (e.g. Hive vs. Iceberg, partitioned vs. non-partitioned).
List actions#
GET /api/v1/catalogs/{catalog_id}/tables/{table_id}/actions
List all actions supported by the platform (e.g. expire snapshots and vacuum, expire partitions, compress, compact, improve layout). Actions can vary by table type (Hive vs. Iceberg), partitioning, etc.
Path Parameters
| Parameter | Description |
|---|
catalog_id | Identifier for the catalog. |
table_id | Identifier for the table. |
Query Parameters
| Parameter | Description |
|---|
| — | This endpoint does not use query parameters. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns list of supported actions. |
| 4xx / 5xx | Error response. |
Trigger/schedule actions#
POST /api/v1/catalogs/{catalog_id}/tables/{table_id}/actions
Trigger or schedule actions for the table.
Path Parameters
| Parameter | Description |
|---|
catalog_id | Identifier for the catalog. |
table_id | Identifier for the table. |
Query Parameters
| Parameter | Description |
|---|
| — | Request body specifies action and schedule. |
Response
| Code | Description |
|---|
| 200 / 202 | Successfully accepted or completed the request. |
| 4xx / 5xx | Error response. |
Get action details#
GET /api/v1/catalogs/{catalog_id}/tables/{table_id}/actions/{action_id}
Get action details.
Path Parameters
| Parameter | Description |
|---|
catalog_id | Identifier for the catalog. |
table_id | Identifier for the table. |
action_id | Identifier for the action. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns action details. |
| 4xx / 5xx | Error response. |
Retry action#
POST /api/v1/catalogs/{catalog_id}/tables/{table_id}/actions/{action_id}
Retry an action.
Path Parameters
| Parameter | Description |
|---|
catalog_id | Identifier for the catalog. |
table_id | Identifier for the table. |
action_id | Identifier for the action to retry. |
Response
| Code | Description |
|---|
| 200 / 202 | Successfully accepted or completed the retry. |
| 4xx / 5xx | Error response. |
Managed Table Onboarding APIs#
Onboarding is done by creating policies. Create a policy at catalog level (default for entire catalog), at prefix level (default for that prefix), or at table level. All tables within a default policy scope inherit that policy unless an explicit table-level policy exists.
Create policy (catalog / prefix / table)#
POST /api/v1/catalogs/{catalog_id}/policies
Create a default policy for a prefix of the catalog (onboard prefix). All tables under that prefix inherit this policy by default.
Path Parameters
| Parameter | Description |
|---|
catalog_id | Identifier for the catalog. |
Query Parameters
| Parameter | Description |
|---|
prefix | Namespace prefix to which the default policy applies. |
Response
| Code | Description |
|---|
| 200 / 201 | Successfully created the policy. |
| 4xx / 5xx | Error response. |
POST /api/v1/catalogs/{catalog_id}/tables/{table_id}/policies
Create a policy for a specific table (onboard table). This is an explicit table-level policy.
Path Parameters
| Parameter | Description |
|---|
catalog_id | Identifier for the catalog. |
table_id | Identifier for the table. |
Response
| Code | Description |
|---|
| 200 / 201 | Successfully created the table-level policy. |
| 4xx / 5xx | Error response. |
Managed Table Discovery APIs#
Managed tables are tables with at least one policy.
List managed tables#
GET /api/v1/managedtables
List all managed tables (tables with at least one policy).
Path Parameters
| Parameter | Description |
|---|
| — | This endpoint does not use path parameters. |
Query Parameters
| Parameter | Description |
|---|
q | Filter or search by pattern (e.g. wildcard on table name). |
page_size | Maximum number of items to return per page. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns list of managed tables. |
| 4xx / 5xx | Error response. |
Get managed table detail#
GET /api/v1/managedtables/{table_id}
Get managed table detail, including opportunities (e.g. potential DRR, missing ordering, small files).
Path Parameters
| Parameter | Description |
|---|
table_id | Identifier for the managed table. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns managed table details and opportunities. |
| 4xx / 5xx | Error response (e.g. not found). |
Managed Table Policy APIs#
List policies#
GET /api/v1/managedtables/{table_id}/policies
List policies applied to a managed table.
Path Parameters
| Parameter | Description |
|---|
table_id | Identifier for the managed table. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns list of policies for the table. |
| 4xx / 5xx | Error response. |
Get policy#
GET /api/v1/managedtables/{table_id}/policies/{policy_id}
Get a single policy by ID.
Path Parameters
| Parameter | Description |
|---|
table_id | Identifier for the managed table. |
policy_id | Identifier for the policy. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns policy details. |
| 4xx / 5xx | Error response (e.g. not found). |
Delete policy#
DELETE /api/v1/managedtables/{table_id}/policies/{policy_id}
Delete a policy. If all policies are deleted for a table, the table becomes an un-managed table.
Path Parameters
| Parameter | Description |
|---|
table_id | Identifier for the managed table. |
policy_id | Identifier for the policy to delete. |
Response
| Code | Description |
|---|
| 200 / 204 | Successfully deleted the policy. |
| 4xx / 5xx | Error response. |
Enable and disable policy#
POST /api/v1/managedtables/{table_id}/policies/{policy_id}:enable
Enable a policy.
POST /api/v1/managedtables/{table_id}/policies/{policy_id}:disable
Disable a policy.
Path Parameters
| Parameter | Description |
|---|
table_id | Identifier for the managed table. |
policy_id | Identifier for the policy to enable or disable. |
Response
| Code | Description |
|---|
| 200 | Successfully enabled or disabled the policy. |
| 4xx / 5xx | Error response. |
Table Activities APIs#
Activity history for a table in a catalog, including policy-triggered and manual runs.
List activity history#
GET /api/v1/catalogs/{catalog_id}/tables/{table_id}/runs
Get activity history for a table, including both policy-triggered and manual runs.
Path Parameters
| Parameter | Description |
|---|
catalog_id | Identifier for the catalog. |
table_id | Identifier for the table. |
Query Parameters
| Parameter | Description |
|---|
start | Start of time range for filtering runs. |
end | End of time range for filtering runs. |
status | Filter by run status. |
trigger | Filter by trigger type: POLICY or ACTION. |
page_size | Maximum number of items to return per page. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns list of runs. |
| 4xx / 5xx | Error response. |
Get run detail#
GET /api/v1/catalogs/{catalog_id}/tables/{table_id}/runs/{run_id}
Get activity run detail.
Path Parameters
| Parameter | Description |
|---|
catalog_id | Identifier for the catalog. |
table_id | Identifier for the table. |
run_id | Identifier for the run. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns run details. |
| 4xx / 5xx | Error response (e.g. not found). |
Retry/rerun activity#
POST /api/v1/catalogs/{catalog_id}/tables/{table_id}/runs/{run_id}
Retry or rerun an activity.
Path Parameters
| Parameter | Description |
|---|
catalog_id | Identifier for the catalog. |
table_id | Identifier for the table. |
run_id | Identifier for the run to retry. |
Response
| Code | Description |
|---|
| 200 / 202 | Successfully accepted or completed the retry. |
| 4xx / 5xx | Error response. |
Metrics APIs#
Time-series metrics such as DUM, bytes crunched, partitions crunched, files reduced, DRR, partitions deleted, bytes deleted, query speed improvements, query failure reduction rate, etc.
Note: Design decision — build custom APIs vs. leverage APIs provided by existing metric tech stack. Endpoints to be defined.
User APIs#
List users#
GET /api/v1/users
List users.
Path Parameters
| Parameter | Description |
|---|
| — | This endpoint does not use path parameters. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns list of users. |
| 4xx / 5xx | Error response. |
Get user details#
GET /api/v1/users/{user_id}
Get user details.
Path Parameters
| Parameter | Description |
|---|
user_id | Identifier for the user. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns user details. |
| 4xx / 5xx | Error response (e.g. not found). |
Create user#
POST /api/v1/users
Create a new user.
Path Parameters
| Parameter | Description |
|---|
| — | User details in request body. |
Response
| Code | Description |
|---|
| 200 / 201 | Successfully created the user. |
| 4xx / 5xx | Error response. |
Delete user#
DELETE /api/v1/users/{user_id}
Delete a user.
Path Parameters
| Parameter | Description |
|---|
user_id | Identifier for the user to delete. |
Response
| Code | Description |
|---|
| 200 / 204 | Successfully deleted the user. |
| 4xx / 5xx | Error response. |
Get user roles#
GET /api/v1/users/{user_id}/roles
Get roles assigned to a user.
Path Parameters
| Parameter | Description |
|---|
user_id | Identifier for the user. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns list of roles for the user. |
| 4xx / 5xx | Error response. |
Revoke user roles#
DELETE /api/v1/users/{user_id}/roles
Revoke user roles. Roles to revoke are specified in the request body.
Path Parameters
| Parameter | Description |
|---|
user_id | Identifier for the user. |
Response
| Code | Description |
|---|
| 200 / 204 | Successfully revoked the specified roles. |
| 4xx / 5xx | Error response. |
System Config API#
Get system configuration#
GET /api/v1/config
Get system configuration (e.g. email addresses, Slack webhooks, minimum table size to qualify for optimization, minimum existence days for tables to qualify, default policy values such as 30 days for partition expiration).
Path Parameters
| Parameter | Description |
|---|
| — | This endpoint does not use path parameters. |
Response
| Code | Description |
|---|
| 200 | Successful response; returns system configuration. |
| 4xx / 5xx | Error response. |
Update system configuration#
POST /api/v1/config
Update system configuration. Configuration in request body.
Path Parameters
| Parameter | Description |
|---|
| — | Configuration in request body. |
Response
| Code | Description |
|---|
| 200 | Successfully updated system configuration. |
| 4xx / 5xx | Error response. |