Use S3-Compatible Storage#

S3-compatible object storage is available from many providers, and they all share the same request-signing scheme: AWS Signature Version 4. The sandbox does not include an AWS SDK, so this example builds a small helper for basic object operations that signs requests with Web Crypto, then uses it to upload and read an object.

The helper supports GET, PUT, HEAD, and DELETE, custom headers, query parameters for list requests, and both path-style and virtual-hosted URLs.

The Helper#

Save this file as /lib/s3.js. It exports createS3Client, which takes the bucket configuration and returns an object with a fetch method.

  • s3.fetch(key, options) signs and sends one request.
  • Supports GET, PUT, HEAD, and DELETE.
  • Accepts custom headers and query parameters for list requests.
  • Works with path-style and virtual-hosted URLs.
/**
 * @fileoverview
 * Minimal AWS Signature Version 4 client for S3-compatible storage.
 *
 * Supports object GET, PUT, HEAD, and DELETE requests, custom headers, and
 * query parameters for list requests. Bodies must be strings or Uint8Arrays
 * and are hashed in memory, so keep requests small.
 *
 * Copy this file into your workbench and import it from a job.
 *
 * @see https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html
 */

const encoder = new TextEncoder();

/**
 * @typedef {Object} S3ClientConfig
 * @property {string|URL} endpoint - S3 endpoint origin, without a path.
 * @property {string} bucket - Bucket name.
 * @property {string} region - Signing region for the bucket.
 * @property {string} accessKeyId - Access key ID.
 * @property {string} secretAccessKey - Secret access key.
 * @property {boolean} [pathStyle=true] - Use path-style URLs.
 * @property {string} [sessionToken] - Token for temporary credentials.
 */

/**
 * @typedef {Object} S3RequestOptions
 * @property {string} [method="GET"] - HTTP method.
 * @property {string|Uint8Array} [body] - Request body.
 * @property {Object<string, string|number|boolean>} [headers] - Headers to sign and send.
 * @property {Object<string, string|number|boolean>} [query] - Query parameters.
 * @property {AbortSignal} [signal] - Cancels the request.
 */

/**
 * @typedef {Object} S3Client
 * @property {(key: string, options?: S3RequestOptions) => Promise<Response>} fetch
 *   Sign and send one S3 request.
 */

/**
 * Convert bytes to lowercase hex. SigV4 uses hex for hashes and signatures.
 *
 * @param {ArrayBuffer|Uint8Array} value
 * @returns {string}
 */
function toHex(value) {
	return [...new Uint8Array(value)]
		.map((byte) => byte.toString(16).padStart(2, "0"))
		.join("");
}

/**
 * Hash a string or byte array with SHA-256.
 *
 * Large inputs count against the sandbox's Web Crypto budget, which also
 * caps the size of a body this client can sign.
 *
 * @param {string|Uint8Array} value
 * @returns {Promise<string>} Lowercase hex digest
 */
async function sha256(value) {
	const bytes = typeof value === "string" ? encoder.encode(value) : value;
	return toHex(await crypto.subtle.digest("SHA-256", bytes));
}

/**
 * Compute HMAC-SHA256. The SigV4 signing key is built by chaining this
 * function over the date, region, service, and "aws4_request".
 *
 * @param {string|Uint8Array} key
 * @param {string} value
 * @returns {Promise<Uint8Array>}
 */
async function hmac(key, value) {
	const rawKey = typeof key === "string" ? encoder.encode(key) : key;
	const cryptoKey = await crypto.subtle.importKey(
		"raw",
		rawKey,
		{ name: "HMAC", hash: "SHA-256" },
		false,
		["sign"],
	);
	return new Uint8Array(
		await crypto.subtle.sign("HMAC", cryptoKey, encoder.encode(value)),
	);
}

/**
 * Encode one path or query component using SigV4's URI rules.
 *
 * encodeURIComponent leaves !'()* alone, and SigV4 requires those characters
 * to be percent-encoded too.
 *
 * @param {string} value
 * @returns {string}
 */
function encodeRfc3986(value) {
	return encodeURIComponent(value).replace(
		/[!'()*]/g,
		(char) => `%${char.charCodeAt(0).toString(16).toUpperCase()}`,
	);
}

// Encode each path segment, keeping the "/" separators intact.
function encodePath(key) {
	return key.split("/").map(encodeRfc3986).join("/");
}

/**
 * Build the canonical query string: percent-encode each name and value, then
 * sort by name and value. SigV4 signs this exact string.
 *
 * @param {Object<string, string|number|boolean>} query
 * @returns {string}
 */
function encodeQuery(query) {
	const pairs = [];
	for (const [name, value] of Object.entries(query)) {
		if (value === undefined || value === null) continue;
		pairs.push([encodeRfc3986(name), encodeRfc3986(String(value))]);
	}
	pairs.sort(([aName, aValue], [bName, bValue]) => {
		if (aName < bName) return -1;
		if (aName > bName) return 1;
		if (aValue < bValue) return -1;
		if (aValue > bValue) return 1;
		return 0;
	});
	return pairs.map(([name, value]) => `${name}=${value}`).join("&");
}

/**
 * Format a Date as YYYYMMDDTHHMMSSZ.
 *
 * @param {Date} date
 * @returns {string}
 */
function amzDate(date) {
	return date.toISOString().replace(/[:-]|\.\d{3}/g, "");
}

/**
 * Create a small S3 client that signs requests with AWS Signature Version 4.
 *
 * @param {S3ClientConfig} config
 * @returns {S3Client}
 * @throws {TypeError} If a required setting is missing or the endpoint includes a path.
 *
 * @example
 * const s3 = createS3Client({
 *   endpoint: "https://s3.example.com",
 *   bucket: "reports",
 *   region: "us-east-1",
 *   accessKeyId: "AKIA...",
 *   secretAccessKey: "...",
 * });
 *
 * const res = await s3.fetch("reports/latest.json", { method: "HEAD" });
 */
export function createS3Client({
	endpoint,
	bucket,
	region,
	accessKeyId,
	secretAccessKey,
	pathStyle = true,
	sessionToken,
}) {
	if (!endpoint) throw new TypeError("endpoint is required");
	if (!bucket) throw new TypeError("bucket is required");
	if (!region) throw new TypeError("region is required");
	if (!accessKeyId) throw new TypeError("accessKeyId is required");
	if (!secretAccessKey) throw new TypeError("secretAccessKey is required");

	const base = new URL(endpoint);
	if (base.pathname !== "/" && base.pathname !== "") {
		throw new TypeError("endpoint must not include a path");
	}

	return {
		/**
		 * Sign and send one S3 request.
		 *
		 * @param {string} key - Object key, or "" for a bucket-level request.
		 * @param {S3RequestOptions} [options]
		 * @returns {Promise<Response>} Raw response; check `ok` before reading the body.
		 * @throws {TypeError} If the body is not a string or Uint8Array.
		 */
		async fetch(key, options = {}) {
			const method = (options.method ?? "GET").toUpperCase();
			const body = options.body;

			if (
				body !== undefined &&
				typeof body !== "string" &&
				!(body instanceof Uint8Array)
			) {
				throw new TypeError("body must be a string or Uint8Array");
			}

			const query = encodeQuery(options.query ?? {});
			const search = query ? `?${query}` : "";
			const encodedKey = key ? encodePath(key) : "";

			// Path-style: https://endpoint/bucket/key
			// Virtual-hosted: https://bucket.endpoint/key
			const root = pathStyle
				? `${base.origin}/${encodeRfc3986(bucket)}`
				: `${base.protocol}//${bucket}.${base.host}`;
			const url = new URL(
				`${root}${encodedKey ? `/${encodedKey}` : ""}${search}`,
			);

			const timestamp = amzDate(new Date());
			const day = timestamp.slice(0, 8);

			// A GET or HEAD has no body, but it still signs the hash of "".
			const payloadHash = await sha256(body ?? "");

			// Host is part of the signature but is added by fetch, so keep it
			// out of the request headers. Authorization is added last and is
			// never signed.
			const signable = {};
			for (const [name, value] of Object.entries(options.headers ?? {})) {
				const lower = name.toLowerCase();
				if (lower === "host" || lower === "authorization") continue;
				if (value === undefined || value === null) continue;
				signable[lower] = String(value).trim().replace(/\s+/g, " ");
			}
			signable.host = url.host;
			signable["x-amz-content-sha256"] = payloadHash;
			signable["x-amz-date"] = timestamp;
			if (sessionToken) {
				signable["x-amz-security-token"] = sessionToken;
			}

			const names = Object.keys(signable).sort();
			const canonicalHeaders = `${names
				.map((name) => `${name}:${signable[name]}`)
				.join("\n")}\n`;
			const signedHeaders = names.join(";");

			// Canonical request: method, path, query, headers, signed header
			// names, and payload hash. canonicalHeaders already ends with a
			// newline, so the join below leaves the required blank line.
			const canonicalRequest = [
				method,
				url.pathname,
				query,
				canonicalHeaders,
				signedHeaders,
				payloadHash,
			].join("\n");

			const scope = `${day}/${region}/s3/aws4_request`;
			const stringToSign = [
				"AWS4-HMAC-SHA256",
				timestamp,
				scope,
				await sha256(canonicalRequest),
			].join("\n");

			// Derive the signing key: secret -> date -> region -> service ->
			// "aws4_request". Only the final signature is sent.
			let signingKey = await hmac(`AWS4${secretAccessKey}`, day);
			signingKey = await hmac(signingKey, region);
			signingKey = await hmac(signingKey, "s3");
			signingKey = await hmac(signingKey, "aws4_request");
			const signature = toHex(await hmac(signingKey, stringToSign));

			const headers = {};
			for (const name of names) {
				if (name !== "host") {
					headers[name] = signable[name];
				}
			}
			headers.authorization =
				`AWS4-HMAC-SHA256 Credential=${accessKeyId}/${scope}, ` +
				`SignedHeaders=${signedHeaders}, Signature=${signature}`;

			const init = { method, headers };
			if (body !== undefined) {
				init.body = body;
			}
			if (options.signal !== undefined) {
				init.signal = options.signal;
			}
			return fetch(url.toString(), init);
		},
	};
}

Using the Helper#

Save this as /upload-to-s3.js. It reads the settings from secrets, creates a client, uploads a JSON object, and reads it back.

// upload-to-s3.js
import env from "env";
import { createS3Client } from "./lib/s3.js";

const sessionToken = env.names().includes("S3_SESSION_TOKEN")
	? env.get("S3_SESSION_TOKEN")
	: undefined;

const s3 = createS3Client({
	endpoint: env.get("S3_ENDPOINT"),
	bucket: env.get("S3_BUCKET"),
	region: env.get("S3_REGION"),
	accessKeyId: env.get("S3_ACCESS_KEY_ID"),
	secretAccessKey: env.get("S3_SECRET_ACCESS_KEY"),
	pathStyle: env.get("S3_PATH_STYLE") !== "false",
	sessionToken,
});

const report = JSON.stringify({
	generatedAt: new Date().toISOString(),
	source: "geartrain",
});

const upload = await s3.fetch("reports/latest.json", {
	method: "PUT",
	body: report,
	headers: { "content-type": "application/json" },
});

if (!upload.ok) {
	throw new Error(`Upload failed: ${upload.status} ${await upload.text()}`);
}

const download = await s3.fetch("reports/latest.json");

if (!download.ok) {
	throw new Error(`Download failed: ${download.status} ${await download.text()}`);
}

const stored = await download.json();
console.log(JSON.stringify(stored, null, 2));

Other operations use the same client:

// Check an object without downloading it.
const head = await s3.fetch("reports/latest.json", { method: "HEAD" });

// List objects under a prefix.
const list = await s3.fetch("", {
	query: { "list-type": "2", prefix: "reports/" },
});

// Delete an object.
await s3.fetch("reports/latest.json", { method: "DELETE" });

How It Works#

  • SigV4 signs the method, path, query string, signed headers, and a SHA-256 hash of the body. The service recomputes the same values and compares the signature.
  • The helper adds x-amz-content-sha256 and x-amz-date, signs every header you pass, and derives host from the URL. fetch adds the Host header itself.
  • The signing key is an HMAC chain over the secret, date, region, s3, and aws4_request. Only the derived signature is sent.
  • pathStyle controls the URL shape. Path-style uses /<bucket>/<key>; virtual-hosted uses <bucket>.<endpoint>/<key>.
  • Query names and values are percent-encoded and sorted before signing, which is required for list requests.
  • The helper returns a normal Response. Check res.ok before reading the body, as with fetch.

Limits#

  • This is a small client for basic object operations, not a full S3 SDK.
  • Requests are single-part. Multipart uploads, streaming bodies, automatic retries, and redirects are not implemented.
  • Bodies are hashed in memory, so keep objects small. Hashing and signing also count against the run’s Web Crypto budget.
  • The plan’s memory, compute, and run limits still apply. See Limits.
  • Third-party modules are not available yet, but support is planned. That would allow a full-featured S3 client to run in your scripts. Until then, call the service’s HTTP API directly, as this example does. See The Sandbox.

Setup#

  1. Create a bucket and an access key with PutObject and GetObject permissions. Add ListBucket and DeleteObject if you use the list or delete snippets. Scope the key to the bucket.

  2. Add the secrets:

    Secret Value
    S3_ENDPOINT Your provider’s S3 endpoint, without a path
    S3_BUCKET The bucket name
    S3_REGION Your provider’s region
    S3_PATH_STYLE true for path-style URLs, false for virtual-hosted
    S3_ACCESS_KEY_ID The access key ID
    S3_SECRET_ACCESS_KEY The secret access key
    S3_SESSION_TOKEN Optional, for temporary credentials
  3. Save the helper at /lib/s3.js and the script at /upload-to-s3.js.

  4. Open Jobs, click Create Job, select upload-to-s3.js as the script, name the job, and choose a schedule. Click Create Job.

  5. Click Run now on the job, then check reports/latest.json in the bucket.

A script does not run on its own. The job points at it and supplies the schedule; Run now starts that job immediately.

For virtual-hosted URLs, use a bucket name without dots so the TLS certificate matches the host.