a manifold extension to port fully onchain an existing collection minted with offchain content
clgsms/README.md
Markdown · 484 lines · 23 KBREADSOURCERAWDOWNLOAD
# COLORGASM — a fully onchain edition, built as a Manifold extension
COLORGASM (`CLGSM`) is a 100-piece edition minted on a Manifold creator
contract, with its metadata and images hosted on Arweave. This repository is
the machinery that moves the artworks themselves into Ethereum contract code,
and lets each holder swap their Arweave-era token for a fully onchain one.
It is three small Solidity contracts, a set of Node scripts that drive them,
and the Python encoder that turns a 60 MB PNG into an AVIF small enough to live
onchain and sharp enough to still be the artwork.
The interesting part — and the reason this README exists — is that **the
creator contract could not be changed**. Everything here is built inside the
one extension point Manifold gives you.
---
## Deployed on Ethereum mainnet
| contract | address | role |
|---|---|---|
| **COLORGASM (`CLGSM`)** creator | [`0x7dC2a81Ba2Bf47548EC2F9645007119dF90ba01b`](https://etherscan.io/address/0x7dC2a81Ba2Bf47548EC2F9645007119dF90ba01b) | the Manifold ERC-721 creator contract — unchanged, not deployed from this repo |
| **ColorgasmRedeemExtension** | [`0xCD5ef63FA405786B2ecb498F83687f6a566CF374`](https://etherscan.io/address/0xCD5ef63FA405786B2ecb498F83687f6a566CF374) | the extension: burn/redeem, artwork pointers, `tokenURI` |
| **ColorgasmStorage** | [`0xE43f8FAC380616bf15968be038eF00Bc01514c00`](https://etherscan.io/address/0xE43f8FAC380616bf15968be038eF00Bc01514c00) | the artwork bytes, as SSTORE2 chunks |
| **ColorgasmRendererV3** | [`0x57a2e9e9a327483A18a37E29F6EF36b96C6Fb5b6`](https://etherscan.io/address/0x57a2e9e9a327483A18a37E29F6EF36b96C6Fb5b6) | assembles the token document |
Deployed at block 25,963,689, source verified on Etherscan. Live configuration:
artwork stored **raw** (the renderer base64s it at read time), document served
as `data:application/json;utf8,`. The same three addresses are recorded in
[`deployments/mainnet.json`](deployments/mainnet.json), which every script in
this repo reads instead of taking addresses on the command line.
---
## 1. The constraint: a contract you cannot change
`CLGSM` is a Manifold **creator contract** — a proxy onto Manifold's ERC-721
Creator Core. The original 100 tokens were minted through Manifold's edition
extension, and their `tokenURI` is an Arweave prefix plus the token index. That
contract is immutable: there is no admin function that rewrites those URIs, no
upgrade path, no way to reach inside and replace what token #57 points at.
So a fully onchain COLORGASM cannot be a migration of the old tokens. It has to
be a **new token that the holder chooses to mint, by destroying the old one** —
and that new token has to get its `tokenURI` from somewhere the creator
contract is willing to ask.
That "somewhere" is an extension.
---
## 2. What a Manifold extension actually is
This is the core mechanism, and it is simpler than it sounds. Manifold's
Creator Core has a registry of **extension contracts**. An extension is an
ordinary contract you deploy yourself. Once the creator's owner or admin calls
```solidity
creator.registerExtension(address(extension), "");
```
three things become true:
**1. The extension can mint.** `creator.mintExtension(to)` mints a new token on
the creator contract and records that this extension minted it
(`creator.tokenExtension(tokenId)`). The token is a real `CLGSM` token: same
contract, same collection, same royalties, same everything a marketplace sees.
**2. The creator asks the extension for the token's metadata.** On every
`tokenURI(tokenId)` call, Creator Core looks up which extension minted that
token and, **via ERC-165**, asks whether that extension implements
`ICreatorExtensionTokenURI`:
```solidity
interface ICreatorExtensionTokenURI {
function tokenURI(address creator, uint256 tokenId) external view returns (string memory);
}
// interfaceId == bytes4(keccak256("tokenURI(address,uint256)"))
```
If it does, the creator **delegates** — it calls back into the extension and
returns whatever string comes out. That single delegation is the whole hinge of
this project: it is the only way to make an immutable creator contract serve a
`tokenURI` that is computed from bytes stored in code, and it applies only to
tokens the extension itself minted. The old tokens are untouched.
**3. The extension can burn, with permission.** `creator.burn(tokenId)` works
for an owner-or-approved caller, so the extension can destroy an original — but
only once the holder has approved it. Nothing here can take a token.
What an extension **cannot** do is equally important: it cannot change the
`tokenURI` of tokens it did not mint, it cannot alter the creator's royalties or
name, and it cannot register itself. Registration is a deliberate act by the
creator's admin wallet.
### The call graph
```
HOLDER MARKETPLACE / anyone
│ │
│ 1. approve(extension, id) │ tokenURI(newId)
│ 2. burnRedeem(id) ▼
▼ ┌──────────────────────┐
┌─────────────────────────┐ burn(id) │ CLGSM creator core │
│ ColorgasmRedeemExtension │ ───────────────────────► │ (Manifold, immutable)│
│ │ mintExtension(holder) └──────────┬───────────┘
│ artworkKeyOf[id] ──────┼──┐ │ ERC-165 check,
│ metadataOf[id] │ │ tokenURI(creator, newId)
│ originalOf[newId] │ │ │ delegated back
└──────────┬──────────────┘ │ ◄──────────┘
│ renderer │ bytes32 key
▼ ▼
┌────────────────────────┐ ┌─────────────────────────┐
│ ColorgasmRendererV3 │─►│ ColorgasmStorage │
│ builds the document │ │ SSTORE2 chunks per key │
└────────────────────────┘ └─────────────────────────┘
```
---
## 3. Three contracts, one job each
The split is not decoration. Each contract exists because something about it
has to be able to change independently of the others — or to never change at
all.
### `ColorgasmRedeemExtension` — authority and bookkeeping
The Manifold-facing contract, and the only one the creator contract knows
about. It holds:
- `creator`, `firstOriginalId`, `editionSize` — immutable, set at deployment
(`CLGSM`, 1, 100).
- `artworkKeyOf[originalTokenId]` → a `bytes32` storage key. **Not the image** —
just where it lives.
- `metadataOf[originalTokenId]` → the token's original JSON minus the image
fields, stored via SSTORE2. Each redeemed token keeps exactly the name,
description and attributes it was minted with.
- `originalOf[newTokenId]` / `redeemedTokenOf[originalTokenId]` — the mapping
between the burned original and its onchain replacement, in both directions.
- `renderer` and `blobStorage` — swappable addresses, until they are locked.
Its two public entry points are `burnRedeem(originalTokenId)` and
`burnRedeemBatch(uint256[])`. Both check `isRedeemable`, check `ownerOf`, burn,
mint, and record. One transaction, ~130k gas, no data arguments — the holder
never sends the artwork, because the artwork is already onchain.
`isRedeemable(id)` is true only when all five conditions hold: an artwork
pointer is set, a metadata fragment is set, the token is not already redeemed,
a renderer is configured, and a storage contract is configured. Surface it in
any frontend rather than letting holders guess.
### `ColorgasmStorage` — the bytes
A deliberately generic, permissionless blob store. Bytes are written under a
caller-chosen `bytes32` key, one chunk per transaction, as **SSTORE2** — the
data is deployed as contract code, which is the cheapest persistent storage the
EVM offers: 200 gas per byte, against 625 for a regular storage slot.
```solidity
append(bytes32 key, bytes calldata chunk) returns (uint256 index)
seal(bytes32 key) // irreversible
chunksOf(bytes32 key) returns (address[]) // what a renderer reads
```
Two design points worth understanding:
- **The first address to write a key owns it.** Nobody can overwrite someone
else's blob, and anyone can use the contract for their own work. There is no
admin.
- **`MAX_CHUNK` is 24,573 bytes** — SSTORE2's 24,575-byte limit rounded down to
a multiple of 3 (3 × 8191). The multiple of 3 is what lets base64 be done
chunk by chunk at render time: 3 bytes become exactly 4 characters, so
3-aligned chunks encode independently and their encodings concatenate with no
padding in the middle.
`seal(key)` is the promise a collector actually cares about: after it, those
bytes can never change, by anyone, ever.
### `ColorgasmRendererV3` — presentation, and the only swappable part
Reads the pointer and metadata fragment from the extension, the chunk pointers
from storage, and builds
```
data:application/json;utf8,{"name":"COLORGASM #057",…,"image":"data:image/avif;base64,…"}
```
in **one exactly-sized buffer**, allocated once, every byte written into its
final place and never moved again. Earlier versions built the image, then its
base64, then the JSON, then sometimes a base64 of that — four full-size buffers,
and the EVM prices memory with the square of how far it stretches. At these
image sizes that difference is the difference between a token that renders and
one that does not.
Three choices it encodes:
- **Raw vs pre-encoded.** `preEncoded` is immutable per deployment. Raw stores
the AVIF bytes and base64s them at read time (cheapest to store). Pre-encoded
stores the base64 characters and renders by straight `extcodecopy` (33% more
to store, 15–20× cheaper to serve). Both produce byte-identical documents.
Mainnet runs raw.
- **UTF-8 vs base64 JSON.** Serving the JSON as UTF-8 rather than base64-ing it
again removes the last full-size copy — at 64 KB that alone is 280k gas
against 5.15M. Switchable with `setBase64Json`.
- **Percent-encoding.** Every COLORGASM name contains a `#`, which starts a URI
fragment and would truncate the document at the name. The metadata fragment
is escaped; base64 output is URI-safe throughout.
The renderer is token-agnostic — it holds no per-token state — so replacing it
is a single `setRenderer()` call, until `lockRenderer()`.
---
## 4. Two numbers that shaped everything
**A transaction cannot use more than 16,777,216 gas.** EIP-7825 caps it,
whatever the block gas limit is. A maximum-size SSTORE2 chunk costs ~5.53M, so
one chunk per transaction is the comfortable shape — and the reason EthFS and
ScriptyStorageV2 work the same way. This is why the artwork never passes
through the extension: uploading is 41 ordinary transactions for the largest
piece, and pointing a token at it afterwards is one ~140k-gas call, however
large the artwork.
**An `eth_call` gets roughly 50,000,000 gas** on a standard node. That is the
budget a marketplace has when it asks for `tokenURI`. Past it the call errors
and the token shows as broken. Measured with `npm run ceiling`, on synthetic
images at the sizes of the real encodes:
| image | store pre-encoded | store raw | **render pre-encoded** | render raw |
|---|---|---|---|---|
| 116 KB | 35.0M | 26.2M | **0.4M** | 7.0M |
| 250 KB | 75.3M | 56.5M | **1.2M** | 15.4M |
| 400 KB | 120.5M | 90.3M | **2.7M** | 25.6M |
| 600 KB | 180.6M | — | **5.8M** | — |
| 964 KB | 290.1M | — | **14.2M** | — |
Every COLORGASM encode renders inside that budget — including the 964 KB one,
and not merely within an `eth_call` but inside a single transaction.
---
## 5. The life of one artwork, end to end
```
CLGSM-57.png 60 MB, 4320 × 7680
│ encode/encode_onchain_v2.py AVIF 4:4:4, constant quality, 1 MB cap
▼
COLORGASM_#57.avif ~600 KB
│ scripts/upload.js 25 chunks of 24,573 B, one per tx
▼
ColorgasmStorage key = bytes32("COLORGASM_57"), then seal()
│ scripts/point.js setArtwork(57, key, metadata)
▼
ColorgasmRedeemExtension isRedeemable(57) == true
│ holder: approve + burnRedeem(57) original burned, new token minted
▼
tokenURI(newId) creator delegates → extension → renderer
▼
data:application/json;utf8,{…,"image":"data:image/avif;base64,…"}
```
The storage key is readable, not a hash: the ASCII of `COLORGASM_<id>`,
right-padded into a `bytes32`. Anyone can recompute it and read the bytes back
themselves — see [§9](#9-verifying-it-yourself).
Nothing about this order is global. Every token is independent: its own key,
its own pointer, its own moment of becoming redeemable. The artist can do one
piece completely, stop for a month, and do another. Holders can redeem finished
pieces while the rest are still being uploaded.
---
## 6. Getting a 60 MB PNG under a megabyte
`encode/encode_onchain_v2.py` turns the source PNGs into the AVIF files that go
onchain. It is Pillow and NumPy only — no shell-out to `avifenc`, no build
dependency.
The policy is **constant quality, capped**: every piece is encoded at one
quantiser (`--q`, default 40) so the collection sits on a single visual tier,
and fits *down* only when a heavy piece would exceed `--budget` (default
1,000,000 bytes). Light pieces land around 0.4–0.6 MB; only the heavy ones pay
the cap. Bytes are spent where the content needs them, not spread evenly.
Getting there cheaply is a rate model: `log(bytes)` as a low-order polynomial
in the quantiser index, fitted on fast speed-7 probes, then shifted to pass
through the real speed-4 encode. That turns what was seven full encodes per
image into one, typically with one or two probes. Metrics (PSNR, edge-energy
retention, gradient PSNR) are streamed in row bands, so a 33 Mpx image does not
need 2.4 GB of float64 copies.
Every codec decision in the file header was measured at matched byte size, and
the ones that lost are listed there too — 4:2:0 (+7% bytes), 10-bit (+2%),
`tune=iq` (+12%), rav1e (+14% and 4× slower), closed-loop residual
compensation (+115%). The one thing that wins for this material is a light
pre-sharpen (unsharp radius 0.6, 50%): +2.5 points of edge energy for 1.7% more
bytes. Edge sharpness is the whole point of these images.
The actual collection, from [`encode/report.csv`](encode/report.csv):
| | |
|---|---|
| encodes | 99 (piece #66 not yet encoded) |
| source | 4320 × 7680 each |
| total | 59.2 MiB |
| per file | 114 KB min, 613 KB mean, 963 KB max |
| at the 1 MB cap | 18 of 99 (encoded below q40) |
| mean PSNR | 36.4 dB |
| raw chunks | 2,577 — i.e. 2,577 upload transactions |
`encode/convert.sh` is the small companion that produces the pre-encoded
(base64) form, with a byte-for-byte round-trip check.
`encode/MANIFEST.sha256` is the checksum of every encode as it was uploaded.
---
## 7. What it costs
Measured all-in, with calldata and transaction overhead: **~226 gas per byte**
of image stored raw, **~301 gas per byte** stored pre-encoded.
| AVIF size | chunks | pre-encoded | @1 gwei | raw | @1 gwei |
|---|---|---|---|---|---|
| 116 KB | 7 | 34.4M | 0.034 ETH | 26.2M | 0.026 ETH |
| 250 KB | 14 | 75.3M | 0.075 ETH | 56.5M | 0.057 ETH |
| 400 KB | 23 | 120.5M | 0.121 ETH | 90.3M | 0.090 ETH |
| 964 KB | 41 | 288.6M | 0.289 ETH | 218M | 0.218 ETH |
For the collection as encoded (99 files, 59.2 MiB):
| | gas | @0.5 gwei | @1 gwei | @3 gwei |
|---|---|---|---|---|
| pre-encoded | 18.26B | 9.13 ETH | **18.26 ETH** | 54.77 ETH |
| raw | 13.70B | 6.85 ETH | **13.70 ETH** | 41.09 ETH |
Pointing all 99 tokens afterwards is ~142k gas each — around 0.014 ETH for the
lot, a rounding error beside the uploads. Redeeming is paid by the holder:
~130k gas, one transaction.
`npm run estimate` prices *your* files rather than these round numbers, by
deploying a real `ColorgasmStorage` on a throwaway local chain and measuring
each one. Run it before committing to anything — sealing is permanent.
---
## 8. Running it
```bash
npm ci # not `npm install` — see below
cp .env.example .env # add ALCHEMY_KEY (or INFURA_KEY) and PRIVATE_KEY
npm run build
npm test # 38 tests, no RPC, no keys
```
Use `npm ci` and keep `package-lock.json`. Every dependency is pinned to an
exact version, because gas figures and EDR behaviour move between patch
releases and the numbers above were measured against exactly these versions.
`npm test` stands up a real Manifold `ERC721Creator` in-process from
`fixtures/ERC721Creator.json`, so the extension is exercised against the genuine
creator core rather than a mock: wiring, storage, pointers, redeem, batch
redeem, all three locks, renderer swapping, and a byte-exact round trip of a
multi-chunk image through SSTORE2 → base64 → `tokenURI`. `npm run test:fork`
repeats the flow against the real `CLGSM` contract on a mainnet fork, with the
real admin and a real current holder.
The fixture is frozen bytecode rather than a source dependency on purpose:
compiling creator-core from npm drags in OpenZeppelin, and the combination is
version-fragile. Dropping it took the install from 873 packages to 128.
### The scripts
All of them read addresses from `deployments/<network>.json` (override with
`CREATOR=` / `STORAGE=` / `EXTENSION=`) and a token range from `IDS=57,3,12` or
`FROM_ID`/`TO_ID`.
| script | what it does |
|---|---|
| `deploy.js` | storage + extension + renderer, wired and recorded |
| `verify.js` | Etherscan source verification for all three |
| `register-extension.js` | `registerExtension` on the creator, from its admin wallet |
| `fetch-metadata.js` | builds the per-token metadata fragments from the original Arweave JSON |
| `upload.js` | **phase 1a** — one chunk per tx, byte-verified, sealed, resumable |
| `point.js` | **phase 1b** — `setArtwork(id, key, metadata)`, one small tx each |
| `redeem.js` | **phase 2** — approve + `burnRedeem` / `burnRedeemBatch` |
| `inspect.js` | audits keys, seals, bytes and the live `tokenURI` against local files |
| `estimate.js` | prices phase 1 per file from a folder of encodes |
| `render-ceiling.js` | measures store and render cost across image sizes |
| `deploy-creator.js` | a Manifold `ERC721Creator` on a testnet (refuses mainnet) |
| `seed-originals.js` | mints Arweave-era stand-ins on a testnet (refuses mainnet) |
Two properties worth relying on:
- **`upload.js` is resumable and verifies bytes, not sizes.** A key whose stored
bytes already match the file is skipped; one that stopped part-way continues
from the chunk it reached. Every chunk is read back with `eth_getCode` and
compared against the file, so a re-encode that happens to land on the same
byte count is never silently accepted. It also waits rather than sending above
`MAX_GWEI`, so it is safe to leave running for days.
- **The measuring scripts refuse to run on a live chain.** `estimate.js` and
`render-ceiling.js` measure by really deploying and really sending, so they
check for chainId 31337 first and stop. A live network is only ever read from,
for its current gas price.
[`RUNBOOK.md`](RUNBOOK.md) is the full procedure — local, Sepolia rehearsal,
mainnet, then freezing — including what to do when something goes wrong.
Note that `AVIF_DIR` defaults to `../onchain` (the layout of the artist's
working folder, where the encoder writes). Point it wherever your encodes live.
---
## 9. Verifying it yourself
None of this asks you to trust the deployment record. For any token:
1. The storage key is the ASCII of `COLORGASM_<id>`, right-padded into a
`bytes32`.
2. `ColorgasmStorage.chunksOf(key)` returns the SSTORE2 pointer addresses in
order, and `isSealed(key)` tells you whether they can ever change.
3. `eth_getCode` on each pointer returns a leading `STOP` byte followed by the
raw data. Strip the first byte, concatenate in order — that is the AVIF file.
4. Its SHA-256 should match the line for that piece in
[`encode/MANIFEST.sha256`](encode/MANIFEST.sha256).
`scripts/inspect.js` does exactly this, and with `DUMP_DIR=dump` writes the
images it decoded out of the live `tokenURI` to disk so you can look at them.
---
## 10. What is permanent
In increasing order of finality:
| action | effect | reversible? |
|---|---|---|
| `storage.append` | writes bytes | no, but harmless — an orphaned blob costs nothing |
| `extension.setArtwork` | points a token at a key | yes, until `lockStorage` |
| `extension.setRenderer` | changes presentation | yes, until `lockRenderer` |
| `renderer.setBase64Json` | JSON wrapper | yes, always (swap the renderer) |
| `storage.seal(key)` | freezes a key's bytes | **never** |
| `extension.lockStorage()` | freezes every pointer, fragment and the storage address | **never** |
| `extension.lockRenderer()` | freezes presentation | **never** |
| `burnRedeem` | burns an original | **never** — and it is the holder's call, not the artist's |
`lockStorage()` is the guarantee that matters to a collector: the art can no
longer change. `lockRenderer()` is deliberately left until much later — it is
the only thing that allows fixing presentation, changing the MIME type, or
moving to a cheaper renderer.
---
## Repository layout
```
contracts/
ColorgasmRedeemExtension.sol the Manifold extension
ColorgasmStorage.sol SSTORE2 blob storage
ColorgasmRendererV3.sol single-pass document assembly
interfaces/IColorgasm.sol Manifold + internal interfaces
scripts/ deploy, upload, point, redeem, inspect, measure
test/ local (28), render (10), mainnet fork (7)
fixtures/ERC721Creator.json Manifold creator core, frozen bytecode, tests only
deployments/ recorded addresses per network
encode/
encode_onchain_v2.py PNG → AVIF, constant quality, capped
convert.sh AVIF → base64 (pre-encoded storage shape)
report.csv per-file sizes and quality metrics
MANIFEST.sha256 checksums of the uploaded encodes
RUNBOOK.md the full operational procedure
```
`.env` is gitignored and contains only provider keys, a deployer key and an
Etherscan key; nothing in this repository needs it to be read.
---
## License
MIT. See [LICENSE](LICENSE). The contracts carry `SPDX-License-Identifier: MIT`.
The artworks are not licensed by this repository — the code is MIT, the images
are not.