a manifold extension to port fully onchain an existing collection minted with offchain content
clgsms/RUNBOOK.md
Markdown · 431 lines · 14 KBREADSOURCERAWDOWNLOAD
# COLORGASM onchain migration — runbook
The procedure, start to finish. Follow it in order; within PHASE 1 each token is
independent, so you can do one completely, stop, and pick up any other later in
any order.
Two things are permanent and cannot be undone by anyone, ever:
- `ColorgasmStorage.seal(key)` — those bytes can never change
- `extension.lockStorage()` — every pointer and metadata fragment freezes
Everything else is reversible while the extension is unlocked. Nothing below
seals or locks anything until §6, and `lockRenderer()` is deliberately last.
**Paths.** The scripts default to `../onchain` for the encoded artwork
(`AVIF_DIR`) and `meta/` for the metadata fragments (`META_DIR`) — the layout
of the artist's working folder, where `encode/encode_onchain_v2.py` writes.
Set `AVIF_DIR` in `.env` if yours live somewhere else.
**Configuration for this run:** artwork stored **RAW** (`PRE_ENCODED` unset),
document served as **UTF-8 JSON** (`UTF8_JSON` unset). Both are the defaults.
The renderer's `preEncoded` is immutable, so `upload.js` and `point.js` read it
off the deployed renderer and refuse to run on a mismatch — you cannot get this
wrong silently, but do not override it on one command and not another.
---
## 0. Before you spend anything
**0.1 Install exactly the pinned toolchain.**
```bash
rm -rf node_modules cache artifacts
npm ci
```
`npm ci`, not `npm install`, and do not delete `package-lock.json`. Every
dependency is pinned to an exact version because gas figures move between patch
releases.
**0.2 Tests green.**
```bash
npm run build
npm test
```
Expect **38 passing**. If you see fewer, or a suite that finishes suspiciously
fast, you are running against stale artifacts — `rm -rf cache artifacts` and
rebuild before going further.
**0.3 Fill in `.env`.**
```bash
cp .env.example .env
```
- `ALCHEMY_KEY` (or `INFURA_KEY`) — covers mainnet, the fork and Sepolia
- `PRIVATE_KEY` — use a throwaway key for Sepolia; swap in the real one only at §4
- `ETHERSCAN_API_KEY` — one V2 key verifies on both chains
- `MAX_GWEI` — leave unset for now; §5 uses it
**0.4 Check the artwork is complete.**
```bash
ls ../onchain/*.avif | wc -l # expect 100
```
`report.csv` currently lists **99 files — #66 has no encode.** Produce it before
starting, or decide deliberately that #66 stays on Arweave. `upload.js` skips a
missing file with a message rather than failing, so a gap will not stop a run —
it will just quietly leave that token unredeemable.
**0.5 Price it.**
```bash
B64_DIR=../onchain EXT=.avif npm run estimate
```
`--network mainnet` may be added to also print the cost at the live base fee.
The measuring itself always happens on a throwaway local chain whatever you
pass — `estimate.js` and `render-ceiling.js` measure by really deploying and
really sending, so they refuse outright to run against any chain that is not
31337.
Expect roughly **13.7 ETH at 1 gwei** for 99 files stored raw. Decide the gas
price you are willing to run at before you start, not while it is running.
**0.6 Fork test against the real contract.**
```bash
npm run test:fork
```
Then pin `FORK_BLOCK` in `.env` to the block it used, so later runs are
reproducible.
---
## 1. Sepolia — full dress rehearsal
Everything here is free. Do not skip it: this is where you find out whether the
whole chain of contracts behaves, and whether marketplaces accept the document.
**1.1 Stand up a creator contract.** COLORGASM only exists on mainnet.
```bash
npx hardhat run scripts/deploy-creator.js --network sepolia
```
**1.2 Mint stand-in originals.**
```bash
FROM_ID=1 TO_ID=5 npx hardhat run scripts/seed-originals.js --network sepolia
```
**1.3 Deploy storage + extension + renderer.**
```bash
npx hardhat run scripts/deploy.js --network sepolia
```
Check the header it prints says `artwork stored raw, served as UTF-8 JSON`.
Addresses land in `deployments/sepolia.json`.
**1.4 Verify, then register the extension.**
```bash
npx hardhat run scripts/verify.js --network sepolia
npx hardhat run scripts/register-extension.js --network sepolia
```
Verify on Sepolia too — it is the rehearsal for §2.2, and it is where you find
out if your `ETHERSCAN_API_KEY` works before it matters.
The Manifold `ERC721Creator` that `deploy-creator.js` puts on Sepolia is
deployed from frozen bytecode, not from source in this project, so
`scripts/verify.js` does not cover it and `hardhat verify` cannot. It is a test
scaffold that exists only on Sepolia — nothing on mainnet depends on it. If you
want it verified anyway, use Etherscan's "similar match" against any of
Manifold's own deployed creator contracts.
**1.5 Metadata fragments.**
```bash
npm run metadata -- meta 1 5
```
Open one and read it. It should be the original JSON minus the image fields,
with no surrounding braces.
**1.6 One token, end to end.**
```bash
IDS=1 npx hardhat run scripts/upload.js --network sepolia
IDS=1 npx hardhat run scripts/point.js --network sepolia
IDS=1 npx hardhat run scripts/inspect.js --network sepolia
IDS=1 npx hardhat run scripts/redeem.js --network sepolia
```
`inspect.js` must say `exact` for both image and metadata. Then open the new
token on [testnets.opensea.io](https://testnets.opensea.io) and **look at it**.
**1.7 Then the biggest one you have.** This is the real test — a 964 KB piece is
where document size, `eth_call` budget and indexer tolerance all get exercised
at once.
```bash
IDS=94 npx hardhat run scripts/upload.js --network sepolia
IDS=94 npx hardhat run scripts/point.js --network sepolia
IDS=94 npx hardhat run scripts/redeem.js --network sepolia
```
Check it renders on OpenSea, and in a plain browser tab by pasting the
`tokenURI` output. If a marketplace chokes on `data:application/json;utf8,`,
this is the moment to find out — deploy a renderer with `UTF8_JSON=0` and
`setRenderer` to it, which costs one transaction and nothing else.
---
## 2. Mainnet — deploy the contracts
Switch `PRIVATE_KEY` in `.env` to the real artist wallet now.
**2.1 Deploy.**
```bash
npx hardhat run scripts/deploy.js --network mainnet
```
Confirm the header again: `artwork stored raw, served as UTF-8 JSON`, and
`creator 0x7dC2a81Ba2Bf47548EC2F9645007119dF90ba01b`.
**2.2 Verify on Etherscan.** Wait a minute for Etherscan to index the
deployment, then:
```bash
npx hardhat run scripts/verify.js --network mainnet
```
It reads `deployments/mainnet.json` and verifies all three with the constructor
arguments they were actually deployed with — including the renderer's
`preEncoded`, which must match or the bytecode will not. Re-running is safe;
contracts Etherscan has already verified are reported and skipped.
Expect:
```
✓ ColorgasmStorage 0x…
✓ ColorgasmRedeemExtension 0x…
✓ ColorgasmRendererV3 0x…
all 3 contracts verified on mainnet
```
The artwork chunks are SSTORE2 pointers — a STOP byte followed by raw data,
with no source to verify. They are checked by reading them back instead, which
is what `inspect.js` does and what anyone else can do independently.
**2.3 Register the extension on CLGSM.** This must come from the wallet that is
owner or admin of the creator contract — your Manifold admin wallet, which may
not be the deploy wallet.
```bash
npx hardhat run scripts/register-extension.js --network mainnet
```
The script checks `owner()` and `isAdmin()` first and tells you who it expected
if the signer is wrong. If your admin wallet is a hardware wallet you would
rather not put in `.env`, call `registerExtension(<extension>, "")` from
Etherscan's write tab instead — it is the same single call.
**2.4 Metadata fragments for the whole edition.**
```bash
npm run metadata
```
---
## 3. Mainnet — one token, all the way
Pick a token you hold. Do this one completely and stop.
```bash
IDS=57 npx hardhat run scripts/upload.js --network mainnet
IDS=57 npx hardhat run scripts/point.js --network mainnet
IDS=57 npx hardhat run scripts/inspect.js --network mainnet
```
`upload.js` will:
- read `../onchain/COLORGASM_#57.avif`
- split it into 24573-byte chunks (a multiple of 3, so the renderer can base64
it chunk by chunk)
- send one `append` per chunk, waiting between them if gas is above `MAX_GWEI`
- read every chunk back with `eth_getCode` and compare against the file
- **seal the key** — permanent
Then redeem it and look at it:
```bash
IDS=57 npx hardhat run scripts/redeem.js --network mainnet
```
Open it on OpenSea. Refresh metadata if needed. **Stop here until you are happy
with what you see.** Everything so far is one artwork's worth of gas.
---
## 4. Mainnet — the rest, at your own pace
Set a gas ceiling you are content to leave running at:
```bash
export MAX_GWEI=2
```
Then work through the edition however you like. One at a time:
```bash
IDS=3 npx hardhat run scripts/upload.js --network mainnet
IDS=3 npx hardhat run scripts/point.js --network mainnet
```
A handful, in any order:
```bash
IDS=12,4,88,7 npx hardhat run scripts/upload.js --network mainnet
IDS=12,4,88,7 npx hardhat run scripts/point.js --network mainnet
```
Or leave it running over a range:
```bash
FROM_ID=1 TO_ID=100 npx hardhat run scripts/upload.js --network mainnet
FROM_ID=1 TO_ID=100 npx hardhat run scripts/point.js --network mainnet
```
Order never matters. Tokens are independent: each has its own storage key, its
own pointer, and becomes redeemable on its own the moment both halves are set.
Holders can redeem the finished ones while you are still uploading the rest.
**Re-running is always safe** while the extension is unlocked:
- an artwork whose stored bytes already match the file is skipped
- an upload that stopped part-way continues from the chunk it reached
- a token already pointed at the right key with the right metadata is skipped
So if a run dies — laptop sleeps, RPC drops, gas spikes — just run the same
command again.
**Check progress at any time:**
```bash
FROM_ID=1 TO_ID=100 npx hardhat run scripts/inspect.js --network mainnet
```
Every row should read `exact` / `exact` / `sealed yes`. Anything else is listed
as a problem at the bottom and the script exits non-zero.
---
## 5. Announce
Once enough tokens are redeemable, tell holders. They need to know:
- redeeming **burns** the original and mints a new tokenId — the old one is gone
- they must `setApprovalForAll(<extension>, true)` on CLGSM first
- `isRedeemable(originalTokenId)` on the extension says whether their piece is
ready; surface this in your frontend rather than letting them guess
- unredeemed tokens keep their Arweave URI indefinitely — there is no deadline
and nothing breaks if they never redeem
Your own tokens can go through `scripts/redeem.js` with `BATCH=1` (the default)
to do several per transaction. Keep batches under ~50 tokens so the transaction
stays well inside the 16,777,216 gas cap.
---
## 6. Freeze it
Only when every artwork is verified onchain and you have looked at a good number
of them rendered.
**6.1 Final audit, with the images decoded out of the live contract:**
```bash
FROM_ID=1 TO_ID=100 DUMP_DIR=dump npx hardhat run scripts/inspect.js --network mainnet
open dump/ # these came out of tokenURI, not off your disk
```
Compare a few against `../onchain/` by eye and by checksum:
```bash
for f in dump/*.avif; do
id=$(basename "$f" .avif)
cmp -s "$f" "../onchain/COLORGASM_#$id.avif" && echo "OK $id" || echo "MISMATCH $id"
done
```
**6.2 Keys are already sealed** by `upload.js`. Confirm no row says `NO`:
```bash
FROM_ID=1 TO_ID=100 npx hardhat run scripts/inspect.js --network mainnet | grep -c " NO "
```
**6.3 Lock the pointers.** Irreversible.
```bash
npx hardhat console --network mainnet
> const ext = await ethers.getContractAt("ColorgasmRedeemExtension", "<extension>")
> await (await ext.lockStorage()).wait()
```
**6.4 Leave `lockRenderer()` for much later** — months, not days. It is the only
thing that lets you fix presentation, change the MIME type, or move to a cheaper
renderer. Locking storage already gives collectors the guarantee that matters:
the art cannot change.
---
## When things go wrong
**An upload stopped part-way.** Run the same command again. It reads what is
onchain, confirms it matches the start of your file, and continues from there.
**Stored bytes do not match the file.** The script refuses and tells you. If the
key is unsealed, it will not append onto mismatched data — change `KEY_PREFIX`
and upload under a fresh key, then re-point the token.
**A key was sealed with the wrong artwork.** The bytes are permanent, but the
*pointer* is not, while the extension is unlocked. Upload the right artwork
under a different key (`KEY_PREFIX=CLGSM2_`) and re-run `point.js` for that
token. The wrong blob stays onchain, orphaned and harmless.
**Wrong metadata.** `setArtwork(id, bytes32(0), <new metadata>)` replaces the
fragment and keeps the pointer — that is what `point.js` does automatically when
only the metadata differs.
**A transaction is stuck.** Wait, or replace it with the same nonce at a higher
fee from your wallet. Nothing here depends on a particular ordering, so a
dropped transaction is never worse than repeating the step.
**Gas spikes mid-run.** `upload.js` polls and waits rather than sending above
`MAX_GWEI`. Leave it running, or Ctrl-C and resume later — same command.
**A shape mismatch error.** You set `PRE_ENCODED` on one command and not
another. The deployed renderer decides; match it or deploy a renderer for the
other shape. Never work around the check — it is the difference between a token
that renders and one that decodes to nothing.
---
## The irreversible list
| action | effect | reversible? |
|---|---|---|
| `blobs.append` | writes bytes | no, but harmless — orphaned blobs cost nothing |
| `blobs.seal` | freezes a key's bytes | **never** |
| `ext.setArtwork` | points a token at a key | yes, until `lockStorage` |
| `ext.setBlobStorage` | moves to another store | yes, until `lockStorage` |
| `ext.setRenderer` | changes presentation | yes, until `lockRenderer` |
| `renderer.setBase64Json` | JSON wrapper | yes, always (swap the renderer) |
| `ext.lockStorage` | freezes all pointers | **never** |
| `ext.lockRenderer` | freezes presentation | **never** |
| `burnRedeem` | burns an original | **never** — by the holder, not you |
Only four scripts ever send a transaction on a live network: `deploy.js`,
`register-extension.js`, `upload.js`, `point.js`, plus `redeem.js` and the two
testnet-only ones. `estimate.js`, `render-ceiling.js` and the test suites are
measurement tools and refuse to run anywhere but a local chain.