a manifold extension to port fully onchain an existing collection minted with offchain content
clgsms/RUNBOOK.md
Markdown · 431 lines · 14 KBREADSOURCERAWDOWNLOAD

COLORGASM onchain migration — runbook

The procedure, start to finish. Follow it in order; within PHASE 1 each token is independent, so you can do one completely, stop, and pick up any other later in any order.

Two things are permanent and cannot be undone by anyone, ever:

  • ColorgasmStorage.seal(key) — those bytes can never change
  • extension.lockStorage() — every pointer and metadata fragment freezes

Everything else is reversible while the extension is unlocked. Nothing below seals or locks anything until §6, and lockRenderer() is deliberately last.

Paths. The scripts default to ../onchain for the encoded artwork (AVIF_DIR) and meta/ for the metadata fragments (META_DIR) — the layout of the artist's working folder, where encode/encode_onchain_v2.py writes. Set AVIF_DIR in .env if yours live somewhere else.

Configuration for this run: artwork stored RAW (PRE_ENCODED unset), document served as UTF-8 JSON (UTF8_JSON unset). Both are the defaults. The renderer's preEncoded is immutable, so upload.js and point.js read it off the deployed renderer and refuse to run on a mismatch — you cannot get this wrong silently, but do not override it on one command and not another.


0. Before you spend anything

0.1 Install exactly the pinned toolchain.

rm -rf node_modules cache artifacts
npm ci

npm ci, not npm install, and do not delete package-lock.json. Every dependency is pinned to an exact version because gas figures move between patch releases.

0.2 Tests green.

npm run build
npm test

Expect 38 passing. If you see fewer, or a suite that finishes suspiciously fast, you are running against stale artifacts — rm -rf cache artifacts and rebuild before going further.

0.3 Fill in .env.

cp .env.example .env
  • ALCHEMY_KEY (or INFURA_KEY) — covers mainnet, the fork and Sepolia
  • PRIVATE_KEY — use a throwaway key for Sepolia; swap in the real one only at §4
  • ETHERSCAN_API_KEY — one V2 key verifies on both chains
  • MAX_GWEI — leave unset for now; §5 uses it

0.4 Check the artwork is complete.

ls ../onchain/*.avif | wc -l          # expect 100

report.csv currently lists 99 files — #66 has no encode. Produce it before starting, or decide deliberately that #66 stays on Arweave. upload.js skips a missing file with a message rather than failing, so a gap will not stop a run — it will just quietly leave that token unredeemable.

0.5 Price it.

B64_DIR=../onchain EXT=.avif npm run estimate

--network mainnet may be added to also print the cost at the live base fee. The measuring itself always happens on a throwaway local chain whatever you pass — estimate.js and render-ceiling.js measure by really deploying and really sending, so they refuse outright to run against any chain that is not 31337.

Expect roughly 13.7 ETH at 1 gwei for 99 files stored raw. Decide the gas price you are willing to run at before you start, not while it is running.

0.6 Fork test against the real contract.

npm run test:fork

Then pin FORK_BLOCK in .env to the block it used, so later runs are reproducible.


1. Sepolia — full dress rehearsal

Everything here is free. Do not skip it: this is where you find out whether the whole chain of contracts behaves, and whether marketplaces accept the document.

1.1 Stand up a creator contract. COLORGASM only exists on mainnet.

npx hardhat run scripts/deploy-creator.js --network sepolia

1.2 Mint stand-in originals.

FROM_ID=1 TO_ID=5 npx hardhat run scripts/seed-originals.js --network sepolia

1.3 Deploy storage + extension + renderer.

npx hardhat run scripts/deploy.js --network sepolia

Check the header it prints says artwork stored raw, served as UTF-8 JSON. Addresses land in deployments/sepolia.json.

1.4 Verify, then register the extension.

npx hardhat run scripts/verify.js            --network sepolia
npx hardhat run scripts/register-extension.js --network sepolia

Verify on Sepolia too — it is the rehearsal for §2.2, and it is where you find out if your ETHERSCAN_API_KEY works before it matters.

The Manifold ERC721Creator that deploy-creator.js puts on Sepolia is deployed from frozen bytecode, not from source in this project, so scripts/verify.js does not cover it and hardhat verify cannot. It is a test scaffold that exists only on Sepolia — nothing on mainnet depends on it. If you want it verified anyway, use Etherscan's "similar match" against any of Manifold's own deployed creator contracts.

1.5 Metadata fragments.

npm run metadata -- meta 1 5

Open one and read it. It should be the original JSON minus the image fields, with no surrounding braces.

1.6 One token, end to end.

IDS=1 npx hardhat run scripts/upload.js  --network sepolia
IDS=1 npx hardhat run scripts/point.js   --network sepolia
IDS=1 npx hardhat run scripts/inspect.js --network sepolia
IDS=1 npx hardhat run scripts/redeem.js  --network sepolia

inspect.js must say exact for both image and metadata. Then open the new token on testnets.opensea.io and look at it.

1.7 Then the biggest one you have. This is the real test — a 964 KB piece is where document size, eth_call budget and indexer tolerance all get exercised at once.

IDS=94 npx hardhat run scripts/upload.js  --network sepolia
IDS=94 npx hardhat run scripts/point.js   --network sepolia
IDS=94 npx hardhat run scripts/redeem.js  --network sepolia

Check it renders on OpenSea, and in a plain browser tab by pasting the tokenURI output. If a marketplace chokes on data:application/json;utf8,, this is the moment to find out — deploy a renderer with UTF8_JSON=0 and setRenderer to it, which costs one transaction and nothing else.


2. Mainnet — deploy the contracts

Switch PRIVATE_KEY in .env to the real artist wallet now.

2.1 Deploy.

npx hardhat run scripts/deploy.js --network mainnet

Confirm the header again: artwork stored raw, served as UTF-8 JSON, and creator 0x7dC2a81Ba2Bf47548EC2F9645007119dF90ba01b.

2.2 Verify on Etherscan. Wait a minute for Etherscan to index the deployment, then:

npx hardhat run scripts/verify.js --network mainnet

It reads deployments/mainnet.json and verifies all three with the constructor arguments they were actually deployed with — including the renderer's preEncoded, which must match or the bytecode will not. Re-running is safe; contracts Etherscan has already verified are reported and skipped.

Expect:

✓ ColorgasmStorage           0x…
✓ ColorgasmRedeemExtension   0x…
✓ ColorgasmRendererV3        0x…
all 3 contracts verified on mainnet

The artwork chunks are SSTORE2 pointers — a STOP byte followed by raw data, with no source to verify. They are checked by reading them back instead, which is what inspect.js does and what anyone else can do independently.

2.3 Register the extension on CLGSM. This must come from the wallet that is owner or admin of the creator contract — your Manifold admin wallet, which may not be the deploy wallet.

npx hardhat run scripts/register-extension.js --network mainnet

The script checks owner() and isAdmin() first and tells you who it expected if the signer is wrong. If your admin wallet is a hardware wallet you would rather not put in .env, call registerExtension(<extension>, "") from Etherscan's write tab instead — it is the same single call.

2.4 Metadata fragments for the whole edition.

npm run metadata

3. Mainnet — one token, all the way

Pick a token you hold. Do this one completely and stop.

IDS=57 npx hardhat run scripts/upload.js  --network mainnet
IDS=57 npx hardhat run scripts/point.js   --network mainnet
IDS=57 npx hardhat run scripts/inspect.js --network mainnet

upload.js will:

  • read ../onchain/COLORGASM_#57.avif
  • split it into 24573-byte chunks (a multiple of 3, so the renderer can base64 it chunk by chunk)
  • send one append per chunk, waiting between them if gas is above MAX_GWEI
  • read every chunk back with eth_getCode and compare against the file
  • seal the key — permanent

Then redeem it and look at it:

IDS=57 npx hardhat run scripts/redeem.js --network mainnet

Open it on OpenSea. Refresh metadata if needed. Stop here until you are happy with what you see. Everything so far is one artwork's worth of gas.


4. Mainnet — the rest, at your own pace

Set a gas ceiling you are content to leave running at:

export MAX_GWEI=2

Then work through the edition however you like. One at a time:

IDS=3 npx hardhat run scripts/upload.js --network mainnet
IDS=3 npx hardhat run scripts/point.js  --network mainnet

A handful, in any order:

IDS=12,4,88,7 npx hardhat run scripts/upload.js --network mainnet
IDS=12,4,88,7 npx hardhat run scripts/point.js  --network mainnet

Or leave it running over a range:

FROM_ID=1 TO_ID=100 npx hardhat run scripts/upload.js --network mainnet
FROM_ID=1 TO_ID=100 npx hardhat run scripts/point.js  --network mainnet

Order never matters. Tokens are independent: each has its own storage key, its own pointer, and becomes redeemable on its own the moment both halves are set. Holders can redeem the finished ones while you are still uploading the rest.

Re-running is always safe while the extension is unlocked:

  • an artwork whose stored bytes already match the file is skipped
  • an upload that stopped part-way continues from the chunk it reached
  • a token already pointed at the right key with the right metadata is skipped

So if a run dies — laptop sleeps, RPC drops, gas spikes — just run the same command again.

Check progress at any time:

FROM_ID=1 TO_ID=100 npx hardhat run scripts/inspect.js --network mainnet

Every row should read exact / exact / sealed yes. Anything else is listed as a problem at the bottom and the script exits non-zero.


5. Announce

Once enough tokens are redeemable, tell holders. They need to know:

  • redeeming burns the original and mints a new tokenId — the old one is gone
  • they must setApprovalForAll(<extension>, true) on CLGSM first
  • isRedeemable(originalTokenId) on the extension says whether their piece is ready; surface this in your frontend rather than letting them guess
  • unredeemed tokens keep their Arweave URI indefinitely — there is no deadline and nothing breaks if they never redeem

Your own tokens can go through scripts/redeem.js with BATCH=1 (the default) to do several per transaction. Keep batches under ~50 tokens so the transaction stays well inside the 16,777,216 gas cap.


6. Freeze it

Only when every artwork is verified onchain and you have looked at a good number of them rendered.

6.1 Final audit, with the images decoded out of the live contract:

FROM_ID=1 TO_ID=100 DUMP_DIR=dump npx hardhat run scripts/inspect.js --network mainnet
open dump/            # these came out of tokenURI, not off your disk

Compare a few against ../onchain/ by eye and by checksum:

for f in dump/*.avif; do
  id=$(basename "$f" .avif)
  cmp -s "$f" "../onchain/COLORGASM_#$id.avif" && echo "OK $id" || echo "MISMATCH $id"
done

6.2 Keys are already sealed by upload.js. Confirm no row says NO:

FROM_ID=1 TO_ID=100 npx hardhat run scripts/inspect.js --network mainnet | grep -c " NO "

6.3 Lock the pointers. Irreversible.

npx hardhat console --network mainnet
> const ext = await ethers.getContractAt("ColorgasmRedeemExtension", "<extension>")
> await (await ext.lockStorage()).wait()

6.4 Leave lockRenderer() for much later — months, not days. It is the only thing that lets you fix presentation, change the MIME type, or move to a cheaper renderer. Locking storage already gives collectors the guarantee that matters: the art cannot change.


When things go wrong

An upload stopped part-way. Run the same command again. It reads what is onchain, confirms it matches the start of your file, and continues from there.

Stored bytes do not match the file. The script refuses and tells you. If the key is unsealed, it will not append onto mismatched data — change KEY_PREFIX and upload under a fresh key, then re-point the token.

A key was sealed with the wrong artwork. The bytes are permanent, but the pointer is not, while the extension is unlocked. Upload the right artwork under a different key (KEY_PREFIX=CLGSM2_) and re-run point.js for that token. The wrong blob stays onchain, orphaned and harmless.

Wrong metadata. setArtwork(id, bytes32(0), <new metadata>) replaces the fragment and keeps the pointer — that is what point.js does automatically when only the metadata differs.

A transaction is stuck. Wait, or replace it with the same nonce at a higher fee from your wallet. Nothing here depends on a particular ordering, so a dropped transaction is never worse than repeating the step.

Gas spikes mid-run. upload.js polls and waits rather than sending above MAX_GWEI. Leave it running, or Ctrl-C and resume later — same command.

A shape mismatch error. You set PRE_ENCODED on one command and not another. The deployed renderer decides; match it or deploy a renderer for the other shape. Never work around the check — it is the difference between a token that renders and one that decodes to nothing.


The irreversible list

actioneffectreversible?
blobs.appendwrites bytesno, but harmless — orphaned blobs cost nothing
blobs.sealfreezes a key's bytesnever
ext.setArtworkpoints a token at a keyyes, until lockStorage
ext.setBlobStoragemoves to another storeyes, until lockStorage
ext.setRendererchanges presentationyes, until lockRenderer
renderer.setBase64JsonJSON wrapperyes, always (swap the renderer)
ext.lockStoragefreezes all pointersnever
ext.lockRendererfreezes presentationnever
burnRedeemburns an originalnever — by the holder, not you

Only four scripts ever send a transaction on a live network: deploy.js, register-extension.js, upload.js, point.js, plus redeem.js and the two testnet-only ones. estimate.js, render-ceiling.js and the test suites are measurement tools and refuse to run anywhere but a local chain.