What We Learned Publishing on a Schedule

A repository full of finished pages makes one specific mistake very easy: publishing all of them on the day they merge. It is one push away, it is not recoverable by reverting because the crawl has already happened, and it is the single strongest scaled-content-abuse signal a young domain can emit. Separating writing from publishing was the most important structural decision in this whole project.

Merging is publishing

In a normal software repository this is a feature. In a content repository it is a trap, and it is invisible until it fires. Four pages merge and nothing about the pattern is apparent. Four hundred merge and four hundred URLs appear in the sitemap the same morning.

Reverting does not undo it. The pages disappear, the crawl does not. Whatever conclusion was drawn about a domain that emitted several hundred URLs in one day has already been drawn, and the only remedy is time.

So writing and publishing have to be two different actions, and the separation has to be structural rather than procedural. “Remember not to merge everything at once” is not a mechanism.

The mechanism

Three parts, none of them complicated:

1. A release date per page. One optional field on the page object.
2. A generated schedule file holding the dates, written by a script rather than typed, because nobody should be assigning a thousand dates by hand. A page’s own date, where present, overrides the schedule — and the generator refuses to write an entry for a page that carries one, so there can never be two answers to the same question.
3. Routes that evaluate the date per request, not at build time. This is the part that is easy to get wrong. A page whose date passes must become visible on the morning it arrives with no rebuild and no deploy, or the schedule is really a deploy schedule wearing a costume.

The result is that a page can be written, reviewed, committed, deployed and still return 404 on its own URL, be absent from its cluster page, be absent from the index and be absent from the sitemap. It was proved on a production build rather than argued: one finished page dated a week out did exactly that while its three published siblings served normally.

Comparing dates as ISO strings rather than as date objects is deliberate. A release date is a calendar day in UTC and so is today; a string comparison of two YYYY-MM-DD values is exactly a date comparison with no timezone to get wrong.

The default has to fail closed

Originally a page with no date published immediately. That was right when four pages existed and a human typed each date, and it became dangerous the moment pages started arriving twenty at a time. One cluster file merged without dates would have put twenty URLs live on merge day — the exact failure the mechanism exists to prevent.

So the default was inverted: no date anywhere means not published. The two failure modes become “forget a date and one page is invisible” and “forget a date and the whole backlog publishes itself”. The first is a page nobody can read; the second is unrecoverable. Pick the first, and make a test fail loudly when a page is in that state so it does not stay invisible.

The scheduler has to be append-only

The second property that made this safe rather than merely clever: a slug that has a date keeps it forever. Re-running the scheduler can extend the queue and can never move, re-order or un-publish anything already scheduled.

That constraint is what makes it safe to run against live, indexed URLs. Without it, adding a new cluster and re-running would re-sort the whole queue, and pages that had been live for weeks could acquire future dates and vanish. There is no sequence of runs that damages what has already been crawled, and that property is worth more than any scheduling cleverness.

The problem it created: links to unpublished pages

Paced publishing introduces a failure that bulk publishing does not have, and it is invisible until it is live.

The pages cite each other constantly — that is what makes a library a connected body of work rather than a heap of orphans. In this case there were 325 cross-links. But the targets release months apart, so at any given moment most cited pages are written and not yet public. A plain link to one of them is an internal link on an indexed page pointing at a 404. It is not caught by a typecheck, not caught by any content test, and visible only by clicking around the live site before a target’s date arrives.

The fix is a component rather than a rule: it renders a link when the target is published and plain text when it is not, decided per request so it starts working on the morning the date arrives. Then a test asserts that no page bypasses it, because the failure mode is somebody writing the raw link out of habit — and the test is the only thing standing between habit and hundreds of broken internal links.

The day the queue was overridden

The queue ran to 2027. The owner asked for everything to be published immediately, and it was — all thousand pages dated to the same day, on a domain that was four days old. That is close to the strongest version of the pattern the cadence existed to avoid.

Three things are worth recording about it, because this is the part these write-ups usually omit.

• The cost was stated before the decision, not after. The mechanism’s job is to make the trade-off visible and reversible, not to overrule the person whose domain it is. The schedule files say at the top that the dates were set this way by instruction, rather than pretending they had always been that.
• Nothing was dismantled. Dates are still honoured, the scheduler is still append-only, and a page written tomorrow still gets a future date and stays hidden until it arrives. Pulling any of it back is a matter of editing a date forward, which takes effect on the next request with no rebuild.
• The fail-closed default earned its place a second time, that same hour. Everything was first dated to the following day. The suite reported zero pages live, because the gate compares in UTC and it was still the previous day there. Every page had been dated one day into the future and would have stayed invisible with nothing broken to find. A fail-open default would have silently published them; the fail-closed one produced a number that was obviously wrong.

Whether the immediate release was expensive is not yet knowable, and this page will not pretend otherwise. The domain is days old. What can be said is that the decision was made with the cost visible, and that the option to return to a cadence was preserved.

What to take from this