Learning

How a routine improves from what people do. Signals, thresholds, rejection memory, levels, and the learn then append then write order.

What it is

Learning is the step where a routine looks at what people did with its last runs, counts the gap between what it did and what they did, and, when the gap is big and steady enough, puts one plain suggestion in the Inbox. A person accepts, rejects or snoozes it. A routine never applies its own suggestion.

Why it exists

People correct routines all the time, and most of those corrections are thrown away. A person who keeps five roles the settings dropped is saying the settings are wrong. Learning turns that into a specific change (“Add hydrogen to title_keywords”) with the evidence attached, so the routine gets better from use, not from someone rewriting the prompt.

Code does the counting and the deciding. The model only phrases. That way thresholds, cooldowns and rejection memory hold on every run.

How it works

Turning it on

The jobs-watch template ships with learning on:

"learning": {"enabled": true}

Every other key is optional. With all defaults written out:

"learning": {"enabled": true, "signals": {"dead_item": false}, "limits": ["max_inbox_per_run"],
             "minSample": 5, "minEffect": 0.6, "cooldownDays": 14, "maxPerRun": 3, "deadAfterRuns": 10}

lint refuses unknown keys, unknown signals and out-of-range numbers. Every signal is on unless turned off. limits is empty unless you declare it; limit_pressure only counts limits named there.

Signals

Details the code holds:

• Drops are counted once per record value, so a record seen in several runs counts once.
• uncaptured_term proposes words from include text fields, skipping stopwords and words under 3 characters.
• On an exclude list, override proposes removing the value that dropped kept records.
• dead_item is skipped when every item of an include list is dead (that points at a broken source, not bad items).
• limit_pressure proposes the cap plus the largest overflow seen, never above the setting’s max. It counts only match and alert messages the contract would keep.

Since 0.5.0: a ninth signal, people_drop. When people drop records the routine kept, and those records share a word in a text field bound to an exclude list, learn proposes adding that word to the exclude list, with the same thresholds and memory. The jobs-watch template gains excluded_title_words (list, exclude, default empty, example “intern”) bound to the role title.

Deciding

A candidate is raised only when all of these hold:

• at least minSample distinct kept records (or pressured runs), and their share is at least minEffect;
• its key was not rejected or dismissed within cooldownDays, is not snoozed, and is not waiting undecided on an earlier run’s card;
• it passes the write contract’s proposal check.

At most maxPerRun are raised per run, strongest first. Rerunning learn for the same run id does not count twice. An include list is never proposed empty.

Levels

Opening the PR for L2 and L3 is a follow-up. Today the card records the decision only.

The run order

Each run has one order: learn, then append, then write.

python3 skills/routineos/scripts/routineos.py learn --docs snapshot.json --run run.json --out learn

learn writes learning.json (one batch setting state/learning, phase learning) and proposals.json (messages). Apply learning.json, put the proposals.json messages at the front of the run’s inbox unchanged (the model may reword title and why only), then run write. A run already marked done writes nothing more, so proposals appended after write are lost.

Guardrails

• Never self-apply. Only a person’s Accept changes a setting.
• Learned-only fields. write recomputes this run’s learn output and refuses any per-item proposal (one with op, value, key, level, target, change, signal or evidence) that the learn step did not raise. The model cannot invent one.
• Rejection memory. Reject (and Inbox Dismiss) blocks the same key for cooldownDays. Snooze blocks it until its until (7 days on the Desk); a snooze without until blocks for good. A set is keyed by setting alone, so rejecting “set to 8” also blocks “set to 9”.
• Never learnable. Any setting whose name, with case and separators removed, contains push, gate, send, alert, permission, budget, maxruns or capoverride is refused in every proposal shape.
• Values only. A proposed value must match ^[A-Za-z0-9][A-Za-z0-9 .&+#/'_-]{0,59}$, so instruction-like text from a fetched page cannot travel through a proposal.

The off switch

Setting values.learning: false in settings/workspace turns learning off for the whole Desk. Tallies still count, nothing is proposed, and write refuses every proposal message, including model-written ones.

Worked example

In one Hiring Signals run, six roles titled “Hydrogen Developer”, “Hydrogen Engineer” and similar were dropped with not in title_keywords. On the Desk, a person kept five of them. On the next run:

python3 $RO learn --docs snap2.json --run run2.json --out l2
ok: 1 proposal(s) in proposals.json; apply learning.json as one ArtifactData batch

l2/proposals.json:

[{
  "kind": "proposal",
  "title": "Add hydrogen to title_keywords",
  "why": "People kept 5 of 6 records dropped for not matching title_keywords; they carry hydrogen.",
  "proposal": {
    "setting": "title_keywords", "op": "add", "value": "hydrogen", "signal": "uncaptured_term",
    "evidence": {"count": 5, "of": 6, "refs": ["role--https-jobs-example-au-200", "...", "role--https-jobs-example-au-204"]}
  }
}]

After the message is appended and write runs, the stored Inbox message carries the computed change:

"proposal": {
  "setting": "title_keywords", "op": "add", "value": "hydrogen",
  "from": ["Shopify", "headless", "commerce", "AI automation"],
  "to": ["Shopify", "headless", "commerce", "AI automation", "hydrogen"],
  "key": "title_keywords:add:hydrogen", "level": "L1", "signal": "uncaptured_term",
  "evidence": {"count": 5, "of": 6, "refs": ["..."]}
}

The person accepts it on Suggestions. The Desk adds hydrogen to settings/hiring-signals and records the decision. From the next run, Hydrogen roles are active.

If the model had instead written its own per-item proposal, write would refuse with exit code 2:

error: proposal message 'x' is a per-item proposal but is not one this run's learn step raised; append proposals.json from routineos.py learn unchanged

Reference

Learn needs an ISO startedAt or endedAt on the run input.

Common pitfalls

• Expecting a suggestion after one run. minSample is 5. Learning needs people to have made enough decisions.
• Writing proposals by hand in the prompt. Per-item proposals are refused. A plain {setting, from, to} proposal is still allowed and still checked.
• limit_pressure never fires. Declare the limit: "limits": ["max_inbox_per_run"].
• Turning learning off per routine to silence one suggestion. Reject it instead; rejection memory holds it back for cooldownDays.

Related

• Tune learning
• The Desk: Suggestions
• Settings
• Write contract reference