Records

Records (entities) are the things a routine finds. How each gets a stable id, how values from several producers combine, status and drop reasons, and links.

What it is

A record is one thing a routine found: a company, a role, a tender, a post. The store calls records entities and keeps each one at entities/<type>--<slug>. Its type and fields come from the definition.

Why it exists

A routine that reports “here is what I found today” cannot say what is new, what changed, or what people already decided. Records give every finding a stable identity across runs, so the same role is the same document next week, with its history, its decisions and its notes.

How it works

Identity

Each type declares an identity rule. The rule turns a raw key into a normalised key, and the id is <type>--<slug(key)>.

So https://www.Northwind.example/careers and northwind.example both key as northwind.example, id company--northwind-example. A company with no known domain keys on its name: Acme Pty. Ltd. becomes acme pty ltd, id company--acme-pty-ltd, and a link that gives the same name resolves to it.

Slugs

Lowercase [a-z0-9-]; runs of other characters collapse to one dash; at most 120 characters. A longer key keeps 111 characters plus a dash and the first 8 hex characters of the key’s SHA-1, so long keys stay distinct and stable. Two keys that slug the same fail the plan with both named.

Values from several producers

Each producer writes its own slice under bySource.<producerId> and replaces only that slice. display is what the Desk shows: the latest sighting’s value wins per field, and bySource (producers in sorted order) fills the rest.

Status

A dropped record is still written and still shown, so people can see what the settings cut.

Since 0.5.0: people can also Drop a record. The Desk writes decisions/<entityId> {decision: "drop", reason, at, by}; the next run marks the record dropped with that reason and never overrides it. Until then the Desk shows it as “Drop pending”. Restore deletes a person’s drop, or writes keep for a record the routine dropped.

Links

A record input may carry links: [{type, key}]. The contract resolves each link with the target type’s identity rule and stores {type, key, id}. A role links to its company, so the Desk can show a company’s roles.

Worked example

From the Quickstart run, the role record entities/role--https-jobs-example-au-1001:

{
  "type": "role",
  "key": "https://jobs.example/au/1001",
  "status": "active",
  "dropReason": null,
  "bySource": {"hiring-signals": {"title": "Senior Shopify Engineer", "company": "Northwind Goods", "market": "AU", "url": "https://jobs.example/au/1001", "...": "..."}},
  "display": {"title": "Senior Shopify Engineer", "company": "Northwind Goods", "market": "AU", "...": "..."},
  "links": [{"id": "company--northwind-example", "key": "northwind.example", "type": "company"}],
  "firstSeen": "2026-11-02T22:00:04Z",
  "lastSeen": "2026-11-02T22:00:04Z",
  "seenCount": 1,
  "sources": ["hiring-signals"],
  "sightings": [{"run": "2026-11-02-2200", "at": "2026-11-02T22:00:04Z", "producer": "hiring-signals", "outcome": "new", "dropReason": null, "values": {"...": "..."}}],
  "agentFields": {},
  "lastRun": "2026-11-02-2200",
  "schema": 1
}

Reference

People’s stars, tags and notes live in userstate/<entityId>, not on the record.

Common pitfalls

• Writing a person’s edit onto the record. The next run replaces this producer’s slice. Notes and tags belong in userstate/.
• Expecting field filters to be fast at scale. Record values live under display, and the store filters on top-level fields only. Field filters read the full set; see Browse.
• Changing identity on a live routine. Existing documents are never re-keyed. New sightings would create new records next to the old ones.

Related

• Sightings
• Merges
• The definition