catalog-extract/README.md · md · download
# catalog-extract A Claude Code skill that builds and runs a recurring competitor catalog tracker (products, prices, specs, store locations, retailer carriage) with plain Nimble Extract. It generalises the Citizen Watch Company of America pipeline (10 brands, 11 retailers, nightly) into a workflow any user can run against a new brand site. No Extract Templates, no Web Search Agents. ``` SKILL.md the router Claude reads first: the eight rules, which workflow file to load, troubleshooting workflows/recon.md Step 0-1: scope the request, site + PDP recon, escalations when UNVERIFIED workflows/products.md Steps 2-4: discovery, spec map, hooks, smoke test, run, sentinel choice, scheduling and modes workflows/stores.md Step 5: the four locator mechanisms, stores.py usage, metro grid, never fabricate workflows/retailers.md Step 6: retailer listing discovery, RETAILER_FIELDS, exact-reference matching, Signet cookies (no scripts yet) scripts/nx.py Nimble Extract helper: fetch with truncation guard, JSON unwrap, in-browser fetch, cookie warm-up scripts/recon.py site + PDP reconnaissance (discovery source, render mode, where the data lives, spec labels) scripts/discover.py sitemap / Shopify products.json / paginated listing -> urls.csv scripts/parse_product.py JSON-LD + __NEXT_DATA__ + spec harvest -> canonical row (+ spec map) scripts/run_catalog.py the product loop: threads, sentinel guard, filtered vs failed, append-only CSV scripts/stores.py locators: JSON GET, in-browser POST/XHR, metro-grid sweep -> stores CSV scripts/csvio.py schemas + IncrementalCsv (per-row flush, header guard) hooks/example_hook.py where site-specific logic goes (prepare / fetch / parse) references/extract-field-notes.md measured Nimble behaviour (truncation, sessions, <pre> JSON, batch) references/site-recipes.md worked recipes for the 10 brands + 11 retailers references/schema.md canonical columns, active-catalog window, SKU lifecycle references/us_metro_grid.csv 28 metro points for N-nearest-capped locators ``` Install for a project: copy this folder to `<repo>/.claude/skills/catalog-extract/` (or `~/.claude/skills/` for all projects). Needs `pip install nimble_python beautifulsoup4` and `NIMBLE_API_KEY` in the environment or a `.env`. Verified 2026-10-19: Tissot sitemap (569 URLs) and 4 rendered PDPs (24 s, all fields populated), Hamilton recon (render=False recommended, 35 spec labels), Grand Seiko store API (100 rows), Hamilton Magento XHR (358 US rows).