Architecture & Key Constraints
Authors merge only on profile URLs or publisher IDs, and a conflicting email is kept in review instead of being picked.
What it is for
Publisher pages do not store an author in one place. One page uses JSON-LD. The next hides the same person in a Next.js hydration payload, or in a Nuxt state blob. Omniscrape is the parser I use so that work is not rewritten per site.
parseHtml takes raw HTML and the final page URL. It returns metadata, the author or confirmed co-authors, articles on the page, a provenance trail, and a review bucket. It does not fetch the page. The caller does that.
npm install @sfsajid91/omniscrape
What it reads
It reads JSON-LD, DOM and meta tags, and hydration payloads: Next.js (__NEXT_DATA__ and recognized App Router Flight records), Nuxt 2 and 3, Quintype/Bold, Fenrir, and static __INITIAL_STATE__ or __PRELOADED_STATE__ blobs.
Authors merge only on an explicit profile URL or a publisher id, not on a matching name. Article authors stay separate from editorial owners and site bylines. If two sources disagree on an email, the addresses stay in author.emails, author.email is left empty, and the conflict is recorded in review. A broken script records an issue and parsing continues.
The defaults refuse HTML over 8 MiB, skip a script over 2 MiB, and stop at 50,000 decode nodes or a depth of 128. Those limits are in the function options. Invalid URLs throw TypeError. Oversized HTML throws RangeError.
Unlike general metadata extractors such as Metascraper, Omniscrape is built for framework hydration payloads and person-level attribution, with conflict handling and provenance trails. I wrote about how it works: Omniscrape: Extract Metadata from Any HTML.
import { parseHtml } from "@sfsajid91/omniscrape";
const result = parseHtml({ html, pageUrl: url });
result.author.name;
result.provenance;
result.review;
The package is @sfsajid91/omniscrape on npm (v0.2.2), MIT licensed, and the source is on GitHub. Tests run with bun test.



