Why Your Image Crawler Collects Blur Placeholders (and Misses the Images Hiding in CSS)

You run a crawl across a product gallery, get 2,000 files, and open the folder. A third of them are 40-pixel smears. Another chunk are solid rectangles. And yet the page you crawled clearly had a full-bleed hero image that never showed up in the results at all.


Both halves of that outcome come from the same mistake: treating "an image element with a URL" as "the image." Modern pages separate the *presentation* of an image from the *asset* behind it, and they keep a second layer of images entirely outside <img> tags. If your collector only walks the DOM for src attributes, it samples the placeholder layer and misses the CSS layer.


Three things that look like images but are not


1. Inline LQIP data URIs. A low-quality image placeholder is a real image, just a deliberately terrible one — often 20×20 pixels encoded as a base64 JPEG of 40 to 300 characters. It is inlined so it paints before any network request resolves. The src attribute is a complete, valid image. Nothing about the element tells you it is temporary.


2. SVG skeletons. Instead of a bitmap, the placeholder is <svg> with a gradient and a few rounded rectangles, inline in the HTML or in a data URI. It renders instantly, weighs almost nothing, and contains no photograph at all.


3. Blurhash-style canvases and 1×1 trackers. Some sites generate the placeholder at runtime — a hash string is decoded into a <canvas>, or a tiny blurred bitmap is drawn — and later cross-fade to the real image. There is no URL to collect because the pixels were computed. Separately, analytics pixels hide in img tags at 1×1, which are images by every technical definition except the one you care about.


Detecting placeholders without paying for a full decode


The cheap instinct is to decode every candidate and check its dimensions. That is also the expensive instinct: decoding 2,000 JPEGs to throw away a third of them burns CPU and memory for nothing.


For JPEG you can read the dimensions from the header alone. The frame header marker 0xFFC0 (baseline) or 0xFFC2 (progressive) is followed by two length bytes, one precision byte, then height and width as big-endian 16-bit values. Read 4 bytes at the right offset and you know the size without touching the DCT coefficients:


scan bytes for 0xFF 0xC0..0xFF 0xC2

skip 2 length bytes + 1 precision byte

height = (b[i] << 8) | b[i+1]

width  = (b[i+2] << 8) | b[i+3]


PNG is similarly cheap: bytes 16–23 of the file are width and height in IHDR. WebP needs a bit of unpacking (VP8 keyframes store 14-bit dimensions), but it is still header work.


Once you have dimensions, the classification is mostly arithmetic:


- area below roughly 100×100 and the element carries filter: blur(...) or sits inside a wrapper with a blur class → placeholder.

- width and height both 1 → tracking pixel, drop it.

- payload under ~200 bytes but a data:image/ scheme → inline placeholder, not a real asset.

- SVG whose root contains no <image> node → vector skeleton, drop it.


The important part is that all of this runs on bytes you already hold, before any decode, and it lets you keep the *good* candidates and skip the rest.


The images that are never in an img tag


A hero banner implemented as background-image on a <div> has no src anywhere in the DOM. To find it you have to ask the browser for computed style:


const bg = getComputedStyle(el).backgroundImage; // url("..."), possibly several


Four pitfalls break naive implementations here:


1. Commas. Multiple backgrounds are comma-separated, but data URIs contain commas themselves, and URLs can be quoted. Splitting on "," corrupts them. Parse the url("...") token, respecting quotes, rather than splitting the whole string.

2. Shorthand. Reading background may hand you a value where the URL is already absorbed; read backgroundImage explicitly, and also check borderImageSource, maskImage and listStyleImage if you want completeness.

3. Pseudo-elements. A huge share of decorative images live on ::before and ::after, reachable only as getComputedStyle(el, '::before').backgroundImage, plus the occasional content: url(...).

4. Cross-origin stylesheets. Reading document.styleSheets[i].cssRules throws a SecurityError on a stylesheet loaded from another origin. That entire rule set — and any URL that exists only in it — is invisible to you. Computed style still works, because you are reading the resolved value rather than the rules.


Relative URLs also need resolving against document.baseURI, and CSS variables resolve into url(var(--bg)) only after computation, so you must read the computed value rather than the declared one.


The timing problem underneath it all


Even a correct extraction can sample the wrong state. Scan too early and you get placeholders; scan too late and the grid may have been recycled by a virtualized list, so images that were visible a minute ago no longer exist in the DOM. The reliable approach is to keep a set keyed by resolved URL, observe mutations, and merge results across time instead of producing one snapshot — which also means you need deduplication that survives the same asset appearing under four different URLs (src, srcset candidate, CSS background, and a thumbnail transform of the original).


Where OmniPic AI Studio Pro fits


Everything above — header-level dimension sniffing, blur and tracker filtering, ::before background extraction, origin-aware URL resolution, cross-time merging, and perceptual-hash dedupe across all four URL shapes — is a few weeks of work that most people building an image workflow never finish. OmniPic AI Studio Pro does it end to end: it inspects a whole site in one pass, separates real assets from the placeholder layer, restores the original behind CDN thumbnail transforms, groups duplicates by perceptual hash rather than by string, and runs entirely locally — no upload, no server-side index of what you browsed. Export goes straight to Eagle or Billfish, or into a streamed ZIP that handles 2,000+ files without freezing the browser.


Available on Chrome, Edge and Firefox: https://chromewebstore.google.com/detail/omnipic-studio-pro/cokoeaaaglgcljahaffjeijomhmkljlc


If you have a crawler that keeps eating blur placeholders, the header-sniffing trick above is the cheapest fix to start with. The CSS layer is where the real work begins.


OmniPic AI Studio Pro running a site-wide image inspection
OmniPic AI Studio Pro - a site-wide inspection that separates real assets from the placeholder layer.

评论

此博客中的热门博文

Running AI in the Browser: WebGPU-Accelerated MobileNet Feature Extraction for Real-Time Image Search

Why Image Scrapers Silently Fail: CORS, Tainted Canvases, and Blob URLs