博文

目前显示的是 十月, 2026的博文

Every Image on the Page: URL Canonicalization, Lazy Loading, and srcset in a Site-Wide Crawler

 If you've ever tried to "save all images from this page" with a naive loop over document.querySelectorAll('img'), you know how much it misses. Lazy-loaded images aren't in the DOM yet. Responsive images hide their real URLs inside srcset. Background images live in CSS, not in any <img> tag. And even when you do find a URL, the same image can appear under ten different URLs — with different query strings, sizes, and tracking parameters — so your "2,300 images" are really 800 unique ones. A serious site-wide image crawler isn't a querySelector loop. It's a pipeline with three hard problems: finding every image, canonicalizing its URL, and surviving pages that load images on demand. This post walks through each one. Problem 1: Finding images that aren't <img> tags The first pass is the obvious one: img[src], img[srcset], <source> inside <picture>, and video[poster]. But three classes of images need special handling: ...
 When you crawl a site and collect 2,000 images, what does the browser actually have to remember about each one? The URL, dimensions, format, source page, download status, a pHash fingerprint for deduplication, and — if you're doing on-device AI — a 1024-dimensional feature vector. That's roughly 5-10 KB of structured metadata per image just for the vectors, before counting the image bytes themselves. localStorage gives you 5-10 MB of synchronous string storage. A single Float32Array of 1024 values, JSON-serialized, is about 8 KB per image — 2,000 images means 16 MB, already past the limit, and it would block the main thread on every read. Plain in-memory objects vanish on refresh. The Cache API is designed for network responses, not queryable records. For a local-first image tool, none of these fit. The right tool is IndexedDB — a real transactional database built into every browser. This post walks through its data model, why it scales to thousands of records without freezing...

On-Device AI, Explained: Depthwise Convolutions, MobileNet, and Why Running Inference in the Browser Is a Privacy Win

 Every time you tap "upload to cloud" for a photo analysis task — reverse image search, object classification, similar-image grouping — you're sending a real image of a real document, a product shot, or a personal screenshot to somebody else's server. The provider logs it, trains on it, or keeps it for "quality purposes." You have no idea. There is a different way: run the model in the browser, on the user's own machine, with the pixels never leaving the tab. This used to be a fantasy — a neural network needs gigabytes of compute, right? Not with the right architecture. This post explains the two engineering ideas that make on-device inference practical (depthwise separable convolutions and WebGPU), and why "local" is not just a performance story but a privacy architecture. The problem with a "normal" convolution A convolutional layer takes a W×H×C input and produces a W×H×N output feature map. A standard 3×3 convolution does this: pa...

Perceptual Hashing, Explained: Turning an Image into a 64-Bit Fingerprint with DCT

 If you've ever tried to deduplicate a folder of downloaded images with a regular hash, you already know the problem: MD5 and SHA-256 are exact hashes. Re-encode a JPEG at 85% quality, resize it by 2%, or strip its EXIF metadata, and the bytes change — the hash changes completely. Two images that look identical to a human produce two unrelated hashes. That's why image tools need a perceptual hash instead: a fingerprint that stays stable under compression, resizing, and color tweaks. The most widely used one is the pHash (perceptual hash) algorithm, which squeezes an image down to 64 bits and can tell you, with a single integer comparison, whether two images are visually "the same." Here's the complete algorithm, step by step, including the math and a practical JavaScript sketch. Step 1: Shrink to 32×32 The image is resized to 32×32 pixels. This does two things at once: it normalizes away resolution differences (a 4000×3000 photo and a 800×600 crop of it both becom...

Why Manifest V3 Forced Every Image-Tooling Extension to Rethink Its Architecture

 If you've built a Chrome extension that touches more than a handful of pages, you've felt the Manifest V3 (MV3) migration. For image-heavy tools — crawlers, scrapers, screenshot managers, batch downloaders — the change wasn't a paperwork exercise. It broke the assumptions the entire category was built on. Here's what actually changed, and why a serious image-collection tool has to be architected differently today. The Three Pillars of MV3 1. Service Workers replaced background pages In Manifest V2, the background page was a persistent HTML page that lived in memory for the whole browser session. Your global state, long-running loops, and in-memory caches just... stayed there. In MV3, the background script runs in a service worker: an event-driven, ephemeral context that can be terminated at any moment — after 30 seconds of inactivity, or earlier if the browser needs memory. It wakes up on events (messages, alarms, web requests) and shuts back down. Consequence for imag...

WebP, AVIF, and the Hidden Cost of Modern Image Codecs

 Open any modern e-commerce site and check the network tab: the vast majority of images are no longer JPEG. WebP has been the default on Chrome and Android for years, and AVIF — the AV1-based codec — is rapidly taking over CDN delivery because it routinely beats both JPEG and WebP by 25–50% at the same visual quality. But here's the tension: the web loves WebP/AVIF because they're small. Your workflow often still needs PNG or JPEG — design tools, asset pipelines, image libraries, and submission forms that reject WebP outright. So almost every content team ends up needing batch conversion. Understanding what actually happens during decode makes the difference between a tool that handles 2,000 images gracefully and one that freezes the tab. How Modern Lossy Codecs Actually Work JPEG: the 8x8 baseline. JPEG splits the image into 8×8 pixel blocks, applies the Discrete Cosine Transform (DCT) to each block, quantizes the frequency coefficients (throwing away high-frequency detail the...

Decoding Images in the Browser: ImageDecoder, createImageBitmap, and Why Canvas Isn't Always the Answer

 When you need to process hundreds of images in a browser extension — thumbnail generation, format conversion, or feature extraction for on-device AI — the naive "draw to Canvas" approach silently becomes the bottleneck. Here's what actually happens under the hood, and how the modern browser image pipeline changes the design. THE PROBLEM: "JUST USE CANVAS" STOPS SCALING Every frontend engineer has converted an image at some point: const canvas = document.createElement('canvas'); canvas.width = targetW; canvas.height = targetH; canvas.getContext('2d').drawImage(img, 0, 0, targetW, targetH); const blob = await new Promise(r => canvas.toBlob(r, 'image/webp', 0.8)); It works — for one image. Now try it for 2,000 images found by a site-wide crawler: 1. Every drawImage forces a full decode of the source into an RGBA bitmap, even if you only need a 200px thumbnail. 2. canvas.toBlob re-encodes on the main thread, blocking UI for tens of second...