博文

目前显示的是 九月, 2026的博文

Reverse-Engineering CDN Thumbnails: Recovering Full-Resolution Originals from Aliyun OSS, Tencent COS, and Qiniu URLs

 Every serious image pipeline serves thumbnails. The main image on a product page might be a 2,000px original, but the page loads a 200px version so it renders fast. When you're building an image collector, a design asset scraper, or a reverse image search tool, you constantly hit this wall: the URL in the HTML points at a thumbnail, not the original. The naive fix — "just crawl the largest version on the page" — is unreliable. The robust fix is to understand how CDN image processing works, because Aliyun OSS, Tencent COS, and Qiniu all implement the same pattern: the original file stays untouched on storage, and thumbnails are generated on-the-fly by appending processing parameters to the URL query string. That means the original URL is usually recoverable by stripping or rewriting those parameters. No re-download, no guessing. ## How Image-Processing CDNs Work All three major Chinese cloud object stores (and most Western ones too — Cloudinary, imgix, Cloudflare Images) ...

From O(n²) to O(log n): Using a BK-Tree for Fast 64-bit Perceptual Hash Near-Neighbor Search

 Your image collector just grabbed 2,000 images from a website. Before exporting, you want to drop the near-duplicates — same photo recompressed at two qualities, the same meme cropped slightly differently, the same screenshot at two resolutions. Perceptual hashing gives you the distance function: compute a 64-bit pHash for every image, then two images are "duplicates" if their Hamming distance is below a threshold (typically ≤ 10). That part is straightforward. The naive dedup loop is not: for (let i = 0; i < hashes.length; i++) {   for (let j = i + 1; j < hashes.length; j++) {     if (hammingDistance(hashes[i], hashes[j]) <= 10) {       // duplicate found     }   } } That's N×(N-1)/2 comparisons. For 2,000 images: ~2 million Hamming-distance computations. Each is fast (a few XOR+popcount ops), but 2 million of them on the main thread of a browser extension is a noticeable jank, and at 20,000 images it becomes 200 million — unu...

Streaming Large Files in the Browser: Web Streams API, Backpressure, and a ZIP Archive That Never Freezes Your Tab

 Collect 2,000 images from a website with a browser extension, and you've got a real engineering problem on your hands. The images add up fast: 2,000 photos at an average of 800KB is 1.6GB. Now the user clicks "Export as ZIP." The naive implementation reads every image into memory, concatenates them into one giant buffer, and compresses. On a machine with 8GB RAM, that's a frozen tab, a beach ball, and a browser that asks "This page isn't responding. Kill it or wait?" The fix isn't a bigger buffer. It's a completely different execution model: streaming. Process data in small chunks, move them through the pipeline as they arrive, and never hold more than a few megabytes in memory at any instant. The browser has a first-class API for exactly this: the Web Streams API. The Mental Model: Pipes, Not Buffers Traditional file processing looks like this: read all bytes, process, write all bytes. Streaming looks like this: read chunk, process chunk, write...

Running AI on Your GPU in a Browser Tab: WebGPU, MobileNet, and Private Reverse Image Search

 You have 2,000 images collected from dozens of websites. You want to find every variant of one product shot — including versions that were resized, recompressed, or slightly cropped. The classic answer is a reverse image search API: upload the image, wait for the server to hash it, get matches back. But there's a fundamental tension. Every image you upload to a reverse image search service is a copy of your data leaving your machine. For designers working with unreleased product shots, e-commerce operators handling supplier catalogs, or researchers collecting reference material under NDA, that's a non-starter. Uploading proprietary assets to a third-party service—even a reputable one—means the vendor can log, analyze, and retain your images. What if the entire similarity pipeline ran on the user's own machine? Not a thin client that calls an API—the actual neural network, executing in a browser tab, on the GPU, with zero bytes leaving the device. That's not science fic...

Beyond the 5MB Myth: Browser Storage Quotas, OPFS, and Building Local-First Image Collections

 If your mental model of browser storage is "localStorage is 5MB, that's all you get," you're missing out on one of the most powerful—and most misunderstood—platforms in the browser. Modern browsers can store gigabytes of data per origin, stream files directly to disk, and even do synchronous file I/O in Web Workers. The catch: every storage technology has different semantics, different quotas, and different eviction rules. If you're building an image collection tool (or any local-first app), getting this right is the difference between "saves 2,000 images" and "crashes on image #300." Let's map the terrain. The Quota: It's Not 5MB Anymore The 5MB limit was localStorage in 2009. Modern browsers use a shared pool: IndexedDB, Cache API, OPFS, and (in some browsers) localStorage all draw from the same origin-level quota, which is typically 60% of available disk space on desktop Chrome and Firefox. The practical number for a typical machine...

ZIP File Format Deep Dive: From Local File Headers to Streaming Archives

 Every developer has zipped a folder. Very few know what actually happens inside that .zip file. It's not a compressed blob — a ZIP is a precisely laid-out binary container with a self-describing structure: a series of Local File Headers, a Central Directory at the end, and fixed magic numbers that any tool can parse. Understanding this format is the key to building streaming ZIP writers that pack thousands of files without ever holding them all in memory. Let's open the file format. The Big Picture: Three Sections A ZIP file has three conceptual sections: [Local File Headers + file data] ... [Central Directory] ... [End of Central Directory] 1. Local File Headers (LFH) — one per file, appearing immediately before each file's compressed data. This is what the ZIP reader walks through sequentially. 2. Central Directory (CD) — a second index of all files, located near the END of the file. It exists so tools can list contents without decompressing everything. 3. End of Central...

The Art of Concurrent Image Downloads: Queue Scheduling and Memory-Safe Bulk Fetch Pipelines

 When you want to save every image from a webpage — say 1,500 images from a design inspiration site — the naive approach is a Promise.all over a fetch loop. It works on small pages and explodes on real ones: the browser runs out of memory, the tab freezes, the CDN rate-limits you, and half the downloads fail with no recovery. Bulk image download is a systems problem: concurrency control, queue scheduling, retry semantics, and memory management. Let's build the pipeline properly. ## The Naive Approach and Why It Fails ```javascript // Naive — don't do this const blobs = await Promise.all(images.map(async (url) => {   const res = await fetch(url);   return res.blob(); // 1,500 blobs in memory at once })); ``` Problems, in order of severity: 1. Memory blowup — 1,500 images × 2MB average = 3GB of blobs held simultaneously 2. No concurrency limit — the browser opens hundreds of connections, triggering rate limiting and connection pool exhaustion 3. No retry — one transient...

Design Asset Management Integration: How Eagle and Billfish Libraries Work Under the Hood

 If you're a designer, you've probably heard of Eagle or Billfish — local-first design asset managers that organize thousands of reference images, screenshots, and illustrations. Unlike cloud tools like Pinterest, they store everything on your local disk in a well-defined library format. That format is the key to integration: if a browser extension can write directly to Eagle's library structure, it can send images straight into your design workflow. Let's explore how these libraries are structured and what it takes to build a seamless integration. ## The Problem: Collect → Organize → Use The designer's image workflow has a hidden cost. When you find a reference image online, the journey to your organized library usually involves: download to Downloads folder → open Eagle → drag into a folder → tag it → rename it. That's five manual steps per image, and when you're collecting 50 images from a single site, it becomes a tedious chore. The solution is direct in...

Smart Image Classification in the Browser: A Layered Architecture from Rule-Based Heuristics to On-Device CNNs

 Designers, e-commerce sellers, and content teams share the same pain: you collect hundreds of images from a website, and they land in one disorganized folder. Product shots mixed with UI mockups, icons mixed with photos, 4K originals mixed with 32px thumbnails. Before you can do anything useful — dedupe, search, export — you need to classify. But what does "classify images" actually mean in a browser, where you have limited memory, no Python ecosystem, and a hard performance budget? Let's break down a layered classification architecture that works entirely on-device. ## The Problem: Why Naive Classification Fails The naive approach is to treat all images the same: ```javascript // Naive: everything is just an <img> document.querySelectorAll('img').forEach(img => {   saveToFolder('all-images', img.src); }); ``` This fails for three reasons: 1. **No dimensional awareness** — a 4K hero banner and a 32px favicon end up together 2. **No format awarene...

From WebP to PNG: How Browser Canvas Decodes and Converts Modern Image Formats

 Every image you see on the web is encoded in a specific format — JPEG, PNG, WebP, AVIF, GIF. Each format has its own compression algorithm, color space, and tradeoffs between quality and file size. But what happens when you need to convert a WebP image to PNG, or decode an AVIF file in the browser? The HTML Canvas API is the hidden workhorse that makes this possible. Let's dive deep into how browsers decode modern image formats and convert them using Canvas. ## The Image Format Landscape Before we get into the mechanics, let's understand why format conversion matters: - **JPEG**: Lossy compression, great for photos, no transparency support, small file sizes - **PNG**: Lossless compression, supports transparency, larger file sizes, ideal for screenshots and graphics - **WebP**: Google's modern format, supports both lossy and lossless, 25-35% smaller than JPEG/PNG, supports transparency and animation - **AVIF**: Next-gen format based on AV1 video codec, even better compressi...

Storing Thousands of Images Locally: IndexedDB, File System Access API, and the Privacy-First Web

 Every time you save an image from the web, your browser makes a copy — and that copy often lives on someone else's server. Between cloud sync, search engines, and image analysis services, your visual data is everywhere. But what if everything could stay on your device? Modern browsers now provide powerful local storage APIs that make true privacy-first web applications possible. Let's explore how IndexedDB and the File System Access API work, and why 100% local processing is the future of privacy. ## The Privacy Problem with Cloud Processing When you use a cloud-based image tool, your images typically take this journey: 1. Your browser uploads the image to a server 2. The server runs analysis (feature extraction, hashing, deduplication) 3. The server stores your image and its derived data 4. Your results come back over the network Every step creates privacy exposure: your images can be logged, used for model training, or breached. For photographers, designers, and businesses, ...