Showcase
Archive
Gemini 4 Neon3D Worlds & Ecosystems2026-11-20

Lumina Canvas — 3D Relighting Vermeer

60 fps 3D Candlelight Relighting Inside Vermeer’s 1665 Girl with a Pearl Earring · Single JPEG

01 / The Story & Idea

Move a live 3D beeswax candle across Johannes Vermeer’s 1665 Girl with a Pearl Earring — built from a single flat 643 KB museum scan with zero AI video generation, zero looping MP4s, and zero audio files. Sweep the flame to watch real directional shadows flip across her nose and warm up her shadowed cheek, snuff the candle into curling 3D smoke and silver moonlight, catch her softly blinking, or drag an antique brass magnifying glass over her portrait to X-ray the live AI depth and 3D surface normals underneath.

02 / How It Was Built

  1. 01

    Local AI Estimates 3D Shape

    Depth-Anything-V2-Large and u2net run offline on Apple Silicon to map how far her nose, cheeks, brow, pearl, and turban stick out while separating her cleanly from the dark canvas void.

  2. 02

    Python Builds Four Physical Maps

    Instead of a fake bump filter, Python smoothly lifts Vermeer’s painted shadows into clean base pigments, computes 3-scale surface normals with 1665 oil craquelure, and separates glossy pearl/eyes from velvety skin and matte silk.

  3. 03

    Real-Time 32-Step Cast Shadows

    At 60 fps in Three.js WebGL2, GPU shaders march 32 shadow rays from every pixel across the AI depth map toward your floating candle flame — flipping cast shadows cleanly as you move the light.

  4. 04

    Living Fire, Brass Loupe & Ritual Audio

    Cursor sweeps bend the beeswax flame with trailing embers; clicking Candle snuffs it into curling smoke and moonlight (Relight strikes a match), paired with a refractive brass magnifying glass, split-compare curtain, and synthesized Web Audio.

Visual Pipeline Proof · Click Any Frame to Inspect6 Stages
Three.js r187WebGL2GLSLDepth-Anything-V2-Large (Local MPS)OpenCV / SciPyWeb Audio API

03 / Under the Hood

Deep Engineering Specs, Shaders & Architecture (6 technical notes)

Engineered end-to-end with Gemini 4 Neon from a single flat 1665 Mauritshuis museum scan with zero AI video generation and zero external audio files: (1) local Apple Silicon MPS Depth-Anything-V2-Large monocular 3D depth estimation paired with offline u2net portrait matting refined at native 1024×1200 via guided color-distance filtering; (2) automated Python OpenCV/SciPy bilateral Retinex intrinsic delighting (lifting Vermeer’s right cheek cleanly with zero shadow noise amplification and healing baked pearl spots), 3-scale RNM tangent-space surface normals with genuine 1665 oil craquelure, and opaque PBR roughness/SSS maps; (3) Three.js r187 WebGL2 192×224 displaced 3D relief canvas running 32-step Interleaved Gradient Noise horizon raymarched shadows, Cook-Torrance GGX specular, porcelain skin SSS, damped saccadic eye-gaze & painterly blinking, aerodynamic beeswax flame physics with trailing embers and curling 3D snuff smoke, a refractive antique-brass Magnifying Loupe, and a bit-exact Before/After Split-Compare curtain; and (4) 100% procedural spatial Web Audio (candle flutter, micro-crackles, swipe air whoosh, breath-snuff hiss, phosphorus match-strike, and museum ambience).

  1. 01Single-Image Local AI & Intrinsic Vision Pipeline: Starts from one flat Mauritshuis museum scan at zero cloud cost; runs Depth-Anything-V2-Large on Apple Silicon MPS and offline portrait matting refined at native 1024×1200 via guided color-distance filtering, then synthesizes base/detail-separated shadow-lifted intrinsic albedo (lifting Vermeer's right cheek cleanly with zero shadow noise amplification), 3-scale tangent-space normals (macro portrait form, soft pearl relief, and genuine 1665 oil craquelure across both subject and canvas), and opaque PBR surface maps separating glossy pearl/eyes/lips from velvety porcelain skin and matte ultramarine/ochre turban cloth.
  2. 02True 3D Raymarched Shadows & Living Portrait Presence: A 192×224 displaced 3D relief canvas (1.7067 × 2.000 world units with catch-lit museum stretcher bevel sides) runs 32-step Interleaved Gradient Noise horizon raymarched cast shadows so sweeping the 3D flame flips directional shadows across her nose bridge, cheeks, chin, and turban folds (contrastDelta = 98.8, polarityFlip = +5186.3), paired with Cook-Torrance GGX specular highlights (4.04× pearl-to-matte ratio), warm porcelain skin subsurface scattering, damped saccadic eye-gaze tracking with pupil light-reflex, an organic 180 ms painterly eyelid blink, and imperceptible 0.16 Hz breathing relief.
  3. 03Living Candle Ritual & Aerodynamic Fire Physics: Sweeping the cursor bends the double-layer 3D beeswax flame backward into the slipstream with 2nd-order spring inertia, trails cooling world-space golden embers, and swirls nearby museum dust motes; clicking Candle (or dragging the inline blackbody dimmer from 150% down to 0%, or pressing L) snuffs the flame into a dying crimson wick ember, a backlit curling 3D beeswax smoke ribbon, and hushed silver-blue Mauritshuis moonlight, while Relight triggers a crisp 120 ms golden match-strike flare.
  4. 04Conservator's Brass Magnifying Loupe & Before/After Split-Compare: Toggling Loupe (O key) glides a curved, candle-reactive antique-brass magnifying glass directly over the live candlelit painting with optical barrel refraction to inspect 3D Normals, AI Depth, Cast Shadows, 2.2× Macro Craquelure, or the 1665 Flat Original inside the lens (click canvas to cycle modes); Compare (C key) opens a draggable vertical split-curtain comparing the bit-exact original 1665 scan (leftRgbCorr = 1.00000, leftMAE = 0.00) directly against the live 3D relit painting.
  5. 05Zero-Asset Spatial Procedural Web Audio: Synthesizes stereo-panned beeswax flame flutter and micro-crackles coupled to light position and swipe air-whoosh turbulence, filtered breath-snuff and hot-wick hiss on candle extinguish, phosphorus match-strike and warm low-end bloom on ignition, hushed museum room ambience, and muted 420 Hz brass/wood tactile UI taps with zero external audio files.
  6. 06Museum-Minimal Smoked-Glass Controls: Bottom-center Candle Ritual capsule (Candle / Relight toggle, inline 0–150% blackbody dimmer, 1-tap 3D relief tilt to 18°, and 1-tap Loupe + inline lens mode chip) paired with a top-right Girl with a Pearl Earring badge and expandable Controls & 6-Mode AI/PBR Inspector drawer (D key), preserving >= 88% uninterrupted artwork on desktop and unclipped 375×812 mobile ergonomics (scrollWidth <= 375, zero drawer clipping) with shortcuts L Snuff/Relight, O Loupe, C Compare, D Inspector, M Audio, Space Pause, H Hide HUD, wheel light-distance Z, and right-drag 3D canvas tilt.

04 / Element-by-Element Build Breakdown

Single-image 3D relighting — Local AI depth & matting, Python intrinsic albedo/normals, 32-step GPU shadows & spatial Web Audio

Scene ElementPipeline / ToolingHow It Was Built
Source Painting & Local AI Depth Map
Local Depth-Anything-V2-Large (Apple Silicon MPS) + Guided FilterOffline monocular 3D depth predicted from one Public Domain 1230×1440 Vermeer scan, smooth-blended with sculpted glabella/nose-bridge relief and gentle feathered pearl height for staircase-free raymarching
Native 1024px Portrait Silhouette Matte
Local u2net + OpenCV Guided Color-Distance MattingFull-resolution 1024×1200 foreground silhouette matte guided directly by museum brushwork with 1.35 px sub-pixel anti-aliasing for silky contour sfumato against the dark oil backdrop
Shadow-Lifted Intrinsic Albedo
Python Bilateral Base/Detail Retinex + Skin-Domain EqualizationSeparates smooth base illumination from fine canvas/cloth texture so dark-side shadow lifting recovers Vermeer's right cheek monotonically (troughDip = 0.5) without amplifying cloth noise (jacketHfStd = 4.15), while healing baked pearl spots into warm ivory nacre
3-Scale Surface Normals & PBR Materials
Python RNM Normal Fusion + Semantic PBR ExtractionFuses macro depth curvature, feathered dielectric pearl cushion, and authentic 1665 face & background craquelure into tangent-space normals alongside opaque A=255 roughness, dielectric metallic, and warm skin SSS maps plus extensible manifest anchors
Displaced 3D Relief Canvas & Stretcher
Runtime Parametric Three.js r187192×224 subdivided canvas plane displaced along +Z by AI depth (0.14 relief pinned smoothly at edges) mounted on a dark matte museum stretcher with catch-lit linen/wood side bevels visible at 18° oblique 3D tilt
32-Step Raymarched PBR Relighting & Living Presence
Custom GLSL (WebGL2)Horizon depth-map soft cast shadows with IGN jitter, Cook-Torrance GGX specular, warm terminator skin SSS, localized antique oil-canvas candle patina, max-channel hue-preserving highlight shoulder, damped saccadic eye gaze, pupil light-reflex, 180 ms painterly blink, and subtle torso breath
Conservator's Brass Magnifying Loupe & Split-Compare
Single-Pass Optical GLSL Post-OverlayRefractive circular glass lens (1.28× diagnostic / 2.15× macro craquelure) framed by a candle-reactive antique-brass bezel cycling live Normals, AI Depth, Shadows, Macro Craquelure, and 1665 Original, alongside a bit-exact draggable Before/After split curtain
Aerodynamic 3D Candle Flame, Snuff Smoke & Embers
Deterministic Kinematics + Custom GLSL ParticlesSculpted double-layer beeswax teardrop flame bending with 2nd-order cursor velocity inertia, blackbody Kelvin dimmer (1450K–2250K), 16-slot world-space trailing golden embers, dying crimson wick ember, backlit curling 3D smoke ribbon, silver-blue nocturnal moonlight, and dark museum wall aura
Spatial Ritual Web Audio Engine
Zero-Asset Web Audio API SynthesisStereo-panned beeswax flame flutter, micro-crackles, swipe air-whoosh turbulence, breath-snuff + hot-wick hiss, phosphorus match-strike + low-end ignition bloom, hushed museum room ambience, and 420 Hz brass/wood tactile clicks
Candle Ritual HUD & 6-Mode AI/PBR Inspector
Responsive TypeScript UI + CSS Smoked GlassUltra-minimal bottom Candle Ritual capsule (Candle / Relight, inline golden dimmer, 3D, Loupe), top-right painting badge, auto-returning Controls & 6-Mode Inspector drawer, draggable Split-Compare handle, and unclipped 375×812 mobile bottom sheet

More Built With Gemini 4 Neon

View Full Archive →

3D Drops & AI Briefing

Don’t miss the next build

Reserve early access to Ashutosh’s 5-minute AI briefing and new WebGPU 3D drops.

Zero spam · Privacy

! Please enter a valid email address (for example, you@domain.com).

Morning Edition Alerts

Get a 1-click alert at 11:00 IST

One quiet browser alert when Ashutosh’s daily AI briefing or Sunday synthesis goes live. No email required. Privacy