Lucas Gelfond • Segment Anything 2, in WebGPU

I was talking with Diego Rodriguez a few weeks ago, who encouraged me to hack together a Segment Anything 2 implementation fulling using WebGPU. That's this! It runs completely client-side, no data is sent to the server! You can read the source code here or try a demo at http://webgpu-sam2.lucasgelfond.online/. I've been curious about some of the lower-level browser APIs (WebAssembly and WebGPU) since I started doing frontend, but haven't gotten much of a chance to play with them! The new model seemed like a great excuse to learn. I started off looking a bit into the model, and trying to find some examples of people who'd gotten things working. Segment Anything 2 only came out a week or so ago, so most of the examples were for SAM-1. In web-sam and SAM-in-Browser others had used a small model called MobileSAM to do inference in the browser; it certianly seemed possible! The common approach to running models in the browser is to use Transformers.js, essentially a JS port of the famous H

saved by

• jadden picardal