Confidential AI From GPU Enclaves

blog.blyss.dev · 2,909 words · saved by 1 readers

Blyss offers a verifiably confidential LLM API at enclave.blyss.dev using special hardware features of the latest NVIDIA GPUs and AMD CPUs. Our goal is to make it impossible for interactions with the model to be monitored by anyone, including us, and to make that claim publicly verifiable. This is a deep-dive on our security model and technology. We start with some background, then describe exactly how we built our service and why we think it’s trustworthy. To trust a computer, we must know exactly what code it is running, and on exactly what hardware. We will derive all later claims of trustworthiness from these two facts, hence the name “roots of trust”. We rely on two hardware-based technologies to establish these roots: AMD SEV-SNP, present on Zen 3 or later CPUs; and the NVIDIA’s “Confidential Computing” mode, currently exclusive to the H100 GPU. SEV-SNP (“Secure Encrypted Virtualization” and “Secure Nested Paging”) is a CPU feature from AMD that lets CPUs launch isolated and encr

Hardware roots of trust To trust a computer, we must know exactly what code it is running, and on exactly what hardware. We will derive all later claims of trustworthiness from these two facts, hence the name "roots of trust". We rely on two hardware-based technologies to establish these roots: AMD SEV-SNP, present on Zen 3 or later CPUs; and the NVIDIA's "Confidential Computing" mode, currently exclusive to the H100 GPU. AMD SEV-SNP SEV-SNP ("Secure Encrypted Virtualization" and "Secure Nested Paging") is a CPU feature from AMD that lets CPUs launch isolated and encrypted virtual machines tha

Confidential AI From GPU Enclaves

related reading