flâneur

Why Care About Natural Latents? — LessWrong

lesswrong.com · 1,701 words · saved by 1 readers

Suppose Alice and Bob are two Bayesian agents in the same environment. They both basically understand how their environment works, so they generally agree on predictions about any specific directly-observable thing in the world - e.g. whenever they try to operationalize a bet, they find that their odds are roughly the same. However, their two world models might have totally different internal structure, different “latent” structures which Alice and Bob model as generating the observable world around them. As a simple toy example: maybe Alice models a bunch of numbers as having been generated by independent rolls of the same biased die, and Bob models the same numbers using some big complicated neural net. Now suppose Alice goes poking around inside of her world model, and somewhere in there she finds a latent variable Λ A with two properties (the Natural Latent properties): In the die/net case, the die’s bias ( Λ A ) approximately mediates between e.g. the first 100 numbers ( X 1 ) a

x Why Care About Natural Latents? — LessWrong AI Frontpage 56 Why Care About Natural Latents? by johnswentworth , David Lorell 9th May 2024 5 min read 3 56 Suppose Alice and Bob are two Bayesian agents in the same environment. They both basically understand how their environment works, so they generally agree on predictions about any specific directly-observable thing in the world - e.g. whenever they try to operationalize a bet, they find that their odds are roughly the same. However, their two world models might have totally different internal structure, different “latent” structures which A

related reading