Eliezer de Souza da Silva

1 followers · 6 following · 840 views

on the atlas — 15

highlights — 13

Variational Diffusion Models Obtains SotA likelihoods on image density estimation benchmarks, outperforming autoregressive models that have dominated these benchmarks for many years, with often significantly faster optimization. https://arxiv.org/abs/2107.00630
ZoubinGhahrama1 · Twitter · @roydanroy @arankomatsuzaki @dpkingma @TimSalimans @poolio @GoogleAI C...
Neural differential equations made easy: from torchdyn import NeuralODE # your preferred torch.nn.Module here f = nn.Sequential(nn.Conv2d(1, 32, 3), nn.Softplus(), nn.Conv2d(32, 1, 3) ) nde = NeuralODE(f)
DiffEqML/torchdyn: A PyTorch based library for all things neural differential equations
DiffMan is a MATLAB toolbox for solving Ordinary Differential Equations on manifolds, based on the concept of ’Coordinate Free Numerics’.
kenthe/DiffMan: DiffMan is a MATLAB toolbox for solving Ordinary Differential Equations on manifolds, based on the concept of ’Coordinate Free Numerics’.
In this work, we highlight an intrinsic correspondence between a simple family of autoregressive normalizing flows and identifiable causal models. We exploit the fact that autoregressive flow architectures define an ordering over variables, analogous to a causal ordering, to show that they are well-suited to performing a range of causal inference tasks, ranging from causal discovery to making interventional and counterfactual predictions. First, we show that causal models derived from both affine and additive autoregressive flows with fixed orderings over variables are identifiable, i.e. the t…
[2011.02268] Causal Autoregressive Flows
Unbiased implicit gradient estimator 1) UIVI - Unbiased Implicit Variational Inference Using semi-implicit family distributions parameterized by 𝜃 we can re-work the ELBO to be expressed as 𝐿(𝜃)=E𝑞(𝑢)𝑞(𝜖)[{log𝑝(𝑥,𝑧)−log𝑞𝜃(𝑧)}|𝑧=ℎ𝜃(𝑢,𝜖)] The way to understand this loss is that we sample 𝜖 and 𝑢 from a simple distribution, pass them through the parameterized NN to generate 𝑧 and evaluate the log-likelihood of your model and of the variational proposal distribution. If we compute the gradients with respect to 𝜃 for the ∇𝜃𝐿(𝜃)=E𝑞(𝑢)𝑞(𝜖)[{∇𝑧log𝑝(𝑥,𝑧)−∇𝑧log𝑞𝜃(𝑧)}|𝑧=ℎ𝜃(𝑢,𝜖)∇𝜃ℎ(𝑢,𝜖)] which we can conc…
implicit_vi.ipynb - Colaboratory
DEEN technique explained in [2] There is some evidence that DEEN is better than DAE in capturing the log-prob of the data, for example in [5] both methods are compared in the context of regularizing trajectories for model-based RL, and DEEN is shown to better model the log-prob. Sliced-score matching as explained in [1]: Seems to rely on trace estimator for the hessian
dae-score-energy-generative-models.ipynb - Colaboratory
4.3. Uncertainty In practice, of course, we do not know with any certainty what the consequences of our actions will be, especially over the long term. Again, from the perspective of determining the objectively morally correct action, one might argue that all that matters is the (unknown) probability according to the universe. For individual decision makers, however, any person's ability to predict the future will be limited, and, indeed, will likely vary across individuals. In other words, it is not just our uncertainty about consequences that is a problem, but our uncertainty about our uncer…
Frontiers | On Consequentialism and Fairness | Artificial Intelligence
Debugging a pipeline¶ river encourages users to make use of pipelines. The biggest pain point of pipelines is that it can be hard to understand what's happening to the data, especially when the pipeline is complex. Fortunately the Pipeline class has a debug_one method that can help out. Let's look at a fairly complex pipeline for predicting the number of bikes in 5 bike stations from the city of Toulouse. It doesn't matter if you understand the pipeline or not; the point of this notebook is to learn how to introspect a pipeline.
Debugging a pipeline - River
1.3 Relation to Up(Down)-Sampling Below we assume that the elements of x are iid given z , p ( x j z ) = Y i =1 d p i ( x i j z ) : Under this assumption, the bound ( 1) becomes F ( x ) = ¡ X i E q ( z j x ) [ ln p i ( x i j z )] + KL ( q ( z j x ) k p ( z )) : (4) Now imagine that we preprocess the data by replicating every element of x n times, x i ! ( x i ;:::; x i ) , while also replicating the densities p i ( x i j z ) . So that, p i (( x i ;:::;x i ) j z ) = p i ( x i j z ) n . The corresponding variational bound F n ( x ) for this modied problem is now given by F n ( x ) = ¡ n X i E q …
RescaledKL.tm
Some of these inequalities introduce new variational parameters. Those should be optimized jointly with all the other parameters to minimize the ELBO.
Useful Inequalities for Variational Inference | Invariance
3. Conclusion In these notes we have discussed divergences, metrics, measures of relative information between two probability densities and how to build coordianate-invariant divergences. As observed in section 1.1, the main reason for introducing the concept of relative information is to consistenly quantify surprise and updates in belief states. If our goal is only to optimize the parameters of a density Q so that it approaches another density P , it is sucient to minimize any divergence D ( Q ; P ) with respect to the parameters of Q . On practice, dierent divergences will emphasize dier…
Short Notes on Divergence Measures
Invariance under a change of measure or coordinate system is a fundamental concept both in information theory and physics. It is widely accepted in both elds that the coordinate system employed to express concepts carries no information. This brings us back to the axiom (ii) of previous section, illustrated in Figure (1). Figure 1. Diagram relating p ( x ) and q ( x ) with their twins p ~ ( x ~ ) and q ~ ( x ~ ) in a dierent coordinate system induced by an invertible smooth map ( x ) . The principle of invariance under change of measures postulates that D ( q ; p ) = D ( q ~ ; p ~ ) for an…
Short Notes on Divergence Measures
Highlight me with your cursor
Curius / Onboarding