Lily Wang

0 followers · 250 views

on the atlas — 7

highlights — 23

If language is an abstraction of the world and pixels are a projection of it, then geometry, physics, and dynamics are the world itself.
A Functional Taxonomy of World Models - Dr. Fei-Fei Li
The categories are not, however, fundamentally separate. The same underlying knowledge of how the world works—geometry, physics, dynamics—sits beneath all of them. A model that can render a cup from any angle ought, in principle, to be able to simulate what happens when the cup is pushed and plan a hand to pick the cup up. Increasingly, the most interesting research deliberately blurs the boundaries between the three.
A Functional Taxonomy of World Models - Dr. Fei-Fei Li
This is the physicist’s and roboticist’s state: a complete description of what is happening in the world at a given moment, including every object, every position, every velocity, every property. State is the underlying reality of the world; complete in principle, but never directly visible to any agent inside it. Observations are an agent’s partial view of that reality. Actions are what the agent does in response.
A Functional Taxonomy of World Models - Dr. Fei-Fei Li
PyTorch does eager-mode (interpreted) execution, which sets it apart from other graph-mode (compiled) ML frameworks like TensorFlow
UW PLSE | How does torch.compile work?
The motors are fairly overpowered relative to loading. The mechanism’s position (not velocity) is being controlled. There are no substantial or varying external forces on the mechanism.
Picking a Control Strategy — FIRST Robotics Competition documentation
The motors are fairly overpowered relative to loading. The mechanism’s position (not velocity) is being controlled. There are no substantial or varying external forces on the mechanism.
Picking a Control Strategy — FIRST Robotics Competition documentation
In other situations, it is necessary to use a feedforward model to reduce the amount of work done by the feedback controller.
Picking a Control Strategy — FIRST Robotics Competition documentation
We are controlling the position of the system, so errors accumulate over time There are a lot of difficult-to-dynamic external forces interacting with the mechanism that the feedforward loop cannot account for (e.g. a flywheel that is launching game pieces).
Picking a Control Strategy — FIRST Robotics Competition documentation
only yields acceptable results for velocity control of mechanisms with little external load. In other cases, errors from the system model will be unavoidable and a feedback controller will be necessary to correct for them.
Picking a Control Strategy — FIRST Robotics Competition documentation
we cannot use a deterministic RNN as it would always decode the same hidden state to the same observation and would therefore not allow to properly model a distribution over states. When the latent states are stochastic, we need to use variational inference to generate future (stochastic) observations
Training agents to plan in latent space — a technical overview | by Lukas Bierling | Medium
The observation model will not be used in the planning process and its only purpose is to provide a rich training signal for the transition model.
Training agents to plan in latent space — a technical overview | by Lukas Bierling | Medium
In most cases the distribution is a Gaussian. For example, the transition model learn mean and variance of the Gaussian
Training agents to plan in latent space — a technical overview | by Lukas Bierling | Medium
By reducing the reliance on raw observations, the agent avoids the risk of overfitting to noisy data and is better equipped to generalize to unseen scenarios.
Training agents to plan in latent space — a technical overview | by Lukas Bierling | Medium
Since the dynamics model operates in the latent space rather than directly on raw observations, its errors are less likely to propagate severely or lead to unrealistic trajectories. This abstraction also simplifies the dynamics model’s learning task, as it no longer needs to account for irrelevant details in the raw observations.
Training agents to plan in latent space — a technical overview | by Lukas Bierling | Medium
We initialize the matrices of the RNN with random numbers and the bulk of work during training goes into finding the matrices that give rise to desirable behavior, as measured with some loss function that expresses your preference to what kinds of outputs y you’d like to see in response to your input sequences x.
The Unreasonable Effectiveness of Recurrent Neural Networks
This RNN’s parameters are the three matrices W_hh, W_xh, W_hy. The hidden state self.h is initialized with the zero vector. The np.tanh function implements a non-linearity that squashes the activations to the range [-1, 1]. Notice briefly how this works: There are two terms inside of the tanh: one is based on the previous hidden state and one is based on the current input. In numpy np.dot is matrix multiplication. The two intermediates interact with addition, and then get squashed by the tanh into the new state vector. If you’re more comfortable with math notation, we can also write the hidden…
The Unreasonable Effectiveness of Recurrent Neural Networks
The LSTM is a particular type of recurrent network that works slightly better in practice, owing to its more powerful update equation and some appealing backpropagation dynamics.
The Unreasonable Effectiveness of Recurrent Neural Networks
Since the RNN consists entirely of differentiable operations we can run the backpropagation algorithm (this is just a recursive application of the chain rule from calculus) to figure out in what direction we should adjust every one of its weights to increase the scores of the correct targets (green bold numbers). We can then perform a parameter update, which nudges every weight a tiny amount in this gradient direction.
The Unreasonable Effectiveness of Recurrent Neural Networks
At test time, we feed a character into the RNN and get a distribution over what characters are likely to come next. We sample from this distribution, and feed it right back in to get the next letter. Repeat this process and you’re sampling text! Lets now train an RNN on different datasets and see what happens.
The Unreasonable Effectiveness of Recurrent Neural Networks
Notice also that the first time the character “l” is input, the target is “l”, but the second time the target is “o”. The RNN therefore cannot rely on the input alone and must use its recurrent connection to keep track of the context to achieve this task.
The Unreasonable Effectiveness of Recurrent Neural Networks
We just trained the LSTM on raw data and it decided that this is a useful quantitity to keep track of. In other words one of its cells gradually tuned itself during training to become a quote detection cell, since this helps it better perform the final task. This is one of the cleanest and most compelling examples of where the power in Deep Learning models (and more generally end-to-end training) is coming from.
The Unreasonable Effectiveness of Recurrent Neural Networks
REINFORCE learning rule that is a special case of policy gradient methods in Reinforcement Learning, which allows one to train models that perform non-differentiable computation (taking glances around the image in this case
The Unreasonable Effectiveness of Recurrent Neural Networks
One problem is that RNNs are not inductive: They memorize sequences extremely well, but they don’t necessarily always show convincing signs of generalizing in the correct way (I’ll provide pointers in a bit that make this more concrete). A second issue is they unnecessarily couple their representation size to the amount of computation per step. For instance, if you double the size of the hidden state vector you’d quadruple the amount of FLOPS at each step due to the matrix multiplication.
The Unreasonable Effectiveness of Recurrent Neural Networks