品翰陳

0 followers · 645 views

on the atlas · 13

highlights · 48

demand-and-supply economics will prevail.
The Rise of the AI Engineer - by swyx - Latent Space
Prompt Engineering and AI Engineering will feel inferior to people with good Data Science/ML backgrounds for a long while.
The Rise of the AI Engineer - by swyx - Latent Space
dequantize them to full precision before any reduction operation
DeepSpeed ZeRO++: A leap in speed for LLM and chat model training with 4X less communication - Microsoft Research
quantize gradients before communication
DeepSpeed ZeRO++: A leap in speed for LLM and chat model training with 4X less communication - Microsoft Research
quantization on weights to shrink down each model parameter on the fly from FP16 (two bytes) to INT8 (one byte) data type before communicating
DeepSpeed ZeRO++: A leap in speed for LLM and chat model training with 4X less communication - Microsoft Research
AlphaDev doesn’t choose moves; instead, it chooses instructions to add to a procedure (in what DeepMind engineers call AssemblyGame).
DeepMind AI creates algorithms that sort data faster than those built by people
Against a human expert, Circuit Training frequently lost. Versus simulated annealing, the contest was a bit more even.
Ending an Ugly Chapter in Chip Design
The benchmarks when Google began its work were already about two decades old, and their relevance to modern chips is debated.
Ending an Ugly Chapter in Chip Design
“One reason that others have not been able to use shiny objects in this fashion is that most prior works require surfaces with known geometry or texture. The authors have derived an intriguing, new formulation that does not require such knowledge.”
Using reflections to see the world from new points of view | MIT News | Massachusetts Institute of Technology
ORCa models the scene as a 5D radiance field, which captures additional information about the intensity and direction of light rays that emanate from and strike each point in the scene.
Using reflections to see the world from new points of view | MIT News | Massachusetts Institute of Technology
ORCa uses machine learning to convert the surface of the object into a virtual sensor that captures light and reflections that strike each virtual pixel on the object’s surface. Finally, the system uses virtual pixels on the object’s surface to model the 3D environment from the point of view of the object.
Using reflections to see the world from new points of view | MIT News | Massachusetts Institute of Technology
ORCa (which stands for Objects as Radiance-Field Cameras)
Using reflections to see the world from new points of view | MIT News | Massachusetts Institute of Technology
“In real life, exploiting these reflections is not as easy as just pushing an enhance button. Getting useful information out of these reflections is pretty hard because reflections give us a distorted view of the world,”
Using reflections to see the world from new points of view | MIT News | Massachusetts Institute of Technology
This method could be especially useful in autonomous vehicles. For instance, it could enable a self-driving car to use reflections from objects it passes, like lamp posts or buildings, to see around a parked truck.
Using reflections to see the world from new points of view | MIT News | Massachusetts Institute of Technology
converts the surface of that object into a virtual sensor which captures reflections
Using reflections to see the world from new points of view | MIT News | Massachusetts Institute of Technology
researchers from MIT and Rice University
Using reflections to see the world from new points of view | MIT News | Massachusetts Institute of Technology
It would requiere sentences of all accepted positions during the training loop but positional encoding allow the model to extrapolate to sequence lengths longer than the ones encountered during training
Attention is all you need: Discovering the Transformer paper | by Eduardo Muñoz | Towards Data Science
The scores measure how much focus to place on other places or words of the input sequence w.r.t a word at a certain position.
Attention is all you need: Discovering the Transformer paper | by Eduardo Muñoz | Towards Data Science
elf-attention mechanism: the Query, the Key and the Value
Attention is all you need: Discovering the Transformer paper | by Eduardo Muñoz | Towards Data Science
Both contains a core block of “an attention and a feed-forward network” repeated N times.
Attention is all you need: Discovering the Transformer paper | by Eduardo Muñoz | Towards Data Science
each sequence must be treated one element at a time.
Attention is all you need: Discovering the Transformer paper | by Eduardo Muñoz | Towards Data Science
when dealing with huge corpus it is very time consuming and computationally inefficient.
Attention is all you need: Discovering the Transformer paper | by Eduardo Muñoz | Towards Data Science
allows the decoder to assign greater weight or importance to a certain element of the input for each element of the output.
Attention is all you need: Discovering the Transformer paper | by Eduardo Muñoz | Towards Data Science
This is what attention does, it extracts information from the whole sequence, a weighted sum of all the past encoder states
Attention is all you need: Discovering the Transformer paper | by Eduardo Muñoz | Towards Data Science
OpenAI still has targets to meet. It has until September 30th to create a harder age-gate to keep out minors under 13 and require parental consent for older underage teens. If it fails, it could see itself blocked again. But it’s provided an example of what Europe considers acceptable behavior for an AI company — at least until new laws are on the books.
OpenAI’s regulatory troubles are just beginning - The Verge
After the explosion of large language models like ChatGPT last year, lawmakers are now racing to add rules for “foundation models” and “General Purpose AI Systems (GPAIs)” — two terms for large-scale AI systems that include LLMs — and potentially classing them as “high risk” services.
OpenAI’s regulatory troubles are just beginning - The Verge
Australian mayor has threatened to sue OpenAI for defamation after ChatGPT falsely claimed he had served time in prison for bribery.
OpenAI’s regulatory troubles are just beginning - The Verge
Companies like Samsung and JPMorgan have banned employees from using generative AI tools over fears they’ll upload sensitive data
OpenAI’s regulatory troubles are just beginning - The Verge
OpenAI was breaking GDPR: allowing ChatGPT to provide inaccurate or misleading information, failing to notify users of its data collection practices, failing to meet any of the six possible legal justifications for processing personal data, and failing to adequately prevent children under 13 years old using the service.
OpenAI’s regulatory troubles are just beginning - The Verge
In the EU, they’re applying the General Data Protection Regulation (GDPR)
OpenAI’s regulatory troubles are just beginning - The Verge
If a action led to a scenario where we are better off than before (positive advantage), the weights are change in the gradients direction, effectively making it more likely that that action is chosen again. If it’s worse, we decrease the weights in the direction of the gradient, making it less lightly
A Graphic Guide to Implementing PPO for Atari Games | by DarylRodrigo | Towards Data Science
to make that action (in a specific state) more likely by changing the neural networks weights in the direction of the gradient
A Graphic Guide to Implementing PPO for Atari Games | by DarylRodrigo | Towards Data Science
To select an action, the agent samples from the distribution of probabilities given by the neural network (called logits) — this action is then used to provide the environment with its next input
A Graphic Guide to Implementing PPO for Atari Games | by DarylRodrigo | Towards Data Science
for a discrete action space, output a probability for each possible action
A Graphic Guide to Implementing PPO for Atari Games | by DarylRodrigo | Towards Data Science
Q-learning methods only indirectly optimize for agent performance
Part 2: Kinds of RL Algorithms — Spinning Up documentation
model-free methods forego the potential gains in sample efficiency from using a model, they tend to be easier to implement and tune
Part 2: Kinds of RL Algorithms — Spinning Up documentation
it allows the agent to plan by thinking ahead, seeing what would happen for a range of possible choices, and explicitly deciding between its options.
Part 2: Kinds of RL Algorithms — Spinning Up documentation
we don’t need to describe how good an action is in an absolute sense, but only how much better it is than others on average.
Part 1: Key Concepts in RL — Spinning Up documentation
always has a neural network that maps from observations to mean actions,
Part 1: Key Concepts in RL — Spinning Up documentation
A diagonal Gaussian distribution is a special case where the covariance matrix only has entries on the diagonal
Part 1: Key Concepts in RL — Spinning Up documentation
halo/obstruction is the placement blockage defined for the standard cells across the boundary of macros
Floorplanning: concept, challenges, and closure - EDN
a hard constraint specifying that only the design module can be placed inside the physical boundary of fence
Floorplanning: concept, challenges, and closure - EDN
region is a hard constraint in the design
Floorplanning: concept, challenges, and closure - EDN
A module guide is the guided placement of a logical module structure in the design. The guide is a soft constraint
Floorplanning: concept, challenges, and closure - EDN
the area defined for standard-cell placement in the design
Floorplanning: concept, challenges, and closure - EDN
Track is a virtual guideline/path for the tool at which the signal routing happens in an SOC design
Floorplanning: concept, challenges, and closure - EDN
macro/block placement, partitioning, I/O-pad placement, and power planning directly or indirectly impact the overall implementation cycle.
Floorplanning: concept, challenges, and closure - EDN
typical SOC can include many hard- and soft-IP macros, memories, analog blocks, and multiple power domains
Floorplanning: concept, challenges, and closure - EDN

品翰 陳

on the atlas · 13

highlights · 48

品翰陳