品翰陳

0 followers · 1 following · 829 views

on the atlas · 16

highlights · 30

ensures that the outputs for a certain position in a sequence is based only on the known outputs at previous positions
Understanding and Coding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs
causal self-attention mechanism is also often referred to as “masked self-attention”.
Understanding and Coding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs
Improves order fulfillment times
5 Ways to Use Artificial Intelligence (AI) in Logistics
Reduces overstocking and understocking
5 Ways to Use Artificial Intelligence (AI) in Logistics
The population of graduate students in ML changes. The financial opportunity costs of doing a PhD in AI went way up with the state of the industry.
The AI research job market shit show (and my experience)
Paper submissions at top ML conferences at least slow down
The AI research job market shit show (and my experience)
, while playing chess and solving problems in integral calculus turn out to be easy even for GOFAI, manual labor remains a major challenge even for today’s most sophisticated AIs.
Artificial General Intelligence Is Already Here
“modern Turing Test”: the ability to quickly make a million dollars online (from an initial $100,000 investment).
Artificial General Intelligence Is Already Here
“General intelligence” must be thought of in terms of a multidimensional scorecard, not a single yes/no proposition.
Artificial General Intelligence Is Already Here
joint training optimizes all parameters simultaneously by taking both the wide and deep part as well as the weights of their sum into account at training time.
Wide & Deep Learning for Recommender Systems | by Atul Anand | Analytics Vidhya | Medium
The wide component and deep component are combined using a weighted sum of their output log odds as the pre-diction, which is then fed to one common logistic loss function for joint training.
Wide & Deep Learning for Recommender Systems | by Atul Anand | Analytics Vidhya | Medium
difficult to learn effective low-dimensional representations for queries and items when the underlying query-item matrix is sparse and high-rank, such as users with specific preferences or niche items with a narrow appeal.
Wide & Deep Learning for Recommender Systems | by Atul Anand | Analytics Vidhya | Medium
Embedding-based models, such as factorization machines or deep neural networks, can generalize to previously unseen query-item feature pairs by learning a low-dimensional dense embedding
Wide & Deep Learning for Recommender Systems | by Atul Anand | Analytics Vidhya | Medium
We can only use the item ID and the user ID in the case of matrix factorization
Introduction To Recommender Systems- 2: Deep Neural Network Based Recommendation Systems | by Abhijit Roy | Towards Data Science
The matrix factorization also had the cold start problem
Introduction To Recommender Systems- 2: Deep Neural Network Based Recommendation Systems | by Abhijit Roy | Towards Data Science
Matrix factorization often tends to recommend popular items
Introduction To Recommender Systems- 2: Deep Neural Network Based Recommendation Systems | by Abhijit Roy | Towards Data Science
users taking life advice from AI tools could experience “diminished health and well-being” and a “loss of agency,”
Google reportedly building A.I. that offers life advice
users taking life advice from AI tools could experience “diminished health and well-being” and a “loss of agency,”
Google reportedly building A.I. that offers life advice
few subterms property—for a class of CNF formulas to have polynomial OBDD size, and show that variable convex formulas satisfy this condition.
On Compiling Structured CNFs to OBDDs - PMC
Logistic regression is a binary classification method. It can be modelled as a function that can take in any number of inputs and constrain the output to be between 0 and 1. This means, we can think of Logistic Regression as a one-layer neural network.
Logistic regression as a neural network - DataScienceCentral.com
As previously stated, L2 regularization only shrinks the weights to values close to 0, rather than actually being 0. On the other hand, L1 regularization shrinks the values to 0.
Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? - neptune.ai
L1 regularization is computationally more expensive, because it cannot be solved in terms of matrix math.
Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? - neptune.ai
L1 regularization uses Manhattan distances to arrive at a single point, so there are many routes that can be taken to arrive at a point. L2 regularization uses Euclidean distances, which will tell you the fastest way to get to a point.
Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? - neptune.ai
L1 regularization is more robust than L2 regularization for a fairly obvious reason. L2 regularization takes the square of the weights, so the cost of outliers present in the data increases exponentially. L1 regularization takes the absolute values of the weights, so the cost only increases linearly.
Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? - neptune.ai
L2 regularization, or the L2 norm, or Ridge (in regression problems), combats overfitting by forcing weights to be small, but not making them exactly 0.
Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? - neptune.ai
Essentially, when we use L1 regularization, we are penalizing the absolute value of the weights.
Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? - neptune.ai
The algorithm will continue to make steps towards the global minimum of a convex function and the local minimum as long as the number of iterations (n_iters) are sufficient enough for gradient descent to reach the global minimum.
Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? - neptune.ai
We use regularization because we want to add some bias into our model to prevent it overfitting to our training data.
Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? - neptune.ai
simplify the model is by adding bias into the model with regularization
Fighting Overfitting With L1 or L2 Regularization: Which One Is Better? - neptune.ai
“Regularization is any modification we make to a learning algorithm that is intended to reduce its generalization error but not its training error.”
Understanding Regularization for Logistic Regression | KNIME

品翰 陳

on the atlas · 16

highlights · 30

品翰陳