Adversarial Reprogramming of Neural Cellular Automata
Reprogramming Neural CA to exhibit novel behaviour, using adversarial attacks.
Last call: Stefan Krawcyzk’s ‘Mastering MLOps’ Live Cohort
Tweet Tweet Share Share Last Updated on August 19, 2022 Sponsored Post This is your last chance to sign up for Stefan Krawczyk’s exclusive live cohort, starting next week (August 22nd). We already have students enrolled from Apple, Amazon, Spotify, Nubank, Workfusion, Glassdoor, ServiceNow, and more. Stefan Krawczky has spent the last 15+ years …
Read more “Last call: Stefan Krawcyzk’s ‘Mastering MLOps’ Live Cohort”
Why Initialize a Neural Network with Random Weights?
Tweet Tweet Share Share Last Updated on August 15, 2022 The weights of artificial neural networks must be initialized to small random numbers. This is because this is an expectation of the stochastic optimization algorithm used to train the model, called stochastic gradient descent. To understand this approach to problem solving, you must first understand …
Read more “Why Initialize a Neural Network with Random Weights?”
When to Use MLP, CNN, and RNN Neural Networks
Tweet Tweet Share Share Last Updated on August 15, 2022 What neural network is appropriate for your predictive modeling problem? It can be difficult for a beginner to the field of deep learning to know what type of network to use. There are so many types of networks to choose from and new methods being …
Difference Between a Batch and an Epoch in a Neural Network
Tweet Tweet Share Share Last Updated on August 15, 2022 Stochastic gradient descent is a learning algorithm that has a number of hyperparameters. Two hyperparameters that often confuse beginners are the batch size and number of epochs. They are both integer values and seem to do the same thing. In this post, you will discover …
Read more “Difference Between a Batch and an Epoch in a Neural Network”
Using Depthwise Separable Convolutions in Tensorflow
Tweet Tweet Share Share Last Updated on August 10, 2022 Looking at all of the very large convolutional neural networks such as ResNets, VGGs, and the like, it begs the question on how we can make all of these networks smaller with less parameters while still maintaining the same level of accuracy or even improving …
Read more “Using Depthwise Separable Convolutions in Tensorflow”
Reverse engineering the NTK: towards first-principles architecture design
Deep neural networks have enabled technological wonders ranging from voice recognition to machine transition to protein engineering, but their design and application is nonetheless notoriously unprincipled. The development of tools and methods to guide this process is one of the grand challenges of deep learning theory. In Reverse Engineering the Neural Tangent Kernel, we propose …
Read more “Reverse engineering the NTK: towards first-principles architecture design”
Why do Policy Gradient Methods work so well in Cooperative MARL? Evidence from Policy Representation
In cooperative multi-agent reinforcement learning (MARL), due to its on-policy nature, policy gradient (PG) methods are typically believed to be less sample efficient than value decomposition (VD) methods, which are off-policy. However, some recent empirical studies demonstrate that with proper input representation and hyper-parameter tuning, multi-agent PG can achieve surprisingly strong performance compared to off-policy …
FIGS: Attaining XGBoost-level performance with the interpretability and speed of CART
FIGS (Fast Interpretable Greedy-tree Sums): A method for building interpretable models by simultaneously growing an ensemble of decision trees in competition with one another. Recent machine-learning advances have led to increasingly complex predictive models, often at the cost of interpretability. We often need interpretability, particularly in high-stakes applications such as in clinical decision-making; interpretable models …
Read more “FIGS: Attaining XGBoost-level performance with the interpretability and speed of CART”
The Berkeley Crossword Solver
We recently published the Berkeley Crossword Solver (BCS), the current state of the art for solving American-style crossword puzzles. The BCS combines neural question answering and probabilistic inference to achieve near-perfect performance on most American-style crossword puzzles, like the one shown below: Figure 1: Example American-style crossword puzzle An earlier version of the BCS, in …