Categories: AI/ML Research

Using Activation Functions in Deep Learning Models

A deep learning model in its simplest form are layers of perceptrons connected in tandem. Without any activation functions, they are just matrix multiplications with limited power, regardless how many of them. Activation is the magic why neural network can be an approximation to a wide variety of non-linear function. In PyTorch, there are many […]

The post Using Activation Functions in Deep Learning Models appeared first on MachineLearningMastery.com.

Activation Functions in PyTorch

Last Updated on May 3, 2023 As neural networks become increasingly popular in the field of machine learning, it is important to understand the role that activation functions play in their implementation. In this article, you’ll explore the concept of activation functions that are applied to the output of each…

May 3, 2023

In "AI/ML Research"

Linear Layers and Activation Functions in Transformer Models

This post is divided into three parts; they are: • Why Linear Layers and Activations are Needed in Transformers • Typical Design of the Feed-Forward Network • Variations of the Activation Functions The attention layer is the core function of a transformer model.

July 1, 2025

In "AI/ML Research"