activation function
Output transformation rule
An activation function is a mathematical function in a neural network that determines whether a neuron should "fire"—pass its signal forward—based on the input it receives.
Common activation functions include ReLU, sigmoid, and tanh.
The choice of activation function affects how well a network learns: The model performed better after switching to a different activation function.
- sigmoidCurving in two directions
- backpropagationReverse gradient computation method
- perceptronLinear binary classifier algorithm
- squashingCrushing by pressure
- nonlinearNot in sequence
threshold · neural network · deep learning · vanishing gradient · output · forward pass · swish · saturation · transform · logistic · derivative · vanishing · leaky · layer · input · gating · neuron · dead
- machine learningneural networks, deep learning, backpropagation, underfitting, CNNs
- neural networksneural network, neural networks, hidden layers, backpropagation, layers
- ML settingshyperparameters, learning rate, epochs, dropout, regularization
active, function, inactive, activate, functional, deactivate, activated, activation, hyperactive