This is the reason why backpropagation requires the activation function to be differentiable.

This activation function is linear, and therefore has the same problems as the binary function.

For this reason, back-propagation can only be applied on networks with differentiable activation functions.

All problems mentioned above can be handled by using a normalizable sigmoid activation function.

These activation functions can take many forms, but they are usually found as one of three functions:

In general, a non-linear, differentiable activation function, , is used.

For a linear neuron, the derivative of the activation function is 1, which yields:

The data showed qualitative agreement with the final form of the BCM activation function.

See Multinomial logit for a probability model which uses the softmax activation function.

Hopfield would use a nonlinear activation function, instead of using a linear function.

SHH also seems to activate the activation function of Gli3 but this activity is not strong enough.

In many applications the units of these networks apply a sigmoid function as an activation function.

A commonly used activation function is the logistic function:

The softmax activation function is a neural transfer function.

The activation function for the hidden layer of these machines is referred to as the inner product kernel, .

Neurons with this kind of activation function are also called artificial neurons or linear threshold units.

Backpropagation requires that the activation function used by the artificial neurons (or "nodes") be differentiable.

However, it is the nonlinear activation function that allows such networks to compute nontrivial problems using only a small number of nodes.

A generalisation and extension of the logistic function to multiple inputs is the softmax activation function.

The activation function that converts a neuron's weighted input to its output activation.

Except for the input nodes, each node is a neuron (or processing element) with a nonlinear activation function.

More specialized activation functions include radial basis functions which are used in another class of supervised neural network models.

The activation function is a point-neuron approximation with both discrete spiking and continuous rate-code output.

Activation function having types:

It has been shown that a feedforward network with nonlinear, continuous and differentiable activation functions have universal approximation capability.