FANN C++ Datatypes

This section includes enums and helper data types used by the two main classes neural_net and training_data

Summary
FANN C++ DatatypesThis section includes enums and helper data types used by the two main classes neural_net and training_data
Types
fann_typefann_type is the type used for the weights, inputs and outputs of the neural network.
Enumerations
error_function_enumError function used during training.
stop_function_enumStop criteria used during training.
training_algorithm_enumThe Training algorithms used when training on training_data with functions like neural_net::train_on_data or neural_net::train_on_file.
activation_function_enumThe activation functions used for the neurons during training.
network_type_enumDefinition of network types used by neural_net::get_network_type
Types
connectionDescribes a connection between two neurons and its weight
callback_typeThis callback function can be called during training when using neural_net::train_on_data, neural_net::train_on_file or neural_net::cascadetrain_on_data.

Types

fann_type

fann_type is the type used for the weights, inputs and outputs of the neural network.

fann_type is defined as a

floatif you include fann.h or floatfann.h
doubleif you include doublefann.h
intif you include fixedfann.h (please be aware that fixed point usage is only to be used during execution, and not during training).

Enumerations

error_function_enum

Error function used during training.

ERRORFUNC_LINEARStandard linear error function.
ERRORFUNC_TANHTanh error function, usually better but can require a lower learning rate.  This error function aggressively targets outputs that differ much from the desired, while not targeting outputs that only differ a little that much.  This activation function is not recommended for cascade training and incremental training.

See also

neural_net::set_train_error_function, neural_net::get_train_error_function

stop_function_enum

Stop criteria used during training.

STOPFUNC_MSEStop criteria is Mean Square Error (MSE) value.
STOPFUNC_BITStop criteria is number of bits that fail.  The number of bits; means the number of output neurons which differ more than the bit fail limit (see neural_net::get_bit_fail_limit, neural_net::set_bit_fail_limit).  The bits are counted in all of the training data, so this number can be higher than the number of training data.

See also

neural_net::set_train_stop_function, neural_net::get_train_stop_function

training_algorithm_enum

The Training algorithms used when training on training_data with functions like neural_net::train_on_data or neural_net::train_on_file.  The incremental training looks alters the weights after each time it is presented an input pattern, while batch only alters the weights once after it has been presented to all the patterns.

TRAIN_INCREMENTALStandard backpropagation algorithm, where the weights are updated after each training pattern.  This means that the weights are updated many times during a single epoch.  For this reason some problems, will train very fast with this algorithm, while other more advanced problems will not train very well.
TRAIN_BATCHStandard backpropagation algorithm, where the weights are updated after calculating the mean square error for the whole training set.  This means that the weights are only updated once during an epoch.  For this reason some problems, will train slower with this algorithm.  But since the mean square error is calculated more correctly than in incremental training, some problems will reach a better solutions with this algorithm.
TRAIN_RPROPA more advanced batch training algorithm which achieves good results for many problems.  The RPROP training algorithm is adaptive, and does therefore not use the learning_rate.  Some other parameters can however be set to change the way the RPROP algorithm works, but it is only recommended for users with insight in how the RPROP training algorithm works.  The RPROP training algorithm is described by [Riedmiller and Braun, 1993], but the actual learning algorithm used here is the iRPROP- training algorithm which is described by [Igel and Husken, 2000] which is a variant of the standard RPROP training algorithm.
TRAIN_QUICKPROPA more advanced batch training algorithm which achieves good results for many problems.  The quickprop training algorithm uses the learning_rate parameter along with other more advanced parameters, but it is only recommended to change these advanced parameters, for users with insight in how the quickprop training algorithm works.  The quickprop training algorithm is described by [Fahlman, 1988].
FANN_TRAIN_SARPROPTHE SARPROP ALGORITHM: A SIMULATED ANNEALING ENHANCEMENT TO RESILIENT BACK PROPAGATION http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.47.8197&rep=rep1&type=pdf

See also

neural_net::set_training_algorithm, neural_net::get_training_algorithm

activation_function_enum

The activation functions used for the neurons during training.  The activation functions can either be defined for a group of neurons by neural_net::set_activation_function_hidden and neural_net::set_activation_function_output or it can be defined for a single neuron by neural_net::set_activation_function.

The steepness of an activation function is defined in the same way by neural_net::set_activation_steepness_hidden, neural_net::set_activation_steepness_output and neural_net::set_activation_steepness.

The functions are described with functions where

  • x is the input to the activation function,
  • y is the output,
  • s is the steepness and
  • d is the derivation.
FANN_LINEARLinear activation function.
  • span: -inf < y < inf
  • y = x*s, d = 1*s
  • Can NOT be used in fixed point.
FANN_THRESHOLDThreshold activation function.
  • x < 0 -> y = 0, x >= 0 -> y = 1
  • Can NOT be used during training.
FANN_THRESHOLD_SYMMETRICThreshold activation function.
  • x < 0 -> y = 0, x >= 0 -> y = 1
  • Can NOT be used during training.
FANN_SIGMOIDSigmoid activation function.
  • One of the most used activation functions.
  • span: 0 < y < 1
  • y = 1/(1 + exp(-2*s*x))
  • d = 2*s*y*(1 - y)
FANN_SIGMOID_STEPWISEStepwise linear approximation to sigmoid.
  • Faster than sigmoid but a bit less precise.
FANN_SIGMOID_SYMMETRICSymmetric sigmoid activation function, aka. tanh.
  • One of the most used activation functions.
  • span: -1 < y < 1
  • y = tanh(s*x) = 2/(1 + exp(-2*s*x)) - 1
  • d = s*(1-(y*y))
FANN_SIGMOID_SYMMETRICStepwise linear approximation to symmetric sigmoid.
  • Faster than symmetric sigmoid but a bit less precise.
FANN_GAUSSIANGaussian activation function.
  • 0 when x = -inf, 1 when x = 0 and 0 when x = inf
  • span: 0 < y < 1
  • y = exp(-x*s*x*s)
  • d = -2*x*s*y*s
FANN_GAUSSIAN_SYMMETRICSymmetric gaussian activation function.
  • -1 when x = -inf, 1 when x = 0 and 0 when x = inf
  • span: -1 < y < 1
  • y = exp(-x*s*x*s)*2-1
  • d = -2*x*s*(y+1)*s
FANN_ELLIOTFast (sigmoid like) activation function defined by David Elliott
  • span: 0 < y < 1
  • y = ((x*s) / 2) / (1 + |x*s|) + 0.5
  • d = s*1/(2*(1+|x*s|)*(1+|x*s|))
FANN_ELLIOT_SYMMETRICFast (symmetric sigmoid like) activation function defined by David Elliott
  • span: -1 < y < 1
  • y = (x*s) / (1 + |x*s|)
  • d = s*1/((1+|x*s|)*(1+|x*s|))
FANN_LINEAR_PIECEBounded linear activation function.
  • span: 0 < y < 1
  • y = x*s, d = 1*s
FANN_LINEAR_PIECE_SYMMETRICBounded Linear activation function.
  • span: -1 < y < 1
  • y = x*s, d = 1*s
FANN_SIN_SYMMETRICPeriodical sinus activation function.
  • span: -1 <= y <= 1
  • y = sin(x*s)
  • d = s*cos(x*s)
FANN_COS_SYMMETRICPeriodical cosinus activation function.
  • span: -1 <= y <= 1
  • y = cos(x*s)
  • d = s*-sin(x*s)

See also

neural_net::set_activation_function_hidden, neural_net::set_activation_function_output

network_type_enum

Definition of network types used by neural_net::get_network_type

LAYEREach layer only has connections to the next layer
SHORTCUTEach layer has connections to all following layers

See Also

neural_net::get_network_type, fann_get_network_type

This enumeration appears in FANN >= 2.1.0

Types

connection

Describes a connection between two neurons and its weight

from_neuronUnique number used to identify source neuron
to_neuronUnique number used to identify destination neuron
weightThe numerical value of the weight

See Also

neural_net::get_connection_array, neural_net::set_weight_array

This structure appears in FANN >= 2.1.0

callback_type

This callback function can be called during training when using neural_net::train_on_data, neural_net::train_on_file or neural_net::cascadetrain_on_data.

typedef int (*callback_type) (neural_net &net, training_data &train,
    unsigned int max_epochs, unsigned int epochs_between_reports,
    float desired_error, unsigned int epochs, void *user_data);

The callback can be set by using neural_net::set_callback and is very useful for doing custom things during training.  It is recommended to use this function when implementing custom training procedures, or when visualizing the training in a GUI etc.  The parameters which the callback function takes is the parameters given to the neural_net::train_on_data, plus an epochs parameter which tells how many epochs the training have taken so far.

The callback function should return an integer, if the callback function returns -1, the training will terminate.

Example of a callback function that prints information to cout

int print_callback(FANN::neural_net &net, FANN::training_data &train,
    unsigned int max_epochs, unsigned int epochs_between_reports,
    float desired_error, unsigned int epochs, void *user_data)
{
    cout << "Epochs     " << setw(8) << epochs << ". "
         << "Current Error: " << left << net.get_MSE() << right << endl;
    return 0;
}

See also

neural_net::set_callback, fann_callback_type

class neural_net
neural_net is the main neural network class used for both training and execution
class training_data
training_data is used to create and manipulate training data used by the neural_net
void train_on_data(const training_data &data,
unsigned int max_epochs,
unsigned int epochs_between_reports,
float desired_error)
Trains on an entire dataset, for a period of time.
void train_on_file(const std::string &filename,
unsigned int max_epochs,
unsigned int epochs_between_reports,
float desired_error)
Does the same as train_on_data, but reads the training data directly from a file.
network_type_enum get_network_type()
Get the type of neural network it was created as.
void cascadetrain_on_data(const training_data &data,
unsigned int max_neurons,
unsigned int neurons_between_reports,
float desired_error)
Trains on an entire dataset, for a period of time using the Cascade2 training algorithm.
void set_train_error_function(error_function_enum train_error_function)
Set the error function used during training.
error_function_enum get_train_error_function()
Returns the error function used during training.
fann_type get_bit_fail_limit()
Returns the bit fail limit used during training.
void set_bit_fail_limit(fann_type bit_fail_limit)
Set the bit fail limit used during training.
void set_train_stop_function(stop_function_enum train_stop_function)
Set the stop function used during training.
stop_function_enum get_train_stop_function()
Returns the the stop function used during training.
void set_training_algorithm(training_algorithm_enum training_algorithm)
Set the training algorithm.
training_algorithm_enum get_training_algorithm()
Return the training algorithm as described by FANN::training_algorithm_enum.
void set_activation_function_hidden(
   activation_function_enum activation_function
)
Set the activation function for all of the hidden layers.
void set_activation_function_output(
   activation_function_enum activation_function
)
Set the activation function for the output layer.
void set_activation_function(activation_function_enum activation_function,
int layer,
int neuron)
Set the activation function for neuron number neuron in layer number layer, counting the input layer as layer 0.
void set_activation_steepness_hidden(fann_type steepness)
Set the steepness of the activation steepness in all of the hidden layers.
void set_activation_steepness_output(fann_type steepness)
Set the steepness of the activation steepness in the output layer.
void set_activation_steepness(fann_type steepness,
int layer,
int neuron)
Set the activation steepness for neuron number neuron in layer number layer, counting the input layer as layer 0.
FANN_EXTERNAL enum fann_nettype_enum FANN_API fann_get_network_type(
   struct fann *ann
)
Get the type of neural network it was created as.
void get_connection_array(connection *connections)
Get the connections in the network.
void set_weight_array(connection *connections,
unsigned int num_connections)
Set connections in the network.
void set_callback(callback_type callback,
void *user_data)
Sets the callback function for use during training.
This callback function can be called during training when using fann_train_on_data, fann_train_on_file or fann_cascadetrain_on_data.
Close