org.apache.mahout.classifier.mlp
Class NeuralNetwork

java.lang.Object
  extended by org.apache.mahout.classifier.mlp.NeuralNetwork
Direct Known Subclasses:
MultilayerPerceptron

public abstract class NeuralNetwork
extends Object

AbstractNeuralNetwork defines the general operations for a neural network based model. Typically, all derivative models such as Multilayer Perceptron and Autoencoder consist of neurons and the weights between neurons.


Nested Class Summary
static class NeuralNetwork.TrainingMethod
           
 
Field Summary
protected  String costFunctionName
           
protected  int finalLayerIdx
           
protected  List<Integer> layerSizeList
           
protected  double learningRate
           
protected  String modelPath
           
protected  String modelType
           
protected  double momentumWeight
           
protected  List<Matrix> prevWeightUpdatesList
           
protected  double regularizationWeight
           
protected  List<String> squashingFunctionList
           
protected  NeuralNetwork.TrainingMethod trainingMethod
           
protected  List<Matrix> weightMatrixList
           
 
Constructor Summary
NeuralNetwork()
          The default constructor that initializes the learning rate, regularization weight, and momentum weight by default.
NeuralNetwork(double learningRate, double momentumWeight, double regularizationWeight)
          Initialize the NeuralNetwork by specifying learning rate, momentum weight and regularization weight.
NeuralNetwork(String modelPath)
          Initialize the NeuralNetwork by specifying the location of the model.
 
Method Summary
 int addLayer(int size, boolean isFinalLayer, String squashingFunctionName)
          Add a layer of neurons with specified size.
protected  Vector forward(int fromLayer, Vector intermediateOutput)
          Forward the calculation for one layer.
 int getLayerSize(int layer)
          Get the size of a particular layer.
protected  List<Integer> getLayerSizeList()
          Get the layer size list.
 double getLearningRate()
          Get the value of learning rate.
 String getModelPath()
          Get the model path.
 String getModelType()
          Get the type of the model.
 double getMomentumWeight()
          Get the momentum weight.
 Vector getOutput(Vector instance)
          Get the output calculated by the model.
protected  List<Vector> getOutputInternal(Vector instance)
          Calculate output internally, the intermediate output of each layer will be stored.
 double getRegularizationWeight()
          Get the weight of the regularization.
 NeuralNetwork.TrainingMethod getTrainingMethod()
          Get the training method.
 Matrix[] getWeightMatrices()
          Get all the weight matrices.
 Matrix getWeightsByLayer(int layerIdx)
          Get the weights between layer layerIdx and layerIdx + 1
 void readFields(DataInput input)
          Read the fields of the model from input.
protected  void readFromModel()
          Read the model meta-data from the specified location.
 NeuralNetwork setCostFunction(String costFunction)
          Set the cost function for the model.
 NeuralNetwork setLearningRate(double learningRate)
          Set the degree of aggression during model training, a large learning rate can increase the training speed, but it also decreases the chance of model converge.
 void setModelPath(String modelPath)
          Set the model path.
 NeuralNetwork setMomentumWeight(double momentumWeight)
          Set the momentum weight for the model.
 NeuralNetwork setRegularizationWeight(double regularizationWeight)
          Set the regularization weight.
 NeuralNetwork setTrainingMethod(NeuralNetwork.TrainingMethod method)
          Set the training method.
 void setWeightMatrices(Matrix[] matrices)
          Set the weight matrices.
 void setWeightMatrix(int index, Matrix matrix)
          Set the weight matrix for a specified layer.
 Matrix[] trainByInstance(Vector trainingInstance)
          Get the updated weights using one training instance.
 void trainOnline(Vector trainingInstance)
          Train the neural network incrementally with given training instance.
 void updateWeightMatrices(Matrix[] matrices)
          Update the weight matrices with given matrices.
 void write(DataOutput output)
          Write the fields of the model to output.
 void writeModelToFile()
          Write the model data to specified location.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

modelType

protected String modelType

modelPath

protected String modelPath

learningRate

protected double learningRate

regularizationWeight

protected double regularizationWeight

momentumWeight

protected double momentumWeight

costFunctionName

protected String costFunctionName

layerSizeList

protected List<Integer> layerSizeList

trainingMethod

protected NeuralNetwork.TrainingMethod trainingMethod

weightMatrixList

protected List<Matrix> weightMatrixList

prevWeightUpdatesList

protected List<Matrix> prevWeightUpdatesList

squashingFunctionList

protected List<String> squashingFunctionList

finalLayerIdx

protected int finalLayerIdx
Constructor Detail

NeuralNetwork

public NeuralNetwork()
The default constructor that initializes the learning rate, regularization weight, and momentum weight by default.


NeuralNetwork

public NeuralNetwork(double learningRate,
                     double momentumWeight,
                     double regularizationWeight)
Initialize the NeuralNetwork by specifying learning rate, momentum weight and regularization weight.

Parameters:
learningRate - The learning rate.
momentumWeight - The momentum weight.
regularizationWeight - The regularization weight.

NeuralNetwork

public NeuralNetwork(String modelPath)
Initialize the NeuralNetwork by specifying the location of the model.

Parameters:
modelPath - The location that the model is stored.
Method Detail

getModelType

public String getModelType()
Get the type of the model.

Returns:
The name of the model.

setLearningRate

public NeuralNetwork setLearningRate(double learningRate)
Set the degree of aggression during model training, a large learning rate can increase the training speed, but it also decreases the chance of model converge.

Parameters:
learningRate - Learning rate must be a non-negative value. Recommend in range (0, 0.5).
Returns:
The model instance.

getLearningRate

public double getLearningRate()
Get the value of learning rate.

Returns:
The value of learning rate.

setRegularizationWeight

public NeuralNetwork setRegularizationWeight(double regularizationWeight)
Set the regularization weight. More complex the model is, less weight the regularization is.

Parameters:
regularizationWeight - regularization must be in the range [0, 0.1).
Returns:
The model instance.

getRegularizationWeight

public double getRegularizationWeight()
Get the weight of the regularization.

Returns:
The weight of regularization.

setMomentumWeight

public NeuralNetwork setMomentumWeight(double momentumWeight)
Set the momentum weight for the model.

Parameters:
momentumWeight - momentumWeight must be in range [0, 0.5].
Returns:
The model instance.

getMomentumWeight

public double getMomentumWeight()
Get the momentum weight.

Returns:
The value of momentum.

setTrainingMethod

public NeuralNetwork setTrainingMethod(NeuralNetwork.TrainingMethod method)
Set the training method.

Parameters:
method - The training method, currently supports GRADIENT_DESCENT.
Returns:
The instance of the model.

getTrainingMethod

public NeuralNetwork.TrainingMethod getTrainingMethod()
Get the training method.

Returns:
The training method enumeration.

setCostFunction

public NeuralNetwork setCostFunction(String costFunction)
Set the cost function for the model.

Parameters:
costFunction - the name of the cost function. Currently supports "Minus_Squared", "Cross_Entropy".

addLayer

public int addLayer(int size,
                    boolean isFinalLayer,
                    String squashingFunctionName)
Add a layer of neurons with specified size. If the added layer is not the first layer, it will automatically connect the neurons between with the previous layer.

Parameters:
size - The size of the layer. (bias neuron excluded)
isFinalLayer - If false, add a bias neuron.
squashingFunctionName - The squashing function for this layer, input layer is f(x) = x by default.
Returns:
The layer index, starts with 0.

getLayerSize

public int getLayerSize(int layer)
Get the size of a particular layer.

Parameters:
layer - The index of the layer, starting from 0.
Returns:
The size of the corresponding layer.

getLayerSizeList

protected List<Integer> getLayerSizeList()
Get the layer size list.

Returns:
The sizes of the layers.

getWeightsByLayer

public Matrix getWeightsByLayer(int layerIdx)
Get the weights between layer layerIdx and layerIdx + 1

Parameters:
layerIdx - The index of the layer.
Returns:
The weights in form of Matrix.

updateWeightMatrices

public void updateWeightMatrices(Matrix[] matrices)
Update the weight matrices with given matrices.

Parameters:
matrices - The weight matrices, must be the same dimension as the existing matrices.

setWeightMatrices

public void setWeightMatrices(Matrix[] matrices)
Set the weight matrices.

Parameters:
matrices - The weight matrices, must be the same dimension of the existing matrices.

setWeightMatrix

public void setWeightMatrix(int index,
                            Matrix matrix)
Set the weight matrix for a specified layer.

Parameters:
index - The index of the matrix, starting from 0 (between layer 0 and 1).
matrix - The instance of Matrix.

getWeightMatrices

public Matrix[] getWeightMatrices()
Get all the weight matrices.

Returns:
The weight matrices.

getOutput

public Vector getOutput(Vector instance)
Get the output calculated by the model.

Parameters:
instance - The feature instance in form of Vector, each dimension contains the value of the corresponding feature.
Returns:
The output vector.

getOutputInternal

protected List<Vector> getOutputInternal(Vector instance)
Calculate output internally, the intermediate output of each layer will be stored.

Parameters:
instance - The feature instance in form of Vector, each dimension contains the value of the corresponding feature.
Returns:
Cached output of each layer.

forward

protected Vector forward(int fromLayer,
                         Vector intermediateOutput)
Forward the calculation for one layer.

Parameters:
fromLayer - The index of the previous layer.
intermediateOutput - The intermediate output of previous layer.
Returns:
The intermediate results of the current layer.

trainOnline

public void trainOnline(Vector trainingInstance)
Train the neural network incrementally with given training instance.

Parameters:
trainingInstance - An training instance, including the features and the label(s). Its dimension must equals to the size of the input layer (bias neuron excluded) + the size of the output layer (a.k.a. the dimension of the labels).

trainByInstance

public Matrix[] trainByInstance(Vector trainingInstance)
Get the updated weights using one training instance.

Parameters:
trainingInstance - An training instance, including the features and the label(s). Its dimension must equals to the size of the input layer (bias neuron excluded) + the size of the output layer (a.k.a. the dimension of the labels).
Returns:
The update of each weight, in form of Matrix list.

readFromModel

protected void readFromModel()
                      throws IOException
Read the model meta-data from the specified location.

Throws:
IOException

writeModelToFile

public void writeModelToFile()
                      throws IOException
Write the model data to specified location.

Throws:
IOException

setModelPath

public void setModelPath(String modelPath)
Set the model path.

Parameters:
modelPath - The path of the model.

getModelPath

public String getModelPath()
Get the model path.

Returns:
The path of the model.

write

public void write(DataOutput output)
           throws IOException
Write the fields of the model to output.

Parameters:
output - The output instance.
Throws:
IOException

readFields

public void readFields(DataInput input)
                throws IOException
Read the fields of the model from input.

Parameters:
input - The input instance.
Throws:
IOException


Copyright © 2008–2014 The Apache Software Foundation. All rights reserved.