org.apache.mahout.math.hadoop.stochasticsvd
Class SSVDHelper

java.lang.Object
  extended by org.apache.mahout.math.hadoop.stochasticsvd.SSVDHelper

public final class SSVDHelper
extends Object

set of small file manipulation helpers.


Method Summary
static Iterator<Pair<org.apache.hadoop.io.Writable,Vector>> drmIterator(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path glob, org.apache.hadoop.conf.Configuration conf, Deque<Closeable> closeables)
           
static DenseMatrix drmLoadAsDense(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path glob, org.apache.hadoop.conf.Configuration conf)
          helper capabiltiy to load distributed row matrices into dense matrix (to support tests mainly).
static double[][] extractRawData(Matrix m)
          extracts row-wise raw data from a Mahout matrix for 3rd party solvers.
static DenseSymmetricMatrix loadAndSumUpperTriangularMatricesAsSymmetric(org.apache.hadoop.fs.Path glob, org.apache.hadoop.conf.Configuration conf)
          Load multiple upper triangular matrices and sum them up.
static Vector loadAndSumUpVectors(org.apache.hadoop.fs.Path glob, org.apache.hadoop.conf.Configuration conf)
           
static UpperTriangular loadUpperTriangularMatrix(org.apache.hadoop.fs.Path glob, org.apache.hadoop.conf.Configuration conf)
          Load only one upper triangular matrix and issue error if mroe than one is found.
static void saveVector(Vector v, org.apache.hadoop.fs.Path vectorFilePath, org.apache.hadoop.conf.Configuration conf)
          save single vector into hdfs file.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Method Detail

saveVector

public static void saveVector(Vector v,
                              org.apache.hadoop.fs.Path vectorFilePath,
                              org.apache.hadoop.conf.Configuration conf)
                       throws IOException
save single vector into hdfs file.

Parameters:
v - vector to save
Throws:
IOException

drmIterator

public static Iterator<Pair<org.apache.hadoop.io.Writable,Vector>> drmIterator(org.apache.hadoop.fs.FileSystem fs,
                                                                               org.apache.hadoop.fs.Path glob,
                                                                               org.apache.hadoop.conf.Configuration conf,
                                                                               Deque<Closeable> closeables)
                                                                        throws IOException
Throws:
IOException

drmLoadAsDense

public static DenseMatrix drmLoadAsDense(org.apache.hadoop.fs.FileSystem fs,
                                         org.apache.hadoop.fs.Path glob,
                                         org.apache.hadoop.conf.Configuration conf)
                                  throws IOException
helper capabiltiy to load distributed row matrices into dense matrix (to support tests mainly).

Parameters:
fs - filesystem
glob - FS glob
conf - configuration
Returns:
Dense matrix array
Throws:
IOException

loadAndSumUpperTriangularMatricesAsSymmetric

public static DenseSymmetricMatrix loadAndSumUpperTriangularMatricesAsSymmetric(org.apache.hadoop.fs.Path glob,
                                                                                org.apache.hadoop.conf.Configuration conf)
                                                                         throws IOException
Load multiple upper triangular matrices and sum them up.

Returns:
the sum of upper triangular inputs.
Throws:
IOException

loadAndSumUpVectors

public static Vector loadAndSumUpVectors(org.apache.hadoop.fs.Path glob,
                                         org.apache.hadoop.conf.Configuration conf)
                                  throws IOException
Returns:
sum of all vectors in different files specified by glob
Throws:
IOException

loadUpperTriangularMatrix

public static UpperTriangular loadUpperTriangularMatrix(org.apache.hadoop.fs.Path glob,
                                                        org.apache.hadoop.conf.Configuration conf)
                                                 throws IOException
Load only one upper triangular matrix and issue error if mroe than one is found.

Throws:
IOException

extractRawData

public static double[][] extractRawData(Matrix m)
extracts row-wise raw data from a Mahout matrix for 3rd party solvers. Unfortunately values member is 100% encapsulated in DenseMatrix at this point, so we have to resort to abstract element-wise copying.



Copyright © 2008–2014 The Apache Software Foundation. All rights reserved.