org.apache.nutch.storage

## Class StorageUtils

- java.lang.Object
- - org.apache.nutch.storage.StorageUtils

-     public class StorageUtils
      extends Object

  Entry point to Gora store/mapreduce functionality. Translates the concept of "crawlid" to the corresponding Gora support.

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `StorageUtils()`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `static <K,V extends org.apache.gora.persistency.Persistent> org.apache.gora.store.DataStore<K,V>` | `createWebStore(Configuration conf, Class<K> keyClass, Class<V> persistentClass)` Creates a store for the given persistentClass.
    `static <K,V extends org.apache.gora.persistency.Persistent> Class<? extends org.apache.gora.store.DataStore<K,V>>` | `getDataStoreClass(Configuration conf)` Return the Persistent Gora class used to persist Nutch Web data.
    `static <K,V> void` | `initMapperJob(Job job, Collection<WebPage.Field> fields, Class<K> outKeyClass, Class<V> outValueClass, Class<? extends org.apache.gora.mapreduce.GoraMapper<String,WebPage,K,V>> mapperClass)`
    `static <K,V> void` | `initMapperJob(Job job, Collection<WebPage.Field> fields, Class<K> outKeyClass, Class<V> outValueClass, Class<? extends org.apache.gora.mapreduce.GoraMapper<String,WebPage,K,V>> mapperClass, Class<? extends Partitioner<K,V>> partitionerClass)`
    `static <K,V> void` | `initMapperJob(Job job, Collection<WebPage.Field> fields, Class<K> outKeyClass, Class<V> outValueClass, Class<? extends org.apache.gora.mapreduce.GoraMapper<String,WebPage,K,V>> mapperClass, Class<? extends Partitioner<K,V>> partitionerClass, boolean reuseObjects)`
    `static <K,V> void` | `initMapperJob(Job job, Collection<WebPage.Field> fields, Class<K> outKeyClass, Class<V> outValueClass, Class<? extends org.apache.gora.mapreduce.GoraMapper<String,WebPage,K,V>> mapperClass, Class<? extends Partitioner<K,V>> partitionerClass, org.apache.gora.filter.Filter<String,WebPage> filter, boolean reuseObjects)`
    `static <K,V> void` | `initMapperJob(Job job, Collection<WebPage.Field> fields, Class<K> outKeyClass, Class<V> outValueClass, Class<? extends org.apache.gora.mapreduce.GoraMapper<String,WebPage,K,V>> mapperClass, org.apache.gora.filter.Filter<String,WebPage> filter)`
    `static <K,V> void` | `initReducerJob(Job job, Class<? extends org.apache.gora.mapreduce.GoraReducer<K,V,String,WebPage>> reducerClass)`
    `static String[]` | `toStringArray(Collection<WebPage.Field> fields)`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### StorageUtils

          public StorageUtils()

  - ### Method Detail

    - #### createWebStore

          public static <K,V extends org.apache.gora.persistency.Persistent> org.apache.gora.store.DataStore<K,V> createWebStore(Configuration conf,
                                                                                                                 Class<K> keyClass,
                                                                                                                 Class<V> persistentClass)
                                                                                                                    throws ClassNotFoundException,
                                                                                                                           org.apache.gora.util.GoraException

      Creates a store for the given persistentClass. Currently supports Webpage and Host stores.
      Parameters:
        `conf` -
        `keyClass` -
        `persistentClass` -
      Returns:
      Throws:
        `ClassNotFoundException`
        `org.apache.gora.util.GoraException`

    - #### getDataStoreClass

          public static <K,V extends org.apache.gora.persistency.Persistent> Class<? extends org.apache.gora.store.DataStore<K,V>> getDataStoreClass(Configuration conf)
                                                                                                                                        throws ClassNotFoundException

      Return the Persistent Gora class used to persist Nutch Web data.
      Parameters:
        `the` - Nutch configuration
      Returns:
        the Gora DataStore persistent class
      Throws:
        `ClassNotFoundException`

    - #### initMapperJob

          public static <K,V> void initMapperJob(Job job,
                                 Collection<WebPage.Field> fields,
                                 Class<K> outKeyClass,
                                 Class<V> outValueClass,
                                 Class<? extends org.apache.gora.mapreduce.GoraMapper<String,WebPage,K,V>> mapperClass)
                                    throws ClassNotFoundException,
                                           IOException

      Throws:
        `ClassNotFoundException`
        `IOException`

    - #### initMapperJob

          public static <K,V> void initMapperJob(Job job,
                                 Collection<WebPage.Field> fields,
                                 Class<K> outKeyClass,
                                 Class<V> outValueClass,
                                 Class<? extends org.apache.gora.mapreduce.GoraMapper<String,WebPage,K,V>> mapperClass,
                                 Class<? extends Partitioner<K,V>> partitionerClass)
                                    throws ClassNotFoundException,
                                           IOException

      Throws:
        `ClassNotFoundException`
        `IOException`

    - #### initMapperJob

          public static <K,V> void initMapperJob(Job job,
                                 Collection<WebPage.Field> fields,
                                 Class<K> outKeyClass,
                                 Class<V> outValueClass,
                                 Class<? extends org.apache.gora.mapreduce.GoraMapper<String,WebPage,K,V>> mapperClass,
                                 Class<? extends Partitioner<K,V>> partitionerClass,
                                 boolean reuseObjects)
                                    throws ClassNotFoundException,
                                           IOException

      Throws:
        `ClassNotFoundException`
        `IOException`

    - #### initMapperJob

          public static <K,V> void initMapperJob(Job job,
                                 Collection<WebPage.Field> fields,
                                 Class<K> outKeyClass,
                                 Class<V> outValueClass,
                                 Class<? extends org.apache.gora.mapreduce.GoraMapper<String,WebPage,K,V>> mapperClass,
                                 Class<? extends Partitioner<K,V>> partitionerClass,
                                 org.apache.gora.filter.Filter<String,WebPage> filter,
                                 boolean reuseObjects)
                                    throws ClassNotFoundException,
                                           IOException

      Throws:
        `ClassNotFoundException`
        `IOException`

    - #### initMapperJob

          public static <K,V> void initMapperJob(Job job,
                                 Collection<WebPage.Field> fields,
                                 Class<K> outKeyClass,
                                 Class<V> outValueClass,
                                 Class<? extends org.apache.gora.mapreduce.GoraMapper<String,WebPage,K,V>> mapperClass,
                                 org.apache.gora.filter.Filter<String,WebPage> filter)
                                    throws ClassNotFoundException,
                                           IOException

      Throws:
        `ClassNotFoundException`
        `IOException`

    - #### initReducerJob

          public static <K,V> void initReducerJob(Job job,
                                  Class<? extends org.apache.gora.mapreduce.GoraReducer<K,V,String,WebPage>> reducerClass)
                                     throws ClassNotFoundException,
                                            org.apache.gora.util.GoraException

      Throws:
        `ClassNotFoundException`
        `org.apache.gora.util.GoraException`

    - #### toStringArray

          public static String[] toStringArray(Collection<WebPage.Field> fields)

Copyright © 2015 The Apache Software Foundation