org.apache.nutch.storage

## Class StorageUtils

- java.lang.Object
- - org.apache.nutch.storage.StorageUtils

-     public class StorageUtils
      extends java.lang.Object

  Entry point to Gora store/mapreduce functionality. Translates the concept of "crawlid" to the corresponding Gora support.

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `StorageUtils()`

  - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static <K,V extends org.apache.gora.persistency.Persistent> org.apache.gora.store.DataStore<K,V>` | `createWebStore(Configuration conf, java.lang.Class<K> keyClass, java.lang.Class<V> persistentClass)` Creates a store for the given persistentClass.
    `static <K,V extends org.apache.gora.persistency.Persistent> java.lang.Class<? extends org.apache.gora.store.DataStore<K,V>>` | `getDataStoreClass(Configuration conf)` Return the Persistent Gora class used to persist Nutch Web data.
    `static <K,V> void` | `initMapperJob(Job job, java.util.Collection<WebPage.Field> fields, java.lang.Class<K> outKeyClass, java.lang.Class<V> outValueClass, java.lang.Class<? extends org.apache.gora.mapreduce.GoraMapper<java.lang.String,WebPage,K,V>> mapperClass)`
    `static <K,V> void` | `initMapperJob(Job job, java.util.Collection<WebPage.Field> fields, java.lang.Class<K> outKeyClass, java.lang.Class<V> outValueClass, java.lang.Class<? extends org.apache.gora.mapreduce.GoraMapper<java.lang.String,WebPage,K,V>> mapperClass, java.lang.Class<? extends Partitioner<K,V>> partitionerClass)`
    `static <K,V> void` | `initMapperJob(Job job, java.util.Collection<WebPage.Field> fields, java.lang.Class<K> outKeyClass, java.lang.Class<V> outValueClass, java.lang.Class<? extends org.apache.gora.mapreduce.GoraMapper<java.lang.String,WebPage,K,V>> mapperClass, java.lang.Class<? extends Partitioner<K,V>> partitionerClass, boolean reuseObjects)`
    `static <K,V> void` | `initMapperJob(Job job, java.util.Collection<WebPage.Field> fields, java.lang.Class<K> outKeyClass, java.lang.Class<V> outValueClass, java.lang.Class<? extends org.apache.gora.mapreduce.GoraMapper<java.lang.String,WebPage,K,V>> mapperClass, java.lang.Class<? extends Partitioner<K,V>> partitionerClass, org.apache.gora.filter.Filter<java.lang.String,WebPage> filter, boolean reuseObjects)`
    `static <K,V> void` | `initMapperJob(Job job, java.util.Collection<WebPage.Field> fields, java.lang.Class<K> outKeyClass, java.lang.Class<V> outValueClass, java.lang.Class<? extends org.apache.gora.mapreduce.GoraMapper<java.lang.String,WebPage,K,V>> mapperClass, org.apache.gora.filter.Filter<java.lang.String,WebPage> filter)`
    `static <K,V> void` | `initReducerJob(Job job, java.lang.Class<? extends org.apache.gora.mapreduce.GoraReducer<K,V,java.lang.String,WebPage>> reducerClass)`
    `static java.lang.String[]` | `toStringArray(java.util.Collection<WebPage.Field> fields)`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### StorageUtils

          public StorageUtils()

  - ### Method Detail

    - #### createWebStore

          public static <K,V extends org.apache.gora.persistency.Persistent> org.apache.gora.store.DataStore<K,V> createWebStore(Configuration conf,
                                                                                                                                 java.lang.Class<K> keyClass,
                                                                                                                                 java.lang.Class<V> persistentClass)
                                                                                                                          throws java.lang.ClassNotFoundException,
                                                                                                                                 org.apache.gora.util.GoraException

      Creates a store for the given persistentClass. Currently supports Webpage and Host stores.
      Parameters:
        `conf` -
        `keyClass` -
        `persistentClass` -
      Returns:
      Throws:
        `java.lang.ClassNotFoundException`
        `org.apache.gora.util.GoraException`

    - #### getDataStoreClass

          public static <K,V extends org.apache.gora.persistency.Persistent> java.lang.Class<? extends org.apache.gora.store.DataStore<K,V>> getDataStoreClass(Configuration conf)
                                                                                                                                                        throws java.lang.ClassNotFoundException

      Return the Persistent Gora class used to persist Nutch Web data.
      Parameters:
        `conf` - Nutch configuration
      Returns:
        the Gora DataStore persistent class
      Throws:
        `java.lang.ClassNotFoundException`

    - #### initMapperJob

          public static <K,V> void initMapperJob(Job job,
                                                 java.util.Collection<WebPage.Field> fields,
                                                 java.lang.Class<K> outKeyClass,
                                                 java.lang.Class<V> outValueClass,
                                                 java.lang.Class<? extends org.apache.gora.mapreduce.GoraMapper<java.lang.String,WebPage,K,V>> mapperClass)
                                          throws java.lang.ClassNotFoundException,
                                                 java.io.IOException

      Throws:
        `java.lang.ClassNotFoundException`
        `java.io.IOException`

    - #### initMapperJob

          public static <K,V> void initMapperJob(Job job,
                                                 java.util.Collection<WebPage.Field> fields,
                                                 java.lang.Class<K> outKeyClass,
                                                 java.lang.Class<V> outValueClass,
                                                 java.lang.Class<? extends org.apache.gora.mapreduce.GoraMapper<java.lang.String,WebPage,K,V>> mapperClass,
                                                 java.lang.Class<? extends Partitioner<K,V>> partitionerClass)
                                          throws java.lang.ClassNotFoundException,
                                                 java.io.IOException

      Throws:
        `java.lang.ClassNotFoundException`
        `java.io.IOException`

    - #### initMapperJob

          public static <K,V> void initMapperJob(Job job,
                                                 java.util.Collection<WebPage.Field> fields,
                                                 java.lang.Class<K> outKeyClass,
                                                 java.lang.Class<V> outValueClass,
                                                 java.lang.Class<? extends org.apache.gora.mapreduce.GoraMapper<java.lang.String,WebPage,K,V>> mapperClass,
                                                 java.lang.Class<? extends Partitioner<K,V>> partitionerClass,
                                                 boolean reuseObjects)
                                          throws java.lang.ClassNotFoundException,
                                                 java.io.IOException

      Throws:
        `java.lang.ClassNotFoundException`
        `java.io.IOException`

    - #### initMapperJob

          public static <K,V> void initMapperJob(Job job,
                                                 java.util.Collection<WebPage.Field> fields,
                                                 java.lang.Class<K> outKeyClass,
                                                 java.lang.Class<V> outValueClass,
                                                 java.lang.Class<? extends org.apache.gora.mapreduce.GoraMapper<java.lang.String,WebPage,K,V>> mapperClass,
                                                 java.lang.Class<? extends Partitioner<K,V>> partitionerClass,
                                                 org.apache.gora.filter.Filter<java.lang.String,WebPage> filter,
                                                 boolean reuseObjects)
                                          throws java.lang.ClassNotFoundException,
                                                 java.io.IOException

      Throws:
        `java.lang.ClassNotFoundException`
        `java.io.IOException`

    - #### initMapperJob

          public static <K,V> void initMapperJob(Job job,
                                                 java.util.Collection<WebPage.Field> fields,
                                                 java.lang.Class<K> outKeyClass,
                                                 java.lang.Class<V> outValueClass,
                                                 java.lang.Class<? extends org.apache.gora.mapreduce.GoraMapper<java.lang.String,WebPage,K,V>> mapperClass,
                                                 org.apache.gora.filter.Filter<java.lang.String,WebPage> filter)
                                          throws java.lang.ClassNotFoundException,
                                                 java.io.IOException

      Throws:
        `java.lang.ClassNotFoundException`
        `java.io.IOException`

    - #### initReducerJob

          public static <K,V> void initReducerJob(Job job,
                                                  java.lang.Class<? extends org.apache.gora.mapreduce.GoraReducer<K,V,java.lang.String,WebPage>> reducerClass)
                                           throws java.lang.ClassNotFoundException,
                                                  org.apache.gora.util.GoraException

      Throws:
        `java.lang.ClassNotFoundException`
        `org.apache.gora.util.GoraException`

    - #### toStringArray

          public static java.lang.String[] toStringArray(java.util.Collection<WebPage.Field> fields)

Copyright © 2019 The Apache Software Foundation