org.apache.nutch.parsefilter.naivebayes

## Class NaiveBayesParseFilter

- java.lang.Object
- - org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter

- All Implemented Interfaces:
    Configurable, HtmlParseFilter, Pluggable

      public class NaiveBayesParseFilter
      extends Object
      implements HtmlParseFilter

  Html Parse filter that classifies the outlinks from the parseresult as relevant or irrelevant based on the parseText's relevancy (using a training file where you can give positive and negative example texts see the description of parsefilter.naivebayes.trainfile) and if found irrelevant it gives the link a second chance if it contains any of the words from the list given in parsefilter.naivebayes.wordlist. CAUTION: Set the parser.timeout to -1 or a bigger value than 30, when using this classifier.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static String` | `DICTFILE_MODELFILTER`
    `static String` | `TRAINFILE_MODELFILTER`

    - ### Fields inherited from interface org.apache.nutch.parse.HtmlParseFilter

      `X_POINT_ID`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `NaiveBayesParseFilter()`

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `boolean` | `classify(String text)`
    `boolean` | `containsWord(String url, ArrayList<String> wordlist)`
    `ParseResult` | `filter(Content content, ParseResult parseResult, HTMLMetaTags metaTags, DocumentFragment doc)` Adds metadata or otherwise modifies a parse of HTML content, given the DOM tree of a page.
    `boolean` | `filterParse(String text)`
    `boolean` | `filterUrl(String url)`
    `Configuration` | `getConf()`
    `void` | `setConf(Configuration conf)`
    `void` | `train()`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### TRAINFILE_MODELFILTER

          public static final String TRAINFILE_MODELFILTER

      See Also:
        Constant Field Values

    - #### DICTFILE_MODELFILTER

          public static final String DICTFILE_MODELFILTER

      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### NaiveBayesParseFilter

          public NaiveBayesParseFilter()

  - ### Method Detail

    - #### filterParse

          public boolean filterParse(String text)

    - #### filterUrl

          public boolean filterUrl(String url)

    - #### classify

          public boolean classify(String text)
                           throws IOException

      Throws:
        `IOException`

    - #### train

          public void train()
                     throws Exception

      Throws:
        `Exception`

    - #### containsWord

          public boolean containsWord(String url,
                                      ArrayList<String> wordlist)

    - #### setConf

          public void setConf(Configuration conf)

      Specified by:
        `setConf` in interface `Configurable`

    - #### getConf

          public Configuration getConf()

      Specified by:
        `getConf` in interface `Configurable`

    - #### filter

          public ParseResult filter(Content content,
                                    ParseResult parseResult,
                                    HTMLMetaTags metaTags,
                                    DocumentFragment doc)

      Description copied from interface: `HtmlParseFilter`
      Adds metadata or otherwise modifies a parse of HTML content, given the DOM tree of a page.
      Specified by:
        `filter` in interface `HtmlParseFilter`

Copyright © 2021 The Apache Software Foundation