org.apache.nutch.util

## Class EncodingDetector

- java.lang.Object
- - org.apache.nutch.util.EncodingDetector

-     public class EncodingDetector
      extends java.lang.Object

  A simple class for detecting character encodings.

  Broadly this encompasses two functions, which are distinctly separate:

  1. Auto detecting a set of "clues" from input text.
  2. Taking a set of clues and making a "best guess" as to the "real" encoding.

  A caller will often have some extra information about what the encoding might be (e.g. from the HTTP header or HTML meta-tags, often wrong but still potentially useful clues). The types of clues may differ from caller to caller. Thus a typical calling sequence is:

  - Run step (1) to generate a set of auto-detected clues;
  - Combine these clues with the caller-dependent "extra clues" available;
  - Run step (2) to guess what the most probable answer is.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static org.apache.avro.util.Utf8` | `CONTENT_TYPE_UTF8`
    `static java.lang.String` | `MIN_CONFIDENCE_KEY`
    `static int` | `NO_THRESHOLD`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `EncodingDetector(Configuration conf)`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `addClue(java.lang.String value, java.lang.String source)`
    `void` | `addClue(java.lang.String value, java.lang.String source, int confidence)`
    `void` | `autoDetectClues(WebPage page, boolean filter)`
    `void` | `clearClues()` Clears all clues.
    `java.lang.String` | `guessEncoding(WebPage page, java.lang.String defaultValue)` Guess the encoding with the previously specified list of clues.
    `static java.lang.String` | `parseCharacterEncoding(java.lang.CharSequence contentTypeUtf8)` Parse the character encoding from the specified content type header.
    `static java.lang.String` | `resolveEncodingAlias(java.lang.String encoding)`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### CONTENT_TYPE_UTF8

          public static final org.apache.avro.util.Utf8 CONTENT_TYPE_UTF8

    - #### NO_THRESHOLD

          public static final int NO_THRESHOLD

      See Also:
        Constant Field Values

    - #### MIN_CONFIDENCE_KEY

          public static final java.lang.String MIN_CONFIDENCE_KEY

      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### EncodingDetector

          public EncodingDetector(Configuration conf)

  - ### Method Detail

    - #### autoDetectClues

          public void autoDetectClues(WebPage page,
                                      boolean filter)

    - #### addClue

          public void addClue(java.lang.String value,
                              java.lang.String source,
                              int confidence)

    - #### addClue

          public void addClue(java.lang.String value,
                              java.lang.String source)

    - #### guessEncoding

          public java.lang.String guessEncoding(WebPage page,
                                                java.lang.String defaultValue)

      Guess the encoding with the previously specified list of clues.
      Parameters:
        `defaultValue` - Default encoding to return if no encoding can be detected with enough confidence. Note that this will not be normalized with `resolveEncodingAlias(java.lang.String)`
      Returns:
        Guessed encoding or defaultValue

    - #### clearClues

          public void clearClues()

      Clears all clues.

    - #### resolveEncodingAlias

          public static java.lang.String resolveEncodingAlias(java.lang.String encoding)

    - #### parseCharacterEncoding

          public static java.lang.String parseCharacterEncoding(java.lang.CharSequence contentTypeUtf8)

      Parse the character encoding from the specified content type header. If the content type is null, or there is no explicit character encoding, `null` is returned.

      This method was copied from org.apache.catalina.util.RequestUtil, which is licensed under the Apache License, Version 2.0 (the "License").

      Parameters:
        `contentTypeUtf8` -

Copyright © 2019 The Apache Software Foundation