org.apache.nutch.segment

## Interface SegmentMergeFilter

-     public interface SegmentMergeFilter

  Interface used to filter segments during segment merge. It allows filtering on more sophisticated criteria than just URLs. In particular it allows filtering based on metadata collected while parsing page.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static String` | `X_POINT_ID` The name of the extension point.

  - ### Method Summary

    All Methods Instance Methods Abstract Methods
    Modifier and Type | Method and Description
    --- | ---
    `boolean` | `filter(Text key, CrawlDatum generateData, CrawlDatum fetchData, CrawlDatum sigData, Content content, ParseData parseData, ParseText parseText, Collection<CrawlDatum> linked)` The filtering method which gets all information being merged for a given key (URL).

- - ### Field Detail

    - #### X_POINT_ID

          static final String X_POINT_ID

      The name of the extension point.

  - ### Method Detail

    - #### filter

          boolean filter(Text key,
                         CrawlDatum generateData,
                         CrawlDatum fetchData,
                         CrawlDatum sigData,
                         Content content,
                         ParseData parseData,
                         ParseText parseText,
                         Collection<CrawlDatum> linked)

      The filtering method which gets all information being merged for a given key (URL).
      Returns:
        `true` values for this `key` (URL) should be merged into the new segment.

Copyright © 2021 The Apache Software Foundation