T - The sketch data type.public abstract class KllItemsSketch<T> extends KllSketch implements QuantilesGenericAPI<T>, PartitioningFeature<T>
KllSketchKllSketch.SketchStructure, KllSketch.SketchTypeEMPTY_MSG, MEM_REQ_SVR_NULL_MSG, NOT_SINGLE_ITEM_MSG, TGT_IS_READ_ONLY_MSG, UNSUPPORTED_MSG| Modifier and Type | Method and Description |
|---|---|
double[] |
getCDF(T[] splitPoints,
QuantileSearchCriteria searchCrit)
Returns an approximation to the Cumulative Distribution Function (CDF) of the input stream
as a monotonically increasing array of double ranks (or cumulative probabilities) on the interval [0.0, 1.0],
given a set of splitPoints.
|
GenericPartitionBoundaries<T> |
getPartitionBoundaries(int numEquallySized,
QuantileSearchCriteria searchCrit)
This method returns an instance of
GenericPartitionBoundaries which provides
sufficient information for the user to create the given number of equally sized partitions, where "equally sized"
refers to an approximately equal number of items per partition. |
double[] |
getPMF(T[] splitPoints,
QuantileSearchCriteria searchCrit)
Returns an approximation to the Probability Mass Function (PMF) of the input stream
as an array of probability masses as doubles on the interval [0.0, 1.0],
given a set of splitPoints.
|
T |
getQuantile(double rank,
QuantileSearchCriteria searchCrit)
Gets the approximate quantile of the given normalized rank and the given search criterion.
|
T |
getQuantileLowerBound(double rank)
Gets the lower bound of the quantile confidence interval in which the quantile of the
given rank exists.
|
T[] |
getQuantiles(double[] ranks,
QuantileSearchCriteria searchCrit)
Gets an array of quantiles from the given array of normalized ranks.
|
T |
getQuantileUpperBound(double rank)
Gets the upper bound of the quantile confidence interval in which the true quantile of the
given rank exists.
|
double |
getRank(T quantile,
QuantileSearchCriteria searchCrit)
Gets the normalized rank corresponding to the given a quantile.
|
double |
getRankLowerBound(double rank)
Gets the lower bound of the rank confidence interval in which the true rank of the
given rank exists.
|
double[] |
getRanks(T[] quantiles,
QuantileSearchCriteria searchCrit)
Gets an array of normalized ranks corresponding to the given array of quantiles and the given
search criterion.
|
double |
getRankUpperBound(double rank)
Gets the upper bound of the rank confidence interval in which the true rank of the
given rank exists.
|
KllItemsSketchSortedView<T> |
getSortedView()
Gets the sorted view of this sketch
|
static <T> KllItemsSketch<T> |
heapify(org.apache.datasketches.memory.Memory srcMem,
Comparator<? super T> comparator,
ArrayOfItemsSerDe<T> serDe)
Factory heapify takes a compact sketch image in Memory and instantiates an on-heap sketch.
|
QuantilesGenericSketchIterator<T> |
iterator()
Gets the iterator for this sketch, which is not sorted.
|
void |
merge(KllSketch other)
Merges another sketch into this one.
|
static <T> KllItemsSketch<T> |
newHeapInstance(Comparator<? super T> comparator,
ArrayOfItemsSerDe<T> serDe)
Create a new heap instance of this sketch with the default k = 200.
|
static <T> KllItemsSketch<T> |
newHeapInstance(int k,
Comparator<? super T> comparator,
ArrayOfItemsSerDe<T> serDe)
Create a new heap instance of this sketch with a given parameter k.
|
void |
reset()
Resets this sketch to the empty state.
|
byte[] |
toByteArray() |
void |
update(T item)
Updates this sketch with the given item.
|
static <T> KllItemsSketch<T> |
wrap(org.apache.datasketches.memory.Memory srcMem,
Comparator<? super T> comparator,
ArrayOfItemsSerDe<T> serDe)
Constructs a thin wrapper on the heap around a Memory (or WritableMemory) already initialized with a
validated sketch image of a type T consistent with the given comparator and serDe.
|
getKFromEpsilon, getMaxSerializedSizeBytes, getNormalizedRankError, getNormalizedRankError, getNumRetained, getSerializedSizeBytes, hasMemory, isCompactMemoryFormat, isDirect, isEmpty, isEstimationMode, isMemoryUpdatableFormat, isReadOnly, isSameResource, toString, toStringclone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, waitgetCDF, getMaxItem, getMinItem, getPMF, getQuantile, getQuantiles, getRank, getRanksgetK, getN, getNumRetained, hasMemory, isDirect, isEmpty, isEstimationMode, isReadOnly, toStringgetPartitionBoundariespublic static <T> KllItemsSketch<T> newHeapInstance(Comparator<? super T> comparator, ArrayOfItemsSerDe<T> serDe)
T - The sketch data type.comparator - to compare itemsserDe - Serializer / deserializer for an array of items, T[].public static <T> KllItemsSketch<T> newHeapInstance(int k, Comparator<? super T> comparator, ArrayOfItemsSerDe<T> serDe)
T - The sketch data typek - parameter that controls size of the sketch and accuracy of estimates.comparator - to compare itemsserDe - Serializer / deserializer for items of type T and T[].public static <T> KllItemsSketch<T> heapify(org.apache.datasketches.memory.Memory srcMem, Comparator<? super T> comparator, ArrayOfItemsSerDe<T> serDe)
T - The sketch data typesrcMem - a compact Memory image of a sketch serialized by this sketch and of the same type of T.comparator - to compare itemsserDe - Serializer / deserializer for items of type T and T[].public static <T> KllItemsSketch<T> wrap(org.apache.datasketches.memory.Memory srcMem, Comparator<? super T> comparator, ArrayOfItemsSerDe<T> serDe)
T - The sketch data typesrcMem - the Memory object that this sketch will wrap.comparator - to compare itemsserDe - Serializer / deserializer for items of type T and T[].public double[] getCDF(T[] splitPoints, QuantileSearchCriteria searchCrit)
QuantilesGenericAPIThe resulting approximations have a probabilistic guarantee that can be obtained from the getNormalizedRankError(false) function.
getCDF in interface QuantilesGenericAPI<T>splitPoints - an array of m unique, monotonically increasing items
(of the same type as the input items)
that divide the item input domain into m+1 overlapping intervals.
The start of each interval is below the lowest item retained by the sketch corresponding to a zero rank or zero probability, and the end of the interval is the rank or cumulative probability corresponding to the split point.
The (m+1)th interval represents 100% of the distribution represented by the sketch and consistent with the definition of a cumulative probability distribution, thus the (m+1)th rank or probability in the returned array is always 1.0.
If a split point exactly equals a retained item of the sketch and the search criterion is:
It is not recommended to include either the minimum or maximum items of the input stream.
searchCrit - the desired search criteria.public GenericPartitionBoundaries<T> getPartitionBoundaries(int numEquallySized, QuantileSearchCriteria searchCrit)
PartitioningFeatureGenericPartitionBoundaries which provides
sufficient information for the user to create the given number of equally sized partitions, where "equally sized"
refers to an approximately equal number of items per partition.getPartitionBoundaries in interface PartitioningFeature<T>numEquallySized - an integer that specifies the number of equally sized partitions between
getMinItem() and
getMaxItem().
This must be a positive integer greater than zero.
searchCrit - If INCLUSIVE, all the returned quantiles are the upper boundaries of the equally sized partitions
with the exception of the lowest returned quantile, which is the lowest boundary of the lowest ranked partition.
If EXCLUSIVE, all the returned quantiles are the lower boundaries of the equally sized partitions
with the exception of the highest returned quantile, which is the upper boundary of the highest ranked partition.GenericPartitionBoundaries.public double[] getPMF(T[] splitPoints, QuantileSearchCriteria searchCrit)
QuantilesGenericAPIThe resulting approximations have a probabilistic guarantee that can be obtained from the getNormalizedRankError(true) function.
getPMF in interface QuantilesGenericAPI<T>splitPoints - an array of m unique, monotonically increasing items
(of the same type as the input items)
that divide the item input domain into m+1 consecutive, non-overlapping intervals.
Each interval except for the end intervals starts with a split point and ends with the next split point in sequence.
The first interval starts below the lowest item retained by the sketch corresponding to a zero rank or zero probability, and ends with the first split point
The last (m+1)th interval starts with the last split point and ends after the last item retained by the sketch corresponding to a rank or probability of 1.0.
The sum of the probability masses of all (m+1) intervals is 1.0.
If the search criterion is:
It is not recommended to include either the minimum or maximum items of the input stream.
searchCrit - the desired search criteria.public T getQuantile(double rank, QuantileSearchCriteria searchCrit)
QuantilesGenericAPIgetQuantile in interface QuantilesGenericAPI<T>rank - the given normalized rank, a double in the range [0.0, 1.0].searchCrit - If INCLUSIVE, the given rank includes all quantiles ≤
the quantile directly corresponding to the given rank.
If EXCLUSIVE, he given rank includes all quantiles <
the quantile directly corresponding to the given rank.QuantileSearchCriteriapublic T[] getQuantiles(double[] ranks, QuantileSearchCriteria searchCrit)
QuantilesGenericAPIgetQuantiles in interface QuantilesGenericAPI<T>ranks - the given array of normalized ranks, each of which must be
in the interval [0.0,1.0].searchCrit - if INCLUSIVE, the given ranks include all quantiles ≤
the quantile directly corresponding to each rank.QuantileSearchCriteriapublic T getQuantileLowerBound(double rank)
QuantilesGenericAPIAlthough it is possible to estimate the probability that the true quantile exists within the quantile confidence interval specified by the upper and lower quantile bounds, it is not possible to guarantee the width of the quantile confidence interval as an additive or multiplicative percent of the true quantile.
getQuantileLowerBound in interface QuantilesGenericAPI<T>rank - the given normalized rankpublic T getQuantileUpperBound(double rank)
QuantilesGenericAPIAlthough it is possible to estimate the probability that the true quantile exists within the quantile confidence interval specified by the upper and lower quantile bounds, it is not possible to guarantee the width of the quantile interval as an additive or multiplicative percent of the true quantile.
getQuantileUpperBound in interface QuantilesGenericAPI<T>rank - the given normalized rankpublic double getRank(T quantile, QuantileSearchCriteria searchCrit)
QuantilesGenericAPIgetRank in interface QuantilesGenericAPI<T>quantile - the given quantilesearchCrit - if INCLUSIVE the given quantile is included into the rank.QuantileSearchCriteriapublic double getRankLowerBound(double rank)
getRankLowerBound in interface QuantilesAPIrank - the given normalized rank.public double getRankUpperBound(double rank)
getRankUpperBound in interface QuantilesAPIrank - the given normalized rank.public double[] getRanks(T[] quantiles, QuantileSearchCriteria searchCrit)
QuantilesGenericAPIgetRanks in interface QuantilesGenericAPI<T>quantiles - the given array of quantilessearchCrit - if INCLUSIVE, the given quantiles include the rank directly corresponding to each quantile.QuantileSearchCriteriapublic final KllItemsSketchSortedView<T> getSortedView()
QuantilesGenericAPIgetSortedView in interface QuantilesGenericAPI<T>public QuantilesGenericSketchIterator<T> iterator()
QuantilesGenericAPIiterator in interface QuantilesGenericAPI<T>public final void merge(KllSketch other)
KllSketchpublic void reset()
QuantilesAPIThe parameter k will not change.
reset in interface QuantilesAPIpublic byte[] toByteArray()
public void update(T item)
QuantilesGenericAPIupdate in interface QuantilesGenericAPI<T>item - from a stream of items. Nulls are ignored.Copyright © 2015–2022 The Apache Software Foundation. All rights reserved.