Overview
  • Namespace
  • Class

Namespaces

  • apemsel
    • AttributedString

Classes

  • AttributedString
  • MutableAttributedString
  • TokenizedAttributedString

Class TokenizedAttributedString

Extends AttributedString to support a tokenized string.

You can mix working with tokens with working on string ranges using the AttributedString methods. The original string is preserved.

apemsel\AttributedString\AttributedString implements Countable, ArrayAccess
Extended by apemsel\AttributedString\TokenizedAttributedString
Namespace: apemsel\AttributedString
Author: Adrian Pemsel apemsel@gmail.com
Located at TokenizedAttributedString.php

Methods summary

public
# __construct( string|apemsel\AttributedString\AttributedString $string, string $tokenizer = "whitespace" )

Parameters

$string
String to work on
$tokenizer
Tokenizer to use, either "whitespace", "word" or a custom regex

Overrides

apemsel\AttributedString\AttributedString::__construct
public string[]
# getTokens( )

Return all tokens

Return all tokens

Returns

string[]
tokens
public int[]
# getTokenOffsets( )

Return all tokens' offsets

Return all tokens' offsets

Returns

int[]
offsets
public integer
# getTokenCount( )

Return the number of tokens

Return the number of tokens

Returns

integer
count
public string
# getToken( integer $i )

Get indicated token

Get indicated token

Parameters

$i
token index

Returns

string
token
public integer
# getTokenOffset( integer $i )

Get indicated token offset

Get indicated token offset

Parameters

$i
token index

Returns

integer
offset
public
# setTokenAttribute( integer $i, string $attribute, boolean $state = true )

Set a token to a given attribute and state

Set a token to a given attribute and state

Parameters

$i
token index
$attribute
attribute name
$state
attribute state
public
# setTokenRangeAttribute( integer $from, integer $to, string $attribute, boolean $state = true )

Set a range of tokens to a given attribute and state

Set a range of tokens to a given attribute and state

Parameters

$from
token start index
$to
token end index
$attribute
attribute name
$state
attribute state
public
# setTokenDictionaryAttribute( string[] $dictionary, string $attribute, boolean $state = true )

Set all tokens matching given dictionary to attribute and state

Set all tokens matching given dictionary to attribute and state

Parameters

$dictionary
dictionary
$attribute
attribute name
$state
attribute state
public string[]
# attributesAtToken( integer $i )

Get all attribute of token at given index

Get all attribute of token at given index

Parameters

$i
index

Returns

string[]
attributes
public
# lowercaseTokens( )

Convert all tokens to lower case

Convert all tokens to lower case
public static array
# tokenizeOnWhitespace( string $string )

Tokenize a string on whitespace

Tokenize a string on whitespace

Parameters

$string
string to be tokenized

Returns

array
array of two arrays, with tokens at index 0 and their offsets at index 1
public static array
# tokenizeOnWords( string $string )

Tokenize a string on words

Tokenize a string on words

Parameters

$string
string to be tokenized

Returns

array
array of two arrays, with tokens at index 0 and their offsets at index 1
public static array
# tokenizeOnRegex( string $string, string $pattern )

Tokenize a string with a given regex

Tokenize a string with a given regex

Parameters

$string
string to be tokenized
$pattern
regex. The token must be captured in the first subgroup.

Returns

array
array of two arrays, with tokens at index 0 and their offsets at index 1
public boolean
# offsetExists( integer $i )

Check if the token at the given index exists

Check if the token at the given index exists

Parameters

$i
token index

Returns

boolean
does the offset exist

Overrides

apemsel\AttributedString\AttributedString::offsetExists
public string
# offsetGet( integer $i )

Get token at given index

Get token at given index

Note: TokenizedAttributedString uses the ArrayAccess interface to access tokens, not chars!

Parameters

$i
token index

Returns

string
token

Overrides

apemsel\AttributedString\AttributedString::offsetGet

Methods inherited from apemsel\AttributedString\AttributedString

__toString(), attributeToString(), attributesAt(), byteToCharOffset(), byteToCharOffsetString(), charToByteOffset(), combineAttributes(), count(), createAttribute(), deleteAttribute(), enablebyteToCharCache(), filter(), hasAttribute(), is(), offsetSet(), offsetUnset(), searchAttribute(), setLength(), setPattern(), setRange(), setSubstring(), substrings(), toHtml(), utf8CharLen()

Properties summary

protected $tokens
#
protected $tokenOffsets
#

Properties inherited from apemsel\AttributedString\AttributedString

$attributes, $byteToChar, $length, $string

API documentation generated by ApiGen