Xapian::Unicode Namespace Reference

Functions associated with handling Unicode characters. More...

## Enumerations

enum

category {
UNASSIGNED , UPPERCASE_LETTER , LOWERCASE_LETTER , TITLECASE_LETTER ,
MODIFIER_LETTER , OTHER_LETTER , NON_SPACING_MARK , ENCLOSING_MARK ,
COMBINING_SPACING_MARK , DECIMAL_DIGIT_NUMBER , LETTER_NUMBER , OTHER_NUMBER ,
SPACE_SEPARATOR , LINE_SEPARATOR , PARAGRAPH_SEPARATOR , CONTROL ,
FORMAT , PRIVATE_USE , SURROGATE , CONNECTOR_PUNCTUATION ,
DASH_PUNCTUATION , OPEN_PUNCTUATION , CLOSE_PUNCTUATION , INITIAL_QUOTE_PUNCTUATION ,
FINAL_QUOTE_PUNCTUATION , OTHER_PUNCTUATION , MATH_SYMBOL , CURRENCY_SYMBOL ,
MODIFIER_SYMBOL , OTHER_SYMBOL
}

| Each Unicode character is in exactly one of these categories. More...

## Functions

unsigned | nonascii_to_utf8 (unsigned ch, char *buf)
| Convert a single non-ASCII Unicode character to UTF-8. More...
unsigned | to_utf8 (unsigned ch, char *buf)
| Convert a single Unicode character to UTF-8. More...
void | append_utf8 (std::string &s, unsigned ch)
| Append the UTF-8 representation of a single Unicode character to a std::string.
category | get_category (unsigned ch)
| Return the category which a given Unicode character falls into.
bool | is_wordchar (unsigned ch)
| Test if a given Unicode character is "word character".
bool | is_whitespace (unsigned ch)
| Test if a given Unicode character is a whitespace character.
bool | is_currency (unsigned ch)
| Test if a given Unicode character is a currency symbol.
unsigned | tolower (unsigned ch)
| Convert a Unicode character to lowercase.
unsigned | toupper (unsigned ch)
| Convert a Unicode character to uppercase.
std::string | tolower (const std::string &term)
| Convert a UTF-8 std::string to lowercase.
std::string | toupper (const std::string &term)
| Convert a UTF-8 std::string to uppercase.

## Detailed Description

Functions associated with handling Unicode characters.

## Enumeration Type Documentation

## category

enum Xapian::Unicode::category

Each Unicode character is in exactly one of these categories.

The Unicode standard calls this the "General Category", and uses a "Major, minor" convention to derive a two letter code.

Enumerator
UNASSIGNED | Other, not assigned (Cn)
UPPERCASE_LETTER | Letter, uppercase (Lu)
LOWERCASE_LETTER | Letter, lowercase (Ll)
TITLECASE_LETTER | Letter, titlecase (Lt)
MODIFIER_LETTER | Letter, modifier (Lm)
OTHER_LETTER | Letter, other (Lo)
NON_SPACING_MARK | Mark, nonspacing (Mn)
ENCLOSING_MARK | Mark, enclosing (Me)
COMBINING_SPACING_MARK | Mark, spacing combining (Mc)
DECIMAL_DIGIT_NUMBER | Number, decimal digit (Nd)
LETTER_NUMBER | Number, letter (Nl)
OTHER_NUMBER | Number, other (No)
SPACE_SEPARATOR | Separator, space (Zs)
LINE_SEPARATOR | Separator, line (Zl)
PARAGRAPH_SEPARATOR | Separator, paragraph (Zp)
CONTROL | Other, control (Cc)
FORMAT | Other, format (Cf)
PRIVATE_USE | Other, private use (Co)
SURROGATE | Other, surrogate (Cs)
CONNECTOR_PUNCTUATION | Punctuation, connector (Pc)
DASH_PUNCTUATION | Punctuation, dash (Pd)
OPEN_PUNCTUATION | Punctuation, open (Ps)
CLOSE_PUNCTUATION | Punctuation, close (Pe)
INITIAL_QUOTE_PUNCTUATION | Punctuation, initial quote (Pi)
FINAL_QUOTE_PUNCTUATION | Punctuation, final quote (Pf)
OTHER_PUNCTUATION | Punctuation, other (Po)
MATH_SYMBOL | Symbol, math (Sm)
CURRENCY_SYMBOL | Symbol, currency (Sc)
MODIFIER_SYMBOL | Symbol, modified (Sk)
OTHER_SYMBOL | Symbol, other (So)

## Function Documentation

## nonascii_to_utf8()

unsigned Xapian::Unicode::nonascii_to_utf8 | ( | unsigned | ch,
|  | char * | buf
| ) |  |

Convert a single non-ASCII Unicode character to UTF-8.

This is intended mainly as a helper method for to_utf8().

Parameters

  ch | The character (which must be > 128) to write to buf.
  buf | The buffer to write the character to - it must have space for (at least) 4 bytes.

Returns
  The length of the resultant UTF-8 character in bytes.

Referenced by to_utf8().

## to_utf8()

unsigned Xapian::Unicode::to_utf8 | ( | unsigned | ch,
|  | char * | buf
| ) |  |

inline

Convert a single Unicode character to UTF-8.

Parameters

  ch | The character to write to buf.
  buf | The buffer to write the character to - it must have space for (at least) 4 bytes.

Returns
  The length of the resultant UTF-8 character in bytes.

References nonascii_to_utf8().

Referenced by append_utf8().