<p>Original by Markus Kuhn, adapted for HTML by Martin Dürst.</p>
<pre>UTF-8 decoder capability and stress test                                      |
----------------------------------------                                      |
                                                                              |
Markus Kuhn &lt;mkuhn@acm.org&gt; - 2000-09-02                                      |
                                                                              |
This test text examines, how UTF-8 decoders handle various types of           |
corrupted or otherwise interesting UTF-8 sequences.                           |
                                                                              |
According to ISO 10646-1, sections R.7 and 2.3c, a device receiving           |
UTF-8 shall interpret a "malformed sequence in the same way that it           |
interprets a character that is outside the adopted subset". This means        |
usually that the malformed UTF-8 sequence is replaced by a replacement        |
character (U+FFFD), which looks a bit like an inverted question mark,         |
or a similar symbol. It might be a good idea to visually distinguish a        |
malformed UTF-8 sequence from a correctly encoded Unicode character           |
that is just not available in the current font but otherwise fully            |
legal. For both cases, a clearly recognisable symbol should be used.          |
Just ignoring malformed sequences or unavailable characters will make         |
debugging more difficult and can lead to user confusion.                      |
                                                                              |
Test sequences:                                                               |
                                                                              |
Check, whether a malformed UTF-8 sequence is (1) represented at all,          |
(2) represented by exactly one replacement character (or equivalent           |
signal), and (3) the following quotation mark after an illegal UTF-8          |
sequence is correctly displayed, i.e. proper resynchronization takes          |
place. This file says "THE END" in the last line, so if you don't see         |
that, your decoder crashed somehow before, which is also not nice.            |
                                                                              |
All lines in this file are exactly 79 characters long (plus the line          |
feed). In addition, all lines end with "|", except for the two test           |
lines 2.1.1 and 2.2.1, which contain non-printable ASCII controls             |
U+0000 and U+007F. If you display this file with a fixed-width font,          |
these "|" characters should all line up in column 79 (right margin).          |
This allows you to test quickly, whether your UTF-8 decoder finds the         |
correct number of characters in every line.                                   |
                                                                              |
Here come the tests:                                                          |
                                                                              |
1  Some correct UTF-8 text                                                    |
                                                                              |
You should see the Greek word 'kosme':       "κόσμε"                          |
                                                                              |
2  Boundary condition test cases                                              |
                                                                              |
You should see a correctly encoded character each time (which is not          |
necessarily an existing character).                                           |
                                                                              |
2.1  First possible sequence of a certain length                              |
                                                                              |
2.1.1  1 byte  (U-00000000):        ""                                        
2.1.2  2 bytes (U-00000080):        ""                                       |
2.1.3  3 bytes (U-00000800):        "ࠀ"                                       |
2.1.4  4 bytes (U-00010000):        "𐀀"                                       |
2.1.5  5 bytes (U-00200000):        "�����"                                       |
2.1.6  6 bytes (U-04000000):        "������"                                       |
                                                                              |
2.2  Last possible sequence of a certain length                               |
                                                                              |
2.2.1  1 byte  (U-0000007F):        ""                                        
2.2.2  2 bytes (U-000007FF):        "߿"                                       |
2.2.3  3 bytes (U-0000FFFF):        "￿"                                       |
2.2.4  4 bytes (U-001FFFFF):        "����"                                       |
2.2.5  5 bytes (U-03FFFFFF):        "�����"                                       |
2.2.6  6 bytes (U-7FFFFFFF):        "������"                                       |
                                                                              |
2.3  Other boundary conditions                                                |
                                                                              |
2.3.1  U-0000D7FF = ed 9f bf = "퟿"                                            |
2.3.2  U-0000E000 = ee 80 80 = ""                                            |
2.3.3  U-0000FFFD = ef bf bd = "�"                                            |
2.3.4  U-0010FFFF = f4 8f bf bf = "􏿿"                                         |
2.3.5  U-00110000 = f4 90 80 80 = "����"                                         |
                                                                              |
3  Malformed sequences                                                        |
                                                                              |
3.1  Unexpected continuation bytes                                            |
                                                                              |
Each unexpected continuation byte should be separately signalled as a         |
malformed sequence of its own.                                                |
