UON notation serialization and parsing support

##### Table of Contents

## 1 - URL encoding support overview

Juneau supports converting arbitrary POJOs to and from URL-encoded strings using ultra-efficient serializers and parsers.
The serializer converts POJOs directly to URL-encoded strings without the need for intermediate DOM objects using a highly-efficient state machine.
Likewise, the parser creates POJOs directly from URL-encoded strings without the need for intermediate DOM objects.

Juneau uses UON (URL-Encoded Object Notation) for representing POJOs. The UON specification can be found here.

Juneau can serialize and parse instances of any of the following POJO types:

- Java primitives and primitive objects (e.g. `String`, `Integer`, `Boolean`, `Float`).
- Java Collections Framework objects (e.g. `HashSet`, `TreeMap`) containing anything on this list.
- Multi-dimensional arrays of any type on this list.
- Java Beans with properties of any type on this list.
- Classes with standard transformations to and from `Strings` (e.g. classes containing `toString()`, `fromString()`, `valueOf()`, `constructor(String)`).
- Non-serializable classes and properties with associated `PojoSwaps` that convert them to serializable forms.

Refer to POJO Categories for a complete definition of supported POJOs.

###### Prerequisites

The Juneau URL-encoding serialization and parsing support does not require any external prerequisites. It only requires Java 1.6 or above.

### 1.1 - URL-encoding support overview - example

The example shown here is from the Address Book resource located in the `org.apache.juneau.sample.war` application.
The POJO model consists of a `List` of `Person` beans, with each `Person` containing zero or more `Address` beans.

When you point a browser at `/sample/addressBook/people/1`, the POJO is rendered as HTML:

By appending `?Accept=application/x-www-form-urlencoded&plainText=true` to the URL, you can view the data as a URL-encoded string:

0=( uri=http://localhost:10000/addressBook/people/1, addressBookUri=http://localhost:10000/addressBook/, id=1, name='Barack+Obama', birthDate='Aug+4,+1961', addresses=@( ( uri=http://localhost:10000/addressBook/addresses/1, personUri=http://localhost:10000/addressBook/people/1, id=1, street='1600+Pennsylvania+Ave', city=Washington, state=DC, zip=20500, isCurrent=true ), ( uri=http://localhost:10000/addressBook/addresses/2, personUri=http://localhost:10000/addressBook/people/1, id=2, street='5046+S+Greenwood+Ave', city=Chicago, state=IL, zip=60615, isCurrent=false ) ), age=56 ) &1=( uri=http://localhost:10000/addressBook/people/2, addressBookUri=http://localhost:10000/addressBook/, id=2, name='George+Walker+Bush', birthDate='Jul+6,+1946', addresses=@( ( uri=http://localhost:10000/addressBook/addresses/3, personUri=http://localhost:10000/addressBook/people/2, id=3, street='43+Prairie+Chapel+Rd', city=Crawford, state=TX, zip=76638, isCurrent=true ), ( uri=http://localhost:10000/addressBook/addresses/4, personUri=http://localhost:10000/addressBook/people/2, id=4, street='1600+Pennsylvania+Ave', city=Washington, state=DC, zip=20500, isCurrent=false ) ), age=71 )

Juneau supports two kinds of serialization:

- Construction of full URL query parameter strings (e.g. `&key=value` pairs) from beans and maps.
- Construction of URL query parameter value strings (e.g. just the `value` portion of `&key=value` pairs) from any POJO.

Top-level beans and maps can serialized as key/value pairs as shown below:

###### Example: A bean with 2 string properties, 'foo' and 'baz', serialized to a query string

http://localhost/sample?foo=bar&baz=bing

Lower-level beans and maps are also serialized as key/value pairs, but are surrounded with a "(...)" construct to denote an object mapping, and uses a comma as the parameter delimiter instead of "&".

###### Example: A bean serialized as a query parameter value.

http://localhost/sample?a1=(foo=bar,baz=bing)

###### General methodology:

Java type | JSON equivalent | UON
--- | --- | ---
Maps/beans | OBJECT | a1=(b1=x1,b2=x2) a1=(b1=(c1=x1,c2=x2))
Collections/arrays | ARRAY | a1=@(x1,x2) a1=@(@(x1,x2),@(x3,x4)) a1=@((b1=x1,b2=x2),(c1=x1,c2=x2))
Booleans | BOOLEAN | a1=true&a2=false
int/float/double/... | NUMBER | a1=123&a2=1.23e1
null | NULL | a1=null
String | STRING | a1=foobar a1='true' a1='null' a1='123' a1=' string with whitespace ' a1='string with ~'escaped~' quotes'

Refer to the UON specification for a complete set of syntax rules.

`PojoSwaps` can be used to convert non-serializable POJOs into serializable forms, such as converting `Calendar` object to ISO8601 strings, or `byte[]` arrays to Base-64 encoded strings.
These transforms can be associated at various levels:

- On serializer and parser instances to handle all objects of the class type globally.
- On classes through the `@Bean` annotation.
- On bean properties through the `@BeanProperty` annotations.

###### Example: A serialized Calendar object using `CalendarSwap.RFC2822DTZ` transform.

http://localhost/sample?a1='Sun,+03+Mar+1901+09:05:06+GMT'

For more information about transforms, refer to org.apache.juneau.transform.

## 2 - UrlEncodingSerializer and UonSerializer classes

{@link org.apache.juneau.urlencoding.UrlEncodingSerializer} and {@link org.apache.juneau.uon.UonSerializer} classes are used to convert POJOs to URL-encoded strings.
The `UonSerializer` class converts parameter values to UON notation. The `UrlEncodingSerializer` class converts a POJO to key/value URL-Encoded pairs using `UonSerializer` to serialize the values. If you're trying to construct complete URL-Encoded entities, use `UrlEncodingSerializer`. If you're constructing your own key/value pairs, use `UonSerializer`.

The serializers include several configurable settings.
Static reusable instances of serializers are provided with commonly-used settings:

- {@link org.apache.juneau.urlencoding.UrlEncodingSerializer#DEFAULT} - All default settings, strict mode.
- {@link org.apache.juneau.urlencoding.UrlEncodingSerializer#DEFAULT_READABLE} - Use whitespace and indentation for readability.
- {@link org.apache.juneau.uon.UonSerializer#DEFAULT} - All default settings, strict mode.
- {@link org.apache.juneau.uon.UonSerializer#DEFAULT_READABLE} - Use whitespace and indentation for readability.
- {@link org.apache.juneau.uon.UonSerializer#DEFAULT_ENCODING} - Same as DEFAULT, but use URL-Encoding on special characters.

The general guidelines on which serializer to use is:

- Use encoding serializers when you're using the results to construct a URI yourself, and therefore need invalid URI characters to be encoded.
- Use un-encoding serializers when you're creating parameter values and passing them off to some other utility class that will itself encode invalid URI characters.
- Use the readable serializer for debugging purposes.

###### Notes about examples

The examples shown in this document will use default strict settings.
For brevity, the examples will use public fields instead of getters/setters to reduce the size of the examples.
In the real world, you'll typically want to use standard bean getters and setters.

To start off simple, we'll begin with the following simplified bean and build upon it.

public class Person { // Bean properties public int id; public String name; // Bean constructor (needed by parser) public Person() {} // Normal constructor public Person(int id, String name) { this.id = id; this.name = name; } }

The following code shows how to convert this to a URL-encoded value:

// Use serializer with readable output, simple mode. UonSerializer s = UonSerializer.DEFAULT; // Create our bean. Person p = new Person(1, "John Smith"); // Serialize the bean to URL-encoded parameter value. String urlencoded = s.serialize(p);

The code above produces the following output:

(id=1,name='John+Smith')

The {@link org.apache.juneau.urlencoding.UrlEncodingSerializer} class converts maps and beans into top-level query parameter strings.

// Use serializer with readable output, simple mode. UrlEncodingSerializer s = UrlEncodingSerializer.DEFAULT; // Serialize the bean to URL-encoded query string. String urlencoded = s.serialize(p);

The code above produces the following output:

id=1&name='John+Smith'

By default, the `UrlEncodingSerializer` class will URL-Encode special characters, and the `UonSerializer` will NOT URL-encode special characters.

### 2.1 - @Bean and @BeanProperty annotations

The {@link org.apache.juneau.annotation.Bean @Bean} and {@link org.apache.juneau.annotation.BeanProperty @BeanProperty} annotations are used to customize the behavior of beans across the entire framework.
They have various uses:

- Hiding bean properties.
- Specifying the ordering of bean properties.
- Overriding the names of bean properties.
- Associating transforms at both the class and property level (to convert non-serializable POJOs to serializable forms).

For example, we now add a `birthDate` property, and associate a transform with it to transform it to an ISO8601 date-time string in GMT time.
We'll also add a couple of `URI` properties.
By default, `Calendars` are treated as beans by the framework, which is usually not how you want them serialized.
Using transforms, we can convert them to standardized string forms.

public class Person { // Bean properties public int id; public String name; public URI uri; public URI addressBookUri; @BeanProperty(swap=CalendarSwap.ISO8601DTZ.class) public Calendar birthDate; // Bean constructor (needed by parser) public Person() {} // Normal constructor public Person(int id, String name, String uri, String addressBookUri, String birthDate) throws Exception { this.id = id; this.name = name; this.uri = new URI(uri); this.addressBookUri = new URI(addressBookUri); this.birthDate = new GregorianCalendar(); this.birthDate .setTime(DateFormat.getDateInstance(DateFormat.MEDIUM).parse(birthDate)); } }

Next, we alter our code to pass in the birthdate:

// Create our bean. Person p = new Person(1, "John Smith", "http://sample/addressBook/person/1", "http://sample/addressBook", "Aug 12, 1946");

Now when we rerun the sample code, we'll get the following:

(id=1,name='John+Smith',uri=http://sample/addressBook/person/1, addressBookUri=http://sample/addressBook,birthDate=1946-08-12T00:00:00Z)

Using `UrlEncodingSerializer` instead would create the following:

id=1&name='John+Smith'&uri=http://sample/addressBook/person/1 &addressBookUri=http://sample/addressBook&birthDate=1946-08-12T00:00:00Z

Another useful feature is the {@link org.apache.juneau.annotation.Bean#propertyNamer()} annotation that allows you to plug in your own logic for determining bean property names.
The {@link org.apache.juneau.PropertyNamerDLC} is an example of an alternate property namer. It converts bean property names to lowercase-dashed format.

###### Example:

@Bean(propertyNamer=PropertyNamerDLC.class) public class Person { ...

###### Results

(id=1,name='John+Smith',uri=http://sample/addressBook/person/1, address-book-uri=http://sample/addressBook,birth-date=1946-08-12T00:00:00Z)

### 2.2 - Collections

In our example, let's add a list-of-beans property to our sample class:

public class Person { // Bean properties public LinkedList<Address> addresses = new LinkedList<Address>(); ... }

The `Address` class has the following properties defined:

public class Address { // Bean properties public URI uri; public URI personUri; public int id; public String street, city, state; public int zip; public boolean isCurrent; }

Next, add some quick-and-dirty code to add an address to our person bean:

// Use serializer with readable output, simple mode. UonSerializer s = UonSerializer.DEFAULT_READABLE; // Create our bean. Person p = new Person(1, "John Smith", "http://sample/addressBook/person/1", "http://sample/addressBook", "Aug 12, 1946"); Address a = new Address(); a.uri = new URI("http://sample/addressBook/address/1"); a.personUri = new URI("http://sample/addressBook/person/1"); a.id = 1; a.street = "100 Main Street"; a.city = "Anywhereville"; a.state = "NY"; a.zip = 12345; a.isCurrent = true; p.addresses.add(a);

Now when we run the sample code, we get the following (in readable format):

( id=1, name='John+Smith', uri=http://sample/addressBook/person/1, addressBookUri=http://sample/addressBook, birthDate=1946-08-12T00:00:00Z, addresses=@( ( uri=http://sample/addressBook/address/1, personUri=http://sample/addressBook/person/1, id=1, street='100+Main+Street', city=Anywhereville, state=NY, zip=12345, isCurrent=true ) ) )

If we were to use `UrlEncodingSerializer` instead, we would get the following:

id=1& name='John+Smith'& uri=http://sample/addressBook/person/1& addressBookUri=http://sample/addressBook& birthDate=1946-08-12T00:00:00Z& addresses=@( ( uri=http://sample/addressBook/address/1, personUri=http://sample/addressBook/person/1, id=1, street='100+Main+Street', city=Anywhereville, state=NY, zip=12345, isCurrent=true ) )

Note how the top level `Person` bean is serialized using the standard '&' delimiter, whereas the lower-level `Address` bean is serialized using the ',' character to prevent the `addresses` field from being incompletely parsed.

### 2.3 - Non-tree models and recursion detection

The URL-encoding serializer is designed to be used against POJO tree structures. It expects that there not be loops in the POJO model (e.g. children with references to parents, etc...).
If you try to serialize models with loops, you will usually cause a `StackOverflowError` to be thrown (if {@link org.apache.juneau.serializer.Serializer#SERIALIZER_maxDepth} is not reached first).

If you still want to use the URL-encoding serializer on such models, Juneau provides the {@link org.apache.juneau.serializer.Serializer#SERIALIZER_detectRecursions} setting.
It tells the serializer to look for instances of an object in the current branch of the tree and skip serialization when a duplicate is encountered.

For example, let's make a POJO model out of the following classes:

public class A { public B b; } public class B { public C c; } public class C { public A a; }

Now we create a model with a loop and serialize the results.

// Clone an existing serializer and set property for detecting recursions. UrlEncodingSerializer s = UrlEncodingSerializer.DEFAULT_READABLE.builder() .detectRecursions(true).build(); // Create a recursive loop. A a = new A(); a.b = new B(); a.b.c = new C(); a.b.c.a = a; // Serialize. String json = s.serialize(a);

What we end up with is the following, which does not serialize the contents of the `c` field:

( b=( c=() ) )

Without recursion detection enabled, this would cause a stack-overflow error.

Recursion detection introduces a performance penalty of around 20%. For this reason the setting is disabled by default.

### 2.4 - Configurable properties

See the following classes for all configurable properties that can be used on this serializer:

- {@link org.apache.juneau.BeanContext} - Bean context properties.
- {@link org.apache.juneau.uon.UonSerializerContext} - UON serializer context properties.
- {@link org.apache.juneau.urlencoding.UrlEncodingSerializerContext} - URL-Encoding serializer context properties.

### 2.5 - Other notes

- Like all other Juneau serializers, the URL-encoding serializers are thread safe and maintain an internal cache of bean classes encountered.
  For performance reasons, it's recommended that serializers be reused whenever possible instead of always creating new instances.

## 3 - UrlEncodingParser and UonParser classes

{@link org.apache.juneau.urlencoding.UrlEncodingParser} and {@link org.apache.juneau.uon.UonParser} classes are used to convert URL-encoded strings back into POJOs.
The `UonParser` class converts UON-encoded parameter values to POJOs. The `UrlEncodingParser` class converts entire URL-Encoded strings to POJOs using `UonSerializer` to serialize individual values. If you're trying to parse an entire URL-Encoded string, use `UrlEncodingParser`. If you're trying to parse an individual value (such as that returned by `RestServlet.getQueryParameter(name)`), use `UonParser`.

The following static reusable instances of `UrlEncodingParser` are provided for convenience:

- {@link org.apache.juneau.urlencoding.UrlEncodingParser#DEFAULT} - Default parser for entire URL-encoded strings, decode `%xx` sequences.
- {@link org.apache.juneau.uon.UonParser#DEFAULT} - Default parser for URL-encoded parameter values, don't decode `%xx` sequences.
- {@link org.apache.juneau.uon.UonParser#DEFAULT_DECODING} - Default parser for URL-encoded parameter values, decode `%xx` sequences.

The general guidelines on which parser to use is:

- Use the `DEFAULT` parser for parameter values that have already had `%xx` sequences decoded, such as when using `HttpServletRequest.getQueryParameter(name)`.
- Use the `DEFAULT_ENCODED` parser if the input has not already had `%xx` sequences decoded.

Let's build upon the previous example and parse the generated URL-encoded string back into the original bean.
We start with the URL-encoded string that was generated.

// Use serializer with readable output. UonSerializer s = UonSerializer.DEFAULT_READABLE; // Create our bean. Person p = new Person(1, "John Smith", "http://sample/addressBook/person/1", "http://sample/addressBook", "Aug 12, 1946"); Address a = new Address(); a.uri = new URI("http://sample/addressBook/address/1"); a.personUri = new URI("http://sample/addressBook/person/1"); a.id = 1; a.street = "100 Main Street"; a.city = "Anywhereville"; a.state = "NY"; a.zip = 12345; a.isCurrent = true; p.addresses.add(a); // Serialize the bean. String urlencoded = s.serialize(p);

This code produced the following:

( id=1, name='John+Smith', uri=http://sample/addressBook/person/1, addressBookUri=http://sample/addressBook, birthDate=1946-08-12T00:00:00Z, addresses=@( ( uri=http://sample/addressBook/address/1, personUri=http://sample/addressBook/person/1, id=1, street='100+Main+Street', city=Anywhereville, state=NY, zip=12345, isCurrent=true ) ) )

The code to convert this back into a bean is:

// Parse it back into a bean using the reusable JSON parser. Person p = UonParser.DEFAULT.parse(urlencoded, Person.class); // Render it back as JSON. json = JsonSerializer.DEFAULT_LAX_READABLE.serialize(p);

We print it back out to JSON to show that all the data has been preserved:

{ id: 1, name: 'John Smith', uri: 'http://sample/addressBook/person/1', addressBookUri: 'http://sample/addressBook', birthDate: '1946-08-12T00:00:00Z', addresses: [ { uri: 'http://sample/addressBook/address/1', personUri: 'http://sample/addressBook/person/1', id: 1, street: '100 Main Street', city: 'Anywhereville', state: 'NY', zip: 12345, isCurrent: true } ] }

### 3.1 - Parsing into generic POJO models

The URL-encoding parser is not limited to parsing back into the original bean classes.
If the bean classes are not available on the parsing side, the parser can also be used to parse into a generic model consisting of `Maps`, `Collections`, and primitive objects.

You can parse into any `Map` type (e.g. `HashMap`, `TreeMap`), but using {@link org.apache.juneau.ObjectMap} is recommended since it has many convenience methods for converting values to various types.
The same is true when parsing collections. You can use any Collection (e.g. `HashSet`, `LinkedList`) or array (e.g. `Object[]`, `String[]`, `String[][]`), but using {@link org.apache.juneau.ObjectList} is recommended.

When the map or list type is not specified, or is the abstract `Map`, `Collection`, or `List` types, the parser will use `ObjectMap` and `ObjectList` by default.

Starting back with our original URL-encoded string:

( id=1, name='John+Smith', uri=http://sample/addressBook/person/1, addressBookUri=http://sample/addressBook, birthDate=1946-08-12T00:00:00Z, addresses=@( ( uri=http://sample/addressBook/address/1, personUri=http://sample/addressBook/person/1, id=1, street='100+Main+Street', city=Anywhereville, state=NY, zip=12345, isCurrent=true ) ) )

We can parse this into a generic `ObjectMap`:

// Parse URL-encoded string into a generic POJO model. ObjectMap m = UonParser.DEFAULT.parse(urlencoded, ObjectMap.class); // Convert it back to JSON. String json = JsonSerializer.DEFAULT_LAX_READABLE.serialize(m);

What we end up with is the exact same output.
Even the numbers and booleans are preserved because they are parsed into `Number` and `Boolean` objects when parsing into generic models.

{ id: 1, name: 'John Smith', uri: 'http://sample/addressBook/person/1', addressBookUri: 'http://sample/addressBook', birthDate: '1946-08-12T00:00:00Z', addresses: [ { uri: 'http://sample/addressBook/address/1', personUri: 'http://sample/addressBook/person/1', id: 1, street: '100 Main Street', city: 'Anywhereville', state: 'NY', zip: 12345, isCurrent: true } ] }

Once parsed into a generic model, various convenience methods are provided on the `ObjectMap` and `ObjectList` classes to retrieve values:

// Parse URL-encoded string into a generic POJO model. ObjectMap m = UonParser.DEFAULT.parse(urlencoded, ObjectMap.class); // Get some simple values. String name = m.getString("name"); int id = m.getInt("id"); // Get a value convertable from a String. URI uri = m.get(URI.class, "uri"); // Get a value using a transform. CalendarSwap transform = new CalendarSwap.ISO8601DTZ(); Calendar birthDate = m.get(transform, "birthDate"); // Get the addresses. ObjectList addresses = m.getObjectList("addresses"); // Get the first address and convert it to a bean. Address address = addresses.get(Address.class, 0);

As a general rule, parsing into beans is often more efficient than parsing into generic models.
And working with beans is often less error prone than working with generic models.

### 3.2 - Configurable properties

See the following classes for all configurable properties that can be used on this parser:

- {@link org.apache.juneau.BeanContext} - Bean context properties.
- {@link org.apache.juneau.uon.UonParserContext} - UON parser context properties.
- {@link org.apache.juneau.urlencoding.UrlEncodingParserContext} - URL-Encoding parser context properties.

### 3.3 - Other notes

- Like all other Juneau parsers, the URL-encoding parsers are thread safe and maintain an internal cache of bean classes encountered.
  For performance reasons, it's recommended that parser be reused whenever possible instead of always creating new instances.

*** fín ***