Autotest: test the error conditions in the parser and the encoder

The encoder only supports two error conditions: out of memory (need more
buffer) and illegal simple types. It's possible for the encoder to
encode an array or map with a size the decoder can't parse (UINT32_MAX),
but that's not an error condition.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>