New project: ZJ - A tiny JSON encoder/decoder

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: New project: ZJ - A tiny JSON encoder/decoder

zxq9-2
On 2018年6月28日木曜日 17時50分04秒 JST [hidden email] wrote:
> On 2018年6月28日木曜日 10時44分13秒 JST you wrote:
> > For testing JSON parsers/encoders I recommend https://github.com/nst/JSONTestSuite
>
> Thanks, Michał!
>
> Over the weekend I might hook the two together.
> If it isn't too complicated I might drop a script for this in there.

Ran through the test set in the shell just to get some idea, and there
are a few cases that do need to be fixed -- all are weird (but legal) or
malformed (and some dangerous) number representations.

I'll probably put a test function in there that does what I just did
in the shell. Including the test set into the repo is a little :-/
Including ZJ into the JSON test project might be a better approach.

Once the handful of failing cases are sorted, I don't expect to be
patching ZJ much.

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: New project: ZJ - A tiny JSON encoder/decoder

Jesper Louis Andersen-2
In reply to this post by Dmitry Belyaev-3
On Thu, Jun 28, 2018 at 6:50 AM Dmitry Belyaev <[hidden email]> wrote:
I could never understand where this conversion JSON null <-> Erlang undefined was coming from.

Of course we generally don't use null atom in the code, but I always considered "undefined" value in Erlang meaning "absent" value. For example when a record is created and some field is not populated explicitly, it is set undefined.

​Both variants are somewhat valid: Common Lisp generally treats NIL as the false value, the empty list, null, ... whereas other languages makes the distinction: [], false and null are different things in Erlang for instance.

In a statically typed language, you probably would split them. In particular you would have

type void
type unit = ()
type json = Int of int | String of string | ...  | Null
​Where 'void' is the uninhabitable type which can never be realized in a program​, 'unit' is the type of one result (and only that result) and a Null is something of type json.

In Erlang, undefined often corresponds to the unit type. It isn't the same as void, because in Erlang every term has to return a value, and you cannot return something of type void.

The tradeoff in the end is that of convenience: null == undefined means you can share some data, but you then lost the ability to discriminate null and undefined which from a type/class perspective might be desirable.



--
J.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: New project: ZJ - A tiny JSON encoder/decoder

zxq9-2
In reply to this post by zxq9-2
On 2018年6月28日木曜日 19時16分42秒 JST [hidden email] wrote:

> On 2018年6月28日木曜日 17時50分04秒 JST [hidden email] wrote:
> > On 2018年6月28日木曜日 10時44分13秒 JST you wrote:
> > > For testing JSON parsers/encoders I recommend https://github.com/nst/JSONTestSuite
> >
> > Thanks, Michał!
> >
> > Over the weekend I might hook the two together.
> > If it isn't too complicated I might drop a script for this in there.
>
> Ran through the test set in the shell just to get some idea, and there
> are a few cases that do need to be fixed -- all are weird (but legal) or
> malformed (and some dangerous) number representations.
>
> I'll probably put a test function in there that does what I just did
> in the shell. Including the test set into the repo is a little :-/
> Including ZJ into the JSON test project might be a better approach.
>
> Once the handful of failing cases are sorted, I don't expect to be
> patching ZJ much.

All cases that MUST work according to the JSON Test Suite project now do.

A handful of cases that SHOULD NOT work do anyway (trailing commas, for
example) -- but are mostly of no consequence.

A handful of cases that MAY work do, but most do not.

Several cases that MAY or SHOULD NOT work return clean or truncated
results in decode/1 but return an error in binary_decode/1 due to the
way unicode:characters_to_list/1 and unicode:characters_to_binary/1
work.

One test case crashes: n_structure_open_array_object.json
It is not supposed to work and and crashes many parsers -- this one
included. The attempted allocation of tens of thousands of nested maps
OOMs the runtime. ZJ does not define (or track) object/array depth.

I might implement depth tracking later just for safety.

Results are here:
https://gitlab.com/zxq9/zj/wikis/json%20test%20suite%20results

I don't have time to formalize all this, but I suspect that the JSON
definition and test cases won't change much in the future.

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
12