[ANN] GraphQL Erlang 0.8.0

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[ANN] GraphQL Erlang 0.8.0

Jesper Louis Andersen-2
Hi,

I'm happy to announce that ShopGun's GraphQL code is now open source:

https://github.com/shopgun/graphql-erlang

GraphQL (http://graphql.org) is a query language for the web which is an alternative to RESTful web services. There are some similarity to Joe's UBF stack as well, though it is a completely different take. Originally from Facebook, we implement a GraphQL engine which is almost fully featured in Erlang.

I have a talk at the Erlang Users Conference 2017 on this project, but we are releasing the system early since we are far enough in its lifetime to do so, and there is little reason to keep it closed anymore. If you are interested in this project, please come see the talk :)

Because GraphQL is a fairly large system, there is an accompanying tutorial to the project, which uses the graphql backend:

https://github.com/shopgun/graphql-erlang-tutorial

and it has a quite complete tutorial book written as well (pending some sections which is still being written)

https://shopgun.github.io/graphql-erlang-tutorial/

It is our hope that the tutorial will help people use the repository and build some great software on top of it.

As always, comments are welcome. Either in this thread or in Issues in the projects if need be. PRs are also accepted and we want to encourage an open development strategy on the system. While the project probably still has some rough edges, we expect to grind them down over the coming months.

Happy Hacking on behalf of the ShopGun team :)



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [ANN] GraphQL Erlang 0.8.0

Son Tran-Nguyen-3
Hi Jesper,

I just want to say thank you for open source this. You can't imagine how grateful I am to read that the library use binary for keys. Other implementations in Elixir use atoms everywhere. With server that can take a dynamic schema from users, using atoms is just bound to fill Erlang's atom table.

Again, thank you. I'm looking forward to using this.

On Mon, May 22, 2017 at 9:26 AM, Jesper Louis Andersen <[hidden email]> wrote:
Hi,

I'm happy to announce that ShopGun's GraphQL code is now open source:

https://github.com/shopgun/graphql-erlang

GraphQL (http://graphql.org) is a query language for the web which is an alternative to RESTful web services. There are some similarity to Joe's UBF stack as well, though it is a completely different take. Originally from Facebook, we implement a GraphQL engine which is almost fully featured in Erlang.

I have a talk at the Erlang Users Conference 2017 on this project, but we are releasing the system early since we are far enough in its lifetime to do so, and there is little reason to keep it closed anymore. If you are interested in this project, please come see the talk :)

Because GraphQL is a fairly large system, there is an accompanying tutorial to the project, which uses the graphql backend:

https://github.com/shopgun/graphql-erlang-tutorial

and it has a quite complete tutorial book written as well (pending some sections which is still being written)

https://shopgun.github.io/graphql-erlang-tutorial/

It is our hope that the tutorial will help people use the repository and build some great software on top of it.

As always, comments are welcome. Either in this thread or in Issues in the projects if need be. PRs are also accepted and we want to encourage an open development strategy on the system. While the project probably still has some rough edges, we expect to grind them down over the coming months.

Happy Hacking on behalf of the ShopGun team :)



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [ANN] GraphQL Erlang 0.8.0

Jesper Louis Andersen-2
Hi Son,

You can create a safe hybrid solution in GraphQL which uses atoms, and it is one of the changes we are considering. The parser will produce binary/string data since it is in the control of the client and we can't trust the client. But in the elaboration step, we can run list_to_existing_atom/1 and binary_to_existing_atom/1. If we don't happen to have that atom we know the query is structurally invalid and we can just reject it. If it is structurally valid, we can safely convert into atoms and use them. GraphQL is a statically typed system and we can use this fact to handle it safely.

The reason we are considering the change is that it would be more idiomatic than the current solution. One particular strong point is that the dialyzer is able to describe maps in which the domain is given by atoms. But it can't describe maps in which the domain are binaries.

The reason we use binaries right now is that it was the easy solution. The system does allow atom-use in the 'enum' types for now, but we plan on folding enum handling into a solution which is more akin to scalar handling. This moves the flexibility onto the developer and we don't have to add code to the graphql system whenever people want a new encoding of enums. GraphQL is a fairly big system and mapping it onto the Erlang world took some trial and error. Some of the trials turned out to be less optimal. But one also has to put a foot down at some point and get something out rather than keeping to optimize for elegance.

Now, having said all that: if you just blindly convert client-side-data into atoms, you can easily get into trouble with clients: either maliciously or innocently, and this is a choice that should be rejected IMO.


On Wed, May 24, 2017 at 4:23 AM Son Tran-Nguyen <[hidden email]> wrote:
Hi Jesper,

I just want to say thank you for open source this. You can't imagine how grateful I am to read that the library use binary for keys. Other implementations in Elixir use atoms everywhere. With server that can take a dynamic schema from users, using atoms is just bound to fill Erlang's atom table.

Again, thank you. I'm looking forward to using this.

On Mon, May 22, 2017 at 9:26 AM, Jesper Louis Andersen <[hidden email]> wrote:
Hi,

I'm happy to announce that ShopGun's GraphQL code is now open source:

https://github.com/shopgun/graphql-erlang

GraphQL (http://graphql.org) is a query language for the web which is an alternative to RESTful web services. There are some similarity to Joe's UBF stack as well, though it is a completely different take. Originally from Facebook, we implement a GraphQL engine which is almost fully featured in Erlang.

I have a talk at the Erlang Users Conference 2017 on this project, but we are releasing the system early since we are far enough in its lifetime to do so, and there is little reason to keep it closed anymore. If you are interested in this project, please come see the talk :)

Because GraphQL is a fairly large system, there is an accompanying tutorial to the project, which uses the graphql backend:

https://github.com/shopgun/graphql-erlang-tutorial

and it has a quite complete tutorial book written as well (pending some sections which is still being written)

https://shopgun.github.io/graphql-erlang-tutorial/

It is our hope that the tutorial will help people use the repository and build some great software on top of it.

As always, comments are welcome. Either in this thread or in Issues in the projects if need be. PRs are also accepted and we want to encourage an open development strategy on the system. While the project probably still has some rough edges, we expect to grind them down over the coming months.

Happy Hacking on behalf of the ShopGun team :)



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [ANN] GraphQL Erlang 0.8.0

Son Tran-Nguyen-3
Hi Jesper,

I always prefer binary use over atom. That's what I like about your library. Like you said, in my use case of handling client-side data, I definitely don't want to convert them to atoms.

If any, please consider letting the develop to choose whether they want to convert the binary to atom. If the library switch itself to using atoms, then the developer cannot really do anything about it.


On Wed, May 24, 2017 at 8:08 AM, Jesper Louis Andersen <[hidden email]> wrote:
Hi Son,

You can create a safe hybrid solution in GraphQL which uses atoms, and it is one of the changes we are considering. The parser will produce binary/string data since it is in the control of the client and we can't trust the client. But in the elaboration step, we can run list_to_existing_atom/1 and binary_to_existing_atom/1. If we don't happen to have that atom we know the query is structurally invalid and we can just reject it. If it is structurally valid, we can safely convert into atoms and use them. GraphQL is a statically typed system and we can use this fact to handle it safely.

The reason we are considering the change is that it would be more idiomatic than the current solution. One particular strong point is that the dialyzer is able to describe maps in which the domain is given by atoms. But it can't describe maps in which the domain are binaries.

The reason we use binaries right now is that it was the easy solution. The system does allow atom-use in the 'enum' types for now, but we plan on folding enum handling into a solution which is more akin to scalar handling. This moves the flexibility onto the developer and we don't have to add code to the graphql system whenever people want a new encoding of enums. GraphQL is a fairly big system and mapping it onto the Erlang world took some trial and error. Some of the trials turned out to be less optimal. But one also has to put a foot down at some point and get something out rather than keeping to optimize for elegance.

Now, having said all that: if you just blindly convert client-side-data into atoms, you can easily get into trouble with clients: either maliciously or innocently, and this is a choice that should be rejected IMO.


On Wed, May 24, 2017 at 4:23 AM Son Tran-Nguyen <[hidden email]> wrote:
Hi Jesper,

I just want to say thank you for open source this. You can't imagine how grateful I am to read that the library use binary for keys. Other implementations in Elixir use atoms everywhere. With server that can take a dynamic schema from users, using atoms is just bound to fill Erlang's atom table.

Again, thank you. I'm looking forward to using this.

On Mon, May 22, 2017 at 9:26 AM, Jesper Louis Andersen <[hidden email]> wrote:
Hi,

I'm happy to announce that ShopGun's GraphQL code is now open source:

https://github.com/shopgun/graphql-erlang

GraphQL (http://graphql.org) is a query language for the web which is an alternative to RESTful web services. There are some similarity to Joe's UBF stack as well, though it is a completely different take. Originally from Facebook, we implement a GraphQL engine which is almost fully featured in Erlang.

I have a talk at the Erlang Users Conference 2017 on this project, but we are releasing the system early since we are far enough in its lifetime to do so, and there is little reason to keep it closed anymore. If you are interested in this project, please come see the talk :)

Because GraphQL is a fairly large system, there is an accompanying tutorial to the project, which uses the graphql backend:

https://github.com/shopgun/graphql-erlang-tutorial

and it has a quite complete tutorial book written as well (pending some sections which is still being written)

https://shopgun.github.io/graphql-erlang-tutorial/

It is our hope that the tutorial will help people use the repository and build some great software on top of it.

As always, comments are welcome. Either in this thread or in Issues in the projects if need be. PRs are also accepted and we want to encourage an open development strategy on the system. While the project probably still has some rough edges, we expect to grind them down over the coming months.

Happy Hacking on behalf of the ShopGun team :)



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions




_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [ANN] GraphQL Erlang 0.8.0

Jesper Louis Andersen-2
On Wed, May 24, 2017 at 4:31 PM Son Tran-Nguyen <[hidden email]> wrote:

If any, please consider letting the develop to choose whether they want to convert the binary to atom. If the library switch itself to using atoms, then the developer cannot really do anything about it.


If we make the switch, I'm pretty sure the developers won't be given a choice. The complexity of managing this will essentially mean every module in the system needs lots of small fixups. The cost/benefit analysis doesn't pan out.

The impedance mismatch is when you have a field as an atom, 'created' for instance. But your object stores it as <<"created">> in a map or proplist. In this case, the code would change from

execute(_, Obj, Field, _) -> {ok, maps:get(Field, Obj, null)}

to

execute(_, Obj, Field, _) -> {ok, maps:get(atom_to_binary(Field, utf8), Obj, null)}

which I don't think is a problem in the long run. It would help if the data is a #record{} type on the other hand.


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [ANN] GraphQL Erlang 0.8.0

duncan
In reply to this post by Jesper Louis Andersen-2
I thought unbounded atom creation was to be avoided. If you switch to
atoms, how do prevent the original concern "using atoms is just bound to
fill Erlang's atom table", especially if it is driven by user data?
I could foresee a hacker crashing the system by just varying lots of
inputs.  If the failure is limited by your architecture to just
processes of that user then you can just 'let it fail'. But if filling
the atom table is a wider crash than just a single user-specific server
process, then it's worth thinking about avoiding.

Duncan Sparrell
sFractal Consulting LLC
iPhone, iTypo, iApologize



-------- Original Message --------
Subject: Re: [erlang-questions] [ANN] GraphQL Erlang 0.8.0
From: Jesper Louis Andersen <[hidden email]>
Date: Wed, May 24, 2017 10:57 am
To: Son Tran-Nguyen <[hidden email]>
Cc: "Erlang (E-mail)" <[hidden email]>

On Wed, May 24, 2017 at 4:31 PM Son Tran-Nguyen <[hidden email]> wrote:

If any, please consider letting the develop to choose whether they want
to convert the binary to atom. If the library switch itself to using
atoms, then the developer cannot really do anything about it.





If we make the switch, I'm pretty sure the developers won't be given a
choice. The complexity of managing this will essentially mean every
module in the system needs lots of small fixups. The cost/benefit
analysis doesn't pan out.


The impedance mismatch is when you have a field as an atom, 'created'
for instance. But your object stores it as <<"created">> in a map or
proplist. In this case, the code would change from


execute(_, Obj, Field, _) -> {ok, maps:get(Field, Obj, null)}


to


execute(_, Obj, Field, _) -> {ok, maps:get(atom_to_binary(Field, utf8),
Obj, null)}


which I don't think is a problem in the long run. It would help if the
data is a #record{} type on the other hand.



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [ANN] GraphQL Erlang 0.8.0

Michał Muskała
There's no unbounded atom creation. Since you have the GraphQL schema, you know upfront which atoms
are allowed and which aren't. You only convert those that are allowed.

Michał.

On 25 May 2017, 15:19 +0200, [hidden email], wrote:
I thought unbounded atom creation was to be avoided. If you switch to
atoms, how do prevent the original concern "using atoms is just bound to
fill Erlang's atom table", especially if it is driven by user data?
I could foresee a hacker crashing the system by just varying lots of
inputs. If the failure is limited by your architecture to just
processes of that user then you can just 'let it fail'. But if filling
the atom table is a wider crash than just a single user-specific server
process, then it's worth thinking about avoiding.

Duncan Sparrell
sFractal Consulting LLC
iPhone, iTypo, iApologize



-------- Original Message --------
Subject: Re: [erlang-questions] [ANN] GraphQL Erlang 0.8.0
From: Jesper Louis Andersen <[hidden email]
Date: Wed, May 24, 2017 10:57 am
To: Son Tran-Nguyen <[hidden email]
Cc: "Erlang (E-mail)" <[hidden email]

On Wed, May 24, 2017 at 4:31 PM Son Tran-Nguyen <[hidden email]> wrote:

If any, please consider letting the develop to choose whether they want
to convert the binary to atom. If the library switch itself to using
atoms, then the developer cannot really do anything about it.





If we make the switch, I'm pretty sure the developers won't be given a
choice. The complexity of managing this will essentially mean every
module in the system needs lots of small fixups. The cost/benefit
analysis doesn't pan out.


The impedance mismatch is when you have a field as an atom, 'created'
for instance. But your object stores it as <<"created">> in a map or
proplist. In this case, the code would change from


execute(_, Obj, Field, _) -> {ok, maps:get(Field, Obj, null)}


to


execute(_, Obj, Field, _) -> {ok, maps:get(atom_to_binary(Field, utf8),
Obj, null)}


which I don't think is a problem in the long run. It would help if the
data is a #record{} type on the other hand.



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [ANN] GraphQL Erlang 0.8.0

Son Tran-Nguyen-3
There are cases when the GraphQL schema is not known at compile time.

SaaS services, for example, like Graphcool, ReIndex.IO, etc..., allow the users to 
define their own GraphQL schema during runtime.

On Thu, May 25, 2017 at 8:28 AM, Michał Muskała <[hidden email]> wrote:
There's no unbounded atom creation. Since you have the GraphQL schema, you know upfront which atoms
are allowed and which aren't. You only convert those that are allowed.

Michał.

On 25 May 2017, 15:19 +0200, [hidden email], wrote:
I thought unbounded atom creation was to be avoided. If you switch to
atoms, how do prevent the original concern "using atoms is just bound to
fill Erlang's atom table", especially if it is driven by user data?
I could foresee a hacker crashing the system by just varying lots of
inputs. If the failure is limited by your architecture to just
processes of that user then you can just 'let it fail'. But if filling
the atom table is a wider crash than just a single user-specific server
process, then it's worth thinking about avoiding.

Duncan Sparrell
sFractal Consulting LLC
iPhone, iTypo, iApologize



-------- Original Message --------
Subject: Re: [erlang-questions] [ANN] GraphQL Erlang 0.8.0
From: Jesper Louis Andersen <[hidden email]
Date: Wed, May 24, 2017 10:57 am
To: Son Tran-Nguyen <[hidden email]
Cc: "Erlang (E-mail)" <[hidden email]

On Wed, May 24, 2017 at 4:31 PM Son Tran-Nguyen <[hidden email]> wrote:

If any, please consider letting the develop to choose whether they want
to convert the binary to atom. If the library switch itself to using
atoms, then the developer cannot really do anything about it.





If we make the switch, I'm pretty sure the developers won't be given a
choice. The complexity of managing this will essentially mean every
module in the system needs lots of small fixups. The cost/benefit
analysis doesn't pan out.


The impedance mismatch is when you have a field as an atom, 'created'
for instance. But your object stores it as <<"created">> in a map or
proplist. In this case, the code would change from


execute(_, Obj, Field, _) -> {ok, maps:get(Field, Obj, null)}


to


execute(_, Obj, Field, _) -> {ok, maps:get(atom_to_binary(Field, utf8),
Obj, null)}


which I don't think is a problem in the long run. It would help if the
data is a #record{} type on the other hand.



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [ANN] GraphQL Erlang 0.8.0

Jesper Louis Andersen-2
Son,

This is a good reason to keep the current state as it is. It has to be taken into consideration at least and weighed. If a user of GraphQL creates new atoms, I'm less concerned than if a random client does. But the dynamic nature is something to think about if we change this to some other representation. The current one, binaries, has served us pretty well at ShopGun, so a priori there is no reason to change this unless the atom solution gives us some kind of advantage down the road.


On Thu, May 25, 2017 at 9:27 PM Son Tran-Nguyen <[hidden email]> wrote:
There are cases when the GraphQL schema is not known at compile time.

SaaS services, for example, like Graphcool, ReIndex.IO, etc..., allow the users to 
define their own GraphQL schema during runtime.

On Thu, May 25, 2017 at 8:28 AM, Michał Muskała <[hidden email]> wrote:
There's no unbounded atom creation. Since you have the GraphQL schema, you know upfront which atoms
are allowed and which aren't. You only convert those that are allowed.

Michał.

On 25 May 2017, 15:19 +0200, [hidden email], wrote:
I thought unbounded atom creation was to be avoided. If you switch to
atoms, how do prevent the original concern "using atoms is just bound to
fill Erlang's atom table", especially if it is driven by user data?
I could foresee a hacker crashing the system by just varying lots of
inputs. If the failure is limited by your architecture to just
processes of that user then you can just 'let it fail'. But if filling
the atom table is a wider crash than just a single user-specific server
process, then it's worth thinking about avoiding.

Duncan Sparrell
sFractal Consulting LLC
iPhone, iTypo, iApologize



-------- Original Message --------
Subject: Re: [erlang-questions] [ANN] GraphQL Erlang 0.8.0
From: Jesper Louis Andersen <[hidden email]
Date: Wed, May 24, 2017 10:57 am
To: Son Tran-Nguyen <[hidden email]
Cc: "Erlang (E-mail)" <[hidden email]

On Wed, May 24, 2017 at 4:31 PM Son Tran-Nguyen <[hidden email]> wrote:

If any, please consider letting the develop to choose whether they want
to convert the binary to atom. If the library switch itself to using
atoms, then the developer cannot really do anything about it.





If we make the switch, I'm pretty sure the developers won't be given a
choice. The complexity of managing this will essentially mean every
module in the system needs lots of small fixups. The cost/benefit
analysis doesn't pan out.


The impedance mismatch is when you have a field as an atom, 'created'
for instance. But your object stores it as <<"created">> in a map or
proplist. In this case, the code would change from


execute(_, Obj, Field, _) -> {ok, maps:get(Field, Obj, null)}


to


execute(_, Obj, Field, _) -> {ok, maps:get(atom_to_binary(Field, utf8),
Obj, null)}


which I don't think is a problem in the long run. It would help if the
data is a #record{} type on the other hand.



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Loading...