Why do "interface to language X" projects all seem to die out?

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Why do "interface to language X" projects all seem to die out?

Matthias Lang
Hi,

Pretty much all of the "not built in to OTP" ways to connect to other
languages seem to have died, i.e.

   - EPI (C++). Last commit 2009.
   - py_interface (Python). Last commit 2014. Freshest of the lot.
   - edtk (C). Last release 2007.
   - dryverl (C). Last release 2008.
   - erlua (Lua). Project page dead. Author's email is dead.
   - erlualib (last commit 2010). Github-linked company page dead.

Can anyone offer some experiences or ideas why?

Could it be that these types of solutions are rarely needed? Maybe
NIFs are unbeatable for the "narrow interface, no state, high
performance" cases and port programs with hand-rolled interfaces most
of the rest.

Matt
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Vans S
You should really be using NIFS to interface with C, C++, Rust and others.

Python is well supported, so apparently is Ruby (I only used python) through erlports. Look at the forks of erlports esp by beano, https://github.com/beano/erlport

Why would you want to use lua. Erlang is a much better replacement. And if you really need Lua then use C Nifs to load + execute the lua. Lua is so simple to integrate, you really don't need an entire library for that, just a simple C NIF.


On Wednesday, August 31, 2016 10:18 AM, Matthias Lang <[hidden email]> wrote:


Hi,

Pretty much all of the "not built in to OTP" ways to connect to other
languages seem to have died, i.e.

  - EPI (C++). Last commit 2009.
  - py_interface (Python). Last commit 2014. Freshest of the lot.
  - edtk (C). Last release 2007.
  - dryverl (C). Last release 2008.
  - erlua (Lua). Project page dead. Author's email is dead.
  - erlualib (last commit 2010). Github-linked company page dead.

Can anyone offer some experiences or ideas why?

Could it be that these types of solutions are rarely needed? Maybe
NIFs are unbeatable for the "narrow interface, no state, high
performance" cases and port programs with hand-rolled interfaces most
of the rest.

Matt
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Pierre Fenoll-2
In reply to this post by Matthias Lang


Cheers,
-- 
Pierre Fenoll


On 31 August 2016 at 16:18, Matthias Lang <[hidden email]> wrote:
Hi,

Pretty much all of the "not built in to OTP" ways to connect to other
languages seem to have died, i.e.

   - EPI (C++). Last commit 2009.
   - py_interface (Python). Last commit 2014. Freshest of the lot.
   - edtk (C). Last release 2007.
   - dryverl (C). Last release 2008.
   - erlua (Lua). Project page dead. Author's email is dead.
   - erlualib (last commit 2010). Github-linked company page dead.

Can anyone offer some experiences or ideas why?

Could it be that these types of solutions are rarely needed? Maybe
NIFs are unbeatable for the "narrow interface, no state, high
performance" cases and port programs with hand-rolled interfaces most
of the rest.

Matt
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Tuncer Ayaz
In reply to this post by Matthias Lang
On 31 August 2016 at 16:18, Matthias Lang <[hidden email]> wrote:

> Hi,
>
> Pretty much all of the "not built in to OTP" ways to connect to other
> languages seem to have died, i.e.
>
>    - EPI (C++). Last commit 2009.
>    - py_interface (Python). Last commit 2014. Freshest of the lot.
>    - edtk (C). Last release 2007.
>    - dryverl (C). Last release 2008.
>    - erlua (Lua). Project page dead. Author's email is dead.
>    - erlualib (last commit 2010). Github-linked company page dead.
>
> Can anyone offer some experiences or ideas why?
>
> Could it be that these types of solutions are rarely needed? Maybe
> NIFs are unbeatable for the "narrow interface, no state, high
> performance" cases and port programs with hand-rolled interfaces most
> of the rest.

https://hackage.haskell.org/package/erlang (alive)
https://github.com/jlouis/erlangnode (alive; not sure how complete)
https://github.com/EchoTeam/ocaml-erlang-port (very stable or abandoned)
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Garrett Smith-5
On Wed, Aug 31, 2016 at 10:36 AM, Tuncer Ayaz <[hidden email]> wrote:

> On 31 August 2016 at 16:18, Matthias Lang <[hidden email]> wrote:
>> Hi,
>>
>> Pretty much all of the "not built in to OTP" ways to connect to other
>> languages seem to have died, i.e.
>>
>>    - EPI (C++). Last commit 2009.
>>    - py_interface (Python). Last commit 2014. Freshest of the lot.
>>    - edtk (C). Last release 2007.
>>    - dryverl (C). Last release 2008.
>>    - erlua (Lua). Project page dead. Author's email is dead.
>>    - erlualib (last commit 2010). Github-linked company page dead.
>>
>> Can anyone offer some experiences or ideas why?
>>
>> Could it be that these types of solutions are rarely needed? Maybe
>> NIFs are unbeatable for the "narrow interface, no state, high
>> performance" cases and port programs with hand-rolled interfaces most
>> of the rest.
>
> https://hackage.haskell.org/package/erlang (alive)
> https://github.com/jlouis/erlangnode (alive; not sure how complete)
> https://github.com/EchoTeam/ocaml-erlang-port (very stable or abandoned)
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

I think in these distributed scenarios the go to protocol is something
language agnostic like something-over-http.

Personally I would not go around trying to get other languages to
speak the Erlang wire protocol. I've used the Java libraries to build
nodes and they worked, but I wouldn't do it again. Instead I'd use
something like 0MQ or again the ponderously inefficient but ubiquitous
something-over-http.

If I need to integrate with other languages more closely, I'll use the
port interface. Here's a terrific example of how that can be extended
into various lanauges:

http://erlport.org/

Unless a NIF is simple and well established as non-crashy, I'll write
performance critical code as a C port. People don't like this because
of the emotionally crushing thought of serializing data over stdio.
While that's laughably inefficient when compared to zippy NIFs
(troll), the code remains isolated and can crash all it wants without
bringing down the entire VM. It also encourages an interface design
that minimizes IO -- you performed as much work as you can on either
end and pass small results back and forth.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Ilya Khaprov
In reply to this post by Vans S

 

Hello,

 

Here is a library for writing nodes in Common Lisp: https://github.com/flambard/CLERIC.  

Last time I checked it worked.

 

Ilya.

 

 

From: [hidden email]
Sent: 31 августа 2016 г. 17:18
To: [hidden email]
Subject: [erlang-questions] Why do "interface to language X" projects all seem to die out?

 

Hi,

Pretty much all of the "not built in to OTP" ways to connect to other
languages seem to have died, i.e.

   - EPI (C++). Last commit 2009.
   - py_interface (Python). Last commit 2014. Freshest of the lot.
   - edtk (C). Last release 2007.
   - dryverl (C). Last release 2008.
   - erlua (Lua). Project page dead. Author's email is dead.
   - erlualib (last commit 2010). Github-linked company page dead.

Can anyone offer some experiences or ideas why?

Could it be that these types of solutions are rarely needed? Maybe
NIFs are unbeatable for the "narrow interface, no state, high
performance" cases and port programs with hand-rolled interfaces most
of the rest.

Matt
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Loïc Hoguin-3
In reply to this post by Garrett Smith-5
On 08/31/2016 05:50 PM, Garrett Smith wrote:

> On Wed, Aug 31, 2016 at 10:36 AM, Tuncer Ayaz <[hidden email]> wrote:
>> On 31 August 2016 at 16:18, Matthias Lang <[hidden email]> wrote:
>>> Hi,
>>>
>>> Pretty much all of the "not built in to OTP" ways to connect to other
>>> languages seem to have died, i.e.
>>>
>>>    - EPI (C++). Last commit 2009.
>>>    - py_interface (Python). Last commit 2014. Freshest of the lot.
>>>    - edtk (C). Last release 2007.
>>>    - dryverl (C). Last release 2008.
>>>    - erlua (Lua). Project page dead. Author's email is dead.
>>>    - erlualib (last commit 2010). Github-linked company page dead.
>>>
>>> Can anyone offer some experiences or ideas why?
>>>
>>> Could it be that these types of solutions are rarely needed? Maybe
>>> NIFs are unbeatable for the "narrow interface, no state, high
>>> performance" cases and port programs with hand-rolled interfaces most
>>> of the rest.
>>
>> https://hackage.haskell.org/package/erlang (alive)
>> https://github.com/jlouis/erlangnode (alive; not sure how complete)
>> https://github.com/EchoTeam/ocaml-erlang-port (very stable or abandoned)
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>
> I think in these distributed scenarios the go to protocol is something
> language agnostic like something-over-http.
>
> Personally I would not go around trying to get other languages to
> speak the Erlang wire protocol. I've used the Java libraries to build
> nodes and they worked, but I wouldn't do it again. Instead I'd use
> something like 0MQ or again the ponderously inefficient but ubiquitous
> something-over-http.

I wouldn't dismiss HTTP as inefficient. It's inefficient for RPC, which
is how most people use HTTP (even when they call it REST). But if you
use HTTP entirely then you have the options of caching proxies and
caching on the client side that can easily be plugged in and get rid of
most issues (other issues being largely fixed by HTTP/2).

--
Loïc Hoguin
http://ninenines.eu
Author of The Erlanger Playbook,
A book about software development using Erlang
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Jesper Louis Andersen-2
In reply to this post by Tuncer Ayaz

On Wed, Aug 31, 2016 at 5:36 PM, Tuncer Ayaz <[hidden email]> wrote:
https://github.com/jlouis/erlangnode (alive; not sure how complete)

Reasonably complete. All the ground types work and have been tested, but some of the more esoteric values (Ref's) have not been checked a lot and if you send them roundtrip they might be garbled.

Another bug I know is that there is no way to send a message larger than the inbound buffer (256kilobytes currently). In principle, the code should expand the buffer itself, but there is a bug which makes this not work. I have not yet investigated.


--
J.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Michael Truog
In reply to this post by Matthias Lang
py_interface is GPL, which may explain less use
(I did https://github.com/okeuday/erlang_py/ to make sure I had a BSD licensed dependency)
edtk depended on an obscure scripting language
(I did https://github.com/okeuday/GEPD/ to get EDTK functionality that only depends on the preprocessor, however clang doesn't like boost preprocessor yet, haven't needed to add NIF generation to it)

Other interface to language X projects that are alive include
http://cloudi.org (C/C++, Erlang/Elixir, Java, JavaScript/node.js, Perl, PHP, Python and Ruby)
https://github.com/okeuday/erlang_rb
https://github.com/okeuday/erlang_js
https://github.com/okeuday/erlang_pl
https://github.com/okeuday/erlang_php

On 08/31/2016 07:18 AM, Matthias Lang wrote:

> Hi,
>
> Pretty much all of the "not built in to OTP" ways to connect to other
> languages seem to have died, i.e.
>
>     - EPI (C++). Last commit 2009.
>     - py_interface (Python). Last commit 2014. Freshest of the lot.
>     - edtk (C). Last release 2007.
>     - dryverl (C). Last release 2008.
>     - erlua (Lua). Project page dead. Author's email is dead.
>     - erlualib (last commit 2010). Github-linked company page dead.
>
> Can anyone offer some experiences or ideas why?
>
> Could it be that these types of solutions are rarely needed? Maybe
> NIFs are unbeatable for the "narrow interface, no state, high
> performance" cases and port programs with hand-rolled interfaces most
> of the rest.
>
> Matt
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Michael Truog
In reply to this post by Garrett Smith-5
On 08/31/2016 08:50 AM, Garrett Smith wrote:

> On Wed, Aug 31, 2016 at 10:36 AM, Tuncer Ayaz <[hidden email]> wrote:
>> On 31 August 2016 at 16:18, Matthias Lang <[hidden email]> wrote:
>>> Hi,
>>>
>>> Pretty much all of the "not built in to OTP" ways to connect to other
>>> languages seem to have died, i.e.
>>>
>>>     - EPI (C++). Last commit 2009.
>>>     - py_interface (Python). Last commit 2014. Freshest of the lot.
>>>     - edtk (C). Last release 2007.
>>>     - dryverl (C). Last release 2008.
>>>     - erlua (Lua). Project page dead. Author's email is dead.
>>>     - erlualib (last commit 2010). Github-linked company page dead.
>>>
>>> Can anyone offer some experiences or ideas why?
>>>
>>> Could it be that these types of solutions are rarely needed? Maybe
>>> NIFs are unbeatable for the "narrow interface, no state, high
>>> performance" cases and port programs with hand-rolled interfaces most
>>> of the rest.
>> https://hackage.haskell.org/package/erlang (alive)
>> https://github.com/jlouis/erlangnode (alive; not sure how complete)
>> https://github.com/EchoTeam/ocaml-erlang-port (very stable or abandoned)
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
> I think in these distributed scenarios the go to protocol is something
> language agnostic like something-over-http.
>
> Personally I would not go around trying to get other languages to
> speak the Erlang wire protocol. I've used the Java libraries to build
> nodes and they worked, but I wouldn't do it again. Instead I'd use
> something like 0MQ or again the ponderously inefficient but ubiquitous
> something-over-http.
>
> If I need to integrate with other languages more closely, I'll use the
> port interface. Here's a terrific example of how that can be extended
> into various lanauges:
>
> http://erlport.org/
>
> Unless a NIF is simple and well established as non-crashy, I'll write
> performance critical code as a C port. People don't like this because
> of the emotionally crushing thought of serializing data over stdio.
> While that's laughably inefficient when compared to zippy NIFs
> (troll), the code remains isolated and can crash all it wants without
> bringing down the entire VM. It also encourages an interface design
> that minimizes IO -- you performed as much work as you can on either
> end and pass small results back and forth.
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
Based on https://github.com/hdima/erlport/issues/15 erlport isn't accepting changes and is dead.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Garrett Smith-5
On Wed, Aug 31, 2016 at 1:49 PM, Michael Truog <[hidden email]> wrote:
> On 08/31/2016 08:50 AM, Garrett Smith wrote:
>>
> Based on https://github.com/hdima/erlport/issues/15 erlport isn't accepting
> changes and is dead.

There aren't many Erlang projects that don't have a bus count of ~ 1 :)

Nonetheless this is a very impressive project and its patterns can be
applied to any language.

In terms of providing access to external libraries, I'd take this
route over the distributed Erlang protocol.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Daniel Goertzen-3
In reply to this post by Matthias Lang
I think you are correct that these solutions are rarely needed.  As author of NIFPP I see very little traffic at https://github.com/goertzenator/nifpp which gives me the impression that few people use it.

Also there are multiple paths to integration (ports vs c-node vs NIFs) so this maybe creates fragmentation and further reduces traction for any one project.  The EPI project you pointed out deals with EI which is completely orthogonal to NIFPP.

And another reason is tooling.  Do I need to jury rig a weird build system to make this thing go, or will it just work with rebar/erlang.mk?  If it's too much hassle to use, then I'll just skip it.  This is an area I've been trying to improve for Rust (https://github.com/goertzenator/rust.mk)



On Wed, Aug 31, 2016 at 9:18 AM Matthias Lang <[hidden email]> wrote:
Hi,

Pretty much all of the "not built in to OTP" ways to connect to other
languages seem to have died, i.e.

   - EPI (C++). Last commit 2009.
   - py_interface (Python). Last commit 2014. Freshest of the lot.
   - edtk (C). Last release 2007.
   - dryverl (C). Last release 2008.
   - erlua (Lua). Project page dead. Author's email is dead.
   - erlualib (last commit 2010). Github-linked company page dead.

Can anyone offer some experiences or ideas why?

Could it be that these types of solutions are rarely needed? Maybe
NIFs are unbeatable for the "narrow interface, no state, high
performance" cases and port programs with hand-rolled interfaces most
of the rest.

Matt
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Garrett Smith-5
In reply to this post by Garrett Smith-5
On Wed, Aug 31, 2016 at 1:55 PM, Garrett Smith <[hidden email]> wrote:

> On Wed, Aug 31, 2016 at 1:49 PM, Michael Truog <[hidden email]> wrote:
>> On 08/31/2016 08:50 AM, Garrett Smith wrote:
>>>
>> Based on https://github.com/hdima/erlport/issues/15 erlport isn't accepting
>> changes and is dead.
>
> There aren't many Erlang projects that don't have a bus count of ~ 1 :)
>
> Nonetheless this is a very impressive project and its patterns can be
> applied to any language.
>
> In terms of providing access to external libraries, I'd take this
> route over the distributed Erlang protocol.

We're also talking about interfaces that rarely change, to someone's
earlier point.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Serge Aleynikov-3
In reply to this post by Matthias Lang
Here's another C++/Erlang interface that has some on-going activity:


As I see it if the author is involved in some work that uses such project, the project is alive, otherwise the community is not that large to get involved and learn new APIs.

On Wed, Aug 31, 2016 at 10:18 AM, Matthias Lang <[hidden email]> wrote:
Hi,

Pretty much all of the "not built in to OTP" ways to connect to other
languages seem to have died, i.e.

   - EPI (C++). Last commit 2009.
   - py_interface (Python). Last commit 2014. Freshest of the lot.
   - edtk (C). Last release 2007.
   - dryverl (C). Last release 2008.
   - erlua (Lua). Project page dead. Author's email is dead.
   - erlualib (last commit 2010). Github-linked company page dead.

Can anyone offer some experiences or ideas why?

Could it be that these types of solutions are rarely needed? Maybe
NIFs are unbeatable for the "narrow interface, no state, high
performance" cases and port programs with hand-rolled interfaces most
of the rest.

Matt
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

David Leach
In reply to this post by Garrett Smith-5

We use erlport in production quite successfully, although we'd also like the map feature we haven't had any major issues with it.


So although there may be little activity it does what it says it does.


From: [hidden email] <[hidden email]> on behalf of Garrett Smith <[hidden email]>
Sent: Thursday, 1 September 2016 6:55 a.m.
To: Michael Truog
Cc: Erlang Questions
Subject: Re: [erlang-questions] Why do "interface to language X" projects all seem to die out?
 
On Wed, Aug 31, 2016 at 1:49 PM, Michael Truog <[hidden email]> wrote:
> On 08/31/2016 08:50 AM, Garrett Smith wrote:
>>
> Based on https://github.com/hdima/erlport/issues/15 erlport isn't accepting

> changes and is dead.

There aren't many Erlang projects that don't have a bus count of ~ 1 :)

Nonetheless this is a very impressive project and its patterns can be
applied to any language.

In terms of providing access to external libraries, I'd take this
route over the distributed Erlang protocol.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Why do "interface to language X" projects all seem to die out?

Kostis Sagonas-2
In reply to this post by Matthias Lang
On 08/31/2016 04:18 PM, Matthias Lang wrote:

> Hi,
>
> Pretty much all of the "not built in to OTP" ways to connect to other
> languages seem to have died, i.e.
>
>    - EPI (C++). Last commit 2009.
>    - py_interface (Python). Last commit 2014. Freshest of the lot.
>    - edtk (C). Last release 2007.
>    - dryverl (C). Last release 2008.
>    - erlua (Lua). Project page dead. Author's email is dead.
>    - erlualib (last commit 2010). Github-linked company page dead.
>
> Can anyone offer some experiences or ideas why?
>
> Could it be that these types of solutions are rarely needed? Maybe
> NIFs are unbeatable for the "narrow interface, no state, high
> performance" cases and port programs with hand-rolled interfaces most
> of the rest.

FYIW, we have developed Nifty [1], a NIF interface generator, i.e. a
tool that automates the process of creating NIF libraries from C header
files containing declarations of types and functions that the C library
supplies.

Among other features, Nifty provides support for dirty schedulers and
support for safe execution via separate Erlang node.

Its code is since 2014 on GitHub [2].

A talk and paper titled "The Nifty Way to Call Hell from Heaven" will be
presented at the Erlang Workshop in late September 2016.

Comments, suggestions for improvements, pull requests, etc. are welcome!


Kostis

[1] http://parapluu.github.io/nifty/
[2] https://github.com/parapluu/nifty
[3] http://parapluu.github.io/nifty/erlang16-preprint.pdf
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions