Erlang VM in Rust

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
79 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Erlang VM in Rust

austin aigbe-2
Hello,

Any consideration for implementing the Erlang VM in Rust?

Regards,
Austin

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Dmytro Lytovchenko
I tried to begin once.

Here's the catch:
  • Rust had no unions at the time of my experiments (added in 1.19 - July 2017)
  • Rust has no bit fields (some libraries exist trying to alleviate this)
  • Erlang process heap is created and maintained by abusing pointers and bits in them. Casting integers to pointers and back i believe is possible but will give you some pain.
  • A really fast VM loop is done using C/C++ goto (void*) extension, in Rust you will probably have to use old good switch/case (slower)
When you overcome these base problems, it should be possible to proceed carefully :)

2017-09-12 22:10 GMT+02:00 austin aigbe <[hidden email]>:
Hello,

Any consideration for implementing the Erlang VM in Rust?

Regards,
Austin

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Loïc Hoguin-3
In reply to this post by austin aigbe-2
Any thoughts on the benefits existing Erlang programs would get from a
VM implemented in Rust?

It may or may not simplify working on the VM and may or may not make it
more solid. But what would it improve for the programs written in Erlang
themselves?

Cheers,

On 09/12/2017 10:10 PM, austin aigbe wrote:

> Hello,
>
> Any consideration for implementing the Erlang VM in Rust?
>
> Regards,
> Austin
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>

--
Loïc Hoguin
https://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

zxq9-2
On 2017年09月13日 水曜日 00:27:00 Loïc Hoguin wrote:
> Any thoughts on the benefits existing Erlang programs would get from a
> VM implemented in Rust?
>
> It may or may not simplify working on the VM and may or may not make it
> more solid. But what would it improve for the programs written in Erlang
> themselves?

In an ideal world, nothing. I would hope that the only benefit to
those coding in Erlang is that maintenance would become more of a sure
thing for the OTP maintainers and that, in particular, new feature
implementation and performance improvements would become more obvious
and safer to implement over time. Those benefits would be beneficial
only indirectly.

That said... I really can't imagine such a large, old, well documented,
well explored runtime known to such a large team would be rewritten to
a new language. I can imagine that some of us (probably including some
of the OTP team itself) would enjoy working on a rewrite in Rust that
is an alternative to, but not a replacement for, the current runtime.

Competing runtimes occasionally expose hidden benefits. But a blanket
replacement would be a massive, end-of-Netscape type upheaval. It is
not as though Erlang is implemented in an obfuscated style of C++ that
suffers from fundamental structure and human incompatability issues.

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Paul Oliver-2


On Wed, Sep 13, 2017 at 10:42 AM zxq9 <[hidden email]> wrote:
On 2017年09月13日 水曜日 00:27:00 Loïc Hoguin wrote:
> Any thoughts on the benefits existing Erlang programs would get from a
> VM implemented in Rust?
>
> It may or may not simplify working on the VM and may or may not make it
> more solid. But what would it improve for the programs written in Erlang
> themselves?

In an ideal world, nothing. I would hope that the only benefit to
those coding in Erlang is that maintenance would become more of a sure
thing for the OTP maintainers and that, in particular, new feature
implementation and performance improvements would become more obvious
and safer to implement over time. Those benefits would be beneficial
only indirectly.

That said... I really can't imagine such a large, old, well documented,
well explored runtime known to such a large team would be rewritten to
a new language. I can imagine that some of us (probably including some
of the OTP team itself) would enjoy working on a rewrite in Rust that
is an alternative to, but not a replacement for, the current runtime.

Competing runtimes occasionally expose hidden benefits. But a blanket
replacement would be a massive, end-of-Netscape type upheaval. It is
not as though Erlang is implemented in an obfuscated style of C++ that
suffers from fundamental structure and human incompatability issues.


Large C projects can potentially be rewritten in Rust gradually. An example of this can be found in Remacs (http://www.wilfred.me.uk/blog/2017/01/11/announcing-remacs-porting-emacs-to-rust/). 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Joshua Barney
In reply to this post by Loïc Hoguin-3
I think the real point of implementing the Erlang VM in Rust is to increase the awesomeness 100 fold

But really, the only thing I can think of that affects erlang programs is that it could make is simpler to write nifs that can't crash your whole vm. I don't know how, however.

I feel like cargo could have a similar community building effect as npm has had for nodejs--an explosion of creativity that resulted in a lot of good quality tools for everyone to use. It's conceivable to me that development of a rust/erl VM could move very quickly by leveraging cargo crates

Sent from my iPhone

> On Sep 12, 2017, at 6:27 PM, Loïc Hoguin <[hidden email]> wrote:
>
> Any thoughts on the benefits existing Erlang programs would get from a VM implemented in Rust?
>
> It may or may not simplify working on the VM and may or may not make it more solid. But what would it improve for the programs written in Erlang themselves?
>
> Cheers,
>
>> On 09/12/2017 10:10 PM, austin aigbe wrote:
>> Hello,
>> Any consideration for implementing the Erlang VM in Rust?
>> Regards,
>> Austin
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>
> --
> Loïc Hoguin
> https://ninenines.eu
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

scott ribe
> On Sep 12, 2017, at 4:48 PM, Joshua Barney <[hidden email]> wrote:
>
> But really, the only thing I can think of that affects erlang programs is that it could make is simpler to write nifs that can't crash your whole vm. I don't know how, however.
>

There's already a Rust library to support exactly that.

--
Scott Ribe
[hidden email]
(303) 722-0567

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Tristan Sloughter-4
In reply to this post by Joshua Barney
Maybe Tony 'the tiger' Ramine would come back to working on Erlang if it
was in Rust!

--
  Tristan Sloughter
  "I am not a crackpot" - Abe Simpson
  [hidden email]
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

David Goehrig
In reply to this post by Loïc Hoguin-3
Based on past experience, every rewrite in solution/language du jour merely fragments the community and introduces new classes of incompatible bugs. The broad cost in social capital outweighs any narrow technical benefits.

I have seen it in my own projects. Even projects like Python, Ruby, and Perl are suffering the social costs of rewrites.

We should remember language is a tool for communication between humans, and they are not as easy to program :)

Dave

On Sep 12, 2017 6:27 PM, "Loïc Hoguin" <[hidden email]> wrote:
Any thoughts on the benefits existing Erlang programs would get from a VM implemented in Rust?

It may or may not simplify working on the VM and may or may not make it more solid. But what would it improve for the programs written in Erlang themselves?

Cheers,

On 09/12/2017 10:10 PM, austin aigbe wrote:
Hello,

Any consideration for implementing the Erlang VM in Rust?

Regards,
Austin


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


--
Loïc Hoguin
https://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Richard A. O'Keefe-2
In reply to this post by Paul Oliver-2
As I see it, the main problem rewriting (or better still, rebuilding)
Erlang in Rust is that I'd rather see that human time spent on
improving the system we already have.

As an open source project, Erlang is or was eligible for free
checking by Coverity.  I see on the Coverity web site that the
last check was in August 2015, a little over 2 years ago.  Is
there a plan to have it checked again?

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

zxq9-2
In reply to this post by Tristan Sloughter-4
On 2017年09月12日 火曜日 17:05:33 Tristan Sloughter wrote:
> Maybe Tony 'the tiger' Ramine would come back to working on Erlang if it
> was in Rust!

Hey!
I really like that guy.

Almost a good enough reason on its own. But I'm being selfish. ;-)

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Thom Cherryhomes
In reply to this post by Dmytro Lytovchenko
For a language that purports to be a replacement for low-level systems programming, Rust not having bitfields or unions, would be a real pain in the ass to do any intricate hardware driver work... I'm guessing that Rust is designed by 20-somethings who have never done a line of assembler in their lives?

sigh.
-Thom

On Tue, Sep 12, 2017 at 3:25 PM Dmytro Lytovchenko <[hidden email]> wrote:
I tried to begin once.

Here's the catch:
  • Rust had no unions at the time of my experiments (added in 1.19 - July 2017)
  • Rust has no bit fields (some libraries exist trying to alleviate this)
  • Erlang process heap is created and maintained by abusing pointers and bits in them. Casting integers to pointers and back i believe is possible but will give you some pain.
  • A really fast VM loop is done using C/C++ goto (void*) extension, in Rust you will probably have to use old good switch/case (slower)
When you overcome these base problems, it should be possible to proceed carefully :)

2017-09-12 22:10 GMT+02:00 austin aigbe <[hidden email]>:
Hello,

Any consideration for implementing the Erlang VM in Rust?

Regards,
Austin

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

zxq9-2
On 2017年09月13日 水曜日 01:47:08 Thom Cherryhomes wrote:
> For a language that purports to be a replacement for low-level systems
> programming, Rust not having bitfields or unions, would be a real pain in
> the ass to do any intricate hardware driver work... I'm guessing that Rust
> is designed by 20-somethings who have never done a line of assembler in
> their lives?

That is curious. I wonder if these have been deferred for the time being in favor of using C as an extension (as Rust conforms to the C ABI)?

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Joshua Barney
In reply to this post by Thom Cherryhomes
Rust supports inline assembly https://doc.rust-lang.org/1.8.0/book/inline-assembly.html

Sent from my iPhone

On Sep 12, 2017, at 9:47 PM, Thom Cherryhomes <[hidden email]> wrote:

For a language that purports to be a replacement for low-level systems programming, Rust not having bitfields or unions, would be a real pain in the ass to do any intricate hardware driver work... I'm guessing that Rust is designed by 20-somethings who have never done a line of assembler in their lives?

sigh.
-Thom

On Tue, Sep 12, 2017 at 3:25 PM Dmytro Lytovchenko <[hidden email]> wrote:
I tried to begin once.

Here's the catch:
  • Rust had no unions at the time of my experiments (added in 1.19 - July 2017)
  • Rust has no bit fields (some libraries exist trying to alleviate this)
  • Erlang process heap is created and maintained by abusing pointers and bits in them. Casting integers to pointers and back i believe is possible but will give you some pain.
  • A really fast VM loop is done using C/C++ goto (void*) extension, in Rust you will probably have to use old good switch/case (slower)
When you overcome these base problems, it should be possible to proceed carefully :)

2017-09-12 22:10 GMT+02:00 austin aigbe <[hidden email]>:
Hello,

Any consideration for implementing the Erlang VM in Rust?

Regards,
Austin

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Jesper Louis Andersen-2
In reply to this post by austin aigbe-2
On Tue, Sep 12, 2017 at 10:10 PM austin aigbe <[hidden email]> wrote:
Hello,

Any consideration for implementing the Erlang VM in Rust?


I think the problem is roughly the same as with any other language: A lot of good effort has been sunk into the current VM in C and this makes a switch harder. Rust has ABI compatibility with C in some ways, which makes interoperability simpler, so one could imagine taking some of the more security-oriented parts of the VM and rewriting those in Rust or some other language of the more restrictive kind.

If you want to rewrite all of it, a good approach is to start by making changes to the current VM which makes it smaller:

* Move more work out into NIFs. Reimplement those in Rust
* Coalesce more work on Dirty Schedulers
* Implement Native Processes, so you can start running Rust-native-processes
* Write an AOT or JIT compiler pass. Move ETS into pure Erlang. Move a lot of the BIF optimized functions into pure Erlang.

A smaller core is way easier to replace.

Personally, I'd just throw some more time after concurrent OCaml, and then write a translator from Erlang to OCaml :P


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Roman Galeev
> Personally, I'd just throw some more time after concurrent OCaml, and then write a translator from Erlang to OCaml :P

Do you think concurrent OCaml has advantages over Erlang? And if yes, what are they, in your opinion?

On Wed, Sep 13, 2017 at 1:11 PM, Jesper Louis Andersen <[hidden email]> wrote:
On Tue, Sep 12, 2017 at 10:10 PM austin aigbe <[hidden email]> wrote:
Hello,

Any consideration for implementing the Erlang VM in Rust?


I think the problem is roughly the same as with any other language: A lot of good effort has been sunk into the current VM in C and this makes a switch harder. Rust has ABI compatibility with C in some ways, which makes interoperability simpler, so one could imagine taking some of the more security-oriented parts of the VM and rewriting those in Rust or some other language of the more restrictive kind.

If you want to rewrite all of it, a good approach is to start by making changes to the current VM which makes it smaller:

* Move more work out into NIFs. Reimplement those in Rust
* Coalesce more work on Dirty Schedulers
* Implement Native Processes, so you can start running Rust-native-processes
* Write an AOT or JIT compiler pass. Move ETS into pure Erlang. Move a lot of the BIF optimized functions into pure Erlang.

A smaller core is way easier to replace.

Personally, I'd just throw some more time after concurrent OCaml, and then write a translator from Erlang to OCaml :P


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions




--
With best regards,
     Roman Galeev,
     +420 702 817 968

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Jesper Louis Andersen-2
On Wed, Sep 13, 2017 at 1:20 PM Roman Galeev <[hidden email]> wrote:
> Personally, I'd just throw some more time after concurrent OCaml, and then write a translator from Erlang to OCaml :P

Do you think concurrent OCaml has advantages over Erlang? And if yes, what are they, in your opinion?

You are making tradeoffs whenever you choose another language.

* Semantically, OCaml is far superior to Erlang in any conceivable way.
* OCaml has a powerful type system, which Erlang doesn't.
* The native code generator in OCaml easily surpasses the bytecode interpreter in Erlang.

On the other hand:

* OCaml is currently not parallel in any way. There has been a long effort in fixing this, but even *if* it gets released, we are looking at years of maturing needed before we can hope for robust operation.
* OCaml currently uses something like Lwt and/or Async for concurrent work. This is Node.js in disguise. I don't believe in indirect concurrency codings in general, far preferring Erlang or Go's direct approaches (in which you favor preemption over cooperation in multitasking).
* OCaml is unlikely to ever support hot code upgrade.
* OCaml currently has no dTrace-style production ad-hoc tracing facilities where Erlang has it naturally.
* OCaml programs are easy to make robust (cope with unknown input), but harder to make resilient (coping gracefully with internal failure).

So proviso OCaml gets a good parallel story, and we can implement the important parts of Erlang on top of that, then I think there are programs which can be written in OCaml with advantage. In particular those which can afford to be restarted once in a while. This is true for many modern systems in which you are deploy clusters of nodes() and provisioning them this way. The prime candidate programs are those which require the efficiency the OCaml native code compiler provides and where you can't just run the OCaml program behind an Erlang Port easily.

The *current* state of the art: use OCaml whenever you have a problem requiring the combination of functional abstraction and efficiency[0]. If you don't require the abstraction levels OCaml provide, go for C, C++, Rust, Go, etc. But chances are that the added productivity of OCaml will get you a far better and faster program when you are working with a limited time frame, since you can try more solutions, quicker.

If your problem doesn't require efficient use of the CPU core, don't underestimate how productive Erlang is. Since everything are just Erlang terms, you can often write a good program in a fraction of the time it requires to come up with a good type model in something like OCaml. All tooling supports Erlang terms. This is a powerful abstraction which can be put to good use. And you get resilience on top. I should really blog about how unfairly effective Erlang is at being productive. Efficiency is far from the only important factor in software development.


[0] Haskell is another excellent candidate here.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Dmytro Lytovchenko
In reply to this post by Jesper Louis Andersen-2
I am a strong proponent for rewriting parts of BEAM VM in C++, and I've even made a list of points to consider and benefits it would bring: http://beam-wisdoms.clau.se/en/latest/otp-cpp-ramblings.html I was personally ready to sink a lot of my time into this both paid (we have our ways to get paid for helping OTP team) and unpaid. But i never started because OTP team wasn't happy with this idea.

Main problem with all these rewritings, is that either you fork and take over the fork completely, i.e. maintain version and all bugfixes and all new developments, or you listen what OTP team has got to say. And so far they say no to any major rewrite.

2017-09-13 13:11 GMT+02:00 Jesper Louis Andersen <[hidden email]>:
On Tue, Sep 12, 2017 at 10:10 PM austin aigbe <[hidden email]> wrote:
Hello,

Any consideration for implementing the Erlang VM in Rust?


I think the problem is roughly the same as with any other language: A lot of good effort has been sunk into the current VM in C and this makes a switch harder. Rust has ABI compatibility with C in some ways, which makes interoperability simpler, so one could imagine taking some of the more security-oriented parts of the VM and rewriting those in Rust or some other language of the more restrictive kind.

If you want to rewrite all of it, a good approach is to start by making changes to the current VM which makes it smaller:

* Move more work out into NIFs. Reimplement those in Rust
* Coalesce more work on Dirty Schedulers
* Implement Native Processes, so you can start running Rust-native-processes
* Write an AOT or JIT compiler pass. Move ETS into pure Erlang. Move a lot of the BIF optimized functions into pure Erlang.

A smaller core is way easier to replace.

Personally, I'd just throw some more time after concurrent OCaml, and then write a translator from Erlang to OCaml :P


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

zxq9-2
On 2017年09月13日 水曜日 13:47:54 Dmytro Lytovchenko wrote:
> I am a strong proponent for rewriting parts of BEAM VM in C++

As a serious thought, or as satire?
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang VM in Rust

Dmytro Lytovchenko
Absolutely serious.
It is all listed in that article I linked, I wrote it.

2017-09-13 13:49 GMT+02:00 zxq9 <[hidden email]>:
On 2017年09月13日 水曜日 13:47:54 Dmytro Lytovchenko wrote:
> I am a strong proponent for rewriting parts of BEAM VM in C++

As a serious thought, or as satire?
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
1234