Erlang package manager

classic Classic list List threaded Threaded
100 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Erlang package manager

Bruce Yinhe-2
Hi everyone,

Industrial Erlang User Group (IEUG) has been working together with the Erlang/OTP team to investigate and create a package management system for Erlang/OTP. 

The lack of a package management system for Erlang has been discussed for a long time in the community. In essence, a straightforward package management system is believed to take Erlang programming language a step forward. Multiple tools will appear in the community. It needs to be supported by a highly visible community behind it.


In order to increase the adoption and to result in a tool widely used in the Erlang ecosystem, we are identifying the most important user categories and use cases, based on what the majority of the community want in a package manager. Therefore we would like to invite an open discussion.

Now you are welcome to share your thoughts, suggestions or proposals about an Erlang package manager. It would be great if you could reply with your motivation, explaining why a feature is necessary to have. There are some example questions to begin the dicussion with, including, but not limited to the following. 
  • What metadata information should an Erlang package include?
  • What functionality do you need in a package manager for Erlang in order to use it in production?
  • What other concerns do you have about an Erlang package management system?
Erlang package manager's brief wish list of features: 
  • Console interface
  • Web interface
  • Package Index and Repository
  • Fetch, Install and Remove Packages
  • Publish packages
  • Versioning and Dependency Management
We are aware of several previous efforts and existing tools that attempt to achieve the similar goal. We want to look at existing things, both from Erlang and Elixir, to see if they fit the requirements. If not, we will then have to make something new, perhaps as a rewrite of an existing tool.

The IEUG members are putting together requirements for a package manager and will work with the community and Ericsson to create a standard and address any voids which exists in the existing tooling, funding necessary efforts required. 

Best regards

Bruce Yinhe

Erlang Community Manager
Industrial Erlang User Group
[hidden email]
+46 72 311 43 89

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Vlad Dumitrescu-2
Hi Bruce and all,

I have one addition to the feature list: it should have an Erlang interface too, so that it can be integrated with other tools. If it only runs as an escript and gets configuration in a half-magic way, then it's not easy to integrate it. 

best regards,
Vlad


On Tue, Dec 16, 2014 at 12:41 PM, Bruce Yinhe <[hidden email]> wrote:
Hi everyone,

Industrial Erlang User Group (IEUG) has been working together with the Erlang/OTP team to investigate and create a package management system for Erlang/OTP. 

The lack of a package management system for Erlang has been discussed for a long time in the community. In essence, a straightforward package management system is believed to take Erlang programming language a step forward. Multiple tools will appear in the community. It needs to be supported by a highly visible community behind it.


In order to increase the adoption and to result in a tool widely used in the Erlang ecosystem, we are identifying the most important user categories and use cases, based on what the majority of the community want in a package manager. Therefore we would like to invite an open discussion.

Now you are welcome to share your thoughts, suggestions or proposals about an Erlang package manager. It would be great if you could reply with your motivation, explaining why a feature is necessary to have. There are some example questions to begin the dicussion with, including, but not limited to the following. 
  • What metadata information should an Erlang package include?
  • What functionality do you need in a package manager for Erlang in order to use it in production?
  • What other concerns do you have about an Erlang package management system?
Erlang package manager's brief wish list of features: 
  • Console interface
  • Web interface
  • Package Index and Repository
  • Fetch, Install and Remove Packages
  • Publish packages
  • Versioning and Dependency Management
We are aware of several previous efforts and existing tools that attempt to achieve the similar goal. We want to look at existing things, both from Erlang and Elixir, to see if they fit the requirements. If not, we will then have to make something new, perhaps as a rewrite of an existing tool.

The IEUG members are putting together requirements for a package manager and will work with the community and Ericsson to create a standard and address any voids which exists in the existing tooling, funding necessary efforts required. 

Best regards

Bruce Yinhe

Erlang Community Manager
Industrial Erlang User Group
[hidden email]
+46 72 311 43 89

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Loïc Hoguin-3
In reply to this post by Bruce Yinhe-2
Personally I do not need a package manager. However a centralized
package index would be good to have. This package index needs to be
exportable in a variety of different formats (tab separated values would
be great).

The metadata should include at least:

  *  name
  *  repository type (git, hg, svn...)
  *  repository uri
  *  recommended version (hopefully we can avoid "everything master")
  *  project uri
  *  description

That's really it. Actual package management is not needed at all in my
workflow where everything is a dep, including test, documentation and
other tools.

Best of luck with it.

On 12/16/2014 01:41 PM, Bruce Yinhe wrote:

> Hi everyone,
>
> Industrial Erlang User Group (IEUG) has been working together with the
> Erlang/OTP team to investigate and create a package management system
> for Erlang/OTP.
>
> The lack of a package management system for Erlang has been discussed
> for a long time in the community. In essence, a straightforward package
> management system is believed to take Erlang programming language a step
> forward. Multiple tools will appear in the community. It needs to be
> supported by a highly visible community behind it.
>
>
> In order to increase the adoption and to result in a tool widely used in
> the Erlang ecosystem, we are identifying the most important user
> categories and use cases, based on what the majority of the community
> want in a package manager. Therefore we would like to invite an open
> discussion.
>
> Now you are welcome to share your thoughts, suggestions or proposals
> about an Erlang package manager. It would be great if you could reply
> with your motivation, explaining why a feature is necessary to have.
> There are some example questions to begin the dicussion with, including,
> but not limited to the following.
>
>   * What metadata information should an Erlang package include?
>   * What functionality do you need in a package manager for Erlang in
>     order to use it in production?
>   * What other concerns do you have about an Erlang package management
>     system?
>
> Erlang package manager's brief wish list of features:
>
>   * Console interface
>   * Web interface
>   * Package Index and Repository
>   * Fetch, Install and Remove Packages
>   * Publish packages
>   * Versioning and Dependency Management
>
> We are aware of several previous efforts and existing tools that attempt
> to achieve the similar goal. We want to look at existing things, both
> from Erlang and Elixir, to see if they fit the requirements. If not, we
> will then have to make something new, perhaps as a rewrite of an
> existing tool.
>
> The IEUG members are putting together requirements for a package manager
> and will work with the community and Ericsson to create a standard and
> address any voids which exists in the existing tooling, funding
> necessary efforts required.
>
> Best regards
>
> Bruce Yinhe
>
> Erlang Community Manager
> Industrial Erlang User Group
> [hidden email] <mailto:[hidden email]>
> +46 72 311 43 89
>
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>

--
Loïc Hoguin
http://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Vlad Dumitrescu-2

On Tue, Dec 16, 2014 at 1:17 PM, Loïc Hoguin <[hidden email]> wrote:
Actual package management is not needed at all in my workflow where everything is a dep, including test, documentation and other tools.

I may be wrong, but I believe that 'package' in this context means what you mean by 'dep'. You still need to fetch the dependencies and you don't care about another tool because (I guess) you are using rebar which implements this thing. My understanding of this package manager thing is that it separates this functionality, unifies the different features that rebar, mix, hex, agner etc have and makes it available so that we don't need to reinvent the wheel.

If I got it wrong, I'm sure that Bruce will correct me :-)

regards,
Vlad


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Loïc Hoguin-3
On 12/16/2014 02:43 PM, Vlad Dumitrescu wrote:

>
> On Tue, Dec 16, 2014 at 1:17 PM, Loïc Hoguin <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Actual package management is not needed at all in my workflow where
>     everything is a dep, including test, documentation and other tools.
>
>
> I may be wrong, but I believe that 'package' in this context means what
> you mean by 'dep'. You still need to fetch the dependencies and you

No. From the first post:

 > Fetch, Install and Remove Packages
 > Versioning and Dependency Management

I have no use for installing packages.

There was also no mention of that package manager doing something like
deps either (nor do I want it to: booting the Erlang VM to then call
"git clone" is not particularly efficient, and efficiency is important
for my workflow).

> don't care about another tool because (I guess) you are using rebar

I am using erlang.mk which implements something similar to rebar, yes.

> which implements this thing. My understanding of this package manager
> thing is that it separates this functionality, unifies the different
> features that rebar, mix, hex, agner etc have and makes it available so
> that we don't need to reinvent the wheel.

Unfortunately in my case I will have to keep maintaining a separate
wheel because I do not want to boot the Erlang VM for these things.

--
Loïc Hoguin
http://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Fred Hebert-2
In reply to this post by Bruce Yinhe-2
On 12/16, Bruce Yinhe wrote:
>    - What metadata information should an Erlang package include?
>    - What functionality do you need in a package manager for Erlang in
>    order to use it in production?
>    - What other concerns do you have about an Erlang package management
>    system?

I'd certainly like to see signing addressed by a would-be package manager.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Gleb Peregud

I would really like to see hermiticity and reproducibility of Nix package manager.

On Dec 16, 2014 2:09 PM, "Fred Hebert" <[hidden email]> wrote:
On 12/16, Bruce Yinhe wrote:
>    - What metadata information should an Erlang package include?
>    - What functionality do you need in a package manager for Erlang in
>    order to use it in production?
>    - What other concerns do you have about an Erlang package management
>    system?

I'd certainly like to see signing addressed by a would-be package manager.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

PAILLEAU Eric
In reply to this post by Bruce Yinhe-2
+1
Why not generating OS packages instead reinvent the wheel ? Each dependance could be a package itself and Ports source packages.
There is also nugets and chocolatey for Windows.

« Envoyé depuis mon mobile » Eric

Gleb Peregud <[hidden email]> a écrit :

I would really like to see hermiticity and reproducibility of Nix package manager.

On Dec 16, 2014 2:09 PM, "Fred Hebert" <[hidden email]> wrote:
On 12/16, Bruce Yinhe wrote:
>    - What metadata information should an Erlang package include?
>    - What functionality do you need in a package manager for Erlang in
>    order to use it in production?
>    - What other concerns do you have about an Erlang package management
>    system?

I'd certainly like to see signing addressed by a would-be package manager.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Leonard Boyce-2
In reply to this post by Bruce Yinhe-2
Hi Bruce,

Glad to hear there will at least be "official" work effort expended on this.

A few things I'd like to see:
- Require semver for all packages in the index
- Add/remove alternative package locations
- The ability to 'pin' a package in your local system to a specific version
- Options for both local and global install locations (ala deps vs lib
install location)
- Support for both 'archive' and 'source' versions of packages (user option)
- Package signing with signature verification
- Ability to mirror the package index and sources
- All package should have explicit license information and as a user I
should be aware of/have to explicitly accept licenses before any
package is installed

Kind regards,
Leonard


On Tue, Dec 16, 2014 at 6:41 AM, Bruce Yinhe
<[hidden email]> wrote:

> Hi everyone,
>
> Industrial Erlang User Group (IEUG) has been working together with the
> Erlang/OTP team to investigate and create a package management system for
> Erlang/OTP.
>
> The lack of a package management system for Erlang has been discussed for a
> long time in the community. In essence, a straightforward package management
> system is believed to take Erlang programming language a step forward.
> Multiple tools will appear in the community. It needs to be supported by a
> highly visible community behind it.
>
>
> In order to increase the adoption and to result in a tool widely used in the
> Erlang ecosystem, we are identifying the most important user categories and
> use cases, based on what the majority of the community want in a package
> manager. Therefore we would like to invite an open discussion.
>
> Now you are welcome to share your thoughts, suggestions or proposals about
> an Erlang package manager. It would be great if you could reply with your
> motivation, explaining why a feature is necessary to have. There are some
> example questions to begin the dicussion with, including, but not limited to
> the following.
>
> What metadata information should an Erlang package include?
> What functionality do you need in a package manager for Erlang in order to
> use it in production?
> What other concerns do you have about an Erlang package management system?
>
> Erlang package manager's brief wish list of features:
>
> Console interface
> Web interface
> Package Index and Repository
> Fetch, Install and Remove Packages
> Publish packages
> Versioning and Dependency Management
>
> We are aware of several previous efforts and existing tools that attempt to
> achieve the similar goal. We want to look at existing things, both from
> Erlang and Elixir, to see if they fit the requirements. If not, we will then
> have to make something new, perhaps as a rewrite of an existing tool.
>
> The IEUG members are putting together requirements for a package manager and
> will work with the community and Ericsson to create a standard and address
> any voids which exists in the existing tooling, funding necessary efforts
> required.
>
> Best regards
>
> Bruce Yinhe
>
> Erlang Community Manager
> Industrial Erlang User Group
> [hidden email]
> +46 72 311 43 89
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

zxq9-2
In reply to this post by Fred Hebert-2
On 2014年12月16日 火曜日 08:08:51 Fred Hebert wrote:
> On 12/16, Bruce Yinhe wrote:
> >    - What metadata information should an Erlang package include?
> >    - What functionality do you need in a package manager for Erlang in
> >    order to use it in production?
> >    - What other concerns do you have about an Erlang package management
> >    system?
>
> I'd certainly like to see signing addressed by a would-be package manager.

With that in mind, I would like to see a signed repo (or optionally signed
repo, with a selectable sensitivity) where deps/packages/whatever are curated
to some degree instead of just being "go fetch $trash from $place, which
hopefully still exists".

In this sense building a base of Erlang deps is not unlike managing a meta OS
distribution, and the tasks of a full-blown build/dep/repo tool are not unlike
the stack of services that underly ports/portage or cpan.

Separation of the "packaging" job from the "code scribbling" job is, in my
opinion, a healthy thing. This has been my experience in packaging/managing
Linux distros in the past, and I see more useful parallels in the curation
model there than in the "grab $shit from $codesite", and felt the pain of not
doing it right in Cabal in the past. In particular, the repo itself should be
versioned collectively -- so you have an API promise within major versions
within revisions of the same dep definition within the overall repo major
release. So you can "write against OTP17R + E-Universe 3" and know that your
stuff  will build without chasing down breakages every time someone types
"rebase" in their upstream project tree. (Of course... this means either
upstream *or* packagers will have to adhere to a reasonable versioning
scheme... otoh, curation gives packagers a chance to interdict idiotic
upstream versioning...)

This is how I've come to view the build/packaging/repo/dep problem after
dealing with it in OSes and language environments -- I don't think a simple
(or separate, greatly divergent, incompatible) tools for each individual
aspect will solve the problem. This is probably a heretical view to many
people (its OK, already wearing fireproof undies).
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

PAILLEAU Eric
In reply to this post by Bruce Yinhe-2
I forgot to say that Erlang packages may need Non Erlang application, like Postgresql, or Varnish or (fill blanks).
Debian Meta package allow this, for instance. This could not be achieved with a specific Erlang packager.



« Envoyé depuis mon mobile » Eric

Eric Pailleau <[hidden email]> a écrit :

+1
Why not generating OS packages instead reinvent the wheel ? Each dependance could be a package itself and Ports source packages.
There is also nugets and chocolatey for Windows.

« Envoyé depuis mon mobile » Eric

Gleb Peregud <[hidden email]> a écrit :

I would really like to see hermiticity and reproducibility of Nix package manager.

On Dec 16, 2014 2:09 PM, "Fred Hebert" <[hidden email]> wrote:
On 12/16, Bruce Yinhe wrote:
>    - What metadata information should an Erlang package include?
>    - What functionality do you need in a package manager for Erlang in
>    order to use it in production?
>    - What other concerns do you have about an Erlang package management
>    system?

I'd certainly like to see signing addressed by a would-be package manager.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

zxq9-2
On 2014年12月16日 火曜日 15:56:22 Eric Pailleau wrote:
> I forgot to say that Erlang packages may need Non Erlang application, like
> Postgresql, or Varnish or (fill blanks). Debian Meta package allow this,
> for instance. This could not be achieved with a specific Erlang packager.

There are no Erlang packages or builds that require Postgres. Some C
interfaces may require Postgres headers to build (and these are not "Erlang
packages" in the strict sense), but most Erlang systems that have anything at
all to do with something like Postgres need to be given the address of a
servicing port with which to connect.

This is not an Erlang build issue. At all. Ever.

The way this is handled is through OS distributions -- and if the decision
were made to include this sort of "build" support (or rather, not "dep
resolution" but "adjacent service provision") then the package manager would
wind up becoming its own OS distro in short order, and from there wind up
trying to be OpenStack. And it can't ever be that. Not even anything remotely
close to that.

The only thing even sort of close would be for Erlang to pick a single distro
of a single OS on a single platform and settle on that as the sole reference
implementation forever and ever, amen, and pour time into maintaining that
(blessed) distro's Erlang subsection of packages through its package manager.
This, actually, is workable from a technical point of view -- but is
guaranteed to not be socially workable, even if it were done in the Belly of
Some Beast (neither MS nor IBM has been able to pull this off, and they own
their entire stack) -- actually, I imagine the annoyance of keeping up with
this sort of thing is why Erlang was spun out of Ericsson in the first place.

And that leaves us back at the original problem, which has been demonstrated
to be twofold:

1- Define an ecosystem within which code can be grown and known to survive with
a minimum of wasted developer effort.
2- Set clear limits of that ecosystem.

The ecosystem definition is really more important than any particular
implementation of a tool within it -- but that gets into standards, and who
follows those without a reference implementation anyway? (And who spends time
writing *and* maintaining new implementations when the reference platform
already works well? We want to do more interesting things with Erlang than
write more Erlang, I imagine...) It has to be complete enough to cover the
problem space so that something can be implemented in the first place -- but
absolutely can't go so far that it suffers from second-system-syndrome.

I've got plenty of ideas what the above statements should mean, but that's not
the point of this message. My purpose is to point out straight away that this
entire idea will be DOA if we let the Good Idea Fairy serve up a big dream
steak smothered in hope peppers with a side of good intentions and a community
love salad before even a line of code has been written.

-Craig
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Tuncer Ayaz
In reply to this post by Bruce Yinhe-2
Hi Bruce,

in addition to what others have said, here's a list of other important
features.

-- must haves --
* archival of published versions

* real version constraint solver (SAT)

* instead of blindly following semver, look at RPM versioning. see
  comments starting here:
  https://github.com/rebar/rebar/issues/240#issuecomment-38035168

* design index _and_ packages folder with mirroring in mind:
  - avoid SPOF (availability and reliability issues)
  - allow local mirrors
  - Ideally avoid the use of a CDN and use mirrors like FreeBSD, CTAN,
    and Linux distros do. This means, master is only written to when
    the index is updated and normally no client operation should
    contact master. So, only mirrors should ever be in contac with
    the master.
  - Avoiding a CDN solves multiple issues:
    + no superfluous hosting costs due to unnecessary centralization.
      hosting costs may not be a problem today, but a package index
      and repo of open source code does not need to be hosted in a
      central location by one administrating org.
    + No accessibility issues for geographical or organizational
      network reasons. This is where the ability to mirror, as in CTAN
      or Debian mirror, is a vital feature.
    + No downtime if, say, S3 is for some reason inaccessible or its
      availability is limited.
  - With a proper mirroring system in place, the only operation that can
    suffer is modification of the index+package_dir. That means, no r/o
    user will be affected if EIUG takes master down for maintenance.
    This may seem like it's not a problem, but compared to centralized
    solutions, I never once had a remote reliability problem when
    operating on Linux, BSD, or CTAN packages.
  - Based on the multitude of mirrors for BSD, Linux, CTAN, etc., I
    believe it's more likely for someone to host another mirror than
    expecting such offers to turn into monetary support for covering
    costs of a centralized system.


-- nice to have --
* pinning as - in stay at this version and do not down-/up-grade

* pinning local package as in 'opam source PKGNAME --pin'

* regardless of .ez limitations, architecture specific variants of a
  package

* delta index (Debian-like) and package updates (Fedora-like)


-- to be careful about --
* generating native packages (FreeBSD pkg, dpkg, rpm, etc.) is nice to
  have, but if existing issues of, say, the way Perl is packaged in
  Linux distros is any indication, I'd say it should only be optional.

* a git repository with its history is not a suitable mechanism to
  store either the index or the packages

-- extra notes --
I had planned to work on .ez-fying all libs/apps in
OTP-18.0, but I haven't had the time to approach that yet. I mention
that because we definitely have to improve support in that space
first, especially:

* Make sure everything works with .ez archives without first
  extracting (like JAR and WAR archives).

* Figure out official extensions of .app fields to properly support
  packaging in general.

* Figure out a convention to load .so/.dll from .ez and extend the
  loaders accordingly. It's up for discussion whether somepkg-1.0.ez
  should bundle all .so/.dll files or if there should be one
  somepkg-1.0-<ARCH>.ez for each supported architecture.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

PAILLEAU Eric
In reply to this post by Bruce Yinhe-2
Hi,
your opinion.

There is already esl-erlang packages for Erlang core and libs.
I write on package for erlang applications.

Doing OS packages for an Erlang application may need other tools , and dependencies to esl could install the right Erlang vm version.
the main problem to address is to be able to install and use several VM version in parallel , like alternatives with java. kerl ?
I do not talk on embedded system  releases.

And not mentioning native code.

« Envoyé depuis mon mobile » Eric

zxq9 <[hidden email]> a écrit :

>On 2014年12月16日 火曜日 15:56:22 Eric Pailleau wrote:
>> I forgot to say that Erlang packages may need Non Erlang application, like
>> Postgresql, or Varnish or (fill blanks). Debian Meta package allow this,
>> for instance. This could not be achieved with a specific Erlang packager.
>
>There are no Erlang packages or builds that require Postgres. Some C
>interfaces may require Postgres headers to build (and these are not "Erlang
>packages" in the strict sense), but most Erlang systems that have anything at
>all to do with something like Postgres need to be given the address of a
>servicing port with which to connect.
>
>This is not an Erlang build issue. At all. Ever.
>
>The way this is handled is through OS distributions -- and if the decision
>were made to include this sort of "build" support (or rather, not "dep
>resolution" but "adjacent service provision") then the package manager would
>wind up becoming its own OS distro in short order, and from there wind up
>trying to be OpenStack. And it can't ever be that. Not even anything remotely
>close to that.
>
>The only thing even sort of close would be for Erlang to pick a single distro
>of a single OS on a single platform and settle on that as the sole reference
>implementation forever and ever, amen, and pour time into maintaining that
>(blessed) distro's Erlang subsection of packages through its package manager.
>This, actually, is workable from a technical point of view -- but is
>guaranteed to not be socially workable, even if it were done in the Belly of
>Some Beast (neither MS nor IBM has been able to pull this off, and they own
>their entire stack) -- actually, I imagine the annoyance of keeping up with
>this sort of thing is why Erlang was spun out of Ericsson in the first place.
>
>And that leaves us back at the original problem, which has been demonstrated
>to be twofold:
>
>1- Define an ecosystem within which code can be grown and known to survive with
>a minimum of wasted developer effort.
>2- Set clear limits of that ecosystem.
>
>The ecosystem definition is really more important than any particular
>implementation of a tool within it -- but that gets into standards, and who
>follows those without a reference implementation anyway? (And who spends time
>writing *and* maintaining new implementations when the reference platform
>already works well? We want to do more interesting things with Erlang than
>write more Erlang, I imagine...) It has to be complete enough to cover the
>problem space so that something can be implemented in the first place -- but
>absolutely can't go so far that it suffers from second-system-syndrome.
>
>I've got plenty of ideas what the above statements should mean, but that's not
>the point of this message. My purpose is to point out straight away that this
>entire idea will be DOA if we let the Good Idea Fairy serve up a big dream
>steak smothered in hope peppers with a side of good intentions and a community
>love salad before even a line of code has been written.
>
>-Craig
>_______________________________________________
>erlang-questions mailing list
>[hidden email]
>http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Tuncer Ayaz
In reply to this post by Loïc Hoguin-3
On Tue, Dec 16, 2014 at 1:52 PM, Loic Hoguin wrote:

> Unfortunately in my case I will have to keep maintaining a separate
> wheel because I do not want to boot the Erlang VM for these things.

Curious, has the issue to do with slow startup time of the VM? If so,
it's something that has regressed after R13, and IIRC, at least part
of is due to new features like line numbers, but I'd have to dig out
an old thread to be sure.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Michael Truog
In reply to this post by Bruce Yinhe-2
1) It needs to be simple, allowing you to publish a package within less than 1 hour of usage.  Good examples are https://hex.pm/ for elixir, https://pypi.python.org/pypi for python, https://rubygems.org/ for ruby, https://www.npmjs.com/ for node.js.  Bad examples are http://search.maven.org/ for java, http://www.cpan.org/ for perl.  Maven is probably the best example of what is worst, due to the process being as complex as possible and taking as long as possible.  The goal of the package manager is not to earn more consulting money (I hope), i.e., consultant-ware.
2) It needs to use source code in the packages, not binaries, to make sure everything is transparent, avoiding black-box binary blobs which lack any ability to be examined easily.  Past erlang package managers have had trouble here.  Along with this, there needs to be signing of the package for the identity of the publisher and the integrity of the package.
3) We don't need package dictators which attempt to decide what packages are important, since that just limits the size of the community, making this less of an open source effort (unless that is the goal).  You will notice the large open source communities don't need to do this.
4) It should be easy to utilize external build tools (like autoconf/automake/makefiles) in a way that is the same as things like rebar and emake due to the obvious problems doing non-Erlang builds within an Erlang package (port, NIF and port driver compilation support is unable to cover complex usage, i.e., it lacks cross-platform support and dependency checking, these shortcomings are likely to always be there, due to the effort involved in tackling them seriously).  It would be good to avoid bikeshedding here and the potential for lock-in.
5) It should not require a server running locally, just basic client usage of a tool.  If the package manager tool's execution lasts beyond its usage, it doesn't look as simple as it should (instead it just looks like a potential security problem).  This could include leaving epmd running which has been an issue with past erlang package managers (puzzling potential users).
6) Ideally, it should be easy to create a private package index for private corporate usage of internal packages.  It would be nice to keep all the programming language usage in Erlang for the sake of simplicity (there have been issues in the past with mixing Python and other programming languages into the tool) and to keep the dependencies limited.  This is a great opportunity to show how Erlang can shine to provide the community with simple package management.

On 12/16/2014 03:41 AM, Bruce Yinhe wrote:
Hi everyone,

Industrial Erlang User Group (IEUG) has been working together with the Erlang/OTP team to investigate and create a package management system for Erlang/OTP. 

The lack of a package management system for Erlang has been discussed for a long time in the community. In essence, a straightforward package management system is believed to take Erlang programming language a step forward. Multiple tools will appear in the community. It needs to be supported by a highly visible community behind it.


In order to increase the adoption and to result in a tool widely used in the Erlang ecosystem, we are identifying the most important user categories and use cases, based on what the majority of the community want in a package manager. Therefore we would like to invite an open discussion.

Now you are welcome to share your thoughts, suggestions or proposals about an Erlang package manager. It would be great if you could reply with your motivation, explaining why a feature is necessary to have. There are some example questions to begin the dicussion with, including, but not limited to the following. 
  • What metadata information should an Erlang package include?
  • What functionality do you need in a package manager for Erlang in order to use it in production?
  • What other concerns do you have about an Erlang package management system?
Erlang package manager's brief wish list of features: 
  • Console interface
  • Web interface
  • Package Index and Repository
  • Fetch, Install and Remove Packages
  • Publish packages
  • Versioning and Dependency Management
We are aware of several previous efforts and existing tools that attempt to achieve the similar goal. We want to look at existing things, both from Erlang and Elixir, to see if they fit the requirements. If not, we will then have to make something new, perhaps as a rewrite of an existing tool.

The IEUG members are putting together requirements for a package manager and will work with the community and Ericsson to create a standard and address any voids which exists in the existing tooling, funding necessary efforts required. 

Best regards

Bruce Yinhe

Erlang Community Manager
Industrial Erlang User Group
[hidden email]
+46 72 311 43 89


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Loïc Hoguin-3
In reply to this post by Tuncer Ayaz
On 12/16/2014 07:05 PM, Tuncer Ayaz wrote:
> * a git repository with its history is not a suitable mechanism to
>    store either the index or the packages

Then what is suitable for the index? (It's obviously not suitable for
packages themselves, yes.)

--
Loïc Hoguin
http://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Loïc Hoguin-3
In reply to this post by Tuncer Ayaz
On 12/16/2014 07:37 PM, Tuncer Ayaz wrote:
> On Tue, Dec 16, 2014 at 1:52 PM, Loic Hoguin wrote:
>
>> Unfortunately in my case I will have to keep maintaining a separate
>> wheel because I do not want to boot the Erlang VM for these things.
>
> Curious, has the issue to do with slow startup time of the VM? If so,
> it's something that has regressed after R13, and IIRC, at least part
> of is due to new features like line numbers, but I'd have to dig out
> an old thread to be sure.

It's mostly about slow startup time, yes.

It's interesting how erlc runs very fast though, despite running Erlang
code. If the package manager was built like erlc, instead of like an
escript, it would ease my concerns very much and I might be able to
integrate it with erlang.mk (assuming there are no other issues that I
can't foresee at the moment).

--
Loïc Hoguin
http://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Tuncer Ayaz
In reply to this post by Loïc Hoguin-3
On Tue, Dec 16, 2014 at 9:23 PM, Loic Hoguin wrote:
> On 12/16/2014 07:05 PM, Tuncer Ayaz wrote:
> >
> > * a git repository with its history is not a suitable mechanism to
> >    store either the index or the packages
>
> Then what is suitable for the index? (It's obviously not suitable
> for packages themselves, yes.)

It would be a good start to take hints from Debian or Fedora's
systems, to name two designs that are better suited than just putting
anything in a git repo. If we take archival of old versions seriously,
it's a good idea to have a look at Hackage(2) as well. Ubuntu's index
does not appear to support delta index updates, and that's why I wrote
Debian, if somebody wonders.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Erlang package manager

Tuncer Ayaz
In reply to this post by Loïc Hoguin-3
On Tue, Dec 16, 2014 at 9:28 PM, Loic Hoguin wrote:

> On 12/16/2014 07:37 PM, Tuncer Ayaz wrote:
> >
> > On Tue, Dec 16, 2014 at 1:52 PM, Loic Hoguin wrote:
> >
> > > Unfortunately in my case I will have to keep maintaining a
> > > separate wheel because I do not want to boot the Erlang VM for
> > > these things.
> >
> >
> > Curious, has the issue to do with slow startup time of the VM? If
> > so, it's something that has regressed after R13, and IIRC, at
> > least part of is due to new features like line numbers, but I'd
> > have to dig out an old thread to be sure.
>
>
> It's mostly about slow startup time, yes.
>
> It's interesting how erlc runs very fast though, despite running
> Erlang code. If the package manager was built like erlc, instead of

Without having checked the code, I suppose it's due to starting with
less apps (modules) and similar settings. From what I recall, I think
it used to start the VM with args like "-mode minimal" and maybe other
switches, but I don't remember it all.

> like an escript, it would ease my concerns very much and I might be
> able to integrate it with erlang.mk (assuming there are no other
> issues that I can't foresee at the moment).

The increase in startup time after R13 is unfortunate and asking for
someone to profile and check for possible improvements. Maybe there's
also a need for an erlc-like escript startup mode that could try to
delay stuff.

Anyway, Loic, this is off-topic, so we should either use a separate
thread or not pollute the pkg mgr thread.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
12345