Make Erlang compile on recent FreeBSD releases (long)

[Giacomo Olgeni]

(I'm reposting as the first attempt didn't show up on the list...)

Hello,

I'm following up to a previous post (http://tinyurl.com/5dtkfb) about
FreeBSD support.

The FreeBSD port of Erlang has been collecting patches for a while.
Some of them are critical build fixes without which it is not possible
to compile Erlang on a recent FreeBSD installation (I'm using 7.0
here), so I thought it would be useful to sort out all the patches by
running through an unpatched build of R12B-4 and checking what happens
without them. Most of the patches were actually obtained from
erlang-patches.

All the patches may be downloaded from here:

http://www.freebsd.org/cgi/cvsweb.cgi/ports/lang/erlang/files/

The port sets --enable-threads and --enable-dynamic-ssl-lib as
arguments for running the configure tool.

The first thing to go wrong is the PCRE library. Apparently, Erlang
ships with its own version of PCRE which is not compatible to the PCRE
version from the FreeBSD ports (7.8 as of today). Thus, erl_bif_re.c
will fail to compile:

beam/erl_bif_re.c: In function 'cleanup_restart_context':
beam/erl_bif_re.c:498: warning: implicit declaration of function
'pcre_free_restart_data'
beam/erl_bif_re.c: In function 're_run_3':
beam/erl_bif_re.c:921: error: 'PCRE_EXTRA_LOOP_LIMIT' undeclared (first use in
this function)
beam/erl_bif_re.c:921: error: (Each undeclared identifier is reported only once
beam/erl_bif_re.c:921: error: for each function it appears in.)
beam/erl_bif_re.c:923: error: 'pcre_extra' has no member named 'loop_limit'
beam/erl_bif_re.c:926: error: 'pcre_extra' has no member named 'loop_limit'
beam/erl_bif_re.c:927: error: 'pcre_extra' has no member named 'loop_limit'
beam/erl_bif_re.c:930: error: 'pcre_extra' has no member named 'restart_data'
beam/erl_bif_re.c:931: error: 'pcre_extra' has no member named 'restart_flags'
beam/erl_bif_re.c:932: error: 'pcre_extra' has no member named
'loop_counter_return'
beam/erl_bif_re.c:972: error: 'PCRE_ERROR_LOOP_LIMIT' undeclared (first use in
this function)
beam/erl_bif_re.c: In function 're_exec_trap':
beam/erl_bif_re.c:1023: error: 'pcre_extra' has no member named 'loop_limit'
beam/erl_bif_re.c:1026: error: 'pcre_extra' has no member named 'loop_limit'
beam/erl_bif_re.c:1027: error: 'pcre_extra' has no member named 'loop_limit'
beam/erl_bif_re.c:1029: error: 'pcre_extra' has no member named
'loop_counter_return'
beam/erl_bif_re.c:1030: error: 'pcre_extra' has no member named 'restart_data'
beam/erl_bif_re.c:1031: error: 'pcre_extra' has no member named 'restart_flags'
beam/erl_bif_re.c:1039: error: 'PCRE_ERROR_LOOP_LIMIT' undeclared (first use in
this function)
gmake[3]: *** [obj/i386-portbld-freebsd7.0/opt/plain/erl_bif_re.o] Error 1
gmake[3]: Leaving directory `/usr/storage/erlang/work/otp_src_R12B-4/erts/emulator'

Part of patch-erts-emulator-Makefile.in deals with this by sorting the
$(CC) command line flag so that the local $(INCLUDES) show up as the
first option.

--- erts/emulator/Makefile.in.orig
+++ erts/emulator/Makefile.in
@@ -544,7 +546,7 @@
  endif

  $(OBJDIR)/%.o: beam/%.c
-       $(CC) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) $(INCLUDES) -c $< -o $@
+       $(CC) $(INCLUDES) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) -c $< -o $@

  else

This allows to get past the PCRE issue.

Then you will probably bump into gethostbyname_r, which is called using
incompatible parameters.

connect/ei_resolve.c: In function 'ei_gethostbyname_r':
connect/ei_resolve.c:629: warning: passing argument 5 of 'gethostbyname_r' from
incompatible pointer type
connect/ei_resolve.c:629: error: too few arguments to function
'gethostbyname_r'
gmake[4]: ***
[/usr/storage/erlang/work/otp_src_R12B-4/lib/erl_interface/obj.mt/i386-portbld-freebsd7.0/ei_resolve.o]
Error 1
gmake[4]: Leaving directory
`/usr/storage/erlang/work/otp_src_R12B-4/lib/erl_interface/src'
gmake[3]: *** [opt] Error 2
gmake[3]: Leaving directory
`/usr/storage/erlang/work/otp_src_R12B-4/lib/erl_interface/src'
gmake[2]: *** [opt] Error 2
gmake[2]: Leaving directory
`/usr/storage/erlang/work/otp_src_R12B-4/lib/erl_interface'
gmake[1]: *** [opt] Error 2
gmake[1]: Leaving directory `/usr/storage/erlang/work/otp_src_R12B-4/lib'
gmake: *** [libs] Error 2
*** Error code 2

The patch in patch-lib_erl__interface_src_connect_ei__resolve.c sorts this out.

--- lib/erl_interface/src/connect/ei_resolve.c.orig
+++ lib/erl_interface/src/connect/ei_resolve.c
@@ -621,7 +621,8 @@

    return result;
  #else
-  return gethostbyname_r(name,hostp,buffer,buflen,h_errnop);
+  struct hostent *dummy;
+  return gethostbyname_r(name,hostp,buffer,buflen,&dummy,h_errnop);
  #endif
  #endif
  #endif

If you rebuild with these patches, memsup.c hits you next:

memsup.c: In function 'get_extended_mem':
memsup.c:456: error: void value not ignored as it ought to be

patch-lib_os__mon_c__src_memsup.c will fix it:

--- lib/os_mon/c_src/memsup.c.orig
+++ lib/os_mon/c_src/memsup.c
@@ -404,7 +404,7 @@
  #endif

  #if defined(BSD4_4)
-static void
+static int
  get_extended_mem_bsd4(memory_ext *me) {
      struct vmtotal vt;
      long pgsz;

The next issue is about odbcserver.c:

odbcserver.c: In function 'connect_to_erlang':
odbcserver.c:1625: error: storage size of 'sin' isn't known

patch-lib_odbc_c_src_odbcserver.c includes the missing header:

--- lib/odbc/c_src/odbcserver.c.orig    Sat Jun 25 17:34:20 2005
+++ lib/odbc/c_src/odbcserver.c Sat Jun 25 17:34:33 2005
@@ -107,6 +107,7 @@
  #include <sys/socket.h>
  #include <sys/uio.h>
  #include <netdb.h>
+#include <netinet/in.h>
  #endif

  #include "ei.h"

These patches will at least get you through the build.

Now, let's try to run "gmake install".

gmake[6]: Entering directory
`/usr/storage/erlang/work/otp_src_R12B-4/lib/crypto/c_src'
/usr/bin/install -c -o root -g wheel -d
/usr/local/lib/erlang/lib/crypto-1.5.2.1/priv/obj
/usr/bin/install -c -o root -g wheel -d
/usr/local/lib/erlang/lib/crypto-1.5.2.1/priv/lib
install  -s -o root -g wheel -m 555 ../priv/Makefile
/usr/local/lib/erlang/lib/crypto-1.5.2.1/priv/obj
strip: /usr/local/lib/erlang/lib/crypto-1.5.2.1/priv/obj/Makefile: File format
not recognized
install: wait: No such file or directory
gmake[6]: *** [release_spec] Error 70

The problem here is that Makefile is treated like an executable, so
INSTALL_PROGRAM tries to run the strip command on it, which fails because
there's no executable to strip.

patch-lib_crypto_c__src_Makefile.in switches INSTALL_PROGRAM to INSTALL_DATA:

--- lib/crypto/c_src/Makefile.in.orig   Thu Oct  7 12:11:06 2004
+++ lib/crypto/c_src/Makefile.in        Thu Oct  7 12:11:07 2004
@@ -121,7 +121,7 @@
  release_spec: opt
         $(INSTALL_DIR) $(RELSYSDIR)/priv/obj
         $(INSTALL_DIR) $(RELSYSDIR)/priv/lib
-       $(INSTALL_PROGRAM) $(DRV_MAKEFILE) $(RELSYSDIR)/priv/obj
+       $(INSTALL_DATA) $(DRV_MAKEFILE) $(RELSYSDIR)/priv/obj
         $(INSTALL_PROGRAM) $(OBJS) $(RELSYSDIR)/priv/obj
         $(INSTALL_PROGRAM) $(DYN_DRIVER) $(RELSYSDIR)/priv/lib

With this last patch you will be able to install a working Erlang.

However, there are still some issues with SCTP support.

I added "--enable-sctp" to the configure parameters, but then...

configure: configuring in erts
configure: running /bin/sh
'/usr/storage/erlang/work/otp_src_R12B-4/erts/configure' --prefix=/usr/local
'--enable-threads' '--enable-dynamic-ssl-lib' '--enable-sctp'
'--prefix=/usr/local' '--mandir=/usr/local/man' '--infodir=/usr/local/info/'
'--build=i386-portbld-freebsd7.0' 'CC=cc' 'CFLAGS=-O2 -fno-strict-aliasing
-pipe -I/usr/local/include' 'LDFLAGS= -L/usr/local/lib'
'build_alias=i386-portbld-freebsd7.0' --cache-file=/dev/null
--srcdir=/usr/storage/erlang/work/otp_src_R12B-4/erts
[...]
checking for sys/devpoll.h... no
test: xyes: unexpected operator
checking valgrind/valgrind.h usability... no
[...]

If you look into erts/configure, you'll see that the test for enable_sctp
lies exactly between the devpoll and valgrind checks:

if test "x$enable_sctp" == "xyes" ; then
     echo "$as_me:$LINENO: checking for netinet/sctp.h" >&5
echo $ECHO_N "checking for netinet/sctp.h... $ECHO_C" >&6
if test "${ac_cv_header_netinet_sctp_h+set}" = set; then
   echo $ECHO_N "(cached) $ECHO_C" >&6
else
   cat >conftest.$ac_ext <<_ACEOF

Notice the "==" operator, which should actually be "=".

Let's patch it in patch-erts_configure:

--- erts/configure.orig
+++ erts/configure
@@ -10620,7 +10620,7 @@



-if test "x$enable_sctp" == "xyes" ; then
+if test "x$enable_sctp" = "xyes" ; then
      echo "$as_me:$LINENO: checking for netinet/sctp.h" >&5
  echo $ECHO_N "checking for netinet/sctp.h... $ECHO_C" >&6
  if test "${ac_cv_header_netinet_sctp_h+set}" = set; then

...and retry:

checking for sys/devpoll.h... no
checking for netinet/sctp.h... yes
checking for sctp_bindx in -lsctp... no
checking whether SCTP_UNORDERED is declared... yes
checking whether SCTP_ADDR_OVER is declared... yes
checking whether SCTP_ABORT is declared... yes
checking whether SCTP_EOF is declared... yes
checking whether SCTP_SENDALL is declared... yes
checking whether SCTP_ADDR_CONFIRMED is declared... yes
checking for struct sctp_paddrparams.spp_pathmtu... yes
checking for struct sctp_paddrparams.spp_sackdelay... no
checking for struct sctp_paddrparams.spp_flags... yes
checking for struct sctp_remote_error.sre_data... yes
checking for struct sctp_send_failed.ssh_data... no
checking valgrind/valgrind.h usability... no
checking valgrind/valgrind.h presence... no

Here we made some progress, however the check for sctp_bindx in -lsctp
causes "WITH_SCTP=" to be written in
erts/emulator/i386-portbld-freebsd7.0/Makefile,
thus disabling SCTP entirely.

According to SCTP_BINDX(3), sctp_bindx is found in -lc rather than -lsctp
on FreeBSD.

So, the following patch is added to patch-erts_configure:

--- erts/configure.orig
+++ erts/configure
@@ -10679,7 +10679,6 @@
    echo $ECHO_N "(cached) $ECHO_C" >&6
  else
    ac_check_lib_save_LIBS=$LIBS
-LIBS="-lsctp  $LIBS"
  cat >conftest.$ac_ext <<_ACEOF
  /* confdefs.h.  */
  _ACEOF

However this should probably be fixed in configure.in, together with the
(now incorrect) log message.

This time we get the correct result:

checking for netinet/sctp.h... yes
>>> checking for sctp_bindx in -lsctp... yes <<<
checking whether SCTP_UNORDERED is declared... yes

If we rebuild, we are stopped again here:

[...] obj/i386-portbld-freebsd7.0/opt/plain/efile_drv.o
obj/i386-portbld-freebsd7.0/opt/plain/inet_drv.o
obj/i386-portbld-freebsd7.0/opt/plain/zlib_drv.o
obj/i386-portbld-freebsd7.0/opt/plain/ram_file_drv.o
obj/i386-portbld-freebsd7.0/opt/plain/ttsl_drv.o -lutil -lm   -lncurses
-L../lib/internal/i386-portbld-freebsd7.0
/usr/storage/erlang/work/otp_src_R12B-4/erts/emulator/zlib/obj/i386-portbld-freebsd7.0/opt/libz.a
/usr/storage/erlang/work/otp_src_R12B-4/erts/emulator/pcre/obj/i386-portbld-freebsd7.0/opt/libepcre.a
-lsctp -lethread -lpthread -lerts_internal_r
/usr/bin/ld: cannot find -lsctp

patch-erts-emulator-Makefile.in is then modified to avoid looking for
libsctp on FreeBSD.

--- erts/emulator/Makefile.in.orig
+++ erts/emulator/Makefile.in
@@ -265,8 +265,10 @@
  endif
  WITH_SCTP=@WITH_SCTP@
  ifdef WITH_SCTP
+ifneq ($(findstring freebsd,$(TARGET)),freebsd)
  LIBS += -lsctp
  endif
+endif

  ORG_THR_LIBS=@EMU_THR_LIBS@
  THR_LIB_NAME=@EMU_THR_LIB_NAME@

After the last patch, it appears that SCTP is working.

Erlang (BEAM) emulator version 5.6.4 [source] [async-threads:0] [hipe]
[kernel-poll:false]

Eshell V5.6.4  (abort with ^G)
1> gen_sctp:open ().
{ok,#Port<0.96>}
2>

On the website above you will find more patches which are not critical to
the build. Specifically, there's a patch named for erl_process_dump.c
(which was obtained from this mailing list - credits due to whoever posted
it!) that fixes a potential crash, so I was wondering if it would be
suitable for inclusion in the next release.

--- erts/emulator/beam/erl_process_dump.c.orig
+++ erts/emulator/beam/erl_process_dump.c
@@ -399,9 +399,11 @@
  static void
  dump_externally(int to, void *to_arg, Eterm term)
  {
-    byte sbuf[1024]; /* encode and hope for the best ... */
-    byte* s;
-    byte* p;
+    byte *sbuf, *s, *p;
+    unsigned size;
+
+    size = encode_size_struct(term, TERM_TO_BINARY_DFLAGS);
+    sbuf = (byte *) erts_alloc(ERTS_ALC_T_TMP, size);

      s = p = sbuf;
      erts_to_external_format(NULL, term, &p, NULL, NULL);
@@ -409,4 +411,6 @@
      while (s < p) {
  erts_print(to, to_arg, "%02X", *s++);
      }
+
+    erts_free(ERTS_ALC_T_TMP, (void *) sbuf);
  }

I would also suggest that the next Erlang releases are tested on a supported
FreeBSD version (today 6.x should be fine) so as to make life easier for users
who are following a customized release process and do not rely on the ports
system to install Erlang.

Kind regards,

   Jimmy
_______________________________________________
erlang-patches mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-patches

[Giacomo Olgeni]

Hello,

Erlang R12B-5 was recently released and there's a new issue about
SCTP. Moreover, the "==" operator problem in configure is still there:
how can anybody manage to build with SCTP?

While R12B-4 was using -lsctp as a linker option, R12B-5 is
specifically looking for libsctp.so.1, which is rather unfortunate
because 1) it expects to find a certain shared library version, and 2)
it does not exist at all on (at least) FreeBSD.

The following patch tricks inet_drv into skipping the ddll code if
HAVE_SCTP is defined and you happen to be using FreeBSD. This should
probably be fixed in configure.in, but I wouldn't know where to start :-)

--- erts/emulator/drivers/common/inet_drv.c.orig
+++ erts/emulator/drivers/common/inet_drv.c
@@ -3460,9 +3460,14 @@
      /* Check the size of SCTP AssocID -- currently both this driver and the
         Erlang part require 32 bit: */
      ASSERT(sizeof(sctp_assoc_t)==ASSOC_ID_LEN);
-#   ifndef LIBSCTP
-#     error LIBSCTP not defined
-#   endif
+#   if defined (__FreeBSD__)
+    /* In FreeBSD, sctp_bindx belongs to libc - see sctp_bindx(3) */
+    inet_init_sctp();
+    add_driver_entry(&sctp_inet_driver_entry);
+#   else
+#     ifndef LIBSCTP
+#       error LIBSCTP not defined
+#     endif
      if (erts_sys_ddll_open_noext(STRINGIFY(LIBSCTP), &h_libsctp) == 0) {
  void *ptr;
  if (erts_sys_ddll_sym(h_libsctp, "sctp_bindx", &ptr) == 0) {
@@ -3471,6 +3476,7 @@
     add_driver_entry(&sctp_inet_driver_entry);
  }
      }
+#   endif
  #  endif
  #endif /* _OSE_ */
      /* remove the dummy inet driver */
@@ -8997,7 +9003,7 @@
     rflag = add_flag ? SCTP_BINDX_ADD_ADDR : SCTP_BINDX_REM_ADDR;

     /* Invoke the call: */
-    if (p_sctp_bindx(desc->s, addrs, n, rflag) < 0)
+    if (sctp_bindx(desc->s, addrs, n, rflag) < 0)
  return ctl_error(sock_errno(), rbuf, rsize);

     desc->state = INET_STATE_BOUND;


Regards,

   Giacomo
_______________________________________________
erlang-patches mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-patches

[Per Hedeland]

Giacomo Olgeni <[hidden email]> wrote:
>
>Erlang R12B-5 was recently released and there's a new issue about
>SCTP. Moreover, the "==" operator problem in configure is still there:
>how can anybody manage to build with SCTP?

It works fine if their /bin/sh happens to be bash - one of the more
unfortunate gratuitous "extensions" in that shell, since it tricks users
into writing non-portable scripts for no reason at all. Though I thought
the OTP group was still pretty Solaris-focused, it certainly doesn't
work in Solaris' /bin/sh - but since there's quite a lot else that
doesn't even though it "should', maybe they've fallen for the temptation
to replace it with bash (we haven't gone that far, but pretty close).

By the way, thanks a lot for all your work and persistence with keeping
the FreeBSD port up-to-date! (Speaking both as personal FreeBSD user and
Erlang-product-on-BSD developer.:-) I hope also the fixes in your post
the other day will make it into the standard OTP release.

--Per Hedeland

_______________________________________________
erlang-patches mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-patches

[Raimo Niskanen-5]

On Thu, Nov 06, 2008 at 10:59:40AM +0100, Per Hedeland wrote:

> Giacomo Olgeni <[hidden email]> wrote:
> >
> >Erlang R12B-5 was recently released and there's a new issue about
> >SCTP. Moreover, the "==" operator problem in configure is still there:
> >how can anybody manage to build with SCTP?
>
> It works fine if their /bin/sh happens to be bash - one of the more
> unfortunate gratuitous "extensions" in that shell, since it tricks users
> into writing non-portable scripts for no reason at all. Though I thought
> the OTP group was still pretty Solaris-focused, it certainly doesn't
> work in Solaris' /bin/sh - but since there's quite a lot else that
> doesn't even though it "should', maybe they've fallen for the temptation
> to replace it with bash (we haven't gone that far, but pretty close).

Well, since we are getting Linux workstations we
have fallen into that trap. But we have not seen
any build problems for SCTP on Solaris (10).

>
> By the way, thanks a lot for all your work and persistence with keeping
> the FreeBSD port up-to-date! (Speaking both as personal FreeBSD user and
> Erlang-product-on-BSD developer.:-) I hope also the fixes in your post
> the other day will make it into the standard OTP release.
>

Aggreed!

I am an OpenBSD fan myself, and we have had issues with
FreeBSD regarding kernel poll so it has been a while
when FreeBSD has been uninteresting as an ErlangOTP
platform. And now that FreeBSD has become usable
we have no hardware to run on; it is not exactly
high priority.

But we are acquiring hardware for VMWare to run
many of these for our paying customers peripheral
OS:es such as Free/Open/NetBSD. Having them
work improves quality.

Hopefully on our way to R13 we will have daily builds
for FreeBSD, and Giacomo's patches will then be included.

> --Per Hedeland
>
> _______________________________________________
> erlang-patches mailing list
> [hidden email]
> http://www.erlang.org/mailman/listinfo/erlang-patches

--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-patches mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-patches

[Per Hedeland]

Raimo Niskanen <[hidden email]> wrote:
>
>Well, since we are getting Linux workstations we
>have fallen into that trap. But we have not seen
>any build problems for SCTP on Solaris (10).

Strange... (assuming that not getting it built is a "problem") -

$ /bin/sh
$ uname -srv
SunOS 5.10 Generic_118833-22
$ enable_sctp=yes
$ if test "x$enable_sctp" == "xyes" ; then
> echo OK
> fi
test: unknown operator ==
$

>I am an OpenBSD fan myself, and we have had issues with
>FreeBSD regarding kernel poll so it has been a while
>when FreeBSD has been uninteresting as an ErlangOTP
>platform.

Uh, "issues regarding kernel poll" => "uninteresting as an ErlangOTP
platform" is a pretty huge leap - there are "a few" Erlang applications
that aren't dependant on being able to have thousands of file
descriptors open with good performance. Erlang has always worked fine on
*BSD IMHO.

Which reminds me, I recently fixed the Erlang "virtual time" to work on
*BSD (and QNX!:-), using clock_gettime(). However that was for R10, and
when I looked at this in R12 (which I had already done, but apparently
in the wrong place) in preparation for sending in a patch, I found that
it had already been done - but only for Linux, and in a way that I don't
think will work in general. I.e. it seemed to assume that
clock_gettime(CLOCK_MONOTONIC) has anything near the actual resolution
possible with the struct timespec that it returns - that is definitely
not true on *BSD or QNX, did you verify it on Linux?

> And now that FreeBSD has become usable
>we have no hardware to run on;

I'm sure it will run just fine on your "Linux workstations".:-)

>Hopefully on our way to R13 we will have daily builds
>for FreeBSD, and Giacomo's patches will then be included.

Do you see this as a requirement? I.e. even if some "trustworthy":-)
user reports that the release doesn't even build on OS Foo, and supplies
patches for the problem(s), and those patches don't break the tests on
the OSes that you do test on - you won't incorporate the patches
unless/until you do regular builds on OS Foo yourselves?

--Per
_______________________________________________
erlang-patches mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-patches

[Raimo Niskanen-5]

On Thu, Nov 06, 2008 at 03:54:40PM +0100, Per Hedeland wrote:

> Raimo Niskanen <[hidden email]> wrote:
> >
> >Well, since we are getting Linux workstations we
> >have fallen into that trap. But we have not seen
> >any build problems for SCTP on Solaris (10).
>
> Strange... (assuming that not getting it built is a "problem") -
>
> $ /bin/sh
> $ uname -srv
> SunOS 5.10 Generic_118833-22
> $ enable_sctp=yes
> $ if test "x$enable_sctp" == "xyes" ; then
> > echo OK
> > fi
> test: unknown operator ==
> $

Our daily builds on Solaris 10 did not run for other reasons.
A build failure is something we notice but some machine
mis-setups just make builds silently vanish.
Sorry about that.
Our daily build system needs to be improved for such cases.

>
> >I am an OpenBSD fan myself, and we have had issues with
> >FreeBSD regarding kernel poll so it has been a while
> >when FreeBSD has been uninteresting as an ErlangOTP
> >platform.
>
> Uh, "issues regarding kernel poll" => "uninteresting as an ErlangOTP
> platform" is a pretty huge leap - there are "a few" Erlang applications
> that aren't dependant on being able to have thousands of file
> descriptors open with good performance. Erlang has always worked fine on
> *BSD IMHO.

My memory failed me. It was a severe problem with writev(),
and has been fixed for quite a while, but there was a
long perion in time Erlang did not work fine on FreeBSD.

>
> Which reminds me, I recently fixed the Erlang "virtual time" to work on
> *BSD (and QNX!:-), using clock_gettime(). However that was for R10, and
> when I looked at this in R12 (which I had already done, but apparently
> in the wrong place) in preparation for sending in a patch, I found that
> it had already been done - but only for Linux, and in a way that I don't
> think will work in general. I.e. it seemed to assume that
> clock_gettime(CLOCK_MONOTONIC) has anything near the actual resolution
> possible with the struct timespec that it returns - that is definitely
> not true on *BSD or QNX, did you verify it on Linux?

I do not know about this one, but can you re-submit your R10 patch,
or let us know how it should be done in R12?

>
> > And now that FreeBSD has become usable
> >we have no hardware to run on;
>
> I'm sure it will run just fine on your "Linux workstations".:-)

Surely. Unfortunately it will no longer be a workstation
since we need ClearCase on the workstations.

>
> >Hopefully on our way to R13 we will have daily builds
> >for FreeBSD, and Giacomo's patches will then be included.
>
> Do you see this as a requirement? I.e. even if some "trustworthy":-)
> user reports that the release doesn't even build on OS Foo, and supplies
> patches for the problem(s), and those patches don't break the tests on
> the OSes that you do test on - you won't incorporate the patches
> unless/until you do regular builds on OS Foo yourselves?

Not a requirement. But since it sometimes is hard to
ensure "trustworthy" and it also is easy to make silly
mistakes when applying the patch; having daily builds
sure facilitates trying a patch, and maintaining it.

>
> --Per

--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-patches mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-patches

[Per Hedeland]

Raimo Niskanen <[hidden email]> wrote:

>>
>> >I am an OpenBSD fan myself, and we have had issues with
>> >FreeBSD regarding kernel poll so it has been a while
>> >when FreeBSD has been uninteresting as an ErlangOTP
>> >platform.
>>
>> Uh, "issues regarding kernel poll" => "uninteresting as an ErlangOTP
>> platform" is a pretty huge leap - there are "a few" Erlang applications
>> that aren't dependant on being able to have thousands of file
>> descriptors open with good performance. Erlang has always worked fine on
>> *BSD IMHO.
>
>My memory failed me. It was a severe problem with writev(),
>and has been fixed for quite a while, but there was a
>long perion in time Erlang did not work fine on FreeBSD.

Actually I think you were more correct the first time:-) - AFAIK, the
problem was with the combination of writev and kqueue (and pipes), see
e.g.
http://www.erlang.org/pipermail/erlang-questions/2007-April/026055.html
(I have no idea whether it is fixed btw).

Given that writev was introduced by the "original" BSD more than 25
years ago and is used all over the place in current *BSD (and in the
Erlang runtime), while kqueue (the "*BSD version" of "kernel poll") is a
relative newcomer and arguably only needed in "unusual" cases, I would
say that this has to be qualified as a problem with kqueue, not writev.
And again I'll claim that the lack of kernel poll is essentially
uninteresting for many if not most Erlang applications.

>Not a requirement. But since it sometimes is hard to
>ensure "trustworthy" and it also is easy to make silly
>mistakes when applying the patch; having daily builds
>sure facilitates trying a patch, and maintaining it.

Very reasonable. So I assume that Jimmy's patches, done against current
Erlang/OTP code, from someone who has been the tireless maintainer of
the Erlang FreeBSD port for more than 7 years (the ports *are* built
daily at freebsd.org btw), will be applied ASAP.:-)

--Per
_______________________________________________
erlang-patches mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-patches

[Per Hedeland]

Raimo Niskanen <[hidden email]> wrote:

>
>On Thu, Nov 06, 2008 at 03:54:40PM +0100, Per Hedeland wrote:
>>
>> Which reminds me, I recently fixed the Erlang "virtual time" to work on
>> *BSD (and QNX!:-), using clock_gettime(). However that was for R10, and
>> when I looked at this in R12 (which I had already done, but apparently
>> in the wrong place) in preparation for sending in a patch, I found that
>> it had already been done - but only for Linux, and in a way that I don't
>> think will work in general. I.e. it seemed to assume that
>> clock_gettime(CLOCK_MONOTONIC) has anything near the actual resolution
>> possible with the struct timespec that it returns - that is definitely
>> not true on *BSD or QNX, did you verify it on Linux?
>
>I do not know about this one, but can you re-submit your R10 patch,
>or let us know how it should be done in R12?

There's no re- since I never submitted it, due to stumbling on the
conflicting change in R12 (the conflict is probably "only" in the
configure test though). After a bit more testing of clock_gettime()
behaviour, I believe that the R12 way (i.e. using clock_gettime() as
replacement for the Solaris-specific gethrtime()) is fine on Linux 2.6
(only place R12 uses it) and FreeBSD 6.x/7.x, where CLOCK_MONOTONIC does
seem to have at least 1 usec resolution, but not on NetBSD (10 msec) or
QNX (1-2 msec) (I have no OpenBSD to test on). Sigh, it's a mess...

Anyway I enclose my R10 patch FYI - I certainly don't expect you to do
anything with it. It uses the more pessimistic approach of
clock_gettime() as replacement for times(), which I believe should work
on all systems that pass the configure test, but I guess the result is
"better" if you can use it as replacement for gethrtime(). If you would
be interested in a "merge" to R12, using this approach for the
non-Linux/Solaris case only, I'll try to find the time to create one (I
will have to at some point anyway...).

And for those that think this is just of academic interest, I'll only
say that neither "Use NTP" nor "Use Linux" are viable answers when your
customer, that is using your product in his (e.g.) BSD-based product
(and has his own customers in turn), complains "I get thrown out of the
CLI with idle-timeout when I set the time forward".:-)

--Per

Index: erts/emulator/beam/erl_time_sup.c
===================================================================
--- erts/emulator/beam/erl_time_sup.c (revision 19340)
+++ erts/emulator/beam/erl_time_sup.c (revision 19523)
@@ -176,7 +176,7 @@
 #define correction (hr_correction/1000000)
 
 #else /* !HAVE_GETHRTIME */
-#if !defined(CORRECT_USING_TIMES)
+#if !defined(CORRECT_USING_TIMES) && !defined(CORRECT_USING_CLOCK_GETTIME)
 #define init_tolerant_timeofday()
 #define get_tolerant_timeofday(tvp) sys_gettimeofday(tvp)
 #else
@@ -191,6 +191,8 @@
 static Milli last_cc;
 static clock_t last_ct;
 
+#if defined(CORRECT_USING_TIMES)
+
 /* sys_times() might need to be wrapped and the values shifted (right)
    a bit to cope with newer linux (2.5.*) kernels, this has to be taken care
    of dynamically to start with, a special version that uses
@@ -208,6 +210,28 @@
 
 #endif
 
+#else  /* CORRECT_USING_CLOCK_GETTIME */
+
+static clock_t clock_gettime_ticks()
+{
+    struct timespec ts;
+
+    clock_gettime(CLOCK_MONOTONIC, &ts);
+#ifdef ERTS_WRAP_SYS_TIMES
+    return ((clock_t)(ts.tv_sec * SYS_CLK_TCK_WRAP +
+                      ts.tv_nsec / (1000000000 / SYS_CLK_TCK_WRAP))) &
+        ((1UL << ((sizeof(clock_t) * 8) - 1)) - 1);
+#else
+    return ((clock_t)(ts.tv_sec * SYS_CLK_TCK +
+                      ts.tv_nsec / (1000000000 / SYS_CLK_TCK))) &
+        ((1UL << ((sizeof(clock_t) * 8) - 1)) - 1);
+#endif
+}
+
+#define KERNEL_TICKS() clock_gettime_ticks()
+
+#endif
+
 static void init_tolerant_timeofday(void)
 {
     last_ct = init_ct = KERNEL_TICKS();
@@ -249,7 +273,12 @@
  ct_wrap += ((Sint64) 1) << ((sizeof(clock_t) * 8) - 1);
     }
     last_ct = current_ct;
-    ct_diff = ((ct_wrap + current_ct) - init_ct) * TICK_MS;
+    /* Can't use TICK_MS here unless ticks is an integer fraction of 1000 */
+#ifdef ERTS_WRAP_SYS_TIMES
+    ct_diff = ((ct_wrap + current_ct) - init_ct) * 1000 / SYS_CLK_TCK_WRAP;
+#else
+    ct_diff = ((ct_wrap + current_ct) - init_ct) * 1000 / SYS_CLK_TCK;
+#endif
 
     /*
      * We will adjust the time in milliseconds and we allow for 1%
@@ -326,7 +355,7 @@
 #undef TICK_MS
 }
 
-#endif /* CORRECT_USING_TIMES */
+#endif /* CORRECT_USING_TIMES || CORRECT_USING_CLOCK_GETTIME */
 #endif /* !HAVE_GETHRTIME */
 
 /*
Index: erts/acconfig.h
===================================================================
--- erts/acconfig.h (revision 19340)
+++ erts/acconfig.h (revision 19523)
@@ -18,6 +18,9 @@
 /* Define if you do not have a high-res. timer & want to use times() instead */
 #undef CORRECT_USING_TIMES
 
+/* Define if you do not have gethrtime() and want
+   to use clock_gettime(CLOCK_MONOTONIC) instead */
+#undef CORRECT_USING_CLOCK_GETTIME
 
 /*
  * HiPE enable or not.
Index: erts/aclocal.m4
===================================================================
--- erts/aclocal.m4 (revision 19340)
+++ erts/aclocal.m4 (revision 19523)
@@ -625,7 +625,10 @@
 dnl is implemented using gettimeofday so it doesn't make much sense to
 dnl use it there...) On second thought, it seems to be safer to do it the
 dnl other way around. I.e. only use times() on OS's where we know it will
-dnl work...
+dnl work... Update: Actually times() works right on FreeBSD 6.0 and later,
+dnl but still not on NetBSD/OpenBSD - and a "better" fallback is to use
+dnl clock_gettime(CLOCK_MONOTONIC) anyway - more deterministic and here we
+dnl don't care about the accounting stuff that times() collects.
 dnl
 
 AC_DEFUN(ERL_TIME_CORRECTION,
@@ -640,8 +643,26 @@
     case $host_os in
       linux*)
         erl_cv_time_correction=times ;;
+      *qnx*)
+        # clock_gettime(CLOCK_MONOTONIC) works on QNX 6 at least
+        # (can't run test below when cross-compiling)
+        erl_cv_time_correction=clock_gettime ;;
       *)
-        erl_cv_time_correction=none ;;
+ AC_TRY_RUN([
+ #include <stdlib.h>
+ #include <unistd.h>
+ #include <string.h>
+ #include <stdio.h>
+ #include <time.h>
+ int main() {
+    struct timespec tp;
+
+    if (clock_gettime(CLOCK_MONOTONIC, &tp) < 0)
+      exit(1);
+    exit(0); return 0;
+  }
+ ], erl_cv_time_correction=clock_gettime, erl_cv_time_correction=none, erl_cv_time_correction=none)
+        ;;
     esac
     ;;
 esac
@@ -650,6 +671,9 @@
   times)
     AC_DEFINE(CORRECT_USING_TIMES)
     ;;
+  clock_gettime)
+    AC_DEFINE(CORRECT_USING_CLOCK_GETTIME)
+    ;;
 esac
 ])dnl
 
_______________________________________________
erlang-patches mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-patches

[Björn Gustavsson]

On Sat, Nov 8, 2008 at 1:15 PM, Per Hedeland <[hidden email]> wrote:
>
> Given that writev was introduced by the "original" BSD more than 25
> years ago and is used all over the place in current *BSD (and in the
> Erlang runtime), while kqueue (the "*BSD version" of "kernel poll") is a
> relative newcomer and arguably only needed in "unusual" cases, I would
> say that this has to be qualified as a problem with kqueue, not writev.
> And again I'll claim that the lack of kernel poll is essentially
> uninteresting for many if not most Erlang applications.

We initially thought that the problem only had to do with kqueue, but it turned
out that we could reproduce the problem without using kqueue at all.

Quote from the README file:

"FreeBSD has a bug which cause kqueue/poll/select to fail to detect
that a writev() on a pipe has been made. This bug should have been fixed
in FreeBSD 6.3 and FreeBSD 7.0. More information can be found at:
* http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/sys_pipe.c
* http://lists.freebsd.org/pipermail/freebsd-arch/2007-September/006790.html
NetBSD and DragonFlyBSD probably have or have had the same bug."

>>Not a requirement. But since it sometimes is hard to
>>ensure "trustworthy" and it also is easy to make silly
>>mistakes when applying the patch; having daily builds
>>sure facilitates trying a patch, and maintaining it.
>
> Very reasonable. So I assume that Jimmy's patches, done against current
> Erlang/OTP code, from someone who has been the tireless maintainer of
> the Erlang FreeBSD port for more than 7 years (the ports *are* built
> daily at freebsd.org btw), will be applied ASAP.:-)
>

We will have a look at the patches as soon as we have set up a test machine
running FreeBSD. Our goal is to do that before the next release.

/Bjorn

--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-patches mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-patches

[Per Hedeland]

"Bjorn Gustavsson" <[hidden email]> wrote:
>
>Quote from the README file:
>
>"FreeBSD has a bug which cause kqueue/poll/select to fail to detect
>that a writev() on a pipe has been made. This bug should have been fixed
>in FreeBSD 6.3 and FreeBSD 7.0. More information can be found at:
>* http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/sys_pipe.c
>* http://lists.freebsd.org/pipermail/freebsd-arch/2007-September/006790.html
>NetBSD and DragonFlyBSD probably have or have had the same bug."

Ouch, that was nasty - thanks for beating me over the head with it. This
could then (at least) cause failure to collect the output from some
arbitrary port program (that might be using writev even though you don't
know it) - not "fine" indeed.

--Per
_______________________________________________
erlang-patches mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-patches

[Kenji Rikitake]

In the message <[hidden email]>
dated Mon, Nov 10, 2008 at 12:05:35PM +0100,
Bjorn Gustavsson <[hidden email]> writes:
> We will have a look at the patches as soon as we have set up a test machine
> running FreeBSD. Our goal is to do that before the next release.

I really want FreeBSD to be a tested platform in Ericsson's Erlang/OTP
team.  I think Giacomo Olgeni's FreeBSD port patch set is a must for
FreeBSD Erlang users now (my appreciation goes to Giacomo); and I hope
the latest patches for TAI of mine and TZ/mktime handling of Paul Guyot
to be incorporated in R13B.

Note that the following patches for R12B4 are equally applicable to
R12B5, tested on FreeBSD 6.3 and 7.0 releases.

My patch:
http://www.erlang.org/pipermail/erlang-bugs/2008-October/001073.html
Paul Guyot's patch:
http://www.erlang.org/pipermail/erlang-bugs/2008-November/001077.html

Regards,
Kenji Rikitake
_______________________________________________
erlang-patches mailing list
[hidden email]
http://www.erlang.org/mailman/listinfo/erlang-patches