Application dependencies in Common Test?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Application dependencies in Common Test?

Roger Lipscombe-2
We run a bunch of system tests using Common Test, orchestrating
(currently) 9 Erlang nodes, 4 golang processes, and 4 NodeJS
processes.

Those tests (and various CT hooks) have application dependencies. For
example, we have something that uses hackney to check a web server
response, and we have something that uses cowboy to _be_ a web server,
so that we can respond to a request, and so on.

We run the tests by invoking ct_run, something like this (trimmed for size):

        ct_run -pa _build/dev/lib/*/ebin \
                -include $(abspath include) \
                -dir $(SUITES_DIR) \
                -ct_hooks $(CT_HOOKS) \
                -erl_args \
                -config sys.config \
                -s system_tests

In order to start the dependencies (hackney, cowboy, etc.), we use "-s
system_tests" (as above), where system_tests:start/0 looks like this:

    start() -> {ok, _} = application:ensure_all_started(system_tests).

...and we have system_tests.app list our dependencies.

Now (the reason I'm asking): this feels like we've unnecessarily
coupled everything to system_tests.app: if we add a new assertion or
fixture or one of our CT hooks wants to do something different, we
need to add the relevant dependency to this one place. In fact, this
one place only *exists* to list the application dependencies.

Is there a better way to do this?

Related: we've got a mixture of application env vars in the
system_tests application, and values in CT config. That feels ...
wrong as well. Any suggestions here?

Thanks,
Roger.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Application dependencies in Common Test?

Mikael Karlsson-7
Tue 28 May 2019  17:34 Roger Lipscombe <[hidden email]> wrote:
>
> We run a bunch of system tests using Common Test, orchestrating
> (currently) 9 Erlang nodes, 4 golang processes, and 4 NodeJS
> processes.
>
..

>
> We run the tests by invoking ct_run, something like this (trimmed for size):
>
>         ct_run -pa _build/dev/lib/*/ebin \
>                 -include $(abspath include) \
>                 -dir $(SUITES_DIR) \
>                 -ct_hooks $(CT_HOOKS) \
>                 -erl_args \
>                 -config sys.config \
>                 -s system_tests
>
> In order to start the dependencies (hackney, cowboy, etc.), we use "-s
> system_tests" (as above), where system_tests:start/0 looks like this:
>
>     start() -> {ok, _} = application:ensure_all_started(system_tests).
>
> ...and we have system_tests.app list our dependencies.
>
...
> Is there a better way to do this?
>
...
> Thanks,
> Roger.

If it doesn't take to much time to start and configure the
dependencies maybe an option would be to add them in init_per_suite
(and teardown in end_per_suite). And/or in the ct_hooks themselves.
Probably more code duplication but the dependencies would be
configured closer to the separate tests. The ct_hooks seem also to be
installable from the suite (suite/0 and init_per_suite/1) if that
helps.

Best Regards
Mikael
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions