Quantcast

Common Test last test report

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Common Test last test report

Loïc Hoguin-3
Hello,

It's still difficult to get to the last test report in Common Test.
Currently the URI looks like this:

file:///home/essen/ninenines/cowboy/logs/ct_run.ct_cowboy@localhost.2017-02-25_00.02.28/ninenines.cowboy.rfc7540_SUITE.logs/run.2017-02-25_00.02.29/suite.log.html

Which is great, I can always go back and compare, and sometimes that's
what I want (when I test across Erlang releases, for example). But when
I'm fixing a bug, I tend to run a small set of tests, maybe a single
suite, or sometimes even just a single group or a single test case. When
I do that, things becomes more difficult than they should be because I
can't simply refresh the page to get the new report, I have to go back,
refresh on the all_runs.html page, and follow the links from there.

Would it be possible for Common Test to maintain a separate directory
always containing the latest report? For example:

file:///home/essen/ninenines/cowboy/logs/[hidden email]/ninenines.cowboy.rfc7540_SUITE.logs/run.latest/suite.log.html

With a URI like this I can repeat the test run, refresh in my browser
and get the new report instantly. (And keep the history separate should
I need to compare.)

Alternatively, have an option to disable the timestamps so I can just
refresh when I need it, although the history wouldn't be preserved in
this case.

Cheers,

--
Loïc Hoguin
https://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Common Test last test report

Raghav Karol
+1 to this request. I love common_test, but and this is my biggest pain point. It's either a few clicks in the browser to the see the last failure or a  `find sort cat` like command to see the latest report.

On Sat, Feb 25, 2017 at 12:20 AM, Loïc Hoguin <[hidden email]> wrote:
Hello,

It's still difficult to get to the last test report in Common Test. Currently the URI looks like this:

file:///home/essen/ninenines/cowboy/logs/ct_run.ct_cowboy@localhost.2017-02-25_00.02.28/ninenines.cowboy.rfc7540_SUITE.logs/run.2017-02-25_00.02.29/suite.log.html

Which is great, I can always go back and compare, and sometimes that's what I want (when I test across Erlang releases, for example). But when I'm fixing a bug, I tend to run a small set of tests, maybe a single suite, or sometimes even just a single group or a single test case. When I do that, things becomes more difficult than they should be because I can't simply refresh the page to get the new report, I have to go back, refresh on the all_runs.html page, and follow the links from there.

Would it be possible for Common Test to maintain a separate directory always containing the latest report? For example:

file:///home/essen/ninenines/cowboy/logs/ct_run.ct_cowboy@localhost.latest/ninenines.cowboy.rfc7540_SUITE.logs/run.latest/suite.log.html

With a URI like this I can repeat the test run, refresh in my browser and get the new report instantly. (And keep the history separate should I need to compare.)

Alternatively, have an option to disable the timestamps so I can just refresh when I need it, although the history wouldn't be preserved in this case.

Cheers,

--
Loïc Hoguin
https://ninenines.eu
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



--
Raghav

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Common Test last test report

Peter Andersson-4
In reply to this post by Loïc Hoguin-3
On 02/25/2017 12:20 AM, Loïc Hoguin wrote:

> Hello,
>
> It's still difficult to get to the last test report in Common Test.
> Currently the URI looks like this:
>
> file:///home/essen/ninenines/cowboy/logs/ct_run.ct_cowboy@localhost.2017-02-25_00.02.28/ninenines.cowboy.rfc7540_SUITE.logs/run.2017-02-25_00.02.29/suite.log.html
>
>
> Which is great, I can always go back and compare, and sometimes that's
> what I want (when I test across Erlang releases, for example). But
> when I'm fixing a bug, I tend to run a small set of tests, maybe a
> single suite, or sometimes even just a single group or a single test
> case. When I do that, things becomes more difficult than they should
> be because I can't simply refresh the page to get the new report, I
> have to go back, refresh on the all_runs.html page, and follow the
> links from there.
>
> Would it be possible for Common Test to maintain a separate directory
> always containing the latest report? For example:
>
> file:///home/essen/ninenines/cowboy/logs/[hidden email]/ninenines.cowboy.rfc7540_SUITE.logs/run.latest/suite.log.html
>
>
> With a URI like this I can repeat the test run, refresh in my browser
> and get the new report instantly. (And keep the history separate
> should I need to compare.)
>
> Alternatively, have an option to disable the timestamps so I can just
> refresh when I need it, although the history wouldn't be preserved in
> this case.
>
> Cheers,
>

Hi Loïc + anyone else interested,

I've made a solution to this problem as follows:

In the top level log directory (i.e. where the top level index files,
all_runs.html and index.html, are stored) an html file,
suite.log.latest.html, is written at the start of each test run, which
contains an redirection (immediate, without a click) to the latest saved
suite.log.html (test case overview). There is a link to this redirection
page in the footer of suite.log.html. This makes it possible to jump
immediately to the latest test results from any test case overview page
without having to navigate to the top level index first. So this is
useful when you're running the same tests repeatedly and want to update
and/or compare results.

The shortcoming of this design, however, is that it ignores the test
name. It's the latest report in 'logdir' that is jumped to, not the
latest report for the current set of tests (like e.g. what you get in
the top level index.html page). Example: If you run a set of tests, A,
then repeat this set, A', then run a new set of tests, B, then if you're
analyzing the results of A and click the latest results link, you're
redirected to the report for B, not A'.

The workaround in the above example is either to use different top level
log directories for A and B test runs, or to navigate from
suite.log.html for A to the top level index.html (via the link in the
footer) and go to A' from there.

My question is, what do you guys prefer? Is the current implementation
as I descibe it sufficient and useful, or would a more complex solution
with "name scoped redirections" make more sense? I'd appreciate the input!

Best,
Peter

Ericsson AB, Erlang/OTP


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Loading...