Quantcast

Understanding dialyzer errors better

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Understanding dialyzer errors better

Raghav Karol
I spent a bit of time yesterday on some dialyzer warnings resulting from the singleton type 'undefined' no longer automatically to record fields types OTP-19. What threw me off were several related but hard to decipher warnings. 

Using OTP-19, `rebar3 dialyzer` on this commit https://github.com/basho/bitcask/commit/cd74f59bfe47a39878d7a56a55ce0ae723723677 produces: 

```
src/bitcask.erl
 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 219: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 520: The created fun has no local return
 529: The call bitcask_fileops:close(FD::{_,_,_,_,_,_,_,_,_,_,_}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 717: The call bitcask_fileops:close(Outfile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 722: The call bitcask_fileops:close(TFile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 774: The call bitcask_fileops:close(F::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
1224: The call bitcask_fileops:close(File::#filestate{mode::'read_write',filename::string(),tstamp::integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::non_neg_integer(),ofs::'undefined' | non_neg_integer(),l_ofs::0,l_hbytes::0,l_hintcrc::0}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
1719: The pattern <_Key, _Value, State, 0, LastErr> can never match the type <_,_,#bc_state{dirname::string(),write_file::'fresh' | 'undefined' | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},write_lock::'undefined' | reference(),read_files::'undefined' | [{_,_,_,_,_,_,_,_,_,_,_}],max_file_size::'undefined' | integer(),opts::'undefined' | [any()],key_transform::'undefined' | fun(),keydir::reference(),read_write_p::'undefined' | integer(),tombstone_version::0 | 2},100,'undefined'>
1870: Function wrap_write_file/1 has no local return
1924: Record construction #filestate{filename::[any()],hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port()

src/bitcask_fileops.erl
 166: Record construction #filestate{mode::'read_only',filename::string() | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},tstamp::integer(),hintfd::'undefined',hintcrc::0,ofs::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port()
 240: Function close_hintfile/1 has no local return
 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _, _, _} tagged with a record name violates the declared type of #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}
 251: Record construction #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} violates the declared type of field hintfd::port()
 314: The pattern 'undefined' can never match the type port()

src/bitcask_merge_delete.erl
 151: Record construction #filestate{hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port()
===> Warnings written to /Users/raghav/github/riak_kv/deps/bitcask/_build/default/19.2.dialyzer_warnings
===> Warnings occured running dialyzer: 17
```

Particularly confusing, are errors like the first one 

```
 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
```

I _think_ dialyzer is saying that the call `bitcask_fileops:close_for_writing(WriteFile#filestate{}, ...)` is not possible because of type violations when creating `#filestate{}` lower down the call stack.

Would appreciate if someone could help understand this better and also suggest how one separates noise from real warnings with dialyzer.

Best,
Raghav


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Understanding dialyzer errors better

Alex S.
The general approach I use in these cases is to slap specs on EVERYTHING and see which stuff Dialyzer complains about.
2 марта 2017 г., в 12:39, Raghav Karol <[hidden email]> написал(а):

I spent a bit of time yesterday on some dialyzer warnings resulting from the singleton type 'undefined' no longer automatically to record fields types OTP-19. What threw me off were several related but hard to decipher warnings. 

Using OTP-19, `rebar3 dialyzer` on this commit https://github.com/basho/bitcask/commit/cd74f59bfe47a39878d7a56a55ce0ae723723677 produces: 

```
src/bitcask.erl
 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 219: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 520: The created fun has no local return
 529: The call bitcask_fileops:close(FD::{_,_,_,_,_,_,_,_,_,_,_}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 717: The call bitcask_fileops:close(Outfile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 722: The call bitcask_fileops:close(TFile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 774: The call bitcask_fileops:close(F::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
1224: The call bitcask_fileops:close(File::#filestate{mode::'read_write',filename::string(),tstamp::integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::non_neg_integer(),ofs::'undefined' | non_neg_integer(),l_ofs::0,l_hbytes::0,l_hintcrc::0}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
1719: The pattern <_Key, _Value, State, 0, LastErr> can never match the type <_,_,#bc_state{dirname::string(),write_file::'fresh' | 'undefined' | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},write_lock::'undefined' | reference(),read_files::'undefined' | [{_,_,_,_,_,_,_,_,_,_,_}],max_file_size::'undefined' | integer(),opts::'undefined' | [any()],key_transform::'undefined' | fun(),keydir::reference(),read_write_p::'undefined' | integer(),tombstone_version::0 | 2},100,'undefined'>
1870: Function wrap_write_file/1 has no local return
1924: Record construction #filestate{filename::[any()],hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port()

src/bitcask_fileops.erl
 166: Record construction #filestate{mode::'read_only',filename::string() | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},tstamp::integer(),hintfd::'undefined',hintcrc::0,ofs::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port()
 240: Function close_hintfile/1 has no local return
 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _, _, _} tagged with a record name violates the declared type of #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}
 251: Record construction #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} violates the declared type of field hintfd::port()
 314: The pattern 'undefined' can never match the type port()

src/bitcask_merge_delete.erl
 151: Record construction #filestate{hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port()
===> Warnings written to /Users/raghav/github/riak_kv/deps/bitcask/_build/default/19.2.dialyzer_warnings
===> Warnings occured running dialyzer: 17
```

Particularly confusing, are errors like the first one 

```
 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
```

I _think_ dialyzer is saying that the call `bitcask_fileops:close_for_writing(WriteFile#filestate{}, ...)` is not possible because of type violations when creating `#filestate{}` lower down the call stack.

Would appreciate if someone could help understand this better and also suggest how one separates noise from real warnings with dialyzer.

Best,
Raghav

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Understanding dialyzer errors better

Brujo Benavides-2
In reply to this post by Raghav Karol
Hi Raghav,

I have a private gig at my company: I dialyze projects for beer (true story).
Let me show you an example of my mental process based on your commit. Let’s tackle like 197…

It reads:
_ = bitcask_fileops:close_for_writing(WriteFile),
And dialyzer is saying that the call to that function will never return since it differs the first (and only) argument can’t be fresh nor undefined.
There are 3 possibilities here:
1. bitcask_fileops:close_for_writing/1 actually accepts other kinds of arguments, but the clauses where those types are accepted are also questioned by dialyzer so, in dialyzer’s eyes, they can never happen.
2. WriteFile can actually be fresh or undefined, but dialyzer is fooled by a misplaced spec somewhere.
3. The code is actually broken (i.e. bitcask_fileops:close_for_writing/1 expects fresh or undefined and WriteFile can never be any of those)

Let’s discard #3… I bet your app is tested with 100% coverage, so that can’t happen.
Let’s aim for #1. We have to check the bitcask_fileops:close_for_writing/1’s code. …and bingo! bitcask_fileops:close_for_writing/1 does have a third clause that, instead of fresh or undefined can receive a #filestate{} record (with some fixed params).

Dialyzer must have produced a warning for that one, too… for sure. We check your list and… we found no warning in these lines (236-238).
That’s bad, but check it out, that function is calling close_hintfile/1 and we do have warnings for that one, specifically:

 240: Function close_hintfile/1 has no local return
 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _, _, _} tagged with a record name violates the declared type of #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}
 251: Record construction #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} violates the declared type of field hintfd::port()

Ok, so the function has no local return, that explains all! (o_O)
Anyway, let’s check the last 2, shall we?
On line 240, header of the first clause of the function, we’re using undefined in a record field that doesn’t accept undefined. The warning can be improved, but it can be roughly translated in this way:
- Matching of pattern {‘filestate’, …} tagged with a record name… => The pattern with record filestate that you are using…
- violates the declared type of #… => can never match an actual #filestate record because it’s using at least one sub-pattern that doesn’t match the type of the corresponding field. Here is the record definition:…

On line 251, where we’re building a filestate record, the warning is much clearer: Dialyzer says we’re using the wrong value for field hintfd which is defined as port().

So, it’s pretty clear that our intention was to allow hintfd to be undefined.

My next step here would be to fix that (as you did) and run rebar3 dialyzer again.

We started from line 197… but we could’ve started this process from like 219 as well and reached the same conclusion, right?

I’m not sure if this is what you were actually looking for, but maybe it helps anyway.

Happy dialyzing! :)


On Mar 2, 2017, at 06:39, Raghav Karol <[hidden email]> wrote:

I spent a bit of time yesterday on some dialyzer warnings resulting from the singleton type 'undefined' no longer automatically to record fields types OTP-19. What threw me off were several related but hard to decipher warnings. 

Using OTP-19, `rebar3 dialyzer` on this commit https://github.com/basho/bitcask/commit/cd74f59bfe47a39878d7a56a55ce0ae723723677 produces: 

```
src/bitcask.erl
 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 219: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 520: The created fun has no local return
 529: The call bitcask_fileops:close(FD::{_,_,_,_,_,_,_,_,_,_,_}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 717: The call bitcask_fileops:close(Outfile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 722: The call bitcask_fileops:close(TFile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 774: The call bitcask_fileops:close(F::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
1224: The call bitcask_fileops:close(File::#filestate{mode::'read_write',filename::string(),tstamp::integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::non_neg_integer(),ofs::'undefined' | non_neg_integer(),l_ofs::0,l_hbytes::0,l_hintcrc::0}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
1719: The pattern <_Key, _Value, State, 0, LastErr> can never match the type <_,_,#bc_state{dirname::string(),write_file::'fresh' | 'undefined' | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},write_lock::'undefined' | reference(),read_files::'undefined' | [{_,_,_,_,_,_,_,_,_,_,_}],max_file_size::'undefined' | integer(),opts::'undefined' | [any()],key_transform::'undefined' | fun(),keydir::reference(),read_write_p::'undefined' | integer(),tombstone_version::0 | 2},100,'undefined'>
1870: Function wrap_write_file/1 has no local return
1924: Record construction #filestate{filename::[any()],hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port()

src/bitcask_fileops.erl
 166: Record construction #filestate{mode::'read_only',filename::string() | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},tstamp::integer(),hintfd::'undefined',hintcrc::0,ofs::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port()
 240: Function close_hintfile/1 has no local return
 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _, _, _} tagged with a record name violates the declared type of #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}
 251: Record construction #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} violates the declared type of field hintfd::port()
 314: The pattern 'undefined' can never match the type port()

src/bitcask_merge_delete.erl
 151: Record construction #filestate{hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port()
===> Warnings written to /Users/raghav/github/riak_kv/deps/bitcask/_build/default/19.2.dialyzer_warnings
===> Warnings occured running dialyzer: 17
```

Particularly confusing, are errors like the first one 

```
 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
```

I _think_ dialyzer is saying that the call `bitcask_fileops:close_for_writing(WriteFile#filestate{}, ...)` is not possible because of type violations when creating `#filestate{}` lower down the call stack.

Would appreciate if someone could help understand this better and also suggest how one separates noise from real warnings with dialyzer.

Best,
Raghav

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Understanding dialyzer errors better

Jesper Louis Andersen-2
The dialyzer often reports an error which really occurs somewhere else. Hence, if given an error which you don't know how to solve, it is often good to look for other errors which you do know how to solve. In turn, fixing those errors can clear up the call chain such that other errors become clearer or get resolved as no-error-in-the-first-place.

The reason it is often confusing is that it is never wrong. Rather, the dialyzer tends to find a path in your code you never thought of. So you are thrown off by the structure of your own code which you think you know how runs. The trick is to throw away assumptions and go chase call chains. And fix those errors which you know how to fix early on. Once they are done, you can turn your eyes on the other errors.

Case in point: I did a fix for a single dialyzer warning today. The error count went from about 43 to 24 on that project. The code looked like:

foo(_, _, null) -> ok;
foo(_, _, X) -> ...CBody...

The dialyzer said that the function wouldn't work because my use differed from foo's success typing: (any(), any(), null). What the dialyzer is saying is "there is a bug in CBody, so the function only works if being passed null". Since foo/3 had many uses all over the code base, they were reported with the same error.

On Thu, Mar 2, 2017 at 6:39 PM Brujo Benavides <[hidden email]> wrote:
Hi Raghav,

I have a private gig at my company: I dialyze projects for beer (true story).
Let me show you an example of my mental process based on your commit. Let’s tackle like 197…

It reads:
_ = bitcask_fileops:close_for_writing(WriteFile),
And dialyzer is saying that the call to that function will never return since it differs the first (and only) argument can’t be fresh nor undefined.
There are 3 possibilities here:
1. bitcask_fileops:close_for_writing/1 actually accepts other kinds of arguments, but the clauses where those types are accepted are also questioned by dialyzer so, in dialyzer’s eyes, they can never happen.
2. WriteFile can actually be fresh or undefined, but dialyzer is fooled by a misplaced spec somewhere.
3. The code is actually broken (i.e. bitcask_fileops:close_for_writing/1 expects fresh or undefined and WriteFile can never be any of those)

Let’s discard #3… I bet your app is tested with 100% coverage, so that can’t happen.
Let’s aim for #1. We have to check the bitcask_fileops:close_for_writing/1’s code. …and bingo! bitcask_fileops:close_for_writing/1 does have a third clause that, instead of fresh or undefined can receive a #filestate{} record (with some fixed params).

Dialyzer must have produced a warning for that one, too… for sure. We check your list and… we found no warning in these lines (236-238).
That’s bad, but check it out, that function is calling close_hintfile/1 and we do have warnings for that one, specifically:

 240: Function close_hintfile/1 has no local return
 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _, _, _} tagged with a record name violates the declared type of #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}
 251: Record construction #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} violates the declared type of field hintfd::port()

Ok, so the function has no local return, that explains all! (o_O)
Anyway, let’s check the last 2, shall we?
On line 240, header of the first clause of the function, we’re using undefined in a record field that doesn’t accept undefined. The warning can be improved, but it can be roughly translated in this way:
- Matching of pattern {‘filestate’, …} tagged with a record name… => The pattern with record filestate that you are using…
- violates the declared type of #… => can never match an actual #filestate record because it’s using at least one sub-pattern that doesn’t match the type of the corresponding field. Here is the record definition:…

On line 251, where we’re building a filestate record, the warning is much clearer: Dialyzer says we’re using the wrong value for field hintfd which is defined as port().

So, it’s pretty clear that our intention was to allow hintfd to be undefined.

My next step here would be to fix that (as you did) and run rebar3 dialyzer again.

We started from line 197… but we could’ve started this process from like 219 as well and reached the same conclusion, right?

I’m not sure if this is what you were actually looking for, but maybe it helps anyway.

Happy dialyzing! :)


On Mar 2, 2017, at 06:39, Raghav Karol <[hidden email]> wrote:

I spent a bit of time yesterday on some dialyzer warnings resulting from the singleton type 'undefined' no longer automatically to record fields types OTP-19. What threw me off were several related but hard to decipher warnings. 

Using OTP-19, `rebar3 dialyzer` on this commit https://github.com/basho/bitcask/commit/cd74f59bfe47a39878d7a56a55ce0ae723723677 produces: 

```
src/bitcask.erl
 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 219: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 520: The created fun has no local return
 529: The call bitcask_fileops:close(FD::{_,_,_,_,_,_,_,_,_,_,_}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 717: The call bitcask_fileops:close(Outfile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 722: The call bitcask_fileops:close(TFile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
 774: The call bitcask_fileops:close(F::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
1224: The call bitcask_fileops:close(File::#filestate{mode::'read_write',filename::string(),tstamp::integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::non_neg_integer(),ofs::'undefined' | non_neg_integer(),l_ofs::0,l_hbytes::0,l_hintcrc::0}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
1719: The pattern <_Key, _Value, State, 0, LastErr> can never match the type <_,_,#bc_state{dirname::string(),write_file::'fresh' | 'undefined' | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},write_lock::'undefined' | reference(),read_files::'undefined' | [{_,_,_,_,_,_,_,_,_,_,_}],max_file_size::'undefined' | integer(),opts::'undefined' | [any()],key_transform::'undefined' | fun(),keydir::reference(),read_write_p::'undefined' | integer(),tombstone_version::0 | 2},100,'undefined'>
1870: Function wrap_write_file/1 has no local return
1924: Record construction #filestate{filename::[any()],hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port()

src/bitcask_fileops.erl
 166: Record construction #filestate{mode::'read_only',filename::string() | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},tstamp::integer(),hintfd::'undefined',hintcrc::0,ofs::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port()
 240: Function close_hintfile/1 has no local return
 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _, _, _} tagged with a record name violates the declared type of #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}
 251: Record construction #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} violates the declared type of field hintfd::port()
 314: The pattern 'undefined' can never match the type port()

src/bitcask_merge_delete.erl
 151: Record construction #filestate{hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port()
===> Warnings written to /Users/raghav/github/riak_kv/deps/bitcask/_build/default/19.2.dialyzer_warnings
===> Warnings occured running dialyzer: 17
```

Particularly confusing, are errors like the first one 

```
 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined')
```

I _think_ dialyzer is saying that the call `bitcask_fileops:close_for_writing(WriteFile#filestate{}, ...)` is not possible because of type violations when creating `#filestate{}` lower down the call stack.

Would appreciate if someone could help understand this better and also suggest how one separates noise from real warnings with dialyzer.

Best,
Raghav

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Loading...