TLS handshake records don't handle fragments?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

TLS handshake records don't handle fragments?

Ryan Stewart
I've been getting handshake_failure alerts when trying to connect to a particular server, and I think I've traced it to the fact that the TLS records aren't being handled correctly with respect to fragments. In particular, this server is sending a rather large "certificate request" to allow for client cert auth, and the list is too long to fit in one TLS record. That's breaking the TLS handshake in at least Erlang 18 and 19, I believe. It's basically a mirror image of the problem described in https://bugs.erlang.org/browse/ERL-83. That issue is with Erlang as the TLS server. I'm seeing the same thing with it being the client. Is this addressed somewhere?

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake records don't handle fragments?

Ingela Andin
Hi!

2018-05-01 19:23 GMT+02:00 Ryan Stewart <[hidden email]>:
I've been getting handshake_failure alerts when trying to connect to a particular server, and I think I've traced it to the fact that the TLS records aren't being handled correctly with respect to fragments. In particular, this server is sending a rather large "certificate request" to allow for client cert auth, and the list is too long to fit in one TLS record. That's breaking the TLS handshake in at least Erlang 18 and 19, I believe. It's basically a mirror image of the problem described in https://bugs.erlang.org/browse/ERL-83. That issue is with Erlang as the TLS server. I'm seeing the same thing with it being the client. Is this addressed somewhere?


Can you give me a possibility to recreate the issue?  That issue you described was fixed in 18 and both the client and the server uses the same code to encode handshakes. The issue was in the encoding and not in the
decoding. Can you tell us more details of how you reached your conclusion?

Regards Ingela Erlang/OTP Team - Ericsson AB



 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake records don't handle fragments?

Ryan Stewart
Hi Ingela! Yeah, let me be more clear. The issue is when Erlang is acting as a client and receives a "certificate request" from a server during the handshake, as described at https://tools.ietf.org/html/rfc5246#section-7.4.4. Since Erlang is receiving the message, it's a decode rather than an encode, but the problem is that it's not handling fragmented messages, similar to the story I linked. I'm able to recreate the issue easily by starting a server that sends certificate requests using the system-installed CAs on a CentOS box, like this:

$ openssl req -x509 -newkey rsa:2048 -keyout private.pem -out public.pem -days 365 -nodes
$ openssl s_server -accept 12345 -cert public.pem -key private.pem -CAfile /etc/pki/tls/certs/ca-bundle.crt -verify 1

Then from Erlang:

1> application:ensure_all_started(ssl).
{ok,[crypto,asn1,public_key,ssl]}
2> ssl:connect("localhost", 12345, []).

=ERROR REPORT==== 2-May-2018::14:30:13 ===
SSL: certify: tls_connection.erl:774:Fatal error: handshake failure
{error,{tls_alert,"handshake failure"}}

There are so many CAs included by default these days, it's enough to trigger this bug. If you add a "-msg" to the s_server, you'll see the (very long) certificate request sent out, followed by a handshake_failure alert received from the client (Erlang). On the off chance you try this on a system without enough CAs, I guess we could just generate a bunch of certs and make our own, big CA file. The certificate request in my test above was 19185 bytes, which is larger than a TLS record is allowed to be, so it forces fragmentation.

On Wed, May 2, 2018 at 5:32 AM Ingela Andin <[hidden email]> wrote:
Hi!

2018-05-01 19:23 GMT+02:00 Ryan Stewart <[hidden email]>:
I've been getting handshake_failure alerts when trying to connect to a particular server, and I think I've traced it to the fact that the TLS records aren't being handled correctly with respect to fragments. In particular, this server is sending a rather large "certificate request" to allow for client cert auth, and the list is too long to fit in one TLS record. That's breaking the TLS handshake in at least Erlang 18 and 19, I believe. It's basically a mirror image of the problem described in https://bugs.erlang.org/browse/ERL-83. That issue is with Erlang as the TLS server. I'm seeing the same thing with it being the client. Is this addressed somewhere?


Can you give me a possibility to recreate the issue?  That issue you described was fixed in 18 and both the client and the server uses the same code to encode handshakes. The issue was in the encoding and not in the
decoding. Can you tell us more details of how you reached your conclusion?

Regards Ingela Erlang/OTP Team - Ericsson AB



 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake records don't handle fragments?

Ingela Andin
Hi!

I have not had time to check, but it sounds like it could be another bug that  I recently fixed. Could you try out the release candidate for OTP 21.

Regards Ingela Erlang/OTP Team - Ericsson AB

2018-05-02 16:38 GMT+02:00 Ryan Stewart <[hidden email]>:
Hi Ingela! Yeah, let me be more clear. The issue is when Erlang is acting as a client and receives a "certificate request" from a server during the handshake, as described at https://tools.ietf.org/html/rfc5246#section-7.4.4. Since Erlang is receiving the message, it's a decode rather than an encode, but the problem is that it's not handling fragmented messages, similar to the story I linked. I'm able to recreate the issue easily by starting a server that sends certificate requests using the system-installed CAs on a CentOS box, like this:

$ openssl req -x509 -newkey rsa:2048 -keyout private.pem -out public.pem -days 365 -nodes
$ openssl s_server -accept 12345 -cert public.pem -key private.pem -CAfile /etc/pki/tls/certs/ca-bundle.crt -verify 1

Then from Erlang:

1> application:ensure_all_started(ssl).
{ok,[crypto,asn1,public_key,ssl]}
2> ssl:connect("localhost", 12345, []).

=ERROR REPORT==== 2-May-2018::14:30:13 ===
SSL: certify: tls_connection.erl:774:Fatal error: handshake failure
{error,{tls_alert,"handshake failure"}}

There are so many CAs included by default these days, it's enough to trigger this bug. If you add a "-msg" to the s_server, you'll see the (very long) certificate request sent out, followed by a handshake_failure alert received from the client (Erlang). On the off chance you try this on a system without enough CAs, I guess we could just generate a bunch of certs and make our own, big CA file. The certificate request in my test above was 19185 bytes, which is larger than a TLS record is allowed to be, so it forces fragmentation.

On Wed, May 2, 2018 at 5:32 AM Ingela Andin <[hidden email]> wrote:
Hi!

2018-05-01 19:23 GMT+02:00 Ryan Stewart <[hidden email]>:
I've been getting handshake_failure alerts when trying to connect to a particular server, and I think I've traced it to the fact that the TLS records aren't being handled correctly with respect to fragments. In particular, this server is sending a rather large "certificate request" to allow for client cert auth, and the list is too long to fit in one TLS record. That's breaking the TLS handshake in at least Erlang 18 and 19, I believe. It's basically a mirror image of the problem described in https://bugs.erlang.org/browse/ERL-83. That issue is with Erlang as the TLS server. I'm seeing the same thing with it being the client. Is this addressed somewhere?


Can you give me a possibility to recreate the issue?  That issue you described was fixed in 18 and both the client and the server uses the same code to encode handshakes. The issue was in the encoding and not in the
decoding. Can you tell us more details of how you reached your conclusion?

Regards Ingela Erlang/OTP Team - Ericsson AB



 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions




_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake records don't handle fragments?

Ryan Stewart
 I actually just checked it in the current release of 20, and it works there. It's just broken in 18 and 19 that I know of.

On Wed, May 2, 2018 at 10:52 AM Ingela Andin <[hidden email]> wrote:
Hi!

I have not had time to check, but it sounds like it could be another bug that  I recently fixed. Could you try out the release candidate for OTP 21.

Regards Ingela Erlang/OTP Team - Ericsson AB

2018-05-02 16:38 GMT+02:00 Ryan Stewart <[hidden email]>:
Hi Ingela! Yeah, let me be more clear. The issue is when Erlang is acting as a client and receives a "certificate request" from a server during the handshake, as described at https://tools.ietf.org/html/rfc5246#section-7.4.4. Since Erlang is receiving the message, it's a decode rather than an encode, but the problem is that it's not handling fragmented messages, similar to the story I linked. I'm able to recreate the issue easily by starting a server that sends certificate requests using the system-installed CAs on a CentOS box, like this:

$ openssl req -x509 -newkey rsa:2048 -keyout private.pem -out public.pem -days 365 -nodes
$ openssl s_server -accept 12345 -cert public.pem -key private.pem -CAfile /etc/pki/tls/certs/ca-bundle.crt -verify 1

Then from Erlang:

1> application:ensure_all_started(ssl).
{ok,[crypto,asn1,public_key,ssl]}
2> ssl:connect("localhost", 12345, []).

=ERROR REPORT==== 2-May-2018::14:30:13 ===
SSL: certify: tls_connection.erl:774:Fatal error: handshake failure
{error,{tls_alert,"handshake failure"}}

There are so many CAs included by default these days, it's enough to trigger this bug. If you add a "-msg" to the s_server, you'll see the (very long) certificate request sent out, followed by a handshake_failure alert received from the client (Erlang). On the off chance you try this on a system without enough CAs, I guess we could just generate a bunch of certs and make our own, big CA file. The certificate request in my test above was 19185 bytes, which is larger than a TLS record is allowed to be, so it forces fragmentation.

On Wed, May 2, 2018 at 5:32 AM Ingela Andin <[hidden email]> wrote:
Hi!

2018-05-01 19:23 GMT+02:00 Ryan Stewart <[hidden email]>:
I've been getting handshake_failure alerts when trying to connect to a particular server, and I think I've traced it to the fact that the TLS records aren't being handled correctly with respect to fragments. In particular, this server is sending a rather large "certificate request" to allow for client cert auth, and the list is too long to fit in one TLS record. That's breaking the TLS handshake in at least Erlang 18 and 19, I believe. It's basically a mirror image of the problem described in https://bugs.erlang.org/browse/ERL-83. That issue is with Erlang as the TLS server. I'm seeing the same thing with it being the client. Is this addressed somewhere?


Can you give me a possibility to recreate the issue?  That issue you described was fixed in 18 and both the client and the server uses the same code to encode handshakes. The issue was in the encoding and not in the
decoding. Can you tell us more details of how you reached your conclusion?

Regards Ingela Erlang/OTP Team - Ericsson AB



 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions




_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake records don't handle fragments?

Ingela Andin
Hi Ryan!


2018-05-02 18:08 GMT+02:00 Ryan Stewart <[hidden email]>:
 I actually just checked it in the current release of 20, and it works there. It's just broken in 18 and 19 that I know of.


Ah, ok the only relevant commit that I can think of that could have fixed it is: 1364c7308e17d43d1a2244e3f2bf11cfec3789ef

Regards Ingela Erlang/OTP team 

 

On Wed, May 2, 2018 at 10:52 AM Ingela Andin <[hidden email]> wrote:
Hi!

I have not had time to check, but it sounds like it could be another bug that  I recently fixed. Could you try out the release candidate for OTP 21.

Regards Ingela Erlang/OTP Team - Ericsson AB

2018-05-02 16:38 GMT+02:00 Ryan Stewart <[hidden email]>:
Hi Ingela! Yeah, let me be more clear. The issue is when Erlang is acting as a client and receives a "certificate request" from a server during the handshake, as described at https://tools.ietf.org/html/rfc5246#section-7.4.4. Since Erlang is receiving the message, it's a decode rather than an encode, but the problem is that it's not handling fragmented messages, similar to the story I linked. I'm able to recreate the issue easily by starting a server that sends certificate requests using the system-installed CAs on a CentOS box, like this:

$ openssl req -x509 -newkey rsa:2048 -keyout private.pem -out public.pem -days 365 -nodes
$ openssl s_server -accept 12345 -cert public.pem -key private.pem -CAfile /etc/pki/tls/certs/ca-bundle.crt -verify 1

Then from Erlang:

1> application:ensure_all_started(ssl).
{ok,[crypto,asn1,public_key,ssl]}
2> ssl:connect("localhost", 12345, []).

=ERROR REPORT==== 2-May-2018::14:30:13 ===
SSL: certify: tls_connection.erl:774:Fatal error: handshake failure
{error,{tls_alert,"handshake failure"}}

There are so many CAs included by default these days, it's enough to trigger this bug. If you add a "-msg" to the s_server, you'll see the (very long) certificate request sent out, followed by a handshake_failure alert received from the client (Erlang). On the off chance you try this on a system without enough CAs, I guess we could just generate a bunch of certs and make our own, big CA file. The certificate request in my test above was 19185 bytes, which is larger than a TLS record is allowed to be, so it forces fragmentation.

On Wed, May 2, 2018 at 5:32 AM Ingela Andin <[hidden email]> wrote:
Hi!

2018-05-01 19:23 GMT+02:00 Ryan Stewart <[hidden email]>:
I've been getting handshake_failure alerts when trying to connect to a particular server, and I think I've traced it to the fact that the TLS records aren't being handled correctly with respect to fragments. In particular, this server is sending a rather large "certificate request" to allow for client cert auth, and the list is too long to fit in one TLS record. That's breaking the TLS handshake in at least Erlang 18 and 19, I believe. It's basically a mirror image of the problem described in https://bugs.erlang.org/browse/ERL-83. That issue is with Erlang as the TLS server. I'm seeing the same thing with it being the client. Is this addressed somewhere?


Can you give me a possibility to recreate the issue?  That issue you described was fixed in 18 and both the client and the server uses the same code to encode handshakes. The issue was in the encoding and not in the
decoding. Can you tell us more details of how you reached your conclusion?

Regards Ingela Erlang/OTP Team - Ericsson AB



 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions





_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: TLS handshake records don't handle fragments?

Ryan Stewart
Yeah, that looks like it would help. When I was tracing the code, this line was returning zero packets because there wasn't enough data:

{Packets, Buf} = tls_handshake:get_tls_handshake(Version,Data,Buf0, Options)
(https://github.com/erlang/otp/commit/1364c7308e17d43d1a2244e3f2bf11cfec3789ef#diff-f053ce3c6b9ffc18f497e4bcbc9661f2R426)

In that commit, it looks like if no packets can be decoded, you go back and look for more data, which is what it wasn't doing before.

On Thu, May 3, 2018 at 1:03 AM Ingela Andin <[hidden email]> wrote:
Hi Ryan!


2018-05-02 18:08 GMT+02:00 Ryan Stewart <[hidden email]>:
 I actually just checked it in the current release of 20, and it works there. It's just broken in 18 and 19 that I know of.


Ah, ok the only relevant commit that I can think of that could have fixed it is: 1364c7308e17d43d1a2244e3f2bf11cfec3789ef

Regards Ingela Erlang/OTP team 

 

On Wed, May 2, 2018 at 10:52 AM Ingela Andin <[hidden email]> wrote:
Hi!

I have not had time to check, but it sounds like it could be another bug that  I recently fixed. Could you try out the release candidate for OTP 21.

Regards Ingela Erlang/OTP Team - Ericsson AB

2018-05-02 16:38 GMT+02:00 Ryan Stewart <[hidden email]>:
Hi Ingela! Yeah, let me be more clear. The issue is when Erlang is acting as a client and receives a "certificate request" from a server during the handshake, as described at https://tools.ietf.org/html/rfc5246#section-7.4.4. Since Erlang is receiving the message, it's a decode rather than an encode, but the problem is that it's not handling fragmented messages, similar to the story I linked. I'm able to recreate the issue easily by starting a server that sends certificate requests using the system-installed CAs on a CentOS box, like this:

$ openssl req -x509 -newkey rsa:2048 -keyout private.pem -out public.pem -days 365 -nodes
$ openssl s_server -accept 12345 -cert public.pem -key private.pem -CAfile /etc/pki/tls/certs/ca-bundle.crt -verify 1

Then from Erlang:

1> application:ensure_all_started(ssl).
{ok,[crypto,asn1,public_key,ssl]}
2> ssl:connect("localhost", 12345, []).

=ERROR REPORT==== 2-May-2018::14:30:13 ===
SSL: certify: tls_connection.erl:774:Fatal error: handshake failure
{error,{tls_alert,"handshake failure"}}

There are so many CAs included by default these days, it's enough to trigger this bug. If you add a "-msg" to the s_server, you'll see the (very long) certificate request sent out, followed by a handshake_failure alert received from the client (Erlang). On the off chance you try this on a system without enough CAs, I guess we could just generate a bunch of certs and make our own, big CA file. The certificate request in my test above was 19185 bytes, which is larger than a TLS record is allowed to be, so it forces fragmentation.

On Wed, May 2, 2018 at 5:32 AM Ingela Andin <[hidden email]> wrote:
Hi!

2018-05-01 19:23 GMT+02:00 Ryan Stewart <[hidden email]>:
I've been getting handshake_failure alerts when trying to connect to a particular server, and I think I've traced it to the fact that the TLS records aren't being handled correctly with respect to fragments. In particular, this server is sending a rather large "certificate request" to allow for client cert auth, and the list is too long to fit in one TLS record. That's breaking the TLS handshake in at least Erlang 18 and 19, I believe. It's basically a mirror image of the problem described in https://bugs.erlang.org/browse/ERL-83. That issue is with Erlang as the TLS server. I'm seeing the same thing with it being the client. Is this addressed somewhere?


Can you give me a possibility to recreate the issue?  That issue you described was fixed in 18 and both the client and the server uses the same code to encode handshakes. The issue was in the encoding and not in the
decoding. Can you tell us more details of how you reached your conclusion?

Regards Ingela Erlang/OTP Team - Ericsson AB



 
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions




_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions