ERL-823: SSL cipher_suites too limited when compiling with OPENSSL_NO_EC=1

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: ERL-823: SSL cipher_suites too limited when compiling with OPENSSL_NO_EC=1

Hans Nilsson R (AL/EAB)
Thanks Per!  I'll cancel the PR.
/Hans

On 1/8/19 1:16 PM, Per Hedeland wrote:

> I fixed the VirtualBox problem (again:-(), and could thus do some
> testing on CentOS 7.6 with headers installed. Using OTP 21.2 and
> Nicholas' VerifyCurve function - first a "normal" OpenSSL build,
> i.e. straight from openssl.org source, no modifications or
> algorithm-related config options:
>
> $ openssl version
> OpenSSL 1.0.2k  26 Jan 2017
> $ openssl ecparam -list_curves | grep ':' | wc -l
>       81
>
> 2> length(ssl:eccs()).
> 28
> 3> length([C || C <- ssl:eccs(), VerifyCurve(C) == ok]).
> 28
> 4> length(crypto:ec_curves()).
> 83
> 5> length([C || C <- crypto:ec_curves(), VerifyCurve(C) == ok]).
> 81
>
> (I believe the diff here, due to
>  {ipsec3,{error,badkey}},
>  {ipsec4,{error,badkey}}]
> may be caused by improper usage rather than lack of support. Those
> curves are also not listed by 'ecparam -list_curves'.)
>
> - and then CentOS:
>
> $ openssl version
> OpenSSL 1.0.2k-fips  26 Jan 2017
> $ openssl ecparam -list_curves | grep ':' | wc -l
> 4
>
> 2> length(ssl:eccs()).
> 14
> 3> length([C || C <- ssl:eccs(), VerifyCurve(C) == ok]).
> 7
> 4> length(crypto:ec_curves()).
> 41
> 5> length([C || C <- crypto:ec_curves(), VerifyCurve(C) == ok]).
> 13
>
> And, the installed openssl/obj_mac.h header file is identical for the
> two, and has all the NID_xxx #defines that your PR adds #ifdefs for
> - thus I'm afraid it makes no difference at all for *this* problem
> (yes, I did test it:-).
>
> --Per
>
> On 2019-01-07 23:49, Per Hedeland wrote:
>> On 2019-01-07 18:36, Hans Nilsson R wrote:
>>> Hi,
>>>
>>> I've made a PR with #ifdefs for the EC names in crypto.c:
>>>       https://github.com/erlang/otp/pull/2085
>>>
>>> The C-function algorithms() uses this for the 'curves' value in crypto:supports/0 which both SSL and SSH is supposed to use.
>>>
>>> Comments?
>>
>> Hm, is the idea that those NID_xxx #defines in the OpenSSL headers
>> will reflect what the library supports? I don't think that is the
>> case, but I guess Nicholas can check if his CentOS systems have a
>> different set of #defines (in /usr/include/openssl/obj_mac.h) from a
>> what a "normal" build of the same OpenSSL version has. (I can atm only
>> check a CentOS "live-cd", which doesn't have the headers - for some
>> reason my VirtualBox install always hangs...)
>>
>> --Per
>>
>>> /Hans
>>>
>>> On 1/3/19 6:53 PM, Nicholas Lundgaard wrote:
>>>> Hi,
>>>>
>>>> I wanted to call ERL-823 (https://bugs.erlang.org/browse/ERL-823) to this list's attention. My company operates Erlang microservices in AWS on a kerl-built OTP installation on Amazon Linux
>>>> (RedHat/CentOS-based), and we've encountered a serious challenge to upgrading to OTP 21: When you disable OpenSSL EC ciphers during an OTP build, which is necessary to build an OTP installation
>>>> that doesn't erroneously think it has a bunch of EC ciphers that aren't built into the underlying OpenSSL, you're no longer able to connect to google.com via https (not to mention many, many other
>>>> web properties, like much of AWS infrastructure).
>>>>
>>>> It confuses me that there is not a simpler way to align the Erlang crypto/ssl cipher support with the underlying openssl installation it's linked to, but that notwithstanding, It would be really
>>>> helpful to have a flag to build OTP with support for RedHat/Fedora's EC cipher subset, or something similar to this.
>>>>
>>>> Thanks,
>>>> Nicholas Lundgaard
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> [hidden email]
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> [hidden email]
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> [hidden email]
>> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: ERL-823: SSL cipher_suites too limited when compiling with OPENSSL_NO_EC=1

Nicholas Lundgaard
In reply to this post by Per Hedeland
It seems that the remaining issue here, after removing the unneeded  OPENSSL_NO_EC=1 in my Erlang/OTP build, has turned out to be a quirk of either OpenSSL with FIPS support in general, or RedHat/CentOS/Fedora's OpenSSL with FIPS support (even without FIPS enabled). It seems that the library categorically enforces a minimum of 256 bits for EC curves, and it rejects the definition of curves smaller than this, resulting in the run-time errors I have observed. I have opened up ERL-825 to cover this very distinct issue, and I wanted to post a final note on this thread to anyone interested in following along:

https://bugs.erlang.org/browse/ERL-825

I'm not sure this behavior is really worth fixing, but hopefully that issue will track whatever final resolution is reached.

I've closed https://bugs.erlang.org/browse/ERL-823.

Thanks & Regards,

—Nicholas Lundgaard

On Sat, Jan 5, 2019, at 3:39 PM, Per Hedeland wrote:

> On 2019-01-05 16:03, Nicholas Lundgaard wrote:
> > "Well, I'm not really an expert on this stuff"
> >
> > That makes two of us, at least... :)
>
> I was very impressed by your VerifyCurve function though.:-)
>
> > You're exactly right, when I compile OTP 21 on CentOS with or without OPENSSL_NO_EC2M, in either case, the built OTP correctly leaves ec_gf2m out of the public keys and filters out the GF2m named EC curves that aren't supported in the CentOS/RHEL OpenSSL.
> >
> > The remaining problem on RHEL/CentOS systems is that (for whatever reason) these systems lack support for a bunch of non-GF2m curves that Erlang/OTP's crypto/ssl modules claim to support, but emit runtime exceptions when you try to use them. It seems that is because they are explicitly hard-coded into the C headers in the OTP applications, when that information could be extracted or extrapolated from the linked SSL library itself (either dynamically at runtime or during compilation).
>
> Traditionally the build of the crypto.c NIF module has been based on
> the #defines in the *OpenSSL* headers, in particular
> openssl/opensslconf.h, which is supposed to describe how the OpenSSL
> libraries were built - this is where both OPENSSL_NO_EC and
> OPENSSL_NO_EC2M originates (i.e. from using the 'no-ec' and 'no-ec2m'
> options, respectively, in the OpenSSL configure/build).
>
> I don't think there are any more specific #defines for the individual
> ECC curves though - and in fact there doesn't seem to be any options
> for that in the OpenSSL build either. But in any case, using only the
> OpenSSL headers at build time, while linking dynamically with
> libcrypto, means that the libcrypto found at run time may not match
> the build, and runtime checks are motivated. And I believe both OTP
> crypto and OTP ssl do some such checks, but maybe they aren't 100%.
>
> > For clarity, I think this situation with these missing EC curves on RHEL/CentOS has been like this for many years with almost no change, so upgrading to a newer version of RHEL/CentOS/Amazon Linux isn't going to help. Here is the output from  'openssl ecparam -list_curves' on the latest (2018.03) Amazon Linux AMI running "OpenSSL 1.0.2k-fips  26 Jan 2017":
> >
> >   openssl ecparam -list_curves
> >    secp256k1 : SECG curve over a 256 bit prime field
> >    secp384r1 : NIST/SECG curve over a 384 bit prime field
> >    secp521r1 : NIST/SECG curve over a 521 bit prime field
> >    prime256v1: X9.62/SECG curve over a 256 bit prime field
> >
> > And here is the output from  'openssl ecparam -list_curves' on a CentOS 6.9 box running "OpenSSL 1.0.1e-fips 11 Feb 2013".  The older OpenSSL library omits the secp256k1 curve, but again, pretty similar output:
> >
> > openssl ecparam -list_curves
> >    secp384r1 : NIST/SECG curve over a 384 bit prime field
> >    secp521r1 : NIST/SECG curve over a 521 bit prime field
> >    prime256v1: X9.62/SECG curve over a 256 bit prime field
>
> I also tried the 'ecparam -list_curves' on various systems, and was
> pretty surprised by the result... As you can see, the list you get on
> CentOS is way shorter than the list of successful invocations of your
> VerifyCurve funcion - and with a "normal" OpenSSL-1.0.1 build, I get
> this:
>
>    secp112r1 : SECG/WTLS curve over a 112 bit prime field
>    secp112r2 : SECG curve over a 112 bit prime field
>    secp128r1 : SECG curve over a 128 bit prime field
>    secp128r2 : SECG curve over a 128 bit prime field
>    secp160k1 : SECG curve over a 160 bit prime field
>    secp160r1 : SECG curve over a 160 bit prime field
>    secp160r2 : SECG/WTLS curve over a 160 bit prime field
>    secp192k1 : SECG curve over a 192 bit prime field
>    secp224k1 : SECG curve over a 224 bit prime field
>    secp224r1 : NIST/SECG curve over a 224 bit prime field
>    secp256k1 : SECG curve over a 256 bit prime field
>    secp384r1 : NIST/SECG curve over a 384 bit prime field
>    secp521r1 : NIST/SECG curve over a 521 bit prime field
>    prime192v1: NIST/X9.62/SECG curve over a 192 bit prime field
>    prime192v2: X9.62 curve over a 192 bit prime field
>    prime192v3: X9.62 curve over a 192 bit prime field
>    prime239v1: X9.62 curve over a 239 bit prime field
>    prime239v2: X9.62 curve over a 239 bit prime field
>    prime239v3: X9.62 curve over a 239 bit prime field
>    prime256v1: X9.62/SECG curve over a 256 bit prime field
>    sect113r1 : SECG curve over a 113 bit binary field
>    sect113r2 : SECG curve over a 113 bit binary field
>    sect131r1 : SECG/WTLS curve over a 131 bit binary field
>    sect131r2 : SECG curve over a 131 bit binary field
>    sect163k1 : NIST/SECG/WTLS curve over a 163 bit binary field
>    sect163r1 : SECG curve over a 163 bit binary field
>    sect163r2 : NIST/SECG curve over a 163 bit binary field
>    sect193r1 : SECG curve over a 193 bit binary field
>    sect193r2 : SECG curve over a 193 bit binary field
>    sect233k1 : NIST/SECG/WTLS curve over a 233 bit binary field
>    sect233r1 : NIST/SECG/WTLS curve over a 233 bit binary field
>    sect239k1 : SECG curve over a 239 bit binary field
>    sect283k1 : NIST/SECG curve over a 283 bit binary field
>    sect283r1 : NIST/SECG curve over a 283 bit binary field
>    sect409k1 : NIST/SECG curve over a 409 bit binary field
>    sect409r1 : NIST/SECG curve over a 409 bit binary field
>    sect571k1 : NIST/SECG curve over a 571 bit binary field
>    sect571r1 : NIST/SECG curve over a 571 bit binary field
>    c2pnb163v1: X9.62 curve over a 163 bit binary field
>    c2pnb163v2: X9.62 curve over a 163 bit binary field
>    c2pnb163v3: X9.62 curve over a 163 bit binary field
>    c2pnb176v1: X9.62 curve over a 176 bit binary field
>    c2tnb191v1: X9.62 curve over a 191 bit binary field
>    c2tnb191v2: X9.62 curve over a 191 bit binary field
>    c2tnb191v3: X9.62 curve over a 191 bit binary field
>    c2pnb208w1: X9.62 curve over a 208 bit binary field
>    c2tnb239v1: X9.62 curve over a 239 bit binary field
>    c2tnb239v2: X9.62 curve over a 239 bit binary field
>    c2tnb239v3: X9.62 curve over a 239 bit binary field
>    c2pnb272w1: X9.62 curve over a 272 bit binary field
>    c2pnb304w1: X9.62 curve over a 304 bit binary field
>    c2tnb359v1: X9.62 curve over a 359 bit binary field
>    c2pnb368w1: X9.62 curve over a 368 bit binary field
>    c2tnb431r1: X9.62 curve over a 431 bit binary field
>    wap-wsg-idm-ecid-wtls1: WTLS curve over a 113 bit binary field
>    wap-wsg-idm-ecid-wtls3: NIST/SECG/WTLS curve over a 163 bit binary field
>    wap-wsg-idm-ecid-wtls4: SECG curve over a 113 bit binary field
>    wap-wsg-idm-ecid-wtls5: X9.62 curve over a 163 bit binary field
>    wap-wsg-idm-ecid-wtls6: SECG/WTLS curve over a 112 bit prime field
>    wap-wsg-idm-ecid-wtls7: SECG/WTLS curve over a 160 bit prime field
>    wap-wsg-idm-ecid-wtls8: WTLS curve over a 112 bit prime field
>    wap-wsg-idm-ecid-wtls9: WTLS curve over a 160 bit prime field
>    wap-wsg-idm-ecid-wtls10: NIST/SECG/WTLS curve over a 233 bit binary field
>    wap-wsg-idm-ecid-wtls11: NIST/SECG/WTLS curve over a 233 bit binary field
>    wap-wsg-idm-ecid-wtls12: WTLS curvs over a 224 bit prime field
>    Oakley-EC2N-3:
>          IPSec/IKE/Oakley curve #3 over a 155 bit binary field.
>          Not suitable for ECDSA.
>          Questionable extension field!
>    Oakley-EC2N-4:
>          IPSec/IKE/Oakley curve #4 over a 185 bit binary field.
>          Not suitable for ECDSA.
>          Questionable extension field!
>
> I haven't done a strict comparison of this output with the results of
> running your VerifyCurve function on the same system, but my
> impression is that all curves that were successful in the latter are
> also included in the former.
>
> I really wonder how the CentOS libcrypto was built. The "-fips" in the
> version might be a clue, but I believe this only means that it was
> built in such a way that "FIPS mode" is *available*, and thus it
> shouldn't affect something like the set of "supported" ECC curves.
> (If "FIPS mode" was *enforced*, it wouldn't work at all before OTP 20
> I believe, and this is definitely not the case.)
>
> Anyway, 'ecparam -list_curves' uses the OpenSSL function
> EC_get_builtin_curves(), which is documented in the OpenSSL man page
> with, among other info:
>
>      Whilst the library can be used to create any curve using the functions
>      described above, there are also a number of predefined curves that are
>      available. In order to obtain a list of all of the predefined curves,
>      call the function EC_get_builtin_curves.
>
> So I guess this isn't really the set of *supported* curves, and the
> OpenSSL ecparam(1) man page does say
>
>     -list_curves
>         If this options is specified ecparam will print out a list of all
>         currently implemented EC parameters names and exit.
>                   ^^^^^^^^^^^
>
> - and there may already be some magic going on in the OTP 'crypto' app
> to create additional curves "on demand". But by now I'm way out of
> whatever expertise I may have...
>
> --Per
>
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
12