EEP Light: Option to override private key lookup when using Erlang ssh client

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

EEP Light: Option to override private key lookup when using Erlang ssh client

Vipin Nair
EEP Light: Option to override private key lookup when using Erlang ssh client


 * Why do we need this new feature?

   Currently, ssh:connect/3,4 gets the user's private key from either the
   default directory (~/.ssh) of the user or from the directory specified in the
   'user_dir' ssh connect option. The files names are always assumed to be
   openssh default file names, for example, `id_rsa` or `id_dsa`. This makes the
   API inflexible as we are required to create a directory and write the key
   data to the file system when we are connecting to a SSH server as a third
   party. An API to specify the private key is immensely useful when the
   private keys are stored in, say, a database.


 * How did you solve it?

   - Implementation description

       To address the above problem, new ssh connect options `rsa_priv_key` and
       `dsa_priv_key` are proposed. The connect API will look like:

         ssh:connect("localhost", 22, [{user, "[USERNAME]"},
                                       {rsa_priv_key,
<<"[BINARY_KEY_DATA]">>}]).

       If `rsa_priv_key` is specified and RSA is used for ssh negotiation,
       instead of reading the key from the `id_rsa` file, we use the key
       specified above. (This key could be protected by a passphrase and it is
       the system's responsibility to decrypt it.) `dsa_priv_key` will work
       similarly. See below for handling of ECDSA keys and known_hosts file.

       A pull request with a partial implementation of this proposal along with
       tests is on github[1].

   - Why you choose to implement it the way you did?

       The above solution is in the similar vein as some of the existing API's.
       For example, when password authentication is used to connect to a server,
       the ssh connect option `password` is used to specify the password.
       Similarly, for encrypted private keys, the passphrase is specified using
       ssh connect options `rsa_pass_phrase` and `dsa_pass_phrase`.

   - Other possible implementations considered

       Binary(iodata) key data can be passed onto file:open with `ram` mode
       specified which lets us treat an object in memory as a file. We can build
       a solution around this feature.

       SSH connect option only lets specify the user directory using the
       `user_dir` option and *not* the individual files. File names are assumed
       to be openssh defaults. We could change user dir to also accept a
       proplist of individual file names/iodata.

       Current definition ->  user_dir() :: string()

       Proposed definition -> user_dir() :: string() |
                              [{key_file() :: string() | iodata()}]

                              Where

                              key_file() :: id_rsa | id_dsa | id_ecdsa |
                                            known_hosts

       Considering we can pass either the file name or iodata with key contents,
       we'll have to limit file API usage to the ones that work with both of
       them. For example, ssh_file:read_ssh_file/1 uses file:read_file/1 which
       is not compatible with iodata() input.

       I haven't given enough thought to this solution so it is possible that I
       could be missing something obvious. In my opinion, the first solution
       proposed above is cleaner way to solve this problem.


 * Risks or uncertain artifacts?

   - Describe known incompatibilities

       Introducing the new connect options does not introduce an
       incompatibility, is fully backward compatible and does not affect
       consumers of the existing API in any way.

   - Describe uncertainties that are not fully understood or unclear.

       ECDSA keys can be handled by introducing similar options above. The
       proposed options names are `ecdsa_sha2_nistp256_priv_key`,
       `ecdsa_sha2_nistp384_priv_key` and `ecdsa_sha2_nistp521_priv_key`.

       A new option `known_hosts` can be introduced for specifying known_hosts
       as an API. The value corresponding to this key would be a list of known
       server public keys.

</proposal>

Some background: I am implementing a ssh multiplexer[2] and as a consumer of
this API, I wasn't very happy. I thought about what a better API (IMO) could
look like and took a stab at implementing it.

[1]: https://github.com/erlang/otp/pull/884
[2]: https://github.com/swvist/hss


--
Regards,
Vipin
_______________________________________________
erlang-patches mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-patches
Reply | Threaded
Open this post in threaded view
|

Re: EEP Light: Option to override private key lookup when using Erlang ssh client

Hans Nilsson R (AL/EAB)
On 11/08/2015 04:01 PM, Vipin Nair wrote:

> EEP Light: Option to override private key lookup when using Erlang ssh client
>
>
>  * Why do we need this new feature?
>
>    Currently, ssh:connect/3,4 gets the user's private key from either the
>    default directory (~/.ssh) of the user or from the directory specified in the
>    'user_dir' ssh connect option. The files names are always assumed to be
>    openssh default file names, for example, `id_rsa` or `id_dsa`. This makes the
>    API inflexible as we are required to create a directory and write the key
>    data to the file system when we are connecting to a SSH server as a third
>    party. An API to specify the private key is immensely useful when the
>    private keys are stored in, say, a database.

You seem unaware of that both the client and the server get the keys
from callbacks defined by the ssh_client_key_api and the
ssh_server_key_api.  The option key_cb defines the module to use for
ssh:connect or ssh:daemon.

The default callback module is ssh_file, which implements reading and
writing of keys to and from files compatible with OpenSSH.

You can write plugins to fetch keys from a database, fetch from files
with names of your choice or even hard code them using those behaviours.

See
  http://www.erlang.org/doc/man/ssh_client_key_api.html
  http://www.erlang.org/doc/man/ssh_server_key_api.html

-Hans

>
>
>  * How did you solve it?
>
>    - Implementation description
>
>        To address the above problem, new ssh connect options `rsa_priv_key` and
>        `dsa_priv_key` are proposed. The connect API will look like:
>
>          ssh:connect("localhost", 22, [{user, "[USERNAME]"},
>                                        {rsa_priv_key,
> <<"[BINARY_KEY_DATA]">>}]).
>
>        If `rsa_priv_key` is specified and RSA is used for ssh negotiation,
>        instead of reading the key from the `id_rsa` file, we use the key
>        specified above. (This key could be protected by a passphrase and it is
>        the system's responsibility to decrypt it.) `dsa_priv_key` will work
>        similarly. See below for handling of ECDSA keys and known_hosts file.
>
>        A pull request with a partial implementation of this proposal along with
>        tests is on github[1].
>
>    - Why you choose to implement it the way you did?
>
>        The above solution is in the similar vein as some of the existing API's.
>        For example, when password authentication is used to connect to a server,
>        the ssh connect option `password` is used to specify the password.
>        Similarly, for encrypted private keys, the passphrase is specified using
>        ssh connect options `rsa_pass_phrase` and `dsa_pass_phrase`.
>
>    - Other possible implementations considered
>
>        Binary(iodata) key data can be passed onto file:open with `ram` mode
>        specified which lets us treat an object in memory as a file. We can build
>        a solution around this feature.
>
>        SSH connect option only lets specify the user directory using the
>        `user_dir` option and *not* the individual files. File names are assumed
>        to be openssh defaults. We could change user dir to also accept a
>        proplist of individual file names/iodata.
>
>        Current definition ->  user_dir() :: string()
>
>        Proposed definition -> user_dir() :: string() |
>                               [{key_file() :: string() | iodata()}]
>
>                               Where
>
>                               key_file() :: id_rsa | id_dsa | id_ecdsa |
>                                             known_hosts
>
>        Considering we can pass either the file name or iodata with key contents,
>        we'll have to limit file API usage to the ones that work with both of
>        them. For example, ssh_file:read_ssh_file/1 uses file:read_file/1 which
>        is not compatible with iodata() input.
>
>        I haven't given enough thought to this solution so it is possible that I
>        could be missing something obvious. In my opinion, the first solution
>        proposed above is cleaner way to solve this problem.
>
>
>  * Risks or uncertain artifacts?
>
>    - Describe known incompatibilities
>
>        Introducing the new connect options does not introduce an
>        incompatibility, is fully backward compatible and does not affect
>        consumers of the existing API in any way.
>
>    - Describe uncertainties that are not fully understood or unclear.
>
>        ECDSA keys can be handled by introducing similar options above. The
>        proposed options names are `ecdsa_sha2_nistp256_priv_key`,
>        `ecdsa_sha2_nistp384_priv_key` and `ecdsa_sha2_nistp521_priv_key`.
>
>        A new option `known_hosts` can be introduced for specifying known_hosts
>        as an API. The value corresponding to this key would be a list of known
>        server public keys.
>
> </proposal>
>
> Some background: I am implementing a ssh multiplexer[2] and as a consumer of
> this API, I wasn't very happy. I thought about what a better API (IMO) could
> look like and took a stab at implementing it.
>
> [1]: https://github.com/erlang/otp/pull/884
> [2]: https://github.com/swvist/hss
>
>
_______________________________________________
erlang-patches mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-patches
Reply | Threaded
Open this post in threaded view
|

Re: EEP Light: Option to override private key lookup when using Erlang ssh client

Vipin Nair
Hello Hans,

> You seem unaware of that both the client and the server get the keys
> from callbacks defined by the ssh_client_key_api and the
> ssh_server_key_api.  The option key_cb defines the module to use for
> ssh:connect or ssh:daemon.
> [.....]
>
> See
>   http://www.erlang.org/doc/man/ssh_client_key_api.html
>   http://www.erlang.org/doc/man/ssh_server_key_api.html

I am aware of this but it is very limited and will not fit my usecase. Let me
give you an example of my usecase and demonstrate the limitation of the above
API:

I have to connect to hosts `host1` and `host2` as user `foo`. I have the private
key corresponding to each user/host pair in a database. For simplicity sake,
let's assume I can call a named process (PRIV_KEY_SERVER) that will return the
private key data for a particular user/host pair. Something like:

  {ok, PrivKey} = gen_server:call(PRIV_KEY_SERVER, {get_priv_key, User, Host}).

Let's say we have a module(KeyCB below) that implements the ssh_client_key_api.
The definition could look like:

  KeyCB:user_key(Algorithm, ConnectOptions) ->
     {ok, PrivKey} = gen_server:call(PRIV_KEY_SERVER,
                                     {get_priv_key, User, Host}).


This function cannot be implemented because I have no way to get the value of
`Host`. The the ssh:connect/3,4 options that are passed as `ConnectOptions` to
user_key/2 do have that value.

Now:

 * I do not have the host name available and since host names need not be unique
   across machines, I have no way to get the private key corresponding to a user
   *and* a particular host from, say, a database.

 * ssh:connect/3,4 validates the connect options so we cannot just add an extra
   {host, "[HOSTNAME]"} to the connect options and get around the above
   limitation.

 * In my opinion, calling a database (or a named process like in the example
   above) from a module implementing ssh_client_key_api creates a high level of
   coupling that I would want to avoid when possible. Pragmatic way to do this
   would be:

     USER_INPUT -> FETCH_RELEVANT_DATA -> CREATE_SSH_CONN -> POST_CONNECT_STUFF

   This flows helps with error handling better. Let's say the named process dies
   for some reason, I do not wish to handle those kinds of error in my module
   implementing ssh_client_key_api. I want to keep external (Non SSH) IO
   elsewhere.

--
Regards,
Vipin
_______________________________________________
erlang-patches mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-patches
Reply | Threaded
Open this post in threaded view
|

Re: EEP Light: Option to override private key lookup when using Erlang ssh client

Hans Nilsson R (AL/EAB)
Hi Vipin,

As I indicated when I closed your pull request, those options forces the
callback modules to handle options that might be strange for a certain
callback module. For example, the purpose of ssh_file is only to handle
keys in the way of OpenSSH.

You have correctly pointed out that to be able to write a specialized
callback module there is a need to have options passed to callbacks.

A more general way to pass options to the key callback module would be
to introduce a second form of the key_cb option:
  -  keep todays {key_cb, Module::atom()} to keep compatibility
  -  introduce the variant {key_cb, {Module::atom(), ModuleOptions}}

ModuleOptions has the usual list of {Tag,Value} format. That list is
passed in the Options argument of the callback functions as
    {key_cb_private,ModuleOptions}

The old
   {key_cb, Module::atom()}
could be transformed to
   {key_cb, {Module,[]}}
already in ssh:handle_options to make it cleaner inside.

In that way it is possible to write a callback module with whatever is
needed as private options.

-Hans



_______________________________________________
erlang-patches mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-patches

smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: EEP Light: Option to override private key lookup when using Erlang ssh client

Vipin Nair
Hello Hans,

> A more general way to pass options to the key callback module would be
> to introduce a second form of the key_cb option:
>   -  keep todays {key_cb, Module::atom()} to keep compatibility
>   -  introduce the variant {key_cb, {Module::atom(), ModuleOptions}}
> [...]
> In that way it is possible to write a callback module with whatever is
> needed as private options.

This will work! In retrospect, I should have done this instead. This is much
simpler and solves my problem. I'll send a new pull request with the suggested
implementation. Thanks!

--
Regards,
Vipin
_______________________________________________
erlang-patches mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-patches
Reply | Threaded
Open this post in threaded view
|

Re: EEP Light: Option to override private key lookup when using Erlang ssh client

Hans Nilsson R (AL/EAB)
Great!  If there also is documentation and a test case the likelihood of
acceptance is higher :)
-Hans

On 11/12/2015 06:31 PM, Vipin Nair wrote:

> Hello Hans,
>
>> A more general way to pass options to the key callback module would be
>> to introduce a second form of the key_cb option:
>>   -  keep todays {key_cb, Module::atom()} to keep compatibility
>>   -  introduce the variant {key_cb, {Module::atom(), ModuleOptions}}
>> [...]
>> In that way it is possible to write a callback module with whatever is
>> needed as private options.
>
> This will work! In retrospect, I should have done this instead. This is much
> simpler and solves my problem. I'll send a new pull request with the suggested
> implementation. Thanks!
>

_______________________________________________
erlang-patches mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-patches

smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: EEP Light: Option to override private key lookup when using Erlang ssh client

Vipin Nair
> Great!  If there also is documentation and a test case the likelihood of
> acceptance is higher :)

Noted :)

--
Regards,
Vipin
_______________________________________________
erlang-patches mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-patches
Reply | Threaded
Open this post in threaded view
|

Re: EEP Light: Option to override private key lookup when using Erlang ssh client

Irina Guberman
In reply to this post by Hans Nilsson R (AL/EAB)
Hello dear Erlang developers,

I would tremendously appreciate any help or if anyone can point me to someone who can help with linking Erlang/OTP.   I created an experimental BIF in my own fork of erlang/otp, but can't get past linking stage (on Mac: Undefined symbols for architecture x86_64,  but same thing happens on ubuntu 14.04)  even though I checked everywhere and my BIF seems to be everywhere an existing BIF from the same module is (with same naming  conventions and all).   I don't know where to post this question either (Erlang Central didn't work which is understandable, it's not like people are creating BIFs everyday), I'm so desperate I'm about to post it on Twitter ;)  

Thanks a million for reading this message and even more so for pointing me to where I can get help with this!

Irina.




On Thu, Nov 12, 2015 at 12:19 PM, Hans Nilsson R <[hidden email]> wrote:
Great!  If there also is documentation and a test case the likelihood of
acceptance is higher :)
-Hans

On 11/12/2015 06:31 PM, Vipin Nair wrote:
> Hello Hans,
>
>> A more general way to pass options to the key callback module would be
>> to introduce a second form of the key_cb option:
>>   -  keep todays {key_cb, Module::atom()} to keep compatibility
>>   -  introduce the variant {key_cb, {Module::atom(), ModuleOptions}}
>> [...]
>> In that way it is possible to write a callback module with whatever is
>> needed as private options.
>
> This will work! In retrospect, I should have done this instead. This is much
> simpler and solves my problem. I'll send a new pull request with the suggested
> implementation. Thanks!
>


_______________________________________________
erlang-patches mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-patches



_______________________________________________
erlang-patches mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-patches
Reply | Threaded
Open this post in threaded view
|

Re: EEP Light: Option to override private key lookup when using Erlang ssh client

Hans Nilsson R (AL/EAB)
I would recommend the list [hidden email] for your
question, see

   http://erlang.org/mailman/listinfo/erlang-questions

-Hans

On 11/12/2015 08:01 PM, Irina Guberman wrote:

> Hello dear Erlang developers,
>
> I would tremendously appreciate any help or if anyone can point me to
> someone who can help with linking Erlang/OTP.   I created an
> experimental BIF in my own fork of erlang/otp, but can't get past
> linking stage (on Mac: Undefined symbols for architecture x86_64,  but
> same thing happens on ubuntu 14.04)  even though I checked everywhere
> and my BIF seems to be everywhere an existing BIF from the same module
> is (with same naming  conventions and all).   I don't know where to post
> this question either (Erlang Central didn't work which is
> understandable, it's not like people are creating BIFs everyday), I'm so
> desperate I'm about to post it on Twitter ;)  
>
> Thanks a million for reading this message and even more so for pointing
> me to where I can get help with this!
>
> Irina.
>
>
>
>
> On Thu, Nov 12, 2015 at 12:19 PM, Hans Nilsson R
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Great!  If there also is documentation and a test case the likelihood of
>     acceptance is higher :)
>     -Hans
>
>     On 11/12/2015 06:31 PM, Vipin Nair wrote:
>     > Hello Hans,
>     >
>     >> A more general way to pass options to the key callback module
>     would be
>     >> to introduce a second form of the key_cb option:
>     >>   -  keep todays {key_cb, Module::atom()} to keep compatibility
>     >>   -  introduce the variant {key_cb, {Module::atom(), ModuleOptions}}
>     >> [...]
>     >> In that way it is possible to write a callback module with
>     whatever is
>     >> needed as private options.
>     >
>     > This will work! In retrospect, I should have done this instead.
>     This is much
>     > simpler and solves my problem. I'll send a new pull request with
>     the suggested
>     > implementation. Thanks!
>     >
>
>
>     _______________________________________________
>     erlang-patches mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://erlang.org/mailman/listinfo/erlang-patches
>
>

_______________________________________________
erlang-patches mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-patches

smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: EEP Light: Option to override private key lookup when using Erlang ssh client

Vipin Nair
In reply to this post by Hans Nilsson R (AL/EAB)
> Great!  If there also is documentation and a test case the likelihood of
> acceptance is higher :)

Hans, I have made the suggested changes and sent a pull request[1].
Surprisingly, there were no tests for key callback. I have added tests for
callback module as well as callback module options. Please review.

[1]: https://github.com/erlang/otp/pull/892

--
Regards,
Vipin
_______________________________________________
erlang-patches mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-patches