Reliable file transfer?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Reliable file transfer?

Lloyd R. Prentice-2
Hello,

I’m evaluating https://www.rsync.net/ as a possible backup/archiving service for Erlang dets and Mnesia data.

rsync.net offers replicated data store on ZFS with frequent snapshots at an attractive price. Data can be transferred via rsync, SFTP, etc.

My thought is to create a data management app that would automate the backup/archive/restore functions. But I don’t understand the data transfer options available in Erlang well enough to implement. 

The options I see are:

— calling rsync via os:cmd/1 — seems simple enough but I’m breaking out of Erlang
— ssh-sftp — just don’t understand how to use this library
— File Transfer Service — docs are totally confusing????

One would think that this is an every day plain vanilla kind of problem. But there’s scant guidance on the web.

Can some kind soul point me in the right direction? 

Many thanks,

LRP





Sent from my iPad

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Reliable file transfer?

Mikael Pettersson-5
On Wed, May 29, 2019 at 9:41 PM Lloyd R. Prentice <[hidden email]> wrote:

>
> Hello,
>
> I’m evaluating https://www.rsync.net/ as a possible backup/archiving service for Erlang dets and Mnesia data.
>
> rsync.net offers replicated data store on ZFS with frequent snapshots at an attractive price. Data can be transferred via rsync, SFTP, etc.
>
> My thought is to create a data management app that would automate the backup/archive/restore functions. But I don’t understand the data transfer options available in Erlang well enough to implement.
>
> The options I see are:
>
> — calling rsync via os:cmd/1 — seems simple enough but I’m breaking out of Erlang
> — ssh-sftp — just don’t understand how to use this library
> — File Transfer Service — docs are totally confusing????
>
> One would think that this is an every day plain vanilla kind of problem. But there’s scant guidance on the web.
>
> Can some kind soul point me in the right direction?

We're using os:cmd and rsync/scp/ssh to transfer our DB backups to
internal backup servers,  rsync in particular because it enables
incremental transfers of our leveldb snapshots.  Stuff is also pushed
to S3, but that's done outside of the main Erlang transaction servers.
(Erlang-based micro-services classify and redistribute event-based
data, while Unix cron jobs transfer plain files.)

I don't think we ever used the other options you mention.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Reliable file transfer?

Lloyd R. Prentice-2
Thanks, Mikael,

That’s reassuring.

Best wishes,

Lloyd

Sent from my iPad

> On May 30, 2019, at 5:12 AM, Mikael Pettersson <[hidden email]> wrote:
>
>> On Wed, May 29, 2019 at 9:41 PM Lloyd R. Prentice <[hidden email]> wrote:
>>
>> Hello,
>>
>> I’m evaluating https://www.rsync.net/ as a possible backup/archiving service for Erlang dets and Mnesia data.
>>
>> rsync.net offers replicated data store on ZFS with frequent snapshots at an attractive price. Data can be transferred via rsync, SFTP, etc.
>>
>> My thought is to create a data management app that would automate the backup/archive/restore functions. But I don’t understand the data transfer options available in Erlang well enough to implement.
>>
>> The options I see are:
>>
>> — calling rsync via os:cmd/1 — seems simple enough but I’m breaking out of Erlang
>> — ssh-sftp — just don’t understand how to use this library
>> — File Transfer Service — docs are totally confusing????
>>
>> One would think that this is an every day plain vanilla kind of problem. But there’s scant guidance on the web.
>>
>> Can some kind soul point me in the right direction?
>
> We're using os:cmd and rsync/scp/ssh to transfer our DB backups to
> internal backup servers,  rsync in particular because it enables
> incremental transfers of our leveldb snapshots.  Stuff is also pushed
> to S3, but that's done outside of the main Erlang transaction servers.
> (Erlang-based micro-services classify and redistribute event-based
> data, while Unix cron jobs transfer plain files.)
>
> I don't think we ever used the other options you mention.

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions