Does anyone have experience with LETS?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Does anyone have experience with LETS?

Charles Hixson-2
https://github.com/norton/lets

If this is good, it may solve some problems, but I'm quite new to
Erlang, and won't be able to judge how good it is.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone have experience with LETS?

dmkolesnikov
I’ve decided to use eleveldb directly. 

- Dmitry

On 13 Nov 2014, at 03:32, Charles Hixson <[hidden email]> wrote:

https://github.com/norton/lets

If this is good, it may solve some problems, but I'm quite new to Erlang, and won't be able to judge how good it is.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone have experience with LETS?

Charles Hixson-2
Riak and eleveldb both look very interesting as solutions to the memory limitation problems.  It's clearly going to take a bit of study, but thanks very much.

On 11/13/2014 05:53 AM, Dmitry Kolesnikov wrote:
I’ve decided to use eleveldb directly. 

- Dmitry

On 13 Nov 2014, at 03:32, Charles Hixson <[hidden email]> wrote:

https://github.com/norton/lets

If this is good, it may solve some problems, but I'm quite new to Erlang, and won't be able to judge how good it is.
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone have experience with LETS?

Ulf Wiger-6
In reply to this post by Charles Hixson-2

Basho has had to work very hard to get around some nasty blocking issues with leveldb.

We used to experience those issues in eleveldb before, but the 2.0 version works very well.

My guess is that LETS has not addressed these issues, at least to the level that Basho has.

For an idea of the problem, watch this presentation:
https://www.youtube.com/watch?v=vo88IdglU_8

BR,
Ulf

On 13 Nov 2014, at 02:32, Charles Hixson <[hidden email]> wrote:

> https://github.com/norton/lets
>
> If this is good, it may solve some problems, but I'm quite new to Erlang, and won't be able to judge how good it is.
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
http://feuerlabs.com



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone have experience with LETS?

Scott Lystig Fritchie
Ulf Wiger <[hidden email]> wrote:

uw> Basho has had to work very hard to get around some nasty blocking
uw> issues with leveldb.

Indeed, there's been a tremendous amount of work on LevelDB ... so much
so that Basho's fork of LevelDB[1] (which is separate from the Erlang
binding[2]) is really a separate & independent fork of Google's original
version.  Matthew has made a lot of changes to make LevelDB useful in
Riak's concurrent environment, which uses *many* LevelDB instances
simultaneously.  The Google folks aren't interested in that kind of use
case.

Matthew still has a good working relationship with the parent project,
AFAIK, so they're still accepting patches as we find and fix bugs.
Basho has a strong interest in continuing development & performance
enhancements.  That Basho fork will remain active for the indefinite
future.

-Scott

[1] https://github.com/basho/leveldb
[2] https://github.com/basho/eleveldb
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone have experience with LETS?

Ulf Wiger-6

Scott,

I can testify that it doesn't take that many instances for leveldb till cause problems. Before upgrading to eleveldb 2.0, we experienced some lethal blocking* conditions using only one instance via kvdb. With 2.0, we've seen none of that.

BR,
Ulf W

* Mainly when running on EC2, where disk access is decidedly slower than with the SSD in your average laptop.



---- Scott Lystig Fritchie skrev ----

Ulf Wiger <[hidden email]> wrote:

uw> Basho has had to work very hard to get around some nasty blocking
uw> issues with leveldb.

Indeed, there's been a tremendous amount of work on LevelDB ... so much
so that Basho's fork of LevelDB[1] (which is separate from the Erlang
binding[2]) is really a separate & independent fork of Google's original
version.  Matthew has made a lot of changes to make LevelDB useful in
Riak's concurrent environment, which uses *many* LevelDB instances
simultaneously.  The Google folks aren't interested in that kind of use
case.

Matthew still has a good working relationship with the parent project,
AFAIK, so they're still accepting patches as we find and fix bugs.
Basho has a strong interest in continuing development & performance
enhancements.  That Basho fork will remain active for the indefinite
future.

-Scott

[1] https://github.com/basho/leveldb
[2] https://github.com/basho/eleveldb


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone have experience with LETS?

Scott Lystig Fritchie
Ulf, I discovered that we have been in good company.  According to the
speaker at this RICON talk a few weeks ago, both Yahoo and Facebook
discovered the exact same things.  :-|

https://www.youtube.com/watch?v=8cYp52oeDx4&list=PL9Jh2HsAWHxLco7V1SjU9hUzP53CBZOYO&index=20

-Scott
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone have experience with LETS?

Michael Gebetsroither-3
On 2014-11-17 04:51, Scott Lystig Fritchie wrote:
> Ulf, I discovered that we have been in good company.  According to the
> speaker at this RICON talk a few weeks ago, both Yahoo and Facebook
> discovered the exact same things.  :-|
>
> https://www.youtube.com/watch?v=8cYp52oeDx4&list=PL9Jh2HsAWHxLco7V1SjU9hUzP53CBZOYO&index=20

And they created rocksdb based on levelDB which is fit for such server workloads.

http://rocksdb.org/


Michael Gebetsroither
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone have experience with LETS?

Gokhan Boranalp-2
RocksDB is an embeddable persistent key-value store for fast storage.
RocksDB can also be the foundation for a client-server database but
our current focus is on embedded workloads.

*What is RocksDB suitable for?*

RocksDB can be used by applications that need low latency database
accesses. A user-facing application that stores the viewing history
and state of users of a website can potentially store this content on
RocksDB.
A spam detection application that needs fast access to big data sets
can use RocksDB. A graph-search query that needs to scan a data set in
realtime can use RocksDB. RocksDB can be used to cache data from
Hadoop, thereby allowing applications to query Hadoop data in
realtime. A message-queue that supports a high number of inserts and
deletes can use RocksDB.

On Mon, Nov 17, 2014 at 2:53 PM, Michael Gebetsroither <[hidden email]> wrote:

> On 2014-11-17 04:51, Scott Lystig Fritchie wrote:
>>
>> Ulf, I discovered that we have been in good company.  According to the
>> speaker at this RICON talk a few weeks ago, both Yahoo and Facebook
>> discovered the exact same things.  :-|
>>
>>
>> https://www.youtube.com/watch?v=8cYp52oeDx4&list=PL9Jh2HsAWHxLco7V1SjU9hUzP53CBZOYO&index=20
>
>
> And they created rocksdb based on levelDB which is fit for such server
> workloads.
>
> http://rocksdb.org/
>
>
> Michael Gebetsroither
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions



--
BR,
\|/ Kunthar
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone have experience with LETS?

t ty
I have a very basic Erlang binding for it. 


On Mon, Nov 17, 2014 at 9:03 PM, Gokhan Boranalp <[hidden email]> wrote:
RocksDB is an embeddable persistent key-value store for fast storage.
RocksDB can also be the foundation for a client-server database but
our current focus is on embedded workloads.

*What is RocksDB suitable for?*

RocksDB can be used by applications that need low latency database
accesses. A user-facing application that stores the viewing history
and state of users of a website can potentially store this content on
RocksDB.
A spam detection application that needs fast access to big data sets
can use RocksDB. A graph-search query that needs to scan a data set in
realtime can use RocksDB. RocksDB can be used to cache data from
Hadoop, thereby allowing applications to query Hadoop data in
realtime. A message-queue that supports a high number of inserts and
deletes can use RocksDB.

On Mon, Nov 17, 2014 at 2:53 PM, Michael Gebetsroither <[hidden email]> wrote:
> On 2014-11-17 04:51, Scott Lystig Fritchie wrote:
>>
>> Ulf, I discovered that we have been in good company.  According to the
>> speaker at this RICON talk a few weeks ago, both Yahoo and Facebook
>> discovered the exact same things.  :-|
>>
>>
>> https://www.youtube.com/watch?v=8cYp52oeDx4&list=PL9Jh2HsAWHxLco7V1SjU9hUzP53CBZOYO&index=20
>
>
> And they created rocksdb based on levelDB which is fit for such server
> workloads.
>
> http://rocksdb.org/
>
>
> Michael Gebetsroither
> _______________________________________________
> erlang-questions mailing list
> [hidden email]
> http://erlang.org/mailman/listinfo/erlang-questions



--
BR,
\|/ Kunthar
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions


_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions