ets:delete slow when using complex keys ?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

ets:delete slow when using complex keys ?

Pascal Brisset
The code below shows that ets:delete can take as much as 70 seconds to
free a 60MB ETS table, and that other processes are frozen during that
time (tested on OTP R7B-1 and P8A-20011007, linux, x86, 400MHz).

The delay seems to depend a lot on the complexity of keys (only 4
seconds if we replace PIDs with atoms). Could there be a problem in
free_hash_table()/free_tree_table() ?

My problem is that mnesia_recover.erl builds such tables and deletes
them every few minutes. Can this problem be avoided with some magic
combination of system_flags, process_flags and mnesia parameters ?

-- Pascal Brisset <pascal.brisset> +33141986741 --
----- Cellicium | 73 avenue Carnot | 94230 Cachan | France -----


-module(etsdelete).
-compile(export_all).

start() ->
    spawn(?MODULE, check_realtime, [1.0e20]),
    T = ets:new(tbl, [{keypos, 2}, set, public]),
    io:format("Filling...~n"),
    fill(T, 1000000),
    io:format("Deleting...~n"),
    ets:delete(T),
    io:format("Done~n").

fill(T, 0) -> ok;
fill(T, N) ->
    ets:insert(T, {transient_decision,{tid,N,self()},committed}),
    %% ets:insert(T, {transient_decision,{tid,N,committed},self()}),
    fill(T, N-1).

check_realtime(Time) ->
    receive after 100 -> ok end,
    {MS,S,US} = now(),
    Time1 = (MS*1000000+S)*1000 + US div 1000,
    Delay = Time1 - Time,
    if Delay > 1000 -> io:format("Been sleeping for ~p ms !~n", [Delay]);
       true -> ok
    end,
    check_realtime(Time1).



Reply | Threaded
Open this post in threaded view
|

ets:delete slow when using complex keys ?

Håkan Mattsson-6
On Tue, 9 Oct 2001, Pascal Brisset wrote:

Pascal> My problem is that mnesia_recover.erl builds such tables and deletes
Pascal> them every few minutes. Can this problem be avoided with some magic
Pascal> combination of system_flags, process_flags and mnesia parameters ?

I believe that the main problem is that the contents of ets tables is
scattered all over the memory. Currently deletion of ets tables as
well scanning of ets tables (match/select and friends) requires random
access of lots of memory pages, since records are not clustered per
table. A memory allocation approach with dedicated ets heaps has the
potential of giving a vast performance improvement.

/H?kan





Reply | Threaded
Open this post in threaded view
|

ets:delete slow when using complex keys ?

Pascal Brisset
 > table. A memory allocation approach with dedicated ets heaps has the
 > potential of giving a vast performance improvement.

Thinking about memory allocation strategies...
I just tried P8B-200110007 with "./configure --enable-unified-heap",
and that seems to bring the deletion time from 70s to 1s.

-- Pascal Brisset <pascal.brisset> +33141986741 --
----- Cellicium | 73 avenue Carnot | 94230 Cachan | France -----