How do mailboxes relate to reduction counts?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

How do mailboxes relate to reduction counts?

Sasha Fonseca
I’ve read the following blog post (http://jlouisramblings.blogspot.pt/2013/01/how-erlang-does-scheduling.html), and it has the following excerpt which got me thinking:

Both processes and ports have a "reduction budget" of 2000 reductions. Any operation in the system costs reductions. This includes function calls in loops, calling built-in-functions (BIFs), garbage collecting heaps of that process[n1], storing/reading from ETS, sending messages (The size of the recipients mailbox counts, large mailboxes are more expensive to send to). 

I’m wondering how does the size of a process’s mailbox relates the reduction count? Is it due to each message having to be matched against several message patterns in a case condition, for example? Also how large would the mailbox have to be to noticed any kind of impact in, let’s say, a server process?

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How do mailboxes relate to reduction counts?

Dmytro Lytovchenko
Increased cost to send to larger mailboxes is a sort of simple approach to work balancing. The idea is that if mailbox of some process grows, we should (naively) try and skew scheduling in such a way so it gets more processing time to clean up that mailbox. So we increase the cost of sending to it, and this makes senders schedule out faster and wait for their turn longer (as they get shorter time to run). This in theory might make the "busy" process with the large mailbox get more CPU and fix the problem faster.

Of course this naive approach won't work if the mailbox grows because the receiver ignores new messages. Such system is sick and must be debugged and code must be modified so that extra messages are flushed (dropped). This is artificial feature, as sending is a very cheap operation.

Increased cost of receive is another thing, and indeed it depends on how many instructions are executed to scan and find the right message in selective receive.

2017-03-05 16:11 GMT+01:00 Sasha Fonseca <[hidden email]>:
I’ve read the following blog post (http://jlouisramblings.blogspot.pt/2013/01/how-erlang-does-scheduling.html), and it has the following excerpt which got me thinking:

Both processes and ports have a "reduction budget" of 2000 reductions. Any operation in the system costs reductions. This includes function calls in loops, calling built-in-functions (BIFs), garbage collecting heaps of that process[n1], storing/reading from ETS, sending messages (The size of the recipients mailbox counts, large mailboxes are more expensive to send to). 

I’m wondering how does the size of a process’s mailbox relates the reduction count? Is it due to each message having to be matched against several message patterns in a case condition, for example? Also how large would the mailbox have to be to noticed any kind of impact in, let’s say, a server process?

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions



_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How do mailboxes relate to reduction counts?

Michel Boaventura
In reply to this post by Sasha Fonseca
On Erlang Programming - A Concurrent Approach to Software Development the authors (Francesco Cesarini and Simon Thompson) explain this:

Processes are said to act as bottlenecks when, over time, they are sent messages at a faster rate than they can handle them, resulting in large mailbox queues. How do processes with many messages in their inbox behave? The answer is badly.
First, the process itself, through a selective receive, might match only a specific type of message. If the message is the last one in the mailbox queue, the whole mailbox has to be traversed before this message is successfully matched. This causes a performance penalty that is often visible through high CPU consumption.
Second, processes sending messages to a process with a long message queue are penalized by increasing the number of reductions it costs to send the message. This is an attempt by the runtime system to allow processes with long message queues to catch up by slowing down the processes sending the messages in the first place. The latter bottleneck often manifests itself in a reduction of the overall throughput of the system.
The only way to discover whether there are any bottlenecks is to observe the throughput and message queue buildup when stress-testing the system. Simple remedies to message queue problems can be achieved by optimizing the code and fine-tuning the operating system and VM settings.
Another way to slow down message queue buildups is by suspending the processes generating the messages until they receive an acknowledgment that the message they have sent has been received and handled, effectively creating a synchronous call. Replacing asynchronous calls with synchronous ones will reduce the maximum throughput of the system when running under heavy load, but never as much as the penalty paid when message queues start building up. Where bottlenecks are known to occur, it is safer to reduce the throughput by introducing synchronous calls, and thus guaranteeing a constant throughput of requests in the system with no degradation of service under heavy loads.

So I think the idea behind this is to give the overloaded process a change to keep up with the other ones. 

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions
Loading...