Right now, the distribution protocol gets to play with toys that the rest of us don’t: it can make sequential
I propose moving this logic to a more Erlang-userland accessible level. Rather than atom caches being something that happen internally in
Users could create new atom caches through a BIF (e.g.
The distribution protocol could then be reimplemented backwards-compatibly in terms of these new BIFs, where the atom cache update in each data message's header would simply be the return value of the "describe atom cache update" BIF.
I've outlined below a few potential API designs for this idea. I'd like to get feedback what people like/dislike about them. If there's clear community consensus that one of these ideas is good, I'll move forward with either a patch or an EEP (I personally feel this isn't the kind of big-deal change that warrants an EEP, but correct me if I'm wrong.)
1 - Erlang-exposed atom cache
Using the hypothetical BIF
2 - Exposed atom cache, but opaque transaction handles
Design #1 assumes you want functional-data-structure semantics for atom caches the whole way through the encoding/decoding process, which can be slow, and which really makes little sense given that there is no real use for the intermediate atom-cache representations.
An alternate design would make the intermediate atom-cache representations opaque—you’d have a handle into an atom-cache “transaction” (a piece of opaque ERTS-mutable data in the owner process’s heap, like a NIF handle), and committing the transaction would destroy this handle and return a regular Erlang-visible representation of the updated atom-cache and a diff from the start of the transaction:
This still introduces a
3 - Opaque atom cache (with or without transactions)
This overhead savings is meaningful, but potentially small if the client is using
Further savings could be made if, rather than the atom cache being exposed as a term between transactions, the atom cache itself was an opaque handle for its entire lifetime. For example, an atom cache could exist as an ETS table using the special table-type
This would handily provide a framework for shared-access concurrency of atom caches—concurrent decoder processes and even concurrent encoder processes could be backed by a shared atom cache, allowing terms from the same port, or destined for the same port, to be encoded/decoded concurrently, without generating redundant atom-cache-update binaries in the encoded stream.
Transactions, if still desirable, would still need to exist as separate opaque state-tracking objects, in order for the transaction-commits to produce valid atom-cache diffs corresponding to the terms that have been encoded.
But, if updates to the
If, under an atomic-updates design,
erlang-questions mailing list
|Free forum by Nabble||Edit this page|