Approximately Correct Notes on Selected Threading Models

I’m not sure when I started thinking about threading this way but I’m fairly sure it has a lot do with too many hours spent thinking about remote procedure calls and stuff. But anyway, these are very useful notions.

So, let’s consider a fairly simple case which illustrates the main kinds of choices you have. The idea here is that you’re going to get an object O from some provider P and then use it. Here are some of the canonical choices in that situation and some their consequences. In the interest of brevity, I’m just going to discuss the top level considerations. It all can get much worse than this.

“Apartment Threaded”

When you ask for an O from P it must allocate the memory for O on your thread. You can then use O but only from your thread. When you return the object to P it again frees the object on your thread. O does not need any locking, it is constrained to be used on one thread. This object must never be given to any other thread that asks for an O but it could be given back to the same thread repeatedly. It’s possible to write P without any locking (assuming the base allocator is free threaded) on this plan.

This situation happens often actually. For instance, most UI objects are apartment threaded — they begin and end their life on one thread and woe unto you if you try to use them from some other thread. They never lock.

“Free Threaded”

When you ask for O from P it can give you any O it has lying around. It may or may not have been allocated on your thread and it may or may not be currently in use by some other thread. O must use locking on all its mutating operations because it is visible to many threads.

When P gives you O, if it’s using reference counting it *must* retain the O before it gives it to you, this is because it is possible that the one and only remaining reference to O could be owned by some other thread which could release it before your thread retains it. There are other strategies for accomplishing this but all of them boil down to ensuring that lifetime is extended at the moment of acquisition.

Because of all this P is going to have some locking going on. And probably when threads release O they go through P rather than release directly. Or else O is in on the game and its finalizer works with P. Or sometimes P hands out proxy objects and those guys are in on the plan.

All of this is quite tricky.

Lots of shared collections and/or caches try to do this business. Mostly they get it wrong. A Garbage Collector is your friend if you are trying to do this stuff.

“Rental”

It’s kind of a blend. This is actually my favorite threading model. P can keep a cache of objects O to hand out. It manages them. Once an O has been given out no other thread is allowed to use that O. That means O doesn’t need any locking. It is being used by exactly one thread, different threads over its lifetime (so it can’t use thread local storage) but only one at a time. When you give O back to P it goes back in the pool and some other thread could get it. Or it could be freed. It could be allocated and freed and used on 3 different threads, or more. But at any given instant exactly one thread owns it.

In this model, P does not have the retention problems associated with free threading because only one thread can possibly have an O. All it has to do is atomically create new objects O or else extract them atomically from a cache. So P needs some elementary locking.

Sometimes database wrappers are rented. Or file wrappers.

Closing Comments

It bears repeating that Free Threaded is hard. And frequently free threading one thing is useless because often you have to atomically do two things. For my money if you think you need Free Threading you probably are doing it wrong…

Interestingly, for the most part, if someone writes a provider P it almost never says what threading model it thinks it is providing. This is a pretty good sign that any threading it thinks it is providing is wrong :D

Also, if you’re getting your O from P, it’s very hard to allow some party other than P to free the O except maybe in apartment threading if O’s are never re-used. It’s otherwise very hard indeed to prevent races between the free operation and the handing one out operation.

If O is itself complicated then this only gets worse. But I did say I would be brief…