Multithreading can be used to dramatically speed up the performance of your application, but no speedup is free. Managing parallel threads requires careful programming. Without the proper precautions, you can encounter racing conditions, deadlocks, and even crashes.
What Makes Multithreading Difficult?
Unless you tell your program otherwise, all code on the “main thread”
However, it’s not that easy to just turn on multithreading. Only certain things (such as loops) can be properly multithreaded and there are many considerations to consider.
The first and foremost problem is Racing conditions. These often occur during writes, when a thread changes a resource that is shared by multiple threads. This leads to behavior where the output of the program depends on which thread terminates or changes something first, which can lead to random and unexpected behavior.
These can be very, very simple – for example, you may need to keep a running count of something between loops. The most obvious way to do this is to create a variable and increment it. However, this is not thread safe.
This race condition occurs because it’s not just about adding an abstract one to the variable. The CPU loads the value of
number insert into the register, add a value to that value, and then save the result as the new value of the variable. It is not known that in the meantime any other thread tried to do the exact same thing and soon loaded an incorrect value of
number. The two threads are in conflict and at the end of the loop
number cannot be equal to 100.
.NET has a feature that lets you manage this: the
lock Keyword. This does not prevent changes from being made in-place, but it does help manage concurrency by allowing only one thread to get the lock at a time. If another thread tries to enter a lock statement while another thread is processing, it will wait up to 300 ms before continuing.
You can only lock reference types. Therefore, in a general pattern, a lock object is created beforehand and used as a replacement for locking the value type.
However, you may find that there is now another problem: Deadlocks. This code is a worst case example, but here it is almost exactly the same as normal code
for Loop (actually a little slower as additional threads and locks add additional overhead). Each thread tries to get the lock, but only one can have the lock at a time, so only one thread can execute code within the lock at a time. In this case, that’s all of the code in the loop. So the lock statement removes all the benefits of threading and just slows everything down.
In general, when you need to do writes, you want to lock down as needed. However, you should consider parallelism when choosing what to lock, as reads are not always thread safe, either. If another thread is writing to the object, reading from another thread can result in an incorrect value or a specific condition to return an incorrect result.
Fortunately, there are a few tricks to get this right that allow you to balance the speed of multithreading while using locks to avoid racing conditions.
Use interlocked for atomic operations
For basic operations, use the
lock Statement can be exaggerated. While it’s very useful for locking out from complex changes, it’s too much overhead for something as simple as adding or replacing a value.
Interlocked is a class that wraps some memory operations like add, replace, and compare. The underlying methods are implemented at the CPU level and are guaranteed to be atomic and much faster than the standard
lock Statement. You should use them whenever possible, although they don’t completely replace the latch.
In the example above, the lock is replaced by a call to
Interlocked.Add() will speed up operations a lot. While this simple example is no faster than using Interlocked, it is useful as part of a larger operation and still speeds things up.
There are also
Decrement to the
-- Operations that save you two solid keystrokes. You literally wrap yourself up
Add(ref count, 1) under the hood so there is no special acceleration to use it.
You can also use Exchange, a generic method of setting a variable that corresponds to the value passed to it. However, you should be careful with this value. Setting a value that you calculated using the original value is not thread safe as the old value may have been changed before Interlocked.Exchange was run.
CompareExchange checks two values for equality and replaces the value if they are the same.
Use thread-safe collections
The standard collections in
System.Collections.Generic can be used with multithreading, but is not completely thread safe. Microsoft offers thread-safe implementations of some collections in
This includes the
ConcurrentBag, an unordered generic collection, and
ConcurrentDictionary, a thread-safe dictionary. There are also concurrent queues and stacks, and
OrderablePartitionerthat can split orderable data sources such as lists into separate partitions for each thread.
Look for parallelization loops
Often the easiest place to multithread is in large, expensive loops. If you can run multiple options at the same time, you can significantly speed up the overall run time.
The best way to deal with this is with
System.Threading.Tasks.Parallel. This class provides substitutes for
foreach Loops that the loop bodies execute in separate threads. It’s easy to use, but requires a slightly different syntax:
The catch here, of course, is that you have to make sure
DoSomething() is thread safe and does not interfere with shared variables. However, it’s not always as simple as replacing the loop with a parallel loop, and in many cases you need to do it
lock shared objects to make changes.
To solve some problems with deadlocks,
Parallel.ForEach provide additional functions for handling condition. Basically, not every iteration is executed in a separate thread. If you have 1000 items, it won’t create 1000 threads. As many threads are created as your CPU can handle, and multiple iterations are performed per thread. This means that when you calculate a grand total, you don’t have to lock on each iteration. You can just pass a subtotal variable and at the very end lock the object and make changes once. This drastically reduces the overhead for very large lists.
Let’s look at an example. The following code has a large list of objects and must serialize each one to JSON, resulting in one
List of all objects. JSON serialization is a very slow process, so splitting each element across multiple threads is a huge speed up.
There are a number of arguments here and a lot to unpack:
- The first argument takes an IEnumerable, which defines the data that will be looped over. This is a ForEach loop, but the same concept applies to basic For loops.
- The first action initializes the local subtotal variable. This variable is shared on each iteration of the loop, but only within the same thread. Other threads have their own subtotals. Here we initialize it with an empty list. If you were to calculate a numeric sum, you could
- The second action is the main loop body. The first argument is the current element (or the index in a for loop), the second is a ParallelLoopState object that you can invoke with
.Break()and the last is the subtotal variable.
- In this loop you can edit the item and change the subtotal. The value you return replaces the subtotal for the next loop. In this case, we serialize the item to a string and then add the string to the subtotal, which is a list.
- Finally, the final action takes the “Result” subtotal after all runs are complete, so you can lock and change a resource based on the final total. This action will end up being performed once, but will still be performed on a separate thread. So you need to lock or use locked methods to change resources. Here we call
AddRange()to append the subtotals list to the final list.
One final note: if you are using the Unity game engine, be careful with multithreading. You cannot call Unity APIs or the game will crash. It is possible to use it sparingly by running API operations on the main thread and switching back and forth whenever you need to parallelize something.
This mainly applies to operations that interact with the scene or the physics engine. Vector3 math is not affected, and you can use it from a separate thread with no problem. You can also change fields and properties of your own objects as long as they don’t invoke Unity under-the-hood operations.