The “Many Core” Problem

We, as developers, have a problem.  CPUs will continue to have more cores, and each core is not going to be any faster.  The only way to write faster applications is to write multithreaded code, which has two challenges:

  • Multithreaded code is complex to write and think about.
  • Multithreaded code is difficult to test.

From what I’ve seen, people are pursuing 4 approaches. 

Better Languages

If languages are the whole answer, the language has to make it impossible to write multithreaded bugs, otherwise you only address the first problem (the complexity).

Erlang is one such language that removes whole class of multithreaded bugs. Erlang’s approach is to isolate the threads, and give them a way to pass messages to each other.  However programming in Erlang is awkward and arguably it makes the code more complex.

Other languages, like Clojure and F#, take a different approach and make it easier to write and deal with multithreaded issues, however they don’t prevent you from writing multithreaded bugs.  These languages do not address the whole problem as you still need a way to test the code.

Better Libraries

Microsoft has a library for .Net called Parallel Task Library to make it easier to write multithreaded code.  Need to operate on each element in an array?  Use the Parallel.For() method.  This library makes it easier to think and write multithreaded code, but it doesn’t make it any easier to test.

Static Code Analyzer

Java has a static code analyzer that is able to find multithreading bugs called FindBugs.  It looks at your code and reports any code that will cause multithreaded bugs.  This would address the second issue, because it, in theory, reports all threading bugs.  It doesn’t however address the first issue, making the code easier to write.

Testing with Deterministic Threads

Microsoft is going in a different direction.  They are working on something called CHESS that makes your unit tests find multithreaded bugs.  It does this by taking over the scheduler and sysmatically repeating each test, weaving the threads differently each time to exercise every possible way the scheduler could run the code.  However it requires us to to write a set of tests that finds *all* the threading bugs.  Since it is common to write unit tests with less than 100% coverage, a code coverage tool should be used to write more tests.


The good news is that there appears to be a variety of approaches to the problem and the whole answer will likely involve a combination of language or library enhancements and analyser or testing enhancements that are already in the works.

About Clay Lenhart

I am a DBA/Technical Architect for Latitude Group and love technology.
This entry was posted in Architecture, Languages and tagged , , . Bookmark the permalink.

8 Responses to The “Many Core” Problem

  1. henk says:

    I agree there’s a problem with the complexity of threading, but it already helps *a lot* if the communication pattern between your objects is well defined instead of chaotic.

    And guess, we should already be adhering to the well defined communication pattern for regular sequential (OO) programming ;)

    Next to that, in some cases building multi-threaded software is easier. Firing off some piece of code in a kind of job and then forgetting about it, may be far simpler than executing it inline of your regular sequential code and trying to interweave some other tasks while processing this piece of code.

  2. Andras says:

    Erlang got one thing right – the concurrency mechanism (I’ve written a post on this earlier: It *IS* a pitty that they chose to make it a functional language (it’s *incorrect* to claim that functional languages enable parallelism – why would they still need process constructs like in Erlang and Haskell?!) – same (otherwise powerful and beautifully simple) principles would have worked with traditional languages.
    Cheers, Andras

  3. Clay Lenhart says:

    > Erlang got one thing right – the concurrency mechanism

    You may be interested to know that .Net copied Erlang style concurrency and put it in a library for any .Net language call MPI.Net:

  4. Ulf Wiger says:

    That Erlang is supposed to be awkward is repeated in many blogs, as if it were an indisputable fact. Yet the University of Kiel uses Erlang to teach programming to high-school girls, and their final task after a week is programming a distributed chat server. There may, or may not, be an awkward transition depending on what you’re used to, but there’s plenty of evidence that Erlang itself can give good productiviy, compact code, and excellent life-cycle economy.

    A very recent study compared implementations of IMAP:

    Its conclusion is no surprise to those familiar with Erlang. On several occasions, Erlang teams participating in standardization work have ended up being identified as the de-facto reference implementation, since they are able to implement working feature-complete code much faster than the others.

    There are other languages that have outstanding qualities, but if you’re going to call Erlang awkward and complex, you should be fairly specific about what you’re comparing it to – it may carry some weight if coming from a seasoned Haskell programmer, but even then it will depend on the problem domain.

  5. Kamil Szot says:

    Most of apps today are multiuser and/or built of multiple software components.
    You can just run different users, or diffrent software components on different cores and you get the boost but almost never have to bother yourself with multithreading. The only case that remains a problem is when some component of your system responds to slow, then you have try to replace it with some equivalent that uses algorithm that can be distributed. This single tiny bit of system can be written as well in erlang regardles of it’s obscurity. There is a chance that you won’t even have to write it by yourself because some erlang hackers wrote it already because they needed it before you and published it as open source.

  6. Ulf Ochsenfahrt says:

    In which category does software transactional memory fall?

  7. Clay Lenhart says:

    > That Erlang is supposed to be awkward is repeated in many blogs, as if it were an indisputable fact.

    I’ve got to admit, you make a fair point here. For what it is worth, my merger explanation is:
    a) Erlang hasn’t taken off yet, so people must have some issue with it.
    b) “Standard” languages can do most anything concurrency-wise, which means that a restrictive model, like actors/message passing, must be more awkward.

    Pretty weak.

    Keep in mind that the point I’m trying to make is we’ll get through this crisis. Heck, Erlang may very well be the answer.

  8. Clay Lenhart says:

    > In which category does software transactional memory fall?

    This is a oversight — thanks for reminding me.

    For .Net people, you may be interested in the Transaction Memory blog

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>