mr-edd.co.uk :: horsing around with the C++ programming language

An asynchronous function call mechanism: handling exceptions

[28th June 2007]

We're on the cusp of seeing multi-core machines becoming the default configuration for desktop computers and yet programming using threads is still considered hard, at least in imperative languages such as C++. So I've been trying to look at a mechanism to make one aspect of multi-threaded programming easier.

For GUI driven applications in particular, responsiveness is important. People don't like waiting around for the interface to come alive again while some task is performed. For this reason, it's often a good idea to look at kicking off that time-consuming task in a background thread so that the user interface remains responsive.

Starting up such a background thread is a common pattern and yet it's actually quite tricky to do. For example, you really want to say call function F with arguments X, Y and Z in another thread and give me the result of type R when I'm ready. This kind of thing is often hard-coded but I believe that it's actually possible to generalise this in to a small re-usable set of components so that performing this asynchronous call is as easy as:

async::future<R> f = async::call(F, X, Y, Z); // call F(X,Y,Z) in another thread. call() returns immediately

// later on...

R r = f.value(); // get the result of the background thread. Block until ready

This formulation was stolen adapted from ideas presented in Eric Niebler's summary of a meeting between C++ standards committee members in August 2006 and from some of the ideas in Herb Sutter's presentations (video) on the Concur project.

So in this post I'm going to start introducing some of the gadgetry we'll need to implement such a feature, in particular the code needed to handle exceptions that are thrown out of the function being called asynchronously.

I should warn you that while I hope the code will be easy to use once finished, the underlying implementation details are really rather advanced and these are what I'll be discussing in this post. In particular we'll be using templates a lot and a number of associated meta-programming techniques.

EDIT 5th August 2007: async now has it's own permanent home, here. If you're more interested in the usage of the library, as opposed to the implementation, you should go there instead.

So what should happen if an exception escapes?

The C++ standard has no notion of threads[1] so there's no hard-and-fast answer to this question as of yet.

Here are some possibilities:

  1. Let the compiler/platform decide (implementation dependent, but probably kills the program somehow)
  2. terminate() the program as soon as the exception escapes the child thread, in the same way as when an exception escapes main()
  3. terminate() the program at the point when the future's value() member function is called
  4. Propagate the exception in to the parent thread when the future's value() member function is called

None of these is the one-true-answer. I personally very much like option number 4, but your own needs may differ. So it would be nice if the asynchronous call mechanism could allow you to choose your exception handling strategy.

It turns out that this is actually possible to a certain degree. We'll see that implementing options 1, 2 and 3 are actually quite simple but number 4 is a bit more troublesome without support from the language.

Before I go any further in to looking at why this is, I'll just revise the way in which an asynchronous call is made. We'll pass in the exception-handling strategy as the first argument:

async::future<R> f = async::call(exception_strategy, F, X, Y, Z);

Perhaps we might decide on a sensible default so that we can have an overload of the async::call() function in which the exception handling object doesn't need to be specified.

All the code I've created for this project so far lives in the async namespace (or another namespace nested within), but I'll drop the async:: qualification where convenient throughout this post.

Propagating exceptions in to the parent thread

As I've mentioned, it's generally not possible to propagate an exception of arbitrary type in to the parent thread. This is because in order to do this, we need to be able to first catch the exception in the child thread and we can't do that unless we know the type of the exception in advance i.e.

try
{
    F(X, Y, Z);
}
catch( /* what goes here? */ )
{
    // ...
}

We could use catch(...) but then we don't know what we've caught.

As it happens there are a couple of proposals for the next revision of the standard that add support to the C++ language in order to propagate an exception of arbitrary type to a parent thread. See N2107 by Jens Maurer and Alisdair Meredith and N2179 by Peter Dimov.

But the next revision of the C++ standard isn't due until 2009 by all accounts and even then it will probably take a while for compiler vendors to implement the new spec. So we need a stop-gap — well /I/ do at least!

I think about the best we can do for the moment is to specify the types of exceptions we want our exception handler to attempt to catch. We could make a layered cage object that would be used as an exception handler like so:

layered_cage<std::runtime_error, std::exception, std::logic_error> cage;

future<R> f = call(cage, F, X, Y, Z);

Internally, this would effectively create the following exception-handling construct (hopefully you'll see why it's called a layered_cage<>):

try
{
    try
    {
        try 
        { 
            F(X, Y, Z); 
        }
        catch(const std::runtime_error &ex) 
        { 
            store_exception(ex); 
        }
    }
    catch(const std::logic_error &ex)
    {
        store_exception(ex);
    }
}
catch(const std::exception &ex)
{
    store_exception(ex);
}

One question is how is this nested construct created? I'll come on to this when I discuss the implementation in full. For now we'll look at the issues we need to solve in order to build the core components. So there are a number of preliminary problems that need our attention before we begin writing layered_cage<>.

  1. What happens when an exception is of a type that wasn't in our list?
  2. The code should try to catch the most-derived exceptions first to avoid slicing, where possible. But how?
  3. Slicing is usually undesirable. Sometimes, it's catastrophic. How can we avoid it?
  4. How does the store_exception() function work?

Fortunately, these questions have adequate answers, I believe. Let's look at them in turn.

Handling exceptions that aren't in layered_cage<>'s list of types

We'll create a special type, catch_all that when placed in the list of template parameters for the layered_cage<> template, will signify that a try{ } catch(...){ } will form the final layer of the cage:

try
{
    // try-catch blocks for the other layers of the cage
    // ...
}
catch(...)
{
    // An unknown_exception object will be stored and propagated in to the parent
    // thread to signify that something was caught, but we don't know what
    store_exception(unknown_exception());
}

In fact we mentioned that we might be able to create an overload of our async::call() function that uses some sensible default exception handling object. I think a good candidate is an object of type layered_cage<catch_all>.

If catch_all isn't in the list of types for a layered_cage<>, we'll leave it up to the implementation to decide what should happen.

Catching the most derived exceptions first

Let's say that we create a layered_cage<std::exception, std::logic_error> as our exception handling object. What we don't want is for the following construct to be built up internally:

try
{
    try
    {
        F(X, Y, Z);
    }
    catch(const std::exception &ex)
    {
        store_exception(ex);
    }
}
catch(const std::runtime_error &ex)
{
    store_exception(ex);
}

Code like this is pointless because the catch(const std::exception &ex) clause will catch everything that the catch(const std::runtime_error &ex) is able to handle because std::exception is a public base of std::runtime_error. Thus the latter clause is redundant.

Somehow we need to make sure that attempts to catch more derived types of exception occur sooner.

Fortunately, using the magic of the boost meta-programming and preprocessor libraries we can actually manipulate a list of types at compile time in such a way as to sort them based on their derived-ness!

The code looks something like this:

template<BOOST_PP_ENUM_PARAMS_WITH_A_DEFAULT(BOOST_MPL_LIMIT_VECTOR_SIZE, typename Ex, void)>
class layered_cage
{
    private:
        typedef typename
            boost::mpl::vector<BOOST_PP_ENUM_PARAMS(BOOST_MPL_LIMIT_VECTOR_SIZE, Ex)>::type
            template_parameters; // mpl::vector of template parameters
			
        typedef typename boost::mpl::remove
        <
            template_parameters,
            void
        >
        ::type template_parameters_no_voids; // voids removed

        typedef typename boost::mpl::find<template_parameters_no_voids, catch_all>::type catch_all_iter;
        static const bool has_catch_all = catch_all_iter::pos::value != boost::mpl::size<template_parameters_no_voids>::type::value;
		
        typedef typename boost::mpl::remove
        <
            template_parameters_no_voids,
            catch_all
        >
        ::type template_parameters_no_specials; // catch_alls and voids removed

        typedef typename boost::mpl::sort
        <
            template_parameters_no_specials,
            boost::is_base_and_derived
            <
                boost::mpl::_2,
                boost::mpl::_1
            >
        >
        ::type sorted_exceptions; // types sorted according to "derived-ness", most derived types toward the front

        // ...

Now this isn't the prettiest piece of code ever to grace a text editor, but it does what we want. First we use the preprocessor library to do the work of writing the following for us:

template<typename Ex1 = void, typename Ex2 = void, ... , typename ExN = void>
class layered_cage
{
    //...

where N is given by the constant BOOST_MPL_LIMIT_VECTOR_SIZE.

Thus we currently have a fixed limit on the number of exceptions we can specify. I'll talk more about how we can remove this restriction later on.

Next we create a type called template_parameters that is effectively a vector of types containing all those given in the template parameter list. Then we use boost::mpl's compile-time remove algorithm to create a new vector, template_parameters_no_voids, with all the void default arguments removed.

Then we create a compile-time boolean constant, has_catch_all, which indicates whether the catch_all type is in the template_parameters_no_voids vector by employing boost::mpl's find algorithm.

Then we create a new vector, template_parameters_no_specials, with any catch_alls removed.

Finally we use the boost::mpl::sort in conjunction with boost::is_base_and_derived to create a new vector, sorted_exceptions that is a copy of template_parameters_no_specials, but sorted according to derived-ness.

There's a lot more to boost::mpl. I'm still relatively new to it all, but there's a book by Dave Abrahams and Aleksey Gurtovoy that I'm currently reading which explains the concepts and techniques of the library for those that are interested.

How do we avoid slicing?

But even now that we have our sorted list of exceptions, we may still get slicing when catching the various types of exception.

Slicing happens when we refer to an object through a reference to one of its bases and then try to copy it e.g.

class B { /* ... */ };
class D : public B { /* ... */ };

void func(const B &b)
{
    // Slicing happens here if/when b really refers to a D, 
    // as we're only copying the parts from the B base object
    B b2(b); 

    // ...
}

int main()
{
    D d;
    func(d);
}

Obviously slicing is often undesirable and in some cases can actually be dangerous as the slice breaks some invariant set-up by the object of the derived type.

So we need a way to protect ourselves against this. One option is to provide a wrapper template dont_slice<> such that putting dont_slice<E> in the type list of layered_cage<> indicates that exceptions of type E should be caught and stored but only if slicing won't occur as a result.

Another option is to specify that all exceptions should be derived from a base class that looks like this, or alternatively add the same pure virtual functions to the std::exception base class:

class exception_base
{
    public:
        exception_base *clone() const = 0;
        void throw_copy() const = 0;
};

clone() is your typical clone function where derived classes override it to create a copy on the heap without slicing. So we can call clone instead of a copy-constructor in order to make a copy of the exception.

Then when we want to propagate the exception in to the parent thread, we can call throw_copy() on our clone()-d copy to do that for us at the appropriate time.

The exception_base solution is intrusive, though. It won't work for existing exceptions, which isn't good. A proposal, N2229 by Beman Dawes, exists for adding this kind of thing to the C++ standard, but it now seems more likely that making minimal changes to the core language is the solution most likely to be adopted by C++09.

So for now, I'll go with the dont_slice<> option. The exception handling strategy object is independent from the rest of the code so we'll be able to integrate the exception_base style code in future, anyway, if need be.

So to implement the dont_slice<> template class we need a couple of things. First we obviously need to define our dont_slice<> template:

template<typename T>
struct dont_slice { typedef T type; };

Simple enough! Now need to know whether or not a copy will slice the object. If a slice will occur, we don't want to catch the exception. This is actually pretty simple to implement:

try
{
    // ...
}
catch(const some_type &ex)
{
    if (typeid(some_type) != typeid(ex)) // the dynamic type differs from the static type
        throw;

    store_exception(ex);
}

However, this method won't work if some_type and all of its base classes have no virtual functions. But in this case, the derived exception really should have used private inheritance or composition rather than public inheritance because having no virtuals suggests an implemented-in-terms-of relationship, rather than an is-a relationship (which is what public inheritance really means).

We'll just have to lump it for now as I can't see a way to work around other people's faulty code ;)

Another thing we need is the ability to extract those types wrapped in dont_slice<> from the list of template parameters given to layered_cage<>. This isn't too hard either. We need to define a meta-function that tells us whether or not a given type looks like dont_slice<T> for some T. We can do this using partial template specialisation:

template<typename T>
struct is_slicable : boost::true_type { };

template<typename T>
struct is_slicable<dont_slice<T> > : boost::false_type { };

I recommend looking at the documentation to the boost type traits and mpl libraries if you don't follow the above code.

Now using boost::mpl's remove_if algorithm we can separate out those entries in the list of types that are wrapped with dont_slice<>:

typedef typename boost::mpl::remove_if
<
    template_parameters_no_specials,
    boost::mpl::not_< detail::is_slicable<boost::mpl::_1> >
>
::type slicable_exceptions; // all those NOT of the form async::dont_slice<T>

typedef typename boost::mpl::remove_if
<
    emplate_parameters_no_specials,
    detail::is_slicable<boost::mpl::_1>
>
::type non_slicable_exceptions; // all those of the form async::dont_slice<T>, need sorting as before

Now when the cage construct is put together, we attempt to catch all those types in the non_slicable_exceptions list first and throw them on their way if their static type does not equal their dynamic type at the catch-point.

Implementing the store_exception() function

What we'll actually do is create an exception_store class that has a store_exception() member function:

class exception_store : boost::noncopyable
{
    public:
        template<typename ExceptionType>
        void store_exception(const ExceptionType &ex);

        bool has_exception() const;

        void rethrow() const;

    private:
        boost::shared_ptr<const exception_holder_base> holder_;
};

The layered_cage<> will have access to such an object, as will any other type of exception handling object. An exception_store object allows us to hold on to an object of arbitrary type and rethrow() it on command.

How do we implement this? We'll use what I like to call the morph-o-matic pattern, where we have an abstract base class and a derived class that is templated to implement the base class interface for any type we like i.e.

class exception_holder_base : boost::noncopyable
{
    public:
        virtual ~exception_holder_base() { }
        virtual void rethrow() const = 0;
};
	
template<typename ExceptionType>
class exception_holder : public exception_holder_base
{
    public:
        exception_holder(const ExceptionType &ex) : ex_(ex) { }
        void rethrow() const { throw ex_; }
			
    private:
        ExceptionType ex_;
};

So an exception_holder<T> implements the exception_holder_base interface to store and rethrow() an object of type T.

This allows us to implement exception_store::store_exception() very simply:

class exception_store : boost::noncopyable
{
    public:
        template<typename ExceptionType>
        void store_exception(const ExceptionType &ex)
        {
            holder_.reset(new detail::exception_holder<ExceptionType>(ex));
        }

    // ...

The interface to an exception handling object

So far I've danced around the issue of what an exception handling object such as layered_cage<> will actually look like. Clearly, there will need to be some kind of common interface that all handlers will share.

In fact this interface is very simple. An exception handling object will be required to provide a single member function, call:

class some_exception_handler
{
    public:
        void call(const boost::function<void ()> &f, exception_store &ex) const;
};

Although it appears restrictive, it turns out that this is sufficient to implement any of the exception handler behaviours we specified at the start of this post.

Note that we are passing in the function to be called via a boost::function<void ()> and yet we want to allow a function of arbitrary signature to be called by our asynchronous call mechanism.

In the implementation of the async::call() function we'll be able to use boost::bind in order to pack the arguments inside the functor as additional context and thus reduce it to the form shown. Of course we'll need to do something similar for the return value, too. But I'll leave the binding of the arguments for a different post. For now, we'll just assume that we're at the stage where we've been handed a boost::function<void ()> to call.

So let's look at our list of possible exception handling policies and see how they're implemented in terms of this interface. Here are the policies we identified:

  1. Let the compiler/platform decide (implementation dependent, but probably kills the program somehow)
  2. terminate() the program as soon as the exception escapes the child thread, in the same way as when an exception escapes main()
  3. terminate() the program at the point when the future's value() member function is called
  4. Propagate the exception in to the parent thread when the future's value() member function is called
class implementation_default // number 1
{
    public:
        void call(const boost::function<void ()> &f, exception_store &) const
        {
            f();
        }
};

class terminate_immediately // number 2
{
    public:
        void call(const boost::function<void ()> &f, exception_store &) const
        {
            try { f(); }
            catch(...) { std::terminate(); }
        }
};

class terminate_on_join // number 3
{
    private:
        struct terminate_on_2nd_copy
        {
            terminate_on_2nd_copy() : n_(0) { }
            terminate_on_2nd_copy(const terminate_on_2nd_copy &other) : n_(other.n_ + 1) 
            {
                if (n_ == 2) std::terminate();
            }
            unsigned n_;
        };

    public:
        void call(const boost::function<void ()> &f, exception_store &ex) const
        {
            try { f(); }
            catch(...) 
            { 
                terminate_on_2nd_copy to2c;

                // 1st copy made here, 2nd when rethrow()-n in parent thread:
                ex.store_exception(to2c); 
            }
        }     
};

What about number 4? This would be our layered_cage<> code. We need to set up the nested try-catch construct and call f() deep down inside of it. So let's have a look at how to do this, keeping in mind that we have an arbitrary number of types that we must attempt to catch and so it becomes impractical to hard-code implementations for various cage-depths.

Using boost::mpl, we've seen that we're able to divide our list of types in to two:

Note that neither list contains the catch_all type, but we know whether or not it was given as a parameter to layered_cage<> because we have a constant has_catch_all that tells us so. If has_catch_all == true we need to add a catch(...) as the outer layer of the cage.

So the idea is to use this information to construct what is essentially a linked-list of objects down which the boost::function is passed until it reaches the end. Once there it is called and any exception that is thrown from this eventual call travels its way back up the list until caught by one of the cage layers.

layered_cage internalsClick to enlarge

Each node in the list is of a different type, but they are all derived from the same base class, basic_cage and implement a virtual void call(const boost::function<void ()> &f, exception_store &store) member function. The derived type is shown above each node in the diagram.

You can see how the function is passed down to the inner-most layer of the cage, where it is called. If an exception is thrown it travels back up the list through all the exception handling code inside the nodes.

Constructing the layers of the cage

Let's look at how this list is constructed. Each layered_cage<> object has a member object boost::shared_ptr<basic_cage> cage_.

When a layered cage is constructed, this is initialised to point to an inner_cage object (far right in the previous diagram). In the body of the constructor, we start adding the layers proper.

We do this using the boost::mpl::for_each compile time algorithm to call the template member function layered_cage<>::add_layer<T>() for each T in the two lists of exception types.

add_layer() looks like this:

template< /* ... */ >
class layered_cage
{
    // ...

    public:
        template<typename Exception>
        void add_layer()
        {
            const bool slicable = is_slicable<Exception>::type::value;
		
            typedef typename remove_dont_slice_wrapper<Exception>::type ex_wo_exact;
            typedef typename boost::remove_reference<ex_wo_exact>::type ex_wo_exact_wo_ref;
            typedef typename boost::remove_cv<ex_wo_exact_wo_ref>::type ex_type;

            boost::shared_ptr<detail::basic_cage> c(new cage<ex_type, slicable>(cage_));
            cage_ = c;
        }

    // ...
};

We've seen the is_slicable meta-function before. The as-yet-unseen remove_dont_slice_wrapper meta-function is implemented in a similar fashion:

template<typename T>
struct remove_dont_slice_wrapper { typedef T type; };

template<typename T>
struct remove_dont_slice_wrapper<dont_slice<T> > { typedef T type; };

Then we remove any const/volatile qualification and reference parts that the Exception type might have. Next we create a cage for the particular type of exception at hand. This new cage has the cage_ as its next_ member (which equates to its first inner layer). Finally we assign this new cage object to the layered_cage<>'s cage_ member to replace the existing one.

Note that we pass slicable as the second template parameter to cage<> in order to activate the appropriate specialisation depending on whether Exception was of the form dont_slice<T> for some T.

So the layered_cage<> constructor looks like this:

layered_cage() : cage_(new detail::inner_cage)
{
    layer_builder b(*this);
    boost::mpl::for_each<non_slicable_exceptions, boost::mpl::make_identity<boost::mpl::_> >(b);
    boost::mpl::for_each<sorted_slicable_exceptions, boost::mpl::make_identity<boost::mpl::_> >(b);
			
    if (has_catch_all)
    {
        boost::shared_ptr<detail::basic_cage> c(new detail::catch_all_cage(cage_));
        cage_ = c;
    }
}

layer_builder is a simple callable class that forwards calls to the add_layer<T>() member function for a given type T. I won't present it here, but it'll be in the example code at the end of the post.

Note that we first add the contents of the non_slicable_exceptions list to create the inner layers for the cage because we don't want the cages for the slicable exceptions to get in their way when catching exceptions. Then the types in sorted_slicable_execptions are added. Finally, if catch_all was specified in the template arguments, we add an outer layer that catches all exceptions.

Now all layered_cage<>::call() has to do is:

void call(const boost::function<void ()> &f, exception_store &s) const
{
    assert(cage_);
    cage_->call(f, s);
}

Removing the fixed restriction on the number of exception types allowed

Currently the implementation of layered_cage<> only allows BOOST_MPL_LIMIT_VECTOR_SIZE different types of exception to be given as template arguments. While we could up this number, it would still ultimately be a fixed constant.

If you need more, you can always add layers to an existing layered_cage<> object by using its add_layer<>() member function.

However, this won't be necessary in future since variadic templates have been accepted as a feature in the new C++ standard (expected 2009). This does for templates what the ellipsis (…) does for regular functions.

Here are the proposals (PDFs):

In fact there's already a version of GCC available that implements this feature.

Clearly this new language feature will also be useful to implement the actual async::call() function that was introduced at the start of this post.

While I'm dropping links I'll also quickly mention Herb Sutter posted a summary of the features that were accepted in to the draft standard, back in May.

Other uses for layered_cage<> and exception_store

I had actually formulated a similar construct to layered_cage<> before I started looking at this asynchronous function stuff. It turns out layered_cage<> and exception_store also find a use in the context of catching and identifying exceptions that are thrown from unit tests.

A unit test might be considered a failure if it throws an exception. To determine the reason for the failure, it would be nice to know the type of that exception so that the test may be debugged. This calls for a similar construct to layered_cage<> and exception_store where the code for the unit test is executed inside the inner-most layer of the cage. But rather than giving exception_store a rethrow() member function, it may instead have a type() member function that would return the std::type_info object for the exception caught. From a std::type_info object you can get the name of the type it represents by calling its name() member function[2].

Still to come

Despite this probably being the longest post I've written so far for this blog, we have only implemented part of the asynchronous call mechanism! In a future post I hope to get around to looking at either how the future<> template class is implemented or how we'll construct the boost::function<void ()> that we're planning to pass down the exception handler object to make the actual call.

EDIT 26/7/2007: The next two posts in this series can be found here and here.

Show me the code!

The code for layered_cage and exception_store along with a little example program that illustrates its features is available for download below. The zip archive includes a slam build file. You'll need a boost installation to compile the code. I've only tried version 1.34, but 1.33.1 will probably work fine, too.

To compile the code using slam:

> slam toolset=msvc8 variant=debug

Replace msvc8 with the supported toolset of your choice and debug with release if you want a release build. I've so far only tested the code with msvc8 and mingw. Also, if you don't have a BOOST_ROOT environment variable set, you need to specify it on the command line e.g.

> slam toolset=gcc variant=debug BOOST_ROOT=/folder/containing/boost/include/folder

Downloads

Footnotes
  1. though work is being done to add the threads to the next revision of the standard. video []
  2. strictly speaking, std::type_info::name() doesn't have to return the name of the type. It can legally return anything it wants. But for all compilers I've seen some kind of sensible name is returned. On GCC, the mangled name is returned but it's possible to de-mangle this name using functions in the cxxabi namespace. Borland and MS Visual C++ 8 both return the actual name of the type []

Comments

(optional)
(optional)
(required, hint)

Links can be added like [this one -> http://www.mr-edd.co.uk], to my homepage.
Phrases and blocks of code can be enclosed in {{{triple braces}}}.
Any HTML markup will be escaped.