Higher-Order functions in C++

The other day, I was writing some C++ and found that I was thinking about how to manipulate the data I had as if I was writing F#. It would have been convenient to turn a std::map into an array of tuples, which I could do in F# like this:

let f (xs : map<int,string) =
  let xs = xs |> Map.toArray
  // now treat xs as array of tuples...

There’s no function in STL to do this off the bat – instead, you have to roll your own (not that it’s much code, but it does break your through processes if you have to stop to code this sort of thing every time).

Of course, this is just one of many helpful F# higher-order functions that are provided in the F# Map module – and there are counterparts for each of the collection classes i.e. Array, Set, List etc. In C++, the nearest equivalent is the STL which provides both collection classes and a number of algorithms that operate on them. Better still, from C++11 onwards we have lambdas, which make using STL algorithms much easier. Even so, in most cases, the F# operations seem much more tailored to the sort of data transformation I see at work – our codebase is littered with map/filter/fold operations as people transform/select and accumulate data. Conversely, our C++ codebase is full of … for loops, evidence in my eyes that STL algorithms aren’t as immediately applicable. In fact, the ease of use of higher-order functions was one of the reasons that F# was quickly adopted in my workplace (along with immutability, strong-typing, conciseness, type inference and syntax checking).

I’ve written one-to-one C++ equivalents of the F# module functions that I use the most for Map and Vector – see below. Interestingly, I found that I really did have to ‘engage brain’ to write some of these, particularly Map.filter. For that one, you can’t use the erase-remove idiom because map keys are both const and strictly ordered (whereas for Vector, erase-remove_if implements filter neatly). A library of functions as per my code below would definitely be a productivity boost.

First, I’ve factored some common utilities into namespace Collection:

namespace MusingStudio
{
    namespace Collection
    {
        template <typename C, typename F>
        C& filter( C& collection, F keep_predicate )
        {
            auto erase_predicate = [&pred=keep_predicate]( auto&& x ){ return !pred( std::forward<decltype(x)>(x) ); };
            collection.erase( std::remove_if( collection.begin(), collection.end(), erase_predicate ), collection.end() );
            return collection;
        }
        
        // This form of filter always takes a copy and applies the filter to it
        // - sometimes you want to preserve the original collection
        template <typename C, typename F>
        C filter_copy( const C& collection, F keep_predicate )
        {
            C target;
            std::copy_if( collection.begin(), collection.end(), std::inserter( target, target.end() ), keep_predicate );
            return target;
        }
        
        template <typename C, typename F, typename A>
        A fold( const C& items, F f, A&& init )
        {
            A acc{ std::forward<A>(init) };
            for ( const auto& item : items )
            {
                f( acc, item );
            }
            return acc;
        }
                
        // F( T ) -> T and collection C is mutated
        template <typename C, typename F>
        C& transform( C& items, F f )
        {
            for ( auto& t : items )
            {
                t = f( t );
            }
            
            return items;
        }

    }
}

Next, here are the higher-order functions that I use for Map:

namespace MusingStudio
{
    namespace Map
    {
        // filter_copy takes a copy of the original collection then applies the filter
        template< typename K, typename V, typename F>
        std::map<K,V> filter_copy( const std::map<K,V>& items, F predicate )
        {
            return Collection::filter_copy( items, predicate );
        }
        
        template< typename K, typename V, typename F>
        std::map<K,V>& filter( std::map<K,V>& items, F predicate )
        {
            // NB the erase-remove_if idiom does not work for std::map
            // because the nodes must remain ordered by key.  This is enforced
            // by std::map<K,V> holding keys as const K.  So any assignment
            // to the key (to effecfively re-order the binary tree) fails to compile.
            // http://stackoverflow.com/questions/9515357/map-lambda-remove-if
            
            // instead, manually iterate over the collection, erasing items
            // for which predicate() returns false
            for ( auto it = items.begin(), itEnd = items.end(); it != itEnd; )
            {
                if ( predicate( *it ) )
                {
                    ++it; // ok - keep this item
                }
                else
                {
                    it = items.erase( it );
                }
            }
            
            return items;
        }
        
        template <typename K, typename V>
        auto to_vector( const std::map<K,V>& collection )
        {
            std::vector< std::pair<K,V> > items;
            
            for ( const auto& item : collection )
            {
                items.push_back( std::make_pair( item.first, item.second ) );
            }
            
            return items;
        }
        
        template <typename K, typename V>
        auto keys( const std::map<K,V>& collection )
        {
            std::set< K > items;
            
            for ( const auto& item : collection )
            {
                items.insert( item.first );
            }
            
            return items;
        }
        
        template <typename K, typename V>
        auto values( const std::map<K,V>& collection )
        {
            std::vector< V > items;
            
            for ( const auto& item : collection )
            {
                items.push_back( item.second );
            }
            
            return items;
        }
        
        template<typename K, typename V, typename F, typename A>
        A fold( const std::map<K,V>& items, F f, A&& init )
        {
            return Collection::fold( items, f, std::forward<A>(init) );
        }
        
        // F( std::pair<K,V> ) -> std::pair< L, U >
        // Construct a new std::map<L,U> mapping from (K,V) to (L,U)
        template <typename K, typename V, typename F>
        auto map( const std::map<K,V>& items, F f )
        {
            using KVP = typename std::map<K,V>::value_type;
            using RVP = decltype( f( KVP() ) );
            
            std::map< decltype( RVP().first ), decltype( RVP().second ) > result;
            
            for ( const KVP& kvp : items )
            {
                result.insert( f( kvp ) );
            }
            
            return result;
        }
        
        // F( K, V ) -> V and std::map<K,V> is mutated
        template <typename K, typename V, typename F>
        std::map<K,V>& transform( std::map<K,V>& items, F f )
        {
            using KVP = typename std::map<K,V>::value_type;
            
            for ( const KVP& kvp : items )
            {
                items[kvp.first] = f( kvp.first, kvp.second );
            }
            
            return items;
        }
    }
}

And here are the higher-order functions that I use for Vector:

namespace MusingStudio
{
    namespace Vector
    {
        template< typename T, typename F>
        std::vector<T>& filter( std::vector<T>& items, F predicate )
        {
            Collection::filter( items, predicate );
            return items;
        }
        
        template< typename T, typename F>
        std::vector<T> filter_copy( const std::vector<T>& items, F predicate )
        {
            return Collection::filter_copy( items, predicate );
        }
        
        // Requires F to have signature void( A&, T )
        template< typename T, typename F, typename A>
        A fold( const std::vector<T>& items, F f, A&& init )
        {
            return Collection::fold( items, f, std::forward<A>(init) );
        }
        
        template< typename T, typename P = std::less<T> >
        std::vector<T>& sort( std::vector<T>& items, P compare = P() )
        {
            std::sort( items.begin(), items.end(), compare );
            return items;
        }
        
        template< typename T, typename P = std::less<T> >
        std::vector<T> sort_copy( const std::vector<T>& items, P compare = P() )
        {
            std::vector<T> result( items );
            std::sort( result.begin(), result.end(), compare );
            return result;
        }
        
        // F( T ) -> U, construct a new vector<U>, mapping from T to U
        template <typename T, typename F>
        auto map( const std::vector<T>& items, F f )
        {
            using U = decltype( f(T()) );
            
            std::vector< U > result;
            std::transform( items.begin(), items.end(), std::inserter( result, result.end() ), f );
            return result;
        }
        
        // F( T ) -> T and std::vector<T> is mutated
        template <typename T, typename F>
        std::vector<T>& transform( std::vector<T>& items, F f )
        {
            return Collection::transform( items, f );
        }
    }
}

Here are some unit tests that show how much easier it is to use the Map/Vector functions instead of going directly to STL – I’d argue that this code is comparable to F# for conciseness (although F# code would still benefit from pipelining subsequent operations).

#include <iostream>

#include <gmock/gmock.h>
#include <Vector.hpp>
#include <Map.hpp>

using namespace testing;
using namespace MusingStudio;

TEST( Map, to_vector )
{
    using Mapped = std::map<int, std::string>;
    using Tuples = std::vector<std::pair<int,std::string> >;
    
    Mapped items{ { 1, "Hi" }, { 2, "Bye" } };
    
    EXPECT_EQ( (Tuples{ { 1, "Hi" }, { 2, "Bye" } }), 
      Map::to_vector( items ) );
}

TEST( Map, keys )
{
    using Mapped = std::map<int, std::string>;
    using Keys = std::set<int>;
    
    Mapped items{ { 1, "Hi" }, { 2, "Bye" } };
    
    EXPECT_EQ( (Keys{ 1, 2 }), 
      Map::keys( items ) );
}

TEST( Map, values )
{
    using Mapped = std::map<int, std::string>;
    using Values = std::vector<std::string>;
    
    Mapped items{ { 1, "Hi" }, { 2, "Bye" } };
    
    EXPECT_EQ( (Values{ "Hi", "Bye" }), 
      Map::values( items ) );
}

TEST( Map, filter )
{
    using Mapped = std::map<int, int>;
    
    Mapped items{ {1,1}, {2,4}, {3,9}, {4,16} };
    
    Mapped even_keys{ {2,4},{4,16} };
    auto lambda = []( const auto& keyvaluepair ){ return keyvaluepair.first % 2 == 0; };
    
    // Map::filter will mutate parameter 'items'
    EXPECT_EQ( even_keys, 
      Map::filter( items, lambda ) );
    EXPECT_EQ( 2, items.size() );
}

TEST( Map, filter_copy )
{
    using Mapped = std::map<int, int>;
    
    Mapped items{ {1,1}, {2,4}, {3,9}, {4,16} };
    
    Mapped even_keys{ {2,4},{4,16} };
    auto lambda = []( const auto& keyvaluepair ){ return keyvaluepair.first % 2 == 0; };
    
    // Map::filter_copy creates a copy, so parameter 'items' is untouched
    EXPECT_EQ( even_keys, 
      Map::filter_copy( items, lambda ) );
    EXPECT_EQ( 4, items.size() );
}

TEST( Map, fold )
{
    using Mapped = std::map<int, int>;
    
    Mapped items{ {1,2}, {3,4}, {5,6} };
    
    // Map::fold takes F( A&, pair<K,V> ) -> void
    EXPECT_EQ( 21, 
      Map::fold( items, 
        []( int& acc, const auto& kvp ){ acc += kvp.first + kvp.second; }, 0 ) );
}

TEST( Map, transform_mutable_values_only )
{
    using Transformed = std::map<int, int>;

    // Map::transform over the values, mutating them
    // Takes F(K,V) -> V i.e. the type of the return value must be V
    // "items" must be a named variable because parameter is non-const
    // (we will mutate it)
    Transformed items = { {1,1}, {2,2}, {3,3} };
    
    EXPECT_EQ( (Transformed{ {1,1}, {2,4}, {3,9} }),
      Map::transform( items, 
        []( int _, int v ){ return v*v; } ) );
}

TEST( Map, map_keys_and_values )
{
    using Mapped = std::map<int,double>;
    
    // Map::map over the pairs<key,value>
    // Takes F( pair<K,V> ) -> pair<K',V'> 
    // i.e. both key and value types can change
    auto lambda =
        []( const auto& kvp )
        {
            return std::make_pair( kvp.first + kvp.second,
                                   (double)kvp.second / (double)kvp.first );
        };
    
    // Map::map - new keys and values, not mutating the original collection, 
    // can be passed as unnamed temporary
    EXPECT_EQ( (Mapped{ {2,1}, {6,2} }),
      Map::map( std::map<int,int>{ {1,1}, {2,4} }, lambda ) );
}

TEST( Vector, filter )
{
    std::vector<int> items{ 1,2,3,4,5,4,3,2,1 };

    // Vector::filter will mutate the input collection
    EXPECT_EQ( (std::vector<int>{1,2,2,1}),
      Vector::filter( items,
        [](int i){ return 0 <= i && i <= 2; } ) );
    EXPECT_EQ( 4, items.size() );
}

TEST( Vector, filter_copy )
{
    std::vector<int> items{ 1,2,3,4,5,4,3,2,1 };
    auto untouched_size = items.size();
    
    // Vector::filter_copy creates a copy, so parameter 'items' is untouched
    EXPECT_EQ( (std::vector<int>{1,2,2,1}),
      Vector::filter_copy( items,
        [](int i){ return 0 <= i && i <= 2; } ) );
    EXPECT_EQ( untouched_size, items.size() );
}

TEST( Vector, fold )
{
    std::vector<int> items{ 1, 2, 3, 4, 5, -4, -6, -2, -1 };
    // Vector::fold takes F( A&, T ) -> void
    auto accumulate_squares = []( std::set<int>& acc, int i ){ acc.insert(i*i); };
    std::set<int> expected{1, 4, 9, 16, 25, 36};
    EXPECT_EQ( expected, 
      Vector::fold( items, accumulate_squares, std::set<int>{} ) );
}

TEST( Vector, sort )
{
    // Vector::sort mutates the input, hence input is non-const reference
    std::vector<int> items{1,2,1,3};
    EXPECT_EQ( (std::vector<int>{1,1,2,3}), 
      Vector::sort( items ) );
}

TEST( Vector, sort_copy )
{
    // Vector::sort_copy copies the input collection,
    // so collection parameter is const& (and can be an unnamed temporary)
    EXPECT_EQ( (std::vector<int>{1,1,2,3}), 
      Vector::sort_copy( std::vector<int>{1,2,1,3} ) );
}

TEST( Vector, map )
{
    // Vector::map takes F(T) -> U
    // Input collection is const and a new collection is returned
    EXPECT_EQ( (std::vector<double>{ 1.1, 2.1, 3.1 }),
      Vector::map( std::vector<int>{ 1,2,3 },
        []( int i ){ return (double)i + 0.1; } ) );
}

TEST( Vector, transform )
{
    // Vector::transform takes F(T) -> T
    // Input collection is mutated
    std::vector<int> items{ 1, 2, 3 };
    EXPECT_EQ( (std::vector<int>{ 2, 4, 6 }),
      Vector::transform( items, []( int i ){ return 2*i; } ) );
}

int main(int argc, char* argv[]) 
{    
    InitGoogleMock( &argc, argv );
    return RUN_ALL_TESTS();   
}

Notice that we can bypass immutability in C++, so whereas in F# Map::filter would always create a copy, it could be preferable in C++ to filter in-place. With that in mind, I’ve written both filter and filter_copy variations. There’s a similar dilemma for map operations – if you want free rein over the output types, then use Map::map or Vector::map. But if you want to transform the data in place (sticking to the existing types), use Map::transform or Vector::transform.

That covers the most popular functions for just Map and Vector, but it would be straight-forward to extend the library to cover List, Set and others. Similarly, I’d like to extend it to include higher-order functions like Choose, but I’ll need C++17’s std::optional for that.