How to serialize a COM object as a Base64 byte array

A component I’m working on needs to be able to serialize its data via JSON.  That’s fine for scalars, arrays and maps, but we also needed to be able to serialize COM objects (there’s a large legacy codebase underneath, much of which is deployed via COM).  It turns out ATL provides API functions for Base 64 encoding (which was a surprise to me and a number of my colleagues).  Here’s some code to make calling it easier (I’ve replaced the production code error handling with simple strings):

std::string to_base_64(int byte_length, const void* bytes  )
{
  if (bytes == 0)
    throw "ToBase64 called with zero bytes";
  
  ATL::CStringA base64;
  int base64_length = ATL::Base64EncodeGetRequiredLength(byte_length);

  if(!ATL::Base64Encode(static_cast<const BYTE*>(bytes), 
                        byte_length, 
                        base64.GetBufferSetLength(base64_length), 
                        &base64_length))
    throw "ATL::Base64Encode failed";

  base64.ReleaseBufferSetLength(base64_length);
  return static_cast<const char*>(CT2A(base64));
}

std::pair<size_t,std::unique_ptr<BYTE>> from_base_64( const std::string& encoded )
{
  // Predict length needed for decoding
  int decoded_length = Base64DecodeGetRequiredLength( encoded.size() );
  std::unique_ptr<BYTE> buffer( new BYTE[decoded_length] );

  // Actual decoded-length may be different to that predicted - method overwrites the value
  if (!Base64Decode(encoded.c_str(), encoded.size(), buffer.get(), &decoded_length))
  {
    throw "ATL::Base64Decode failed";
  }

  return std::make_pair( decoded_length, std::move(buffer) );
}

That leaves the requirement to turn a serializable COM object into a byte array.  The following code does the job provided the COM object implements IPersistStream:

std::pair<size_t, std::unique_ptr<BYTE>> object_to_bytes( IUnknown* pUnk )
{
  // Create an in-memory stream, HGlobalMem will be allocated internally
  HRESULT hr;
  CComPtr<IStream> pIStream;

  if (FAILED(hr = CreateStreamOnHGlobal(0, TRUE, &pIStream)))
    throw "Could not CreateStreamOnHGlobal";

  CComPtr<IPersistStream> pPersistStream;
  if (FAILED(hr = pUnk->QueryInterface(IID_IPersistStream, &pPersistStream ) )
    throw "Failed to QI to IPersistStream";

  CLSID clsid;
  if (FAILED(hr = pPersistStream->GetClassID(&clsid) )
    throw "Failed to get clsid";

  // Serialise CLSID into stream
  if (FAILED(hr = WriteClassStm( pIStream, clsid )))
    throw "WriteClassStm failed";

  // Serialise object into stream
  if (FAILED(hr = pPersistStream->Save( pIStream, TRUE ) )
    throw "Save object into IPersistStream failed";

  // Find out how big the stream is
  LARGE_INTEGER liStart = {0};
  if (FAILED(hr = pIStream->Seek( liStart, STREAM_SEEK_SET, NULL ) )
    throw "IPersistStream::Seek failed";

  STATSTG statstg;
  memset( &statstg, 0, sizeof(statstg) );

  if (FAILED(hr = pIStream->Stat( &statstg, STATFLAG_NONAME ) )
    throw "IPersistStream::Stat failed";

  // Populate a byte array from the stream
  std::unique_ptr<BYTE> buffer( new BYTE[statstg.cbSize.LowPart] );
  unsigned long size_read;
  if (FAILED(hr = pIStream->Read( (void*)buffer.get(), statstg.cbSize.LowPart, &size_read ) )
    throw "IPersistStream::Read failed";

  ASSERT( size_read == statstg.cbSize.LowPart );

  return std::make_pair( size_read, std::move(buffer) );
}

CComPtr<IUnknown> bytes_to_object( size_t length, void* bytes )
{
  HRESULT hr;
  CComPtr<IStream> pIStream;

  HGLOBAL handle = GlobalAlloc( GMEM_MOVEABLE | GMEM_NODISCARD, length );

  if (FAILED(hr = CreateStreamOnHGlobal(handle, TRUE, &pIStream)))
    throw "Could not CreateStreamOnHGlobal";

  unsigned long size_written;
  if (FAILED(hr = pIStream->Write( bytes, length, &size_written ) )
    throw "Failed to write to stream";

  // Reset the stream
  LARGE_INTEGER liStart = {0};

  if (FAILED(hr = pIStream->Seek( liStart, STREAM_SEEK_SET, NULL ) )
    throw "IPersistStream::Seek failed";

  // Serialise CLSID out of stream
  CLSID clsid;

  if (FAILED(hr = ReadClassStm( pIStream, &clsid )))
    throw "ReadClassStm failed";

  // Create object from the clsid and stream from IStream into the object, using IPersistStream
  CComPtr<IUnknown> pUnk;
  if (FAILED(hr = ::CoCreateInstance(clsid, NULL, CLSCTX_ALL, &pUnk ) )
    throw "Failed to CoCreate clsid";

  CComPtr<IPersistStream> pStream;
  if (FAILED(hr = pUnk->QueryInterface(IID_IPersistStream, &pStream ) )
    throw "Failed to QI to IPersistStream";

  if (FAILED(hr = pStream->Load( pIStream ) )
    throw "IPersistStream::Load failed";

  return pUnk;
}

Then it’s a matter of putting the methods together to achieve a simple API:

std::string to_base64_encoded_byte_array( IUnknown* pUnk )
{
  auto bytes = object_to_bytes( pUnk );
  return to_base_64( bytes.first, bytes.second.get() );
}

CComPtr<IUnknown> from_base64_encoded_byte_array( const std::string& encoded)
{
  auto bytes = from_base_64( encoded );
  return bytes_to_object( bytes.first, bytes.second.get() );
}

Leave a comment

Filed under C++, C++ Code, Programming

Book Review: The Stone Monkey, Jeffery Deaver

StoneMonkey
This is another in the Lincoln Rhyme series by Jeffery Deaver. Highlights of that series include The Bone Collector and The Coffin Dancer. This one falls short of that standard in my view, mainly because the whole plot falls down on a basic question – why is The Ghost (a sinister, violent figure from the Chinese underworld) going around trying to kill all the illegal immigrants that he was attempting to smuggle to America? Unusually, I was way ahead of the hero, Lincoln Rhyme, in questioning this motive – he only worked it out at the end of the book. Yet without knowing why this was going on, the book didn’t make sense to me.
20130418-193203.jpg

Leave a comment

Filed under Book Review

How to downcast a shared_ptr

A lot of the codebase I work on is deployed as COM servers. This means that it isn’t uncommon to pass an array of objects as an of pointers to base (i.e. IUnknown) and then downcast to the specific interface you want using QueryInterface.

I found myself in a similar situation with some more modern C++, where I had a std::vector<std::shared_ptr> and wanted to convert each pointer to a shared_ptr in order to call into some API method. The question is – if you have a std::shared_ptr<T1>, how to cast it to a std::shared_ptr<T2>  (for suitably related T1 and T2)?

This question was already asked on StackOverflow and the answers pointed me at std::dynamic_pointer_cast and std::static_pointer_cast, which are analogous to static_cast/dynamic_cast for raw pointers.

class B
{
public:
  virtual void Print(){ std::cout << "Object type B\n"; }
  virtual ~B(){};
};

class C : public B
{
public:
  virtual void Print(){ std::cout << "Object type C\n"; }
  void BehaveLikeC()  {std::cout << "Behave like C\n";}
};

class D : public B
{
public:
  virtual void Print(){ std::cout << "Object type D\n"; }
  void BehaveLikeD() { std::cout << "Behave like D\n"; }
};

void DoSomething( const std::shared_ptr<D>& d )
{
    d->BehaveLikeD();
}

int _tmain(int argc, _TCHAR* argv[])
{
  std::vector<std::shared_ptr<b>> elements;
  elements.push_back( std::make_shared<C>() );
  elements.push_back( std::make_shared<D>() );

  // Call base class method on all elements
  std::for_each( elements.begin(), elements.end(),
    []( const std::shared_ptr<b>& b ){ b->Print(); });

  // Call C behaviour on all C elements
  for ( size_t i = 0; i < elements.size(); ++i )
  {
    auto c = std::dynamic_pointer_cast( elements[i] );
    if (c ){ c->BehaveLikeC(); }

    auto d = std::dynamic_pointer_cast<D>( elements[i] );
    if (d){ DoSomething(d); }
  }

  return 0;
}

1 Comment

Filed under C++, C++ Code, Programming

Morgan Stanley HFT Overhaul

TheTradeNews.com reports that Morgan Stanley’s HFT software re-write is promising:

“This is the first time we’ve done a full re-write of our equity trading infrastructure – it’s brand new software running on brand new hardware, and we’ve specifically brought in expertise from low latency trading firms to achieve this”

Leave a comment

Filed under Finance, Programming

Clean Code cheat sheet

This Clean Code cheat sheet was posted on LinkedIn. I’m guessing it’s a summary of “Clean Code” by Robert Martin.

Leave a comment

Filed under Programming

Book Review: The Governor’s Wife, Mark Gimenez

GovernorsWife

Initially, I was concerned that this book was going to be a bit dull – it starts with a lot of scene setting about American politics and there’s much coverage of the history/geography of the border with Mexico. Having read the book, I totally forgive the author for spending the time to introduce the reader to these matters – it brings realism to the motivations and main characters: the Governor, the Mexican doctor and the bandit (El Diablo). Interesting that the title is The Governor’s Wife – she’s the focal point of the plot, but not necessarily involved in much of the story line.

It’s a terrific book. The action sequence at the end that brings together the three male protaganists is on a par with Lee Child’s Jack Reacher series, yet his handling of the journey taken by the Governor and his change of outlook is sensitively written. He gives great colour to even minor roles – if a film is made of this book, I bet many stars will covet the role of the political advisor to the Governor.

Definitely worth reading – I’m looking forward to another by Mark Gimenez.

FiveStars

Leave a comment

Filed under Book Review

Herb Sutter GotW91: Smart Pointer parameters

Herb Sutter’s guidelines on passing smart pointers as parameters include the following:

  • Copying smart pointers incurs two performance hits:
    • Cost of increment and decrement on the internally synchronised reference count
    • Scalability woes due to cache contention on the shared reference count
  • Passing a shared_ptr by value implies taking shared ownership. A copy is needed anyway, so incurring copying cost is fine.
  • Don’t use a const smart_ptr& parameter because it exposes the function to the caller’s lifetime management policy. Use a Widget* instead.

Guideline: Don’t pass a smart pointer as a function parameter unless you want to use or manipulate the smart pointer itself, such as to share or transfer ownership.

Guideline: Prefer passing objects by value, *, or &, not by smart pointer.

Leave a comment

Filed under C++, Programming

Herb Sutter GotW89: Smart Pointers

Herb Sutter’s GotW89 Smart Pointers post includes good reasons to use make_shared/make_unique instead of naked new. Avoiding memory fragmentation is one of those:

Separate allocation

auto sp1 = shared_ptr<Widget>{ new widget{} };
auto sp2 = sp1;

Separate allocation

Single allocation
If you use make_shared to allocate the object and the shared_ptr all in one go, then the implementation can fold them together in a single allocation.

auto sp1 = make_shared<Widget>();
auto sp2 = sp1;

Single allocation

Leave a comment

Filed under C++, Programming

Fake Tweet causes Dow Jones to drop 1%

TheTradeNew.com reports on a mini-crash last month:

Shortly after 1pm on Tuesday 23 April, a tweet from the verified Twitter account of US newswire Associated Press stated explosions at the White House had injured President Obama. Within minutes, the markets had bottomed out, with the Dow Jones Industrial Average sliding 145 points, or 1%, before rebalancing pre-drop, four minutes later.

These days, even Tweets and blog posts may be polled for market sensitive content in order to inform trading strategies:

Automated news reading services scan web-based news sources and social media for breaking news relevant to markets and specific securities. This can be fed via an application program interface (API), to a trading algorithm, which may act as a circuit breaker to stop trading or accelerate participation in a certain stock depending on the news.

Leave a comment

Filed under Finance

decltype and declval

Good examples of decltype and declval posted on reddit and TheNewCpp.com.

Leave a comment

Filed under C++, Programming