Facebook invited me (and a few hundred others!) to a private event at their London offices to network and hear presentations from several of their engineers.
Allan Mertner – Welcome
Allan was previously at King (makers of Candy Crush) and joined just five months ago. He leads the Ads group in London – the largest group in Facebook London. The culture is to move fast and ship things, to make things better – at scale.
Damien Lefortier – Causal Modeling: Delivering Incremental Advertiser Value
Incrementality
A conversion is any event of interest happening on the advertiser’s side (subscribing, purchasing etc). Incrementality is a conversion caused by the ad that would not have happened otherwise. So reporting these incremental conversions is a goal for any advertising platform. But it’s hard to assess – how to measure people that would have discovered the advertiser anyway.
Attribution frameworks
Used as a proxy to incrementality for determining which conversions are considered for both reporting and delivering. Eg. 1-day or 7-day click-through (so record that the person saw the ad, then count it if they convert within some period if going to the advertiser’s site). Or 1-day View-through – the whole thing from showing the ad to converting within a day. But not all attributed conversions are incremental – may have happened anyway.
Delivering incremental value
First need to measure incremental conversions! Then you can assess if you’re improving your delivery. So you can measure random samples of people that are shown/not shown the ad, and see how many convert.
Causal modelling – model both the conversion probability given ad exposure and given non ad exposure. No need for attribution with this approach. E.g. 1 – Natural Experiments – use the natural randomisation in the system to get data. Use this to train a model – but needs correction for biases. E.g. 2 – Small randomised trials – collect data, but typically means less data to teach models.
A model can not only be used for measuring, but also predicting the impact of an ad campaign. It’s a hard, and interesting, machine learning problem. But it actually works – such as for Brand Awareness following ad campaigns.
Brian Rosenthal – How Facebook culture influenced our ad system evolution
Brian has been at Facebook for over 8 years and has worked on much of the Ads codebase.
Mission driven, monetisation business. Facebook don’t want you to see ads that waste your time and the advertiser’s money. This helps frame the problem as one of efficiency – which is appealing to engineers!
Stage 1: Launching Quickly
Ads started in 2004/2005 – it was initially a college site, so the adverts were related to colleges, storage and moving companies. Ads were integrated as part of the social media interface, not just as banner ads. They had an Advert creation interface, where you could add your ad – with identification of who is running it. Then in 2007, an auction where you could bid per click (where a higher bid increased the likelihood of it being seen). In 2009, you could finally edit your ads (previously, they were only created or bulk uploaded)
Stage 2: Embrace best practices
Figured out the right principles – unlike stage 1 (which was quick and dirty to launch), they really understood the problems. Fixed bugs due to database inconsistencies – instead, adopted modular, transactional units – taking 12 months to ensure consistency (half of that time writing tests). Enforced a strict API layer (so internal code also had to use the API as well as outside users). The auction functionality matured – multi-objective optimisation. You could now bid on all the actions you wanted.
Stage 3: Deeper lessons
Try things again, decouple concerns. In 2011, the world was moving to mobile, but ads didn’t work on mobile! They tested ads on mobile and it was a failure. Trying again with deeper knowledge helped to get it right. Decoupled concerns – an early design error was to have objectives and ad formats entangled. These were separated so that every format supported every objective and the code was independent. Dynamic Ads introduced – made the ad more suited to the person (‘customer segmentation’). Objective-based ad buying – do you want to increase adds to cart or checkouts?
Current problems
Scaling, fighting fraud, quality, malicious actors. Biggest thing has been to be adaptive, to change what they do to meet demand.
Ben Savage – Reducing Unintentional Clicks
Ben joined Facebook in 2013 – initially thinking it would be for a year or so, but has stayed because it’s a fun place to work!
Facebook has a long term view – they need their advertisers to be happy, getting return on investment and increasing their budget year-on-year. This enables Facebook to increase the sophistication of their offering – they used to bill by Clicks, now can model conversions.
Audience Network – the problem was that the value of a click was lower than that on Facebook news feed. They saw that the number of clicks reported was not well associated with the number of significant website sessions. Hypothesis – this poor conversion was due to unintentional clicks. This was frustrating for the users and reduced the value per click.
Tinder uses audience network for its ads. The click goes to Web View – so you can measure the time spent in the browser (before they go back to the launching app). At Facebook, Data Wins Arguments. If there are short sessions, it’s likely the click was unintentional. Could they remove the financial incentive to get these unintentional clicks?
Well, what if they stop billing advertisers for such clicks? Blogged that the company would stop billing for this, even though it reduced revenue, because it’s the right thing to do. As an engineer, that’s pretty rewarding. The hope is that advertisers would then re-design ads (such as removing active whitespace areas) because they wouldn’t count clicks for that anyway.
Wrap up
The panel answered questions from the audience. One development they would love to see is that, for users who allow mobile Facebook to track their location, associating showing an ad with the user physically visiting the store to make a purchase. Another feature that’s already available is “Why am I seeing this?” if you’re wondering why a particular ad was selected for you.
musingstudio 