Since Google disabled third-party cookies on 1% of Chrome traffic in January, we’ve been busy testing the Protected Audience (PA) and Topics Privacy Sandbox APIs. One of the commitments we made was to share the results of our testing with our customers and partners, as well as with Google Chrome and the UK Competition and Markets Authority (CMA).
It’s clear that Privacy Sandbox is a complex solution and requires a significant shift to existing programmatic processes.
Our tests have proven that millions of Sandbox impressions can be purchased and delivered, however, many open questions and issues remain to be addressed. For instance, we don’t yet have data on attribution or measurement, which is one of the areas we hope to evaluate with partners in the future. Google’s delayed plans to fully deprecate third-party cookies offer much-needed time for the industry to continue testing and for Google to address some of the concerns raised to date.
Here are some of the key findings and challenges we’ve observed through our initial Privacy Sandbox testing period.
Initial findings from Privacy Sandbox testing
1. Limited testing scope: Our first phase of testing involved over 100 publishers, thousands of domains, and 10 DSPs. This was one of the larger-scale testing initiatives outside of Google and is a significant testament to our partners given the investment and onboarding time required. It’s important to point out that while Sandbox-enabled impressions have grown to a large enough volume where we can draw some conclusions from testing, overall impressions and spend are still quite limited.
2. Common topics observed: We’ve been collecting performance data around the Topics API, including the topics we see most often, and have released a new Topics reporting feature to share these insights within our UI. It’s still unclear how useful observed topics are to marketers to inform bidding or optimise campaigns, though we are encouraged by recent feedback from buyers who have begun testing the API.
3. Effect on CPMs: Most worrisome, throughout our testing, we’ve observed a 33% decline in CPMs on Sandbox-enabled impressions compared to impressions where a third-party cookie was present. This compares to 36% lower CPMs in impressions without both Sandbox and cookies. So, while the Sandbox APIs did help (+3%), they’re not closing the gap enough at the current scale, available feature set, and level of adoption to prevent significant revenue consequences for publishers.
With its current limitations, Privacy Sandbox may not be an effective solution yet for general use, or it may be too costly for tech companies to ready their implementations for general availability. There are major risks to publishers and the overall programmatic ecosystem that we must address to make it easier and more efficient to scale.
Publisher-focused issues and feature requests
Our Privacy Sandbox testing has shown there’s still a lot of work to do to refine the design, and we’ve highlighted some of the key issues we’ve shared with the Google Chrome team below. The team has been open to feedback and is engaging with companies across the industry. We’ll continue to test new features and raise issues to help progress Privacy Sandbox as a viable and scalable solution.
1. Format, channel, and transaction functionality
For Sandbox to be effective, it must support different ad formats. We understand Google is working on designs for many of these feature requests, and we’re eager to test them when released.
Currently, there isn’t viable support for video, which accounts for a meaningful amount of cookied ad spend. We’ve also advocated for support for multi-size requests to reduce the total number of auctions for an impression opportunity and help curb latency.
It’s been more than a few years since we entered “the year of mobile.” From our vantage, mobile testing across the industry has been limited, and we haven’t yet participated in testing Privacy Sandbox on Android. We’ll need more traction here to evaluate the design and effectiveness.
Finally, Sandbox’s current design doesn’t offer adequate support for deals. Programmatic transactions are increasingly executed through deals, so support is a must.
2. Latency
Latency has shown to be a significant issue, as Mediavine reported early on. We’re seeing 28% more latency for Privacy Sandbox auctions compared to others.
This latency is primarily due to the requirement for Google to be the topSeller in PA auctions, which in turn requires all non-Google bids to be processed by Google Ad Manager (GAM) before an auction proceeds. All auction participants must also wait for Google to finalise the winning bid.
Latency could be reduced by allowing other publishers and ad exchanges to compete directly in a client-side auction via Prebid. This would create a more level playing field and speed up web transactions significantly. Addressing issues like the lack of multi-size and multi-format auctions and optimising URL handling for globally distributed speed could further improve latency.
We recommend publishers monitor latency and ensure their Privacy Sandbox testing takes these risks into account. For example, this could mean restricting testing to less-trafficked pages.
3. TEE solutions and cloud competition
To protect privacy, Privacy Sandbox requires auctions to run on-device within the Chrome browser or through a trusted execution environment (TEE), which securely runs the auction server-side in the cloud. TEEs can provide more computing power than is available on a user’s device and reduce latency.
However, Google has produced quite sophisticated technical requirements to move auctions to a TEE, and as of now, only permits TEEs that run on AWS or Google Cloud Platform. We view this as a competitive concern as it levies an unfair potential tax on independent publishers, exchanges, and other ad tech platforms while boosting revenue for a few large cloud providers.
For instance, we run our own cloud-based TEE at Index which allows us to aggressively optimise our costs while driving significant efficiency gains for our customers. Supporting Privacy Sandbox would mandate that we move a large portion of our auction infrastructure to the public cloud at a much higher cost, which would ultimately mean higher costs for our customers and partners.
4. Publisher-controlled audiences
Today, only media buyers and DSPs can create interest groups, the generalised audience groups that the PA API is built around. This limits potential use cases and restricts publishers from monetising their own audiences effectively. If a publisher were to curate audience offerings, they’d have to provision that audience to each DSP they work with.
We propose a Sandbox design that gives publishers more control and allows them to provision interest groups to their sell-side partners, who can in turn provision them to one or many DSPs at once through deal IDs.
5. Related cookieless initiatives
We’re also concerned about fringe initiatives related to Sandbox, such as IP truncation. While these initiatives won’t replace a cookie, they do have implications for regulation compliance. For instance, it may be difficult to resolve users to geographic regions or states with the way IP truncation works in cookieless browsers today, making it harder to comply with local privacy regulations.
Looking ahead
Overall, we’re encouraged to see the growing number of companies from across the programmatic and open web ecosystems who have joined together to test such a significant solution in the span of a few months.
Privacy Sandbox is complicated and materially different than how programmatic has been executed to date. While it may not be perfect, it demonstrates potential over other proposed alternatives or no replacement at all. We’ve proven we can deliver millions of impressions, but haven’t been able to test all aspects and will need to continue to invest further to build out our infrastructure. We hope Google, the CMA, and others will consider testing feedback and evolve existing designs to help improve post-cookie addressability across the open web. We’re also eager to see more buy-side participation to help further accelerate testing and adoption.
The extended cookie deprecation timeline provides the industry with a needed buffer for readiness. We’ll continue to support and test Privacy Sandbox, and will keep you updated on our progress and learnings. If you have any questions or want to begin testing, please contact our Privacy Sandbox team.
For more support, be sure to check out some of the resources we’ve created, including how to get started with Privacy Sandbox, our Index Explains video series on the APIs, and our Protected Audiences demo and technical diagram.
