Like anything in business, it’s one thing to say it…it’s another thing to do it.
One of C3 Metrics’ clients shared his process playbook for attribution buy-in and organizational adoption.
In the sixth video, Matt DiAntonio of Carbonite shares how he tackled one particularly challenging marketing channel, saving $750k each year, and then growing revenue from that channel by 2x.
But results come with process. The below videos illustrate his process which he shared at the C3 Metrics Summit, which received the highest scores and accolades we’ve ever seen.
Play any of these seven short videos, which are accompanied by transcribed text:
1. VP Analytics, Matt DiAntonio (Belichick of the Playbook)
Spent 10 years at Silicon Valley’s Trend Micro which is a security vendor that vendor competed with Norton and Kaspersky and McAfee, working with a really bright CMO and a really stingy CFO.
That CFO and I had an accord where I would sit down and take him through the numbers and be data first, and then he would give me the money I wanted.
My CMO told me I was really good at analytics: grabbing all the data, telling the story, and being concise.
I thought the CMO was politely telling me I was a sh*tty marketer and was trying to get rid of me…but he asked me to build an analytics team for Trend Micro and I did that for four years.
At some point I had a bigger vision for what I wanted to do, which was to bring all of the data together to sit at the vortex of the company and tell cross-functional stories.
I love marketing analytics: but it doesn’t mean a heck of a lot if you can’t impact the top + bottom line…driving positive, accretive behavior.
2. Start Small: Tangible Wins With Creative
I think a lot of people ignore the upper funnel phase and jump right to the bottom part of the funnel. But for our business, cultivation between upper funnel to conversion is really important to a comprehensive marketing measurement program.
We call this our prospect to lead space where we have inquiries that are becoming MQLs that become revenue.
All of that happens with touches. The touches aren’t just Marketo, or Pardot, or whatever your marketing automation platform is… these are people.
We like to think of them as weird rows in the spreadsheet, but they’re people and they’re living and engaging in a world where your brand is present and every once in a while you’re going to see some piece of creative that pops with something meaningful and relevant for them to digest, and the brand becomes a trusted adviser.
In that conversation, you build some equity with stakeholders across the marketing team, and what our analytics team does is use C3 Metrics to be able to corral all of these disparate touches, then we drop them into a data Lake and we start looking for correlation.
We want to see if the creative assets (videos, touch points, etc) that are correlated to driving someone deeper into the success metrics that the CFO cares about.
When we find that correlation, we say “let’s juice the visibility” and it’s really for marketing guys to deploy that particular creative element. If the correlation holds, you start to build more assets like it, and you start to spend more marketing dollars with evidence based metrics.
3. Discovering Trends and Exceptions (Good + Bad)
I start every morning reviewing every single important trend in the business.
At Carbonite, I need to be able to go through an awful lot because it centers me and I’m doing something that’s paying attention to the business.
But if there’s an issue, it’s better to jump on that early.
So what we’ve done is take the C3 Metrics data from FTP, create a Tableau server dashboard which takes all the things that I care about…putting them in one line of sight with dates all aligned, so I can see where there’s correlation and exception. We have server side impressions, server side clicks, marketing spend by channel, and the same for C3 which measures consumer side actual views, consumer side clicks, and spend that’s in C3, and ultimately the attributed value that comes out of it at a high level.
I’m going to talk a little bit about why we built this so what you’re seeing here is a tableau dashboard where the ORAC sections of the funnel (Originator, Roster, Assist, Convert) are split out and that red line is the Mendoza line that tracks ROAS or AVSR…did I spend a dollar did I get more than a dollar back in terms of attributed value.
One of the important things you have to keep in mind as we go through and look at either a campaign centric view or in this case we’re looking at a medium specific view this is the affiliate program mediums and tactics play different roles in the process of converting.
You have to be really conscious of not clipping your R’s (Roster touchpoints) in favor of your O’s (Originators). You want to be able to look at it and see a consistent performance across these trend lines, and if something goes jumping way up or drops way down…we can look up here at spend and say we spent less and we’re getting less of this value.
But the good news is overall ROAS or AVSR is still hovering around 2.3 to 1.4. When we talk about this, we want to talk about outcomes.
Before C3 Metrics data was available to us…somebody with a suit and tie would show up at your desk and says. “what’s happening with this outcome something’s up something’s down?”
Then, in the old days, everybody would run over to Google Analytics or whatever web session stuff you’re looking at will never agree with C3 Metrics, because the lasts click lies!
When people say (based on Google Analytics) something’s really kicking ass…let’s just put that aside because we need a vehicle to be able to speak intelligently to outcomes though C3 data and the process of discovering trends and exceptions, and it can only happen quickly if done proactively.
Now, with C3 Metrics data plus this process, you begin to be able to speak intelligently to outcomes with minimal amount of effort. Ideally you have marketing managers and directors who are in there looking at these trends who can tell that story pretty actively so when the suit shows up at my door I give them the answer.
But this can be done with any outcome. For us, it’s a trial registration but this could be video views, or MQLs generated. It could be for any of these things appropriate to your business.
4. Ion Cannon = Integrity = Rowing In Same Direction
We put together a data integrity process to make sure we’re proactively looking at this stuff when you think about C3 as a tool for us.
We have Commission Junction, we have Marin doing our search, and we have an ad server for display; we look at impressions, clicks and spend.
Let me show you what that looks like and take you through this project we call Ion Cannon because everything in our world has a cheesy Star Wars name if anybody’s familiar with what ion cannon is extra credit to you.
What you’re seeing in the green bars are the spend in commission junction and in the blue bars is the spend in C3 Metrics.
That gray bar down at the bottom is the variance between the two of them, and it’s really important to note that we’re not looking for constant zero percent variance…we’re looking for a consistent variance.
For example, think about impressions. C3 Metrics measures impressions very differently than the way an ad server measures an impression (serve side versus consumer side).
We’re not trying to get Tigers to change stripes, we just want to see a consistent variance and I’ll show you what that looks like.
Here’s when the alignment and variance looks correct, but here’s where it looks off…so what’s going here?
This story is pretty interesting: it’s a rogue (fraudulent) vendor in Commission Junction who started botting their own site driving false conversions which looked glorious to Commission Junction (and they were paying them) until we showed them this data…saying, take a look at our run rate and take a look at this anomalous jump. This is BS
This is all fraud, and we showed Commission Junction this exact report: that green line coming back down is Commission Junction getting them out of the by cutting them completely.
They agreed that the rogue vendor was botting and they cut them. Without C3 Metrics, a typical marketing manager or director would look at the siloed, last click, not fraud-corrected silo (not C3), and say it’s converting well, but because we control the data in our tableau data, we scrubbed out the fraud that C3 caught, so we don’t have to constantly have and our trend goes back to a nice flat normal expected line and everybody has confidence in the data.
Now here’s a different situation which is much more common where we have this steady 0% variance on the left-hand side; green and blue bars hug and agree doing all sorts of good stuff. But all of a sudden this variance starts to pop up, and within 12 hours we had somebody on the operational side saying did you release something that didn’t have tagging?
What happened here right is a really good story for the business, and it shows how we have to talk about things as a tea. What occurred was that the B2B side of our business found good success for “click-to-call.” So think about how that would align in the data: they started turning on “click-to-call” which increases spend from Marin search, and increases clicks because click-to-call counts as a search click but there’s no outcome that we can tie that to in our world.
That side of our business is smiling and dialing, but before you jump to conclusions and say the ROAS or AVSR is are down and spend is up, you need to color this differently and everybody comes to the table prepared, and see this is good for the health of the business, and keep on going
5. Running After Walking (Very Advanced)
We built a tool called Viceroy that we’re really close to open sourcing for anybody who wants to be able to use it in conjunction with C3, and it’s a tool that anchors on a standard UTM value that assigns a pid into the tactic right we assign one individual ID to the tactic that creates the link that can get either pushed through an API to an ad server or it can get sourced by marketers so they can move unfettered without being slowed down.
I see this all the time: someone has to jump through three different offices, and it takes ten days to get a link.
I think it took fourteen days at the start of last year in Carbonite.
We put all that stuff in and we have people agree with it. It’s mutable, it’s not immutable…so we can go in and change it and then we push a publish button and everything that was published goes ouinto the data lake, and as these things start moving and shaking in the world around you can start to show reporting for a campaign just by following the process the right way.
You want how me all the pages that came from you know these PIDs that were under a campaign pivoted on its head. Then show me all the activity we’re seeing which had lead-generation intent and it starts to pivot the cube and works amazingly well wrapping the right context around your tool.
6. The Challenging Marketing Channel: $750,000 Savings + 2x Growth
When I got to Carbonite, there was a golden goose…a sacred cow, and that was Affiliate.
This marketing channel hid behind last touch attribution for so damn long, they were impossible to move away from the notion that failure could ever be touched. They had this group of people singing “cumbaya” with flawed last click metrics.
We all raised an eyebrow and said, “We’re data people, we’re not going to come in with emotional assertions of their ridiculous nature, we’re going to come in with some data that is going to prove to the team that we can do something better.”
About two and a half months into the process with C3 Metrics, and absolutely mind-bendingly terrible results of last click were exposed.
This golden goose that for so long had been the sacred cow was returning pennies on every dollar spent and we said, “Listen guys, I know this is going to be hard. We’re going to take the blanket away and make some really tough decisions.”
The initial response, “we can’t do that, because they’ve been loyal for so long!”
And we said, “Sorry, we’re cutting their commissions and we’re going to cut them drastically.”
Everybody came in and we see this really strong jump upward in terms of the ROAS or AVSR.
I don’t know how many people are familiar with what that number means, but it’s an Attributed Value to Spend Ratio.
For every dollar we spent, we’re getting back an accretive business value.
They declared victory, and tried to plant the flag and on to the next issue.
But I said, “No, no. This is an evolution.” We need to come back and say, “What’s the next lever to pull?”
We see total volume increasing and we said, “Let’s take that extra money and go make some bets.”
Here’s the big success story in the slide. It isn’t the big jump in ROAS or AVSR, it’s the fact that as a group we were willing to take some risks.
We had a really solid level of confidence in our listening device and worked through finding a new lever that was able to drive business value and ultimately if you were to look at this, this is the first couple of quarters of C3’s existence, the affiliate program now drives quite a bit more volume.
Now…it’s about 2x total volume.
The exact same level of pay in terms of the cash out and we have this up to about 2-2.5 at any point in time.
Really, really cool success story.
7. Three Final Secrets
There are three things I want to tell everybody in this space whenever I have the opportunity. The first one is, adopt an executive strategy. Take ownership of the conversation and be concise. What does that mean?
I’ve sat in enough board meetings and I can help you all through this. Don’t come to the table with a ream of paper. If I had a nickel for every time I saw the marketing team show up with every click-through and, stop. Just stop.
There’s three questions that you should answer at every one of these meetings. One is more important than the other.
What’s working? What’s not? Don’t run away from what’s not. Executives and leaders, they don’t want cheer leaders. They don’t want doom sayers, your job isn’t to nit pick every possible thing that sucks.
Your job is to say what’s working, what’s not and what are we doing about it?
That last point is the absolute most important piece.
When you start talking about what you’re doing about it and you keep an active log of it and say, “Listen, we had this conversation. As a team we’re driving change in the organization. Is it going to work? I don’t know, but we’re doing something that shows that we have ownership and accountability in this company.”
All of a sudden what you’re going to see is the entire C-Suite start showing up at these meetings. We saw it.
Literally, we have a Thursday meeting and our entire C-Suite sits in the back of the room because they want to hear what’s going to happen next. They want to see who’s playing ball and who’s owning it and that’s the most important part. If you leave this out, the value that you’re bringing to the organization starts to go.
I hear a lot a lot of people say “we have this awesome system there’s all sorts of good stuff coming in here but we don’t see any change happening.”
When was the last time you actually helped someone else own some of the chains so they look good. If I’m the one to ring the bell, other people are going to say, “f*ck you, I’m here to do a good job.”
Make the other people in your company successful, but don’t lose the paper trail and what we’re doing about the paper trail, who you’re empowering, and how it’s changing the company.
Connect the dots for everyone and show them the business value. That’s the piece that’s always missing.
Celebrate success and don’t take all the credit.
Holds quarterly and annual reviews. It may feels a little bit like you’re opening up your eye and putting a needle in there. It’s a lot of work and the reality in today’s world to be an impactful leader or analyst is to know your numbers inside out.
QBR’s quarterly business reviews don’t need to be a twenty page diatribe, just: what’s working, what’s not, and what are we doing about it?
Get that momentum started and then you become the Exec Team’s best friend because they need all the information you have: to say what’s working, what’s not, and what are we doing about it.
If you’ve taken every meeting that popped up onto your calendar, you’re not making a difference in your organization. You become a consensus member of these people who hold meetings because they need to have everybody there giving a thumbs-up for them to drive change. It’s the wrong way to do it.
I’m here to tell you it’s okay to say “not right now.” Don’t just say no, just not right now.
These are my priorities, here’s what we’re doing, and I need time to be able to do this in a way that’s meaningful right…and “not right now.” Your priorities aren’t mine: the company’s priorities should be what we’re looking at, and if there’s something that needs to change let’s do that, but make time to stop and think.
Schedule think time. Put think time on your calendar for these C3 tools whether you’re using customized dashboards like ours or the standard C3 dashboards.
If you put the work into building the right mindset which takes effort, and then you don’t do anything about it… you become one of the problems (which is I’m running around doing so much I can’t figure out what I should be doing and that’s the biggest problem).
If you can’t say what it is you should be doing and making a little bit of an impact on that on a daily basis you’re not helping your company.
We don’t need more people eating more snacks in the middle of meetings, we need people to make an impact.