A Novel Approach to Data Quality in Consumer Insights

Priscilla McKinney:Hello and welcome to Ponderings from the Perch, the Little Bird Marketing Company podcast. I'm Priscilla McKinney. I am the CEO and mama bird here. And I just love going out to market research and finding interesting people. I've got a doozy for you.

We're going to have maybe a little bit of a spicy. Henry and I agreed before that we would be trying to keep it a little bit more jalapeno and not quite ghost pepper spicy show today. But I think we've got a little bit of a controversy going on. I think we have some interesting point of views today and we're going to talk about it.

Let me tell you who I have here with me. I have Joey Maddox. He is the chief strategy officer over at Verisoul and I've Henry Legard and he is the co-founder and CEO over there. So they did, I'm going to tell you how this podcast even came about first is because they did a very funny video on LinkedIn and y'all know how much of an influencer I am on LinkedIn and how I like it when people come and actually be real humans on LinkedIn. And I loved the message and there was just a genuine excitement and you guys were a little bit nerdy, which also was very, you know, appealing.

Henry Legard:It's stage. I can assure you, we're not nerdy at all. It was purely stage for audience effect.

Priscilla McKinney:Okay, I love it, love it, it was fun. So I reached out to Joey and I'm like, you know what, strategy officer to strategy officer, I wanna hear what happened behind the scenes here, but we're gonna have a really interesting conversation. So let's kick off since I've got multiple people on, I'm gonna kind of like round robin us a little bit to get the juice out of this conversation, but it's gonna be super fun.

Joey, tell me a little bit about just like how that little commercial came up. And then we're going to come back to Henry about the origin story and the vision for Verisoul, but tell us about that funny little clip and I'll go ahead and link it in the show notes also.

Henry Legard:Yeah, the CRO announcement video, is that the one? Yeah, so with Verisoul, we always try to do something a little bit different in terms of our marketing. Like we wanna stand out, we don't wanna just be the same old company that you've seen a thousand times.

And so for the video, we had probably a hundred different ideas. Some were pretty crazy, some were really expensive, some were really cheap. And we ultimately landed on kind of breaking the fourth wall and just coming in and looking at the camera directly and trying to say, hey, look. You know, we are Verisoul. Here's exactly what we do. Have it be fun, but also very informative. And to your point, a little bit nerdy.

Priscilla McKinney:Yeah, loved it. Loved it. Okay. So now a lot of people don't know what Verisoul is. So Henry, this is back to you. Tell us a little bit about the origin story and what the vision is. Because it is interesting now as it's entering market research, but it existed long before market research. So tell us that story.

Henry Legard:Yeah, absolutely. Thanks, Priscilla, A, for having us on. Happy to be a bird or worm, whichever is the right... Great. You're the mom of bird and you were feeding your guests good foods. Maybe we're the... I don't know what it is. Super happy to be on and thanks for having us.

Priscilla McKinney:Bird, bird. The worm gets eaten, Henry. We don't want to be the worm.

Henry Legard:As you mentioned, I'm the CEO and one of the co-founders here at Verisoul. Prior to starting the company actually worked at an identity and fraud company. And then with the onset of AI, just started to realize how big of an issue trust online broadly is and will continue to be.

So as you start to think about kind of the fabric of online interactions between businesses and consumers, it relies on trust. Businesses trust that you're a real human responding with real insights that they want for their business. Businesses like social platforms trust that you're a good user providing your actual point of view and not kind of like a paid fake account trying to subvert governments or political ideas, things like that. Payments companies trust that you're a real actor sending real money to real people around the internet.

And that problem has trust has continued to erode with the onset of AI. And so we thought with given our background, mine from an identity fraud company, our chief product officer and co-founder from Facebook's and Meta's user risk team, and then our CTO from Capital One in the banking space thought we had a really good team to build a category defining company in helping businesses detect fraud, fake accounts, bots, AI agents, all of the nefarious actors. And I'm super, yeah, sorry, go ahead.

Priscilla McKinney:No, and Joey you obviously thought this was a great idea because you were not only the number one employee but you were an angel investor before that so this this this sounded like a good idea to you and as opposed to some people who are friends with co-founders say that's a good idea. Yeah, but will you invest? That's an important question. How good of an idea do you think this is?

Henry Legard:Totally. Totally. And I actually, we were friends, but we also worked at Bain & Company together. And so I had seen Henry in a work setting. I'd seen him in a school setting and also in a friend setting.

And really, you know, I actually worked investing after Bain & Company had a PE fund called Serent Capital that only invests in B2B software businesses. And so, you know, I had good pattern recognition for what makes a great business. And really looking at Verisoul, I mean, what I saw, I trusted the team, the incredible backgrounds. It was obviously a necessary problem, growing problem with a lot of tailwinds, given AI, and it's a broad problem, so it's a huge market.

Really, the reason of joining full-time, which was actually, so I made the angel investment and then joined two years later, the reason I joined full-time is I actually got to see all the updates as an investor and see they are so consistently beating products, not just in market research, but also broader identity products, digital fingerprinting solutions, bot prevention, broader software platforms that serve payments, e-commerce, a bunch of other verticals. And so I thought, okay, if this product with a small team is winning on accuracy, and it's just a better product than these large, well-funded incumbents, they're gonna take off. And so that's why I made the leap and joined full-time.

Priscilla McKinney:I love it. Okay, so let's get to the spice here. This is where if you tuned in and you wanted to, you know, have the, the, you know, the knockdown drag out, this is where it's going to start.

So Henry, tell us a little bit about, you know, the foray and the decision to come into market research. And then we're going to get even spicier with what have people in market research been doing in terms of data quality and fraud prevention and then what's different here. Like, what was the journey over into market research? How did it get identified as a possibility?

Henry Legard:Yeah, so Joey and I actually both worked at Bain and Company together and Bain is one of the largest buyers of research on the planet. And so we ran surveys and studies there. And this is back in 2017. And we started to see firsthand as a consultant the data quality issues.

And we didn't realize at the time, we didn't even have the concept of fraud on surveys at all back then, but we started to see weird things happening in kind of data quality. And fast forward to last year, beginning of the year, we had essentially no customers in market research at all. But what we started to hear and remember is just how big of a challenge it was.

And so we decided let's test this industry, ended up getting kind of one or two early customers who were pivotal in understanding the incentive model, which I think is one of the things we'll dive into, Priscilla, but the incentive model, which creates weird disincentives for data quality. And so we realized we identified it as a super interesting industry and then have just taken off. We have over 50 customers. Many of the large providers today trust us. And we win over 95 % of the deals that we go head to head against the current competitors. I think we've lost like two.

Priscilla McKinney:Right, right. Okay.

Henry Legard:And just to add on to that, I mean, when I was at Bain, we literally had to throw out a survey that, you know, we had a three week diligence for a PE client. They're very demanding. The survey workstream is one of four. Obviously, it's critical.

And at the end of two weeks, we were reviewing the data. We had one more week to cut it all and put it in slides and none of the data made sense. And so we ended up having to toss the survey. Of course, our client's upset. They were trying to make a call on a hundred million plus dollar investment into a company and they don't have a survey because all the data was junk.

And so, Henry and I were talking through, we did a full market mapping exercise of all these different verticals that we could potentially try to enter. And market research was always up and to the right of large market and also large problem in it.

Priscilla McKinney:Yeah. Okay, let's talk about that large problem. Okay. So let's kind of leave Verisoul for a minute. Which one of you wants to kind of tackle and tell me what your opinion was when you look from the outside, whether it was at Bain or then over at Verisoul, and you looked out to the data quality landscape, like who are the players who and how they were going about solving it.

Give us that general idea of how they were attacking it and then give me the, but this is actually not necessarily the only other way to go like go the opposite, but this is a far more complimentary way to go about the fraud problem.

Henry Legard:Yeah, so there's two things I'll hit on here. One is the fact that I would say people were using some naive approaches with respect to checking for, you know, whether the user is human using like trap questions and honey pots and things like that, which are super easy for bots and sophisticated fraud farms to get around.

And then the second thing is just most interestingly, I think the incentive set up today, right? One of the big challenges of the industry is that the incentive at every layer is to pass as many respondents through as possible, right? So data quality is critical in so far as you're still able to pass through a lot of respondents.

And so downstream people would trust, I'm already paying this provider to give me high quality data, but their incentive is actually to pass as much traffic as possible. And so, you know, it's kind of the Charlie Munger quote of show me the incentive and I'll show you, or show me the, yeah, show me the incentive and I'll show you the behavior. And you start to get this industry fervor around data quality because there's an incentive challenge and everyone is talking about it, but not actually doing that much or doing naive approaches that were not, you know, sophisticated enough to catch high grade fraud.

Priscilla McKinney:Right. And it has morphed over time, know, fraudsters got more sophisticated data quality checks and fraud checks got more sophisticated, but they were always behind. And maybe the nature is that the way that is set up is that they will always be behind. I mean, that seems like it's a structural issue.

So, Joey, help me understand then what the different approaches that Verisoul brings to the table. And really, I think kind of, you know, it's the mic drop in the room of wait, why? You know, it's like what I would think of like the upside down, you know, ketchup bottle. Like it's embarrassing how long this took, you know, the human race to come up with this, right? Like, why?

Henry Legard:Totally. Yeah, it's a great question. I mean, inside of market research, you can kind of think about preventing fraud in two different lenses. You have respondent integrity. So that's the person taking the survey. And then you have response integrity.

I think historically, a lot of market research is focused more on the response. Hey, does this look like a good response? Is it in the right language? Is it relevant to the question? Are they consistent across different questions or different surveys? And the issue with AI is it's gotten much easier to pretend to be an expert in the space that you don't know at all.

And so historically, I think it was easier to clean out data. I mean, at Bain, we would have those kind of trap questions, and we would do those types of checks and have a number of survey flags, and if they cross more than three flags, they're out of the survey, things like that. But now it's a lot harder to do that. And so you have to look a lot more at the respondent.

And what's difficult about that and kind of where Verisoul takes a really unique approach, I mean, Verisoul is solely focused on the respondent. We're making sure that it's a real person in the right geolocation. They haven't taken a survey before. They're not using automation. Like it is exactly who you want to target for the survey.

And the way that we do that is by a series of active network checks and active device checks that essentially are trying to uncover immutable signals. So if you're a fraudster, you can't just change your behavior and change your answer to a question and suddenly you got blocked the first time, but you're in the second time. The idea is these are signals and things that you cannot change about your connection type or about your device. And it just makes fraud very expensive and difficult to actually commit.

The other thing I'll add Priscilla is just that, to your point around, do you always, are you always behind? And I think the answer is no. I think it's just that a lot of people in research are focused on research only solutions rather than solutions that see cross industry fraud.

So one of the things about Verisoul is we work with 12 different industry types. So we're seeing fraud of all levels of sophistication. Several of our customers are actually large AI companies that are defending against Chinese AI regime like fraudsters, creating fake accounts to scrape US frontier AI models to train their models. So it's a national security and it's a state level actor of sophistication.

And I think the other thing we've seen in the research space is this kind of really quick cycles of data quality providers where they get customers and then they get purchased, they're research only and then they stop investing in the latest generation of fraud signals. And so that's how they get behind is they stop investing, they get complacent, they get purchased and therefore are just not focused on staying ahead.

One thing I'll add to that, Henry mentioned the different industries that we serve. There is a huge benefit, not just in terms of the level of sophistication, but also in terms of what actually qualifies as fraud or not.

Priscilla McKinney:Hmm.

Henry Legard:I think inside of market research, one of the biggest challenges, it's hard to have truly disposition data of, hey, is this person real or are they fake? In other industries, it's more black and white. You can look at the number of support tickets to figure out how many false positives you have. You can look at chargebacks to see how much fraud is being committed. There's more of a scoreboard and we can use that data to label huge data sets, train our models and be even more accurate.

Priscilla McKinney:You you brought that up in one of our pre-calls, Joey, and I just kind of walked away going, I have never heard that before. And it was when you mentioned there about, you know, companies that are coming to you and there are their accounts, that they're trying to verify if they're real people. And one of those ways is I just hadn't thought about it is does this account create a support ticket? Because a real person has needs and create support tickets. Right?

And so of course, you know, of course that behavior can be faked or whatever, but you glossed over something, Joey, when you, that I want to come back to, you said something about trying to find out if someone had taken the survey before. And that is something that is one of those low brow things, but very important, whereas Henry just brought us to like the highest brow, you know, of like, you know, it's just a matter of national security. Don't worry about it. You know, we're just going to see if we have supremacy in the world any longer, if we can get these accounts verified.

But catching duplicate accounts in the survey world is very powerful, especially when we've moved to really a big aggregator model where no one sample provider really has enough for a global survey. And so a little bit of this, a little bit of that, a little bit of this and so you might come up to your sample number that you need, but maybe 30 % of them are duplicates. They're taking your same survey on this platform as they were on this one.

And this is also another way that data gets skewed. And I think it's about really understanding it's not always because people are committing fraud. There's also just to the Charlie Munger quote, show me the incentive, I'll show you the behavior. People are like, well, I don't know why I was able to take this same survey over here and here. It's not my business. Someone wants to pay me twice. Cool. I'm going to go do it. And it is the, that is the world we live in. And so you glossed right over it, but I found it to be really almost a worth highlighting and also an interesting layer of what's going on.

Henry Legard:Totally, yeah, and kind of as part of that, I think a lot of the challenges with accuracy have actually been tied to this multi-accounting or duplicate survey piece. There are kind of two historical approaches that companies have used that don't work very well. Number one is the very simple browser-based cookie. Hey, have you taken this survey before on this exact same browser? Any fraudster can get around that, it's very easy.

The second is what a lot of other providers will do is IP de-duplication. They'll say, if we've seen this IP before, we're gonna block the second user that we see from the same IP. And there are a lot of issues with both of those approaches. The first one, you have a lot of fraud that's able to get around that.

The second one, you have a lot of false positives because if it's a broad network, let's say you're running a hospital study, you're probably gonna have multiple respondents from the same hospital IP. It's a huge IP. You could have thousands of different providers that are on that same IP.

And so what Verisoul does is pretty unique. We're collecting so many different signals on the device, the browser, the network, everything. We can use all of that to do a probability-based clustering model. So it's a completely different approach. It's not just a yes or a no, a binary decision. It's probability-based. And what's nice about that is it reduces false positives. And as fraud starts to scale, it becomes very easy to catch these massive clusters of fraud.

Priscilla McKinney:I'm just going to put a little finer point on this and I don't want to go down this rabbit hole, but just I do like being very specific. Just for example, you know, when you're trying to get a B2B audience, which is obviously people know a lot harder to get in market research, you know, a lot of those more sophisticated respondents do use VPNs. You know, we are concerned about our, our privacy and so we're smart and we use a VPN so that people can't track us. I mean, you know, my husband's had identity fraud. He doesn't get online without a VPN, you know.

Henry Legard:All right.

Priscilla McKinney:But that doesn't mean he's not a legitimate user. And we run into a lot of problems because of that. And so it's not really understanding why people would use these kinds of things in the real world. And so I love that point. It puts like a, it's not relying on any one data signal. It's coming together and saying, look, there's a lot going on here and we need to make a lot better sense of it.

Okay, we've kind of a little bit at market research, but let's get some positive stuff to come back around. You guys have come in and you've won accounts at an alarming rate and that's wonderful, but you've also...

Henry Legard:Alarming. We don't want to call it alarming. Maybe for the other competitors.

Priscilla McKinney:Yeah, yeah, I would say. I mean, that's all I do is competitive landscape work. So that's what I would call it is alarming. But tell me about some of the successes and the partnerships and some of the things that you see going on in the industry that are important and complementary to, you know, how you feel that you're helping to advance this idea of protecting businesses.

Henry Legard:Yeah, absolutely. So what I can say is that the customers we work with care really deeply about data quality. And so I know I mentioned the incentive problem earlier, which I think is an industry, like an industry wide problem, but is not necessarily specific to every company. So the companies that do choose to work with us take data quality super seriously.

In many cases, they're testing five different solutions because they want to figure out which one works. In some cases, they might even use two different products. For example, we have a partnership with Sentry Cloud Research. They're a pre-screener where they'll look at, know, do you say yes all the time, things like that that we don't really cover. So some companies will pay for two different solutions.

And that we're starting to see, and Joey and I have actually done a huge body of research on this, but a lot of the larger players have a lot more resources and therefore can devote more of an internal team to high quality. And so I think a lot of people get worried about aggregators and exchanges and things like that. But in reality, they have a lot of resource and they put focus on data quality.

And so, you know, generally our clients are sophisticated, they're testing many solutions and they care really deeply about quality.

Priscilla McKinney:I love that example. I did work for a time for cloud research and I worked on the Sentry marketing messaging. So it's fun to see like this idea of what they did was a lot of it was behavioral science attached to it and understanding a little bit deeper of the behavior, you know, and helping tie that to technology. So you constantly see a little bit of a move forward, a little bit of move forward, but I love this idea that you're kind of coming at it and saying, look, it's not just one solution.

It is, you you have to really look at your particular problem, what you're trying to do and decide how many checks do you need? How, what, what will be the magic semblance for what you're doing? Are you working on national security? Are you trying to sell more lipstick for the Kardashians? You know, like there's gotta be a level that is acceptable for, you know, getting the best quality of data that makes sense in that moment. So I love that, but I want to give a little bit of a free form right here.

So I don't usually do this on my show and I just want everybody to understand. They haven't paid to have this spot, but I really find this to be interesting in this industry. And I think if you're in market research in any way, shape or form, you need to listen to this conversation. You need to, you know, you need to rethink not only where you're getting your sample from, but how it is getting to you.

And you know what where where is our world going and I know that we feel this personally because what is the bane of our existence now is my gosh I got to my phone to sign in and then it's sending me a text and then it's you know verified this device and then use my fingerprint on my laptop and I mean just personally we're feeling it right and so we need to think about that if we're feeling that personally on a couple of accounts these companies are ahead of the curve. What do we need to be doing in terms of checking our users and seeing if they're legitimate?

So what I want to do is just give both of you a chance. Wave your magic wand. What do you want to happen as you move into this market research world? What are your thoughts about it? As much as you want to share. We'll start with you, Joey.

Henry Legard:Wow, that is a big platform, big question. I think one of the things that we've talked about before, Priscilla, is really around reducing false positives because there are so many people that are, they wanna contribute to the market research space, they wanna voice their opinion, and they'll be going in to try to take a survey, and suddenly they're going to cash out and it's saying, hey, we think you're a fake person. And they're a real person who's actually trying to contribute to the industry.

And next thing you know, they're going to look for other gigs and other ways to make money outside of the industry. And you start to lose a lot of those high quality people that are taking a bunch of surveys. And so to me, I think one of the crusades that we've had at Verisoul is how do you block as much fraud as possible with essentially zero false positives?

Kind of like we talked about before with the different industries, we have a benefit from working outside of market research. Some of our other customers have rates like 0.1 % false positive rates, 0.5 % false positive rates in these PLG SaaS companies where they're looking at support tickets and people will reach out if they get blocked, right? Cause they're all real people just trying to sign up for a platform. And so that's been really helpful to train our models.

And that is to me the focus of the industry as a whole is we need to block as much fraud as possible, but we can't just go willy nilly blocking a bunch of people. Oh, we're not sure about this person, block them. You need to have certainty when you're choosing to block someone to make sure that everyone has a good experience and the industry continues to do well.

Priscilla McKinney:I love this approach. Lisa Wilding Brown, the CEO of Innovate MR is famed for having said that we need to be thinking about that respondent. They are a non-renewable resource, right? We have to quit thinking that we can treat them badly and then they'll just keep wanting to participate in market research.

And I think that your approach is very proactive and very positive. And instead it says, look, these are resources that we should be taking care of, not attacking. I think that's a really different thought process because people got very scared about survey farms. You know, cloud research was the first one to be like, well, we've got footage from, you know, an actual, my gosh, stuff is so scary to watch, right?

Henry Legard:Bye. I know, we are planning a trip to a few different bot farms later this year. So catch us in Nigeria and Bangladesh pretty soon.

Priscilla McKinney:I still don't understand why they let you in, anyway, that's a whole different podcast. Like why would they do this? Yeah. Show me the incentive. I'll show you the behavior. Right. Okay. Well, Henry, what's your response there? What do you want to see happen? What do you feel, you know, is your focus as you move into this, into this industry?

Henry Legard:It's. Yeah. It's funny because I was actually going to hit on the false positives. I it's a great one, Joey. No, no, no, it's great one to hit on and I'm glad we got to touch on that.

Priscilla McKinney:Sorry.

Henry Legard:One thing that is super interesting to me is the idea that with each pass of the respondent, you get less clarity on the identity or who that is or the veracity of that user such that when they get to the client, they have no clues there. Now, one of the things that's actually very interesting is many different people see these respondents and there's a lot of data on them, at least on the front end, panel level or app level.

And so I'm starting to think about what would kind of a passport or some sort of like identity protocol look like in market research where you can share more information, validated information about these users such that you can clarify, hey, this is a known identity with a long standing history of taking surveys. We feel very confident about it so that you can reduce those false positives, right?

Your husband, he uses a VPN. Maybe he takes surveys every day. He's got an extremely strong robust history of no fraud, really good responses. And then he goes and gets blocked for using a VPN. But if you have some sort of longitudinal data rather than just this session. I mean that's one of the things we do uniquely is we look at users over time and you can generate a much more robust perspective of that user.

And so how do we create a longitudinal perspective for every single respondent in real time is a really interesting question that no one solved and is something that we might be in a really good position to do. So who knows about the coming time.

Priscilla McKinney:I gotta say that, that like, it harkens back to something I overheard and I wish I could attribute it to the right person. But I was at Pure Spectrum's Connect. They had a, like, they have a yearly, you know, kind of their own internal meeting. And I was lucky enough to get invited to speak once and, and, and to attend. But it is this inside industry, but somebody there said something I thought was so provocative. And I, it has been just you know just rattling around in my brain for a while this idea that at some point, you know the individual so me Priscilla McKinney the the the entity that I am I get to have monetization agency over who I am like when when I am verified as a human and my identity I I need to be paid for that like I I need to own all the things that are around that own all the identity pieces. I know I'm just butchering this to death.

But I feel like where you're going with this is this idea of, you know, kind of what Joey said. And then into what you said is like treating the respondent, like, look, what we're doing is we're not just protecting businesses from fraud. We're protecting you from being able to monetize and truly own your identity and be able to do with it what you will tell people as much as you want, or keep as much private as you want. And this kind of idea is in the end respecting the individual.

Henry Legard:Right. It's a great point and it creates choice, right? Many users, many companies will avoid asking for additional information in effort to reduce friction, right? Consumers will often trade, they often just don't want friction, but they don't care as much about privacy as many people think in a lot of cases and oftentimes might choose additional monetization for providing much more certainty around their identity, right?

If I can validate that I'm a real human with an ID and a job and this type of thing, and I've got a passport that can be reused around the market research industry, and I can choose to share X, Y, Z things with X, Y, Z people, earn additional monetization, the business then gets much more clarity that it's a real user. There's a huge win-win that can exist that there's no protocol for today, but someone like us is in a really interesting and potentially good position to create something in that space.

Priscilla McKinney:Okay. I love this. Now I know I've got some people mad at me right now. So if you have a different opinion, that's fine. Reach out to me. That's what we're about at Ponderings from the Perch. I'm always looking to pull the curtain back and say, this is what's going on behind the scenes. And I want to be transparent about it.

I want to have people come on and say, look, we have a different approach. I think we've been thinking about this maybe too shallowly and there is something else, you know, to talk about here. So definitely. If you're mad right now, go ahead, email me. That's fine. And we'll keep going. But if you want to know more about Verisoul, you can go check them out at verisoul.ai. So you can find them and figure out, you know, maybe do some kind of an assessment. What are you doing? Do you think it's enough?

And as they've said, they have other partnerships and are interested in, you know, how much I love a good collab. So you can certainly reach out to them. Of course, they're going to be linked in the show notes. But if you have a different opinion, reach out to me, come get on the show and let's talk about, you know, what you think the other approach is.

I do think we all get better when we're honest about what we're doing. Of course, as Melanie Courtright is famed to say, we're not sharing trade secrets, but we are sharing with each other what we're seeing and what we're learning so that we can make this a better industry. So, Henry, Joey, thank you so much for your time.

Henry Legard:Thank you so much. Thank you. Really appreciate it.

Priscilla McKinney:And Joey, keep making some funny videos, okay? I like, let's, we're making LinkedIn better one post at a time.

Henry Legard:We'll be right back. We have to counter the AI sloth with some actual good human content.

Priscilla McKinney:For sure, for sure. Okay, well, from all the peeps here at Little Bird Marketing, have a great day and happy marketing.