Episode 86

Max Métral on Future-Proof Data Hiring: AI, Soft Skills & Industry Passion

TIM: We are live on the Objective Hiring Show with Max. Max, thanks so much for joining us.

MAX: Thank you for having me, Tim.

TIM: It is absolutely our pleasure. And Max, it'd be great if our audience could hear just a little bit more about yourself to kind of frame the discussion.

MAX: So my name is Max Métral. I'm French, if you can hear it from my accent. I'm a data analytics director Activision Business right now in gaming. And prior to this, I was working in the sports industry. So I worked for Formula One, from City Football Group, which is the holding company of Manchester City Football Club. So my experience is mainly around entertainment.

TIM: That's an awesome summary and a great place to start actually, because before this call, I was thinking about something. And yeah, the last three roles you've been in for the last decade, basically, have been for very iconic brands, beloved brands. And I was wondering how that affects the hiring process in any way. Like, I would have thought it would attract a certain type of person. Does it make it easier to get candidates? Does it create more noise? Like, I'd love to get your thoughts.

MAX: both of what you said, it does make it much easier. But it does attract a lot of candidates and not all of them always best suited for the roles. At F1, you put any job down, you get 500 applicants in like a day or two, not even. a bit less active visitors, but in similar numbers. So it gets very difficult. When trying to analyze CVs, I try not to get too bogged down into it, but it is quite challenging, and I think there is what I might call the serial applicants, people that just click, apply, apply, apply, and especially when there is passion involved, like in gaming or sports, a lot of people want to get into the industry or into the dream company, and they apply for any job. data analytics or any other. And unfortunately it shows, and I don't think it maximizes their chances, but they think in terms of volume, not a bad thing to think of, but it actually doesn't really work. So it does make a lot of noise and it's very challenging when going through CVs for sure.

TIM: What about when you're thinking of the candidates and evaluating them? Does it play a factor at all? Their demonstrated passion for, for gaming, for football. For, for example, back in your Man City days, if a Man United fan had applied, would that be like, Oh, I don't think so, mate.

MAX: So, fandom doesn't, technically. Especially when I was in the city, I had many United fans working there Arsenal and any other club, so that was definitely not a factor. However, it does show, passion shows interest and interest leads to curiosity, which I believe is extremely important. So having someone in Data Analyst role for a game that knows the game and that plays the game is definitely a positive and will help in a day to day job. That shouldn't be taken also at, at face value instead of, what I mean is, you also have people in interviews that spend 30 minutes telling you how much they love the brand. That is time taken away for setting themselves as analysts or as candidates. So it's, it's, I think it's the balance. It does, it is definitely positive and I think if you have a passion for sports gaming, you need to show it. But you shouldn't overlook you setting yourself in the process. And, but it's definitely positive and it is something that I am personally looking for, for sure.

TIM: Yeah, it must surely make it easier to be engaged. And to really understand the kind of story behind the numbers. If you've actually been a user yourself of the product. If you have an inherent interest, it must make everything so much easier to understand. I would have thought.

MAX: Absolutely, absolutely. And it's, it's not, again, it's not a mandatory thing. I'm not saying if you don't. If you don't play the game, you can't work for the company, right? It's not the case. A lot of people who work here are not active super gamers. But it does definitely help ramp up the process and it does make things easier, especially when you're looking at the data. If you see like a weird spike, but you know exactly what happened yesterday in the last update in the game that is definitely making things much easier and faster. So yeah, totally agree with you.

TIM: Thinking about it now, it's almost like you've got the domain knowledge, you're a bit ahead of the curve on the onboarding. You're not coming in on day zero, wondering what on earth does this company even do? If you've been playing in a World of Warcraft for 30 years, then you don't need to be told the basics.

MAX: Yes, I i think you, you, you point towards an interesting point here, which is the more complex the game, the more useful it is to have prior knowledge. and by complex, I mean World of Warcraft is definitely on the more complex side in terms of how the game plays, different versions and then the different activities within the game. So if you listen to World of Warcraft, I would say it does. It adds a bit more value to be a World of Warcraft player than if you go for more mainstream games like Call of Duty or Overwatch, FPS multiplayer games. And again, we have had analysts that haven't had prior knowledge of playing the game or they didn't sell themselves as a hardcore gamers. And they got the job as well, so I'm not trying to say that this is the only thing we're looking for, but it does definitely help.

TIM: I'm wondering if you've had any candidates on the opposite end of the spectrum in, in any of these roles. So either at City Football Group or, or Formula One or Or in your current role that have come in and gone, you know what, I can't stand football. I just can't stand sports in general. I can't stand watching cars go around a track. Like, it's not me, but I'm bloody good at numbers. Like, would there be any benefit to that? Like a pure outsider who's like, almost approaching it with a fresh set of eyes and an almost a disinterest could almost become a benefit in some weird way. Would you consider that or, or does the lack of domain knowledge outweigh any pro they might have?

MAX: It can benefit especially so you use the different examples. I think in gaming, especially for what we're doing, it is more useful than what I've done in sports because in sports, I was focused on, there's two different type of analytics in sports. For those who may not know is this sports analytics side, which is improving the performance on the pitch on the track and really trying to. On the sports performance and there's a business scientific side and I was definitely on the on the lattice of business wise Which means that theoretically you don't really have to know who is winning the grand prize to do a good job at analyzing tv rights Or or database or upsell Potentials or lead scoring or churn scoring for, for the, the F1 TV product, for instance. So there is a bit more of an upside on the gaming side because you analyze gaming data. Whereas on the sports side, the business data, you could argue it's, it's selling tickets for events or selling tickets for shows. So you don't, it, can be useful, but I see a better added benefit on the gaming side because it does get quite complex on the gaming side.

TIM: I'm fascinated now because like I've recently started playing a little bit of games here and there and I was a full addict when I was a kid. So I, I was looking through the list of, of the games in the portfolio. I'm like, Oh, this has given me cool nineties memories, like playing the original Diablo. I'm like, Oh, that was so cool. And. They were before the days of data science or analytics, really, I imagine 30 years ago there wasn't much going on in terms of analytics in the company. What, what kind of things do you analyze? Like, what, what are the main insights you're looking at when you're looking at the kind of, do you call it game analytics? Is that what it is called? does it work?

MAX: So similarly to, to, to sports the different ways to do analytics in gaming and the way is usually being split actually on the business side is what we call the studios, which is the. Beautiful craft minded people that make the games very creative. And you have usually what we call the publishers, which is the evil people that are trying to make money out of it. And fortunately I work on the latter as well. And so there are different ways. Obviously there are, we have dedicated teams in the U S I'm based in London that work on the games themselves where they're really focusing on making the game experience better reducing lead time or like improving things if there's some. Problems in the game or such that's not what me and my team are working on. I'm on the publisher side So the type of questions we were trying to answer is how can we make better decisions? When we do a launch for a game, which market should we invest more or less of? How should we invest? How long should we start from pre order? Should we wait for the launch? Is there some mediums that work better than others? There's a lot on the marketing side, but also from the in game monetization side. Some games now are free to play with the likes of Fortnite and others. Brings in a whole different type of analytics that what the previous as you're mentioning Diablo Previous model was which is you set a box for for with a disk with a specific price and they don't really Analyze the data or how people play because you're focusing on seeing the next box for the next year or in how many years So the free to play and the monetization element, which we call microtransaction, also brings a lot of different questions and analysis that we're trying to do. why is our battle pass doing better or worse in X market versus Y market? Is there anything we can do about it? is there a specific product that we should push more or less of, based on what we see from the data? And that's very much on the publishing side.

TIM: You've been in the role for a few years now. I'm wondering, was there anything that you can remember looking back when you started? That was like, Oh, really different or insightful or like an unusual aspect of how people look at gaming from an analytics perspective that, that kind of stands out in your mind now,

MAX: Not specifically, on the other way, I was very, even though. I was expecting it, but I was very surprised by the very similar commonalities between sports and gaming. because some people say, Oh, why did you leave F1 for gaming? Like some people don't make the link and they don't see how similar it is. And from a data perspective, the similarity is. Gaming used to be selling boxes, so B2B, you sell to a retailer, they sell the disc to players that you don't know, that you have no way to keep tracking, the era before like always on internet connections and others. And same for sports, the main `revenue for sports, if I think of F1, is selling to companies, so selling TV rights, selling sponsorship, selling the races to promoters, and you don't collect data on the end users. With these B2B deals in both environments, sports and gaming, as you have more direct to consumer, you start collecting a lot more data, which then you can leverage for many other things. In F1, when I was there, we launched the F1 TV, which is an OTT platform, which is selling directly to end consumers, the ability to watch formula one, instead of going through whichever is the main TV broadcaster in your market. And that generated a lot of, a lot of data, a lot of insights that you can action on versus setting to. B2B business and same for gaming with the age of everything digital. You can monetize and you can do micro transaction as well. We start knowing our players on a one to one basis versus the old days where we had no clue who was playing and how to contact them. And the similarities are actually really astonishing.

TIM: and you've been able to let, excuse me, been able to leverage those similarities, I guess, to then do your job better and, and apply the learnings you've had in the past.

MAX: Absolutely. Especially in terms of change management how people think about data, how people think about how we can use data and how we can leverage it. It's a similar, similar process, even though gaming is studied earlier, I would say, than in sports. It is a similar mindset process.

TIM: You'd mentioned earlier that part of the pro and con of working for these big brands is you get a lot of applicants. For any role, as you said of, let's, let's call it varied quality in the applicants. Let's put it nicely. And from what I'm hearing from a lot of people in Europe and North America. From other industries as well. In the last six months in particular, there's been just an inundation of candidate applications coming in. Are you also feeling this? Like, is it even more than what you would have experienced a year ago? And if so, do you have any sense of why there's such a high volume?

MAX: I, I wish I had first hand data. My team is small enough, or is small, that I don't really have a big sample size of recruiting. So I can't really compare young years. I wish I had to be a team. Don't get me wrong. But I can tell you, but I'm not surprised, but I don't have first hand data to back that up. Let's put it that way.

TIM: One thing that I've heard might be behind this general trend, the kind of market trend is Okay, so maybe suppressed hiring conditions. Maybe there's some good people who are looking for a job. Fair enough. That's kind of typical macro trend. But then also apparently a lot of candidates would be using an LLM to create their CV, which is fair enough, but also apply to many roles. So, you know, you mentioned that I can't remember the phrase you use, like the, the, the button clickers.

MAX: yeah,

TIM: applicants. That's a nice way of putting it. Yeah, the serial applicants have been emboldened. And yeah, with this kind of AI technology, which could allow them to apply on mass which is causing this as far as I know. And I'm wondering how we're gonna get out of this. Like if the average job 800 applicants can't really manually read all of them. You'd be there all day. Where do you think this is going to go? If, if just the number of applicants per role is now, I don't know, 10 times what it was a year ago because technology has allowed us to apply so easily.

MAX: I guess we're going to have to train our own LLMs to, to look for what we want in the CVs. But have been through 500 CVs myself before, but if it doubles or triples, that's going to be a lot more challenging for sure. That's the only way I can see that or, or, and that's, don't think there's any other way, any way to trick LLMs so that you have like ways to trick them into knowing that if it's an application, then you reject it and you make it very clear, but they're going to get smarter and smarter. So whatever you put, they're going to find a way to bypass, but if it's just you show the only company doing this and not the others. they, until they catch on, then maybe people are not going to bother and you will get less of these. But it's going to be a challenge. I think we're going to, our HR, if we have any HR analytics team actually don't know. I think we could be started, starting on doing LLMs for reading CVs for sure.

TIM: Yeah, that's where I assume it will go. I feel like one of the challenges with this is going to be that. So let's say for argument's sake, that a lot of CVs are written with ChatGPT or at least augmented by ChatGPT And then you've got an LLM on the other side doing the screening to try to compare it to the job description or whatever. But if the LLM in the first place has written it to optimize for the GD, then every CV is going to look very good. Suddenly it's going to have a very high match score. Which I don't know how this ends up. What's the equilibrium in this scenario? thoughts?

MAX: you, you explain the doom scenario team. I hope that's not going to happen. It's a really tough one. I can tell you what I look for. Obviously now if, if people start training their llms this is what I'm saying, I'm screwed. But personally, I, I, I like to look for differences. I like to look for Extra curricular activities or passion projects that sets them apart, whatever the topic, honestly, if it's. Doing analysis on airlines or whatever because they're massive airline, plane fans or, I, I really don't know. But anything that shows this for me is very important because it shows curiosity, sneaking out of the box. And then I love asking questions in the TV process about this to see how the answer, if it's all true. Sometimes, you know, people like to emphasize a bit, what they've done. That's what I try to look for. It's not easy. It's not a home run because you have something you're going to get through the interview process. But, it is hopefully the type of things that you can't manufacture with LLMs, but maybe they will.

TIM: Yeah that seems like a fair approach. So, so it sounds like you'd be an advocate of having a hobbies section on your CV or something like that to help you stand out a little bit from the crowd?

MAX: 100%. I personally value it very much. Because if not, it's really The same, right? It's very hard to see any personality trait in the cv. Not

TIM: yes.

MAX: don't get me wrong, but if, if we can or if the candidate scan, I think that that is the negative value for sure.

TIM: Any other ways that a candidate could stand out in this market? So, you know, almost if you were a candidate now, you were looking at LinkedIn, you were saying, Oh my God, there's 1000 applicants for this role. Are you joking me? Do I have to now to apply to a thousand jobs to get one interview? Like what you know, I could imagine that thought process coming in. How else would you look to stand out from the crowd? Would you even apply to jobs through traditional channels anymore? Would you try another method? I mean, how would you think about the job search in this market?

MAX: I would say the work comes before that, so I think it's upstream of this, which is just applying, which is really understanding what is it I want, what am I, what am I good at, and how do I make sure that I match? the job description that people are looking for. That being said, I know that apparently, studies show that there is gender disparity, in that apparently when men see they fit 5 out of 10 bullet points, great,

TIM: let's go for

MAX: Where apparently female candidates, they need to fit maybe 8 or 9 out of 10 to feel comfortable applying. But I think it's really understanding, going through the introspection as to what I want to do, why do I want to do it and re understanding what is required for the role and for that, for that work. What I encourage candidates to do is reach out to industry experts. Some of them would never reply, but at least trying to ask questions and not you pass on my CV to the HR writer is I think a terrible way to do it, but more what is your job like? What do you like? What don't you like? What do you look for in candidates? Same questions you're asking me right now. How can I set myself apart? Do you mind having a look at my CV and seeing if it's something that you think is interesting? are the type of things I did myself. I remember being in an exchange program in the U. S. towards the end of my business school and having phone calls and towards the end of phone calls with industry experts. So I was looking for a job in data and sports. I was asking a question along the lines of, going to be in the job market in X many months. Based on what I told you, do you think my, I would fit the roles you have in your team, or do you think there's something missing that I should be working And you get some interesting answers. Some people say, well, actually, I don't see X type of experience on your CV, but based on what you tell me, it looks like you might have it, so try and maybe show it a bit better. Or people say, well, actually, I really need someone to have at least SQL Python, blah, blah, blah. So I, I'm tick box exercise. I'm looking at the software to make sure they're there. You could have very interesting feedback. And hopefully, having done that preemptive work, then candidates can really go for I'm always for quantity quality over quantity. so the total opposite of the NNM multi application thing. I'm not saying spend three hours doing your CV before applying for every single job. That is obviously not really scalable. But at least go for what really means something to you. I've had the experience where I select CVs for screening or for interview and people don't even reply. Maybe they've been lucky and they got a job in the meantime, all good for them and they don't want to bother applying. But my guess is sometimes people can apply to a job they don't really want and then they're like, well actually, you know, I can't be bothered to do that interview. So that's for even for them. I think I don't complain right when you're the hiring person You're the one having the decision. So I'm not saying I wish they didn't apply good for them. They got selected They didn't show up. It's fine But I think doing this work ahead of time for candidates will have them be more efficient in what they want in how how can? They get it.

TIM: Yeah, that's such a great shout and ties into a conversation I was having this morning with someone actually who was describing their onboarding process for when they had a new joiner. And they'd normally spend a couple of hours really digging into the details. Of the candidates history saying, well, like, what are the kinds of projects you like? What were the ones you didn't like? Why was that almost like a meditative exercise? And then we were then chatting, thinking about, oh, hang on, that would be really helpful if candidates did that before they even applied in the first place. So to your point, you're doing that work further upstream to really think about these things. And I'm wondering whether an LLM might be a good partner in this. Like you can start talking to ChatGPT and say, Hey, like, this is what I'm thinking. What would you ask me? What should I be thinking about? so much. Like you don't have to necessarily do this alone. You can kind of partner up with AI.

MAX: Absolutely and also further down the line. I think you will help candidates ask the right questions in the interview process Let me pull up a random example. If someone is a very, very competent data analyst, but potentially extremely shy and doesn't really enjoy Part talking to people and actually doesn't want to do it and just is enjoying having a very big problem to solve from a data perspective and not talking to people. Being self aware of this will help in the interview process, asking the right questions to you. To what extent is the analyst required to talk to the key business stakeholders? Is this the analyst who has to present everything and talk to everyone? For some people, it will empower them and make them excited. For some, it will make them stressed. I think having the candidates better understanding what they want and what they were they looking for is helpful for everyone in the process.

TIM: I'm interested to hear a little bit more about the process you were describing before. So you mentioned, so you're at the end of business school or near the end of business school and you're kind of contemplating your next moves and then you managed to have some conversations with some industry leaders like what, how did you go about teeing those up? What were the mechanics of that and why did you do that? And, and almost What, what made you a self reflective person that would even want that in the first place, I guess, is also what I'm trying to get to.

MAX: So as I think a lot of people, I was very interested by the sports industry. I had no idea what jobs I could be doing and even less what would I enjoy. So I went to business school thinking, well, it's going to open up doors like that. See what I like, what I don't like. I knew it was very good at math and something I enjoyed, but I never really made the link throughout the very class, normal classes, strategy, accounting, finance, and others. One of them was a marketing professor that was teaching marketing with a lot of data, like a, I don't make sales to like some clustering, segmentation, and like survey information. I love that, that I found that to be amazing. I had a really good rate. And then the professor told us that he's opening a specialized program. And so that summer, and I was starting in September, I think we finished the classes in like May or June, I was thinking over the summer, should I do it? Because why not? And that's when I started reaching out to people in the sports industry that work in data analytics, which at the time was mostly in the US and trying to understand, is it new, is it not new, do they like, don't they like? And I managed to get some phone calls. So literally sending LinkedIn messages all at the time. Trying to find the email addresses. First name dot last name at website. com. So I did that a lot. So that was definitely more of a volume play, even though everything was personalized and when I managed to get like a quick phone call or a coffee, if I was around Trying to do my homework to get prepared that are trying to be also useful to them. that had a remember an interaction in person for a coffee with someone working for the Washington Wizard Reserves and Capitals. So the NBA and and NHL holding company which is called MSC, Sports Entertainment. that really gave me the the. motivation to go into is doing data and sports. I came back at school. So, okay, cool. I'm just a specialization program. And over the next 18 months, I kept doing this not on and off, but contacting people not to get jobs just to ask questions and to understand where is this going, especially as the U. S. Was far more advanced than Europe and cities. I believe today Europe is just lagging by five years, I would say. Yeah. And so that gave me the understanding of what people are looking for. And as I was doing these fun goals towards the late end of my business school, having done specialization in data analytics, that's how I got my first job. I had a call with someone who was working for Citi and I did ask that question about, Oh, I'm going to be in the job market next month. What do you think about my profile? the person replied, well, actually, we have an open role. We'd like to apply. I applied, I got the role, I changed my plane tickets. From the U. S. of going back to France to go to, to the U. K. And that's how it started. I'm, I'm not sure it's really scalable to be honest with you. I don't think, I think I got extremely lucky to be the right person at the right time. But you have to give yourself the options, right? If you don't do that, it cannot happen. I'm not saying if you do it, it will happen. But that helped me a lot.

TIM: Speaking of sports analytics, actually given I've, I've got you here I'd love to pick your brains a little bit more one, one area and one kind of analogy. So if you have a. seen the book or seen the movie Moneyball by any chance about baseball analytics? Probably one of your favorites, I imagine, actually.

MAX: Yep.

TIM: And I've been thinking a lot about that in terms of recruitment for just professional roles, like for data roles or technical roles. And for me, I feel like the way Baseball recruitment was completely overhauled from this very intuition based gut feel approach, you know, how good looking is the guy's girlfriend that dictates how confident they are, this kind of stuff towards some actual numbers and facts, and that improvement that is now filtered through into football finally in the last five years, it seems as though clubs like Brighton and Liverpool seem to be doing this quite well. Why don't companies do this for professional roles? Is there a limiting factors? The lack of data or is it just a matter of time before anyone is hired in a data driven method?

MAX: You know what, as much as I love Moneyball, and I never thought about it that way. I think the analogy is exceptionally accurate in the sense that the story in Moneyball is that they were optimizing the salary they were offering to players to their own KPIs. KPIs they're looking for, as you clearly stated, were just not correlated with success, or not enough. And so smart guys found better metrics to, to, to To find undervalued assets in the market. And the true goal here is what metrics should I look for if I'm trying to get to go for success. So what are the metrics I can optimize for to get great candidates. you measure this is a whole different topic. I wouldn't even know how to start. but the analogy leads up also to the next stage which is the next phase of Moneyball. So that was in 2001. The story of the book was published in 2002 or 2003 I think. Is really about finding the best undervalued assets in the market, which is we can apply to mark to to recruiting phase of money ball as a lot of players a lot of teams It's not applicable yet in the business sphere, but as people get better the the The the gain you get of doing this, it gets smaller and smaller as people get better at it. So the next step was, how can I make my players progress better with the use of data and insights? And, and at the same starting point, how can I have someone that get, that get better than in a different club, because I'm, I'm, I'm a different franchise, because I'm developing them better. I think that analogy is also very, very interesting in the business side, which I never thought about it, but you saying this made me thought about it. Yeah, maybe that's something that people should invest time and effort into. I'd be very keen to see what people come up with, but I agree with you. I think it's very challenging and maybe the data collection I'm not really expert in HR analytics, but I think that could be interesting. But I think the, the, the What I find to be the limiting factor is how do you measure success is if someone is employed

TIM: yeah.

MAX: at five years, is that a measure of success? Yes or no? If someone is so good that he leaves that he or she leaves after two years, it's a great hire, right? Someone did a great job. I think that would, for me, really very difficult thing in trying to train any model is what is the KPI you optimize against? I wouldn't know, to be honest.

TIM: Yeah, I completely agree. I think that is one of the fundamental challenges compared to how many goals to this player score, how many assisted they have, was that correct pass ratio, all that's available and fairly obvious. But well, that then makes me think actually that we should be measuring what people do in their job better than we currently are. I know this is something we obsessed over for years here, was measuring exactly what our engineering team were doing, down to a very precise detail, like every ticket would have story points, we'd know exactly who was the tester, the code reviewer, who did this, who did that, so we know exactly what everyone contributed. In a way that I think is atypical, but then we, we, we defined our success as who, who did the most, who produced the most products, maybe there's better ways or better metrics than that, but I think that was pretty reasonable. So then we're optimizing our process around who can do the most of that. Whereas a lot of jobs, maybe ones that are less black and white than engineering, maybe it's harder to measure what people do. Maybe leadership roles in particular, if you're not an individual contributor.

MAX: Yeah, and I think to follow on to the analogy with Moneyball, it was really taking advantage of, of biases because people had a lot of bias in, in the way they think and the way they do things. And I think we are riddled with biases in the way we evaluate our own team. We have a proximity bias with people we know, depending on if you have a much bigger team with different levels, you're, you're biased towards people you're close to and, and Over overcoming these, these biases, I think is very challenging in, in, in that specific scenario of analyzing performance. And super harsh. I mean, every company has some kind of calibration process where people are trying to argue, how do we put people on a, on the distribution line distribution curve for, for ratings, it's nightmare because of these. And I think, no I, I'm yet to find a company that is easy that, or that is finding easy to do. I don't know how to make it easy.

TIM: Yeah, that's a challenge. I personally feel like, so the way hiring is typically done Is a lot closer to the end of the gut feel, let's say, pre 2000 pre money ball environment than it is to the post 2000 environment, and there's a lot of quick wins of things that could be measured. Like you can measure people's skills. You can measure their intelligence. You can measure their personality. Maybe the other thing actually to say that to your point around biases is finding a way to not allow those biases to be introduced in the first place. So it's, it's like we need to collect more information and not collect more noise, maybe. So classic one would be the CV, like, I don't need to know their name. I don't need to know where they're from. These are not relevant factors at that stage. Of course, eventually they need to know their name, but maybe we need to think more about your moving the noise.

MAX: Totally agree. And, and also making it like for like, right? If and when possible, I try to give a task to applicants depending on the job. I try to give the questions ahead of time so that there's preparation time and trying to make sure everyone has the same preparation time, the same presentation time so that you can make assessments if you're in a good mood one day, bad mood another day, and you don't ask the same questions. How on hell are you supposed to evaluate two different candidates? And on top of this, and again in the spirit of trying to measure, the way I personally analyze the candidates is, I see the hard skills as a demivariable. There is a or I don't have the exact but there is a level of hard skills that I think is good enough. So I treat that as a 0 1 variable. If someone is really like super, super advanced and I find it useful, maybe it gets a bonus point. But I feel like our skills is quite easy to teach. And for someone to get up to skill or improve on it, I have a soft skills that are much harder. And this is where I have a more linear continuous variable where, and again, hard to measure and hard to put into numbers. But this is how I mentally visualize it. So that I don't over, overweigh too much the hard skills because I find to be the performance in the role is less about hard skills and is much more about soft skills. And in my specific case scenario, I think it's because I work in industries where which i don't have a big team, or at least I've never had a massive team. So I need to have people that can be checkable traits, that can talk, present, and do the work and do the analysis. and so the soft skills for me is extremely important. And that's something I very much look forward to more in the interviews as it's very hard to see in the CVs. But I try to put a lot of emphasis on it and try to be very, to be to be listening to what people are saying, to try and find some cues if there is any. Of these, it's not perfect, I'm not perfect in any way, but I try to.

TIM: Thinking about that now what's going to happen with AI? So let's say we get to, I don't know, a year in the future. And Claude, Chachipiti, what have you, might be able to produce SQL or Python code with 100 percent accuracy. It's possible. And so then for an analyst or data scientist who's maybe considers that a part of their almost identity to be a coder What are they going to do? Do they have to kind of have a reframing about the value they bring to the business? Would you almost start to look for different candidates? Or what does the soft skills become maybe even more important than it is at the moment? How are you thinking about this?

MAX: I think, I think you're on point here. Whatever can be done by a computer without any help, which I think is a very key point, will be, will be disrupted. But the most challenging thing, and I have, I think I have an example for it, is what do you do once you get the data? And that, I think, so far, maybe, I hope I would be wrong, but I haven't seen any LLM model doing it very well, which is Once you have your prediction model about leads, about, about churn for a specific item that is more of a recurring revenue generation. So you have a probability for x many users between 0 and 1 of how much, how they're going to churn. What is it you do with it? How do you implement this? How do you sell this to the business? Do I do A, B tests or do and I think these are the things that are required to be more, to be business savvy. I think as of now, it's not something that I don't see being, being disrupted by AI. Maybe it will, maybe my job would be on the line, I don't know. But I was definitely trying to focus on where, what is business additive. And in itself, doing a SQL query adds nothing to anyone. Data doesn't speak for itself. It's also the one reason I really don't like the phrase being data driven. As much as there was in F1, people loved using this. doesn't drive, data doesn't do anything. There's someone that's piloting this, and it's not data. However, I do like being data informed. So having the right piece of information and insight when making decisions. also being aware of our biases. For some decisions, sometimes guts can be a good thing. Sometimes someone that has 20 year experience in a specific domain, even if they can't explain why, might be someone to listen to. And being also recognizing these type of things. And so I would definitely try to lean more as to value additive. In your job and in your role and trying to find whatever it is. can be done with AI, yet.

TIM: Yeah, I it's funny. I spoke to someone a few weeks ago who was doing some initial exploratory work on could they Like are there any AI products out there that would allow them to answer BI queries? So could they automate the BI step? Their initial finding was like, yeah, not yet. We're still a little way off that a couple of products that they tried sat in the dashboarding layer itself rather than in like the warehouse having row level data access. So they thought, well, it's kind of flawed. They can't possibly get access to all the data they would need to answer the questions. They felt like it was still a ways off, and the BI analysts of the world should not worry that much. I'm not sure if you've thought about this.

MAX: I, I think it is definitely possible. I've heard about Stripe doing it, I don't have any first hand experience, I don't know anyone there, so I don't know if it's true, but I've been told that they do it, that they have models that anyone can go and ask, give me x number based on the y, z variables and apparently it extracts correctly. But to be fair, that I think opens up a whole Kind of warm, in my view, because more data to people is not always and sometimes the solution, and I think it's more dangerous than anything. It also depends on what are the capabilities of the people you have in the business. But I'll be more scared for more people to have more data than the opposite, whereas we can be the guardians of how it's being used and what is it you do it for. Because, again, biases, right? Confirmation bias. If someone asks, JDPT, prove to me that so and so Belief that I have is true It's going to find a number to to to you know to confirm that claim and then convert conservation bias 101 We're in trouble and people are just going to use this to do whatever they want anyway so i'll be more worried if there's a model that enables to get bi directly i'd rather have my team use it only so we can be more efficient I trust us more in making sure that this is being used in the right way again doesn't drive, so how do we make it drive business decision, drive value? And I think it will happen, for sure. Like, some people will be able to drop all the data in the model, spend a couple months training it, and then you can just tell it what to do, and you don't even have to use SQL anymore. Honestly, I'd love that. It'd make my life easier.

TIM: Yeah, I think your view of having it in the BI team and making them maybe five or ten times more productive rather than just, hey, As you say, hey, executive who's already made up their own mind. No, no, no. Don't use this to give it the answer you want it to give you. For the pitch deck on Thursday afternoon. No, no, no, no. Let's keep our feet on the ground. And I think, yeah, ChatGPT is very good or bad at that. Like, it pretty much tells you what you want to hear. You have to be quite brutal with it for it to be brutal to you. You know, it doesn't want to offend you. I'd say it's been quite polite.

MAX: agree, but I think the way it works is really, that's why it's called artificial intelligence, right? It's really replicating how we think, and that's how they make the leapfrog advancement in LLMs with transformers, is by mirroring how our mind works, and applying this to data to try and make sure. now it's really to words and sentences, but who's to say that they can do that with, with math and how we think and analyze data. think it's a big leap to, to find someone who's going to do it someday. But again, I'm more concerned about how it's being used and about that more business savvy, how to apply data in what business context. And this is where I think business and insights and data analytics people can focus the energy towards too. Because I, I see this as a lot more difficult to disrupt with AI. But again, who knows, maybe.

TIM: Is there any bit of the hiring process at the moment that you would just love AI to take over or that you see it's kind of like a quick win for AI to improve in some way?

MAX: I don't recruit often enough to feel like it's a burden. So once in a while I have someone in my team moving to another role or moving to do a different job. I'm happy to go through myself to also see like first hand raw data type of things, looking at the CVs. I would feel worried as of right now to give anything to a model that I'm less empowered and it's less my decision. but again, it comes from the fact that I don't do hiring. If I were to, I'm pretty sure we'd have a different, a different answer. But at the moment I'm happy once. once a year or whenever there's an open role in my team to go through 500 CDs. I scan through these very, through these very fast. So it's okay. But so I would say no, but again, based on small sample size.

TIM: What about like an interview co pilot, like someone sitting there helping you maybe take notes maybe helping provide feedback after the interview, even to you, to the candidates would something like that be helpful, do you think?

MAX: Yes, so then going into making requests for people that build great models. I'd love to have more of an image recognition thing. A facial cues or, or body language, I think is the most thing to do and, and potentially saying that when I ask a specific question, that person has a specific reaction that I didn't really see because all I was looking at my nose, I was taking notes that I'd be super keen on to see how it can enhance. I think if it's just doing things that I would do anyway, a bit less so again, because I don't that often, but if it's to give me things that I wouldn't see or that I wouldn't know, definitely I'd be interested.

TIM: One thing I've always thought about interviews is it's quite a cognitively complex thing to do because you're trying to Ask the person questions, listen to their answers, take notes, keep track of time, observe their body language, as you say, like all the nonverbal stuff as well. Like, there's a lot going on in that, so I wonder if even just having some, someone else, okay, something else there by your side even if they're doing ostensibly what you're doing at the same time, it might almost be just a good quick sanity check at the end of it to go, yeah, actually, I didn't notice that or. You picked this up, but I didn't. Like, in theory, maybe there could be like a nice a nice other interviewer there, just behind the scenes.

MAX: Absolutely. Yeah. And also on top of that, one thing that you didn't mention, but I personally do a lot is you have to sell the job too, right? you have to also show what is it they can, they can gain from it. You, you always need to pick interest of the applicants as well. I think it's a two way street. When you're an applicant, you also need to get the, whoever is doing the hiring process to sell you the job and to sell you the team and to sell you why is it good for you? And I think it's important for candidates not to forget this. And one thing that I, as much as it sounds simple, but I find quite correlated with success, not in a super high way, but it is correlated, is, you have any questions for me? Someone who does have any questions, usually it's not a good sign.

TIM: Yeah, I've found the same. I wonder why that is. Is it because it implies a lack of interest that they haven't thought about it? Does it, it's almost actually like when you speak to customers, you're trying to sell them something and their questions are very superficial or you get to the end and they have no questions. That is like there's 0 percent chance they're buying because they're not thinking at that depth of level. Is that, is that what's behind it? Do you think

MAX: I think it's because they don't project themselves into the role. Once someone projects themselves, it's like, oh, right? the office, or, you know, like, it could be any random question. It doesn't have a back end question to sell themselves, which some people do, which is not a bad thing, by the way. But it could be really, like, random stuff, like, oh, by the way, where exactly is the office? Like, hmm. like, if they project themselves, they're gonna ask these type of questions.

TIM: if you had the proverbial magic wand to like a genie in a bottle magically solve hiring and all of its problems what would you do? What would this wand do? How would you solve it?

MAX: that's a tough one. I think I'll try to make sure to surface the hidden gems. my, my fear is always of what if in the people that haven't selected for the interview there's one hidden gem that just didn't pop out for whatever reason. Yeah. and that, that kind of fear of missing out for me, I really don't like it. so that's definitely would be what I use the one for, because I'm sure there are lots of candidates that have very different very different, like, pro stories and history and they went from different things and Maybe I just overlook it without thinking too much in the CV, but there's like so much more to it that just doesn't pop out and that can't be translated into the CV. That's what I use it for because I think these candidates are the most interesting. And it's very hard to find, I don't think there's any blueprint to find them because by definition, they're different.

TIM: That's a great one. Yeah, I feel like our hiring process is often set up rightly. to reduce the chance of a false positive. But the false negatives we just miss forever. You never know who applied and you rejected. You would because no one ever does that analysis retrospectively. You never track. Oh, look, these 30 people ended up leading these teams at Google or whatever. There's none of that, maybe

MAX: exactly that. And you can tailor for this because then your model is crude. But I'd definitely, yeah, I'd love to find these.

TIM: Max, it's been a great conversation today. I think it's gone really smoothly and it's been very insightful for our audience to hear all your different experiences at lots of interesting companies as well and iconic brands. And we've heard a lot today, so thank you so much for sharing all this with our audience.

MAX: Appreciate it. Thank you so much, Tim.