Showing posts with label recruiting. Show all posts
Showing posts with label recruiting. Show all posts

Sunday, January 1, 2023

My advice to students looking for a job

I can't talk to you all

Every week, I get several contacts on LinkedIn from students looking for their first job in Data Science.  I'm very sympathetic, but I just don't have the time to speak to people individually. I do want to help though, which is why I put together this blog post. I'm going to tell you what hiring managers are looking for, the why behind the questions you might be asked, and what you can do to stand out as a candidate.

(Chiplanay, CC BY-SA 4.0, via Wikimedia Commons)

Baseline expectations

I don't expect new graduates to have industry knowledge and context, but I do expect them to have a basic toolkit of methods and to know when to apply them and when not to. In my view, the purpose of completing a postgraduate degree is to give someone that toolkit; I'm not going to hire a candidate who needs basic training when they've just completed a couple of years of study in the field.  

Having said that, I don't expect perfect answers, I've had people who've forgotten the word 'median' but could tell me what they mean and that's OK, everyone forgets things under pressure. I'm happy if people can tell me the principles even if they've forgotten the name of the method.

The goal of my interview process is to find candidates who'll be productive quickly. Your goal as a candidate is to convince me that you can get things done quickly, right, and well.

Classroom vs. the real world

In business, our problems are often poorly framed and ambiguous, it's very rare that problems look like classroom problems. I ask questions that are typical of the problems I see in the real world.

I usually start with simple questions about how you analyze a small data set with lots of outliers. It's a gift of a question; anyone who's done analysis themselves will have come across the problem and will know the answer, it's easy. Many candidates give the wrong answers, with some giving crazily complex answers. Of course, sales and other business data often have small sample sizes and large outliers. 

My follow-up is usually something more complex. I sometimes ask about detecting bias in a data set. This requires some knowledge of probability distributions, confidence intervals, and statistical testing. Bear in mind, I'm not asking people to do calculations, I'm asking them how they would approach the problem and the methods they would use (again, it's OK if they forget the names of the methods). This problem is relevant because we sometimes need to detect whether there's an effect or not, for example, can we really increase sales through sales training? Once again, some candidates flounder and have no idea how to approach the problem.

Testing knowledge and insight

I sometimes ask how you would tell if two large, normally distributed data sets are different. This could be an internet A/B test or something similar. Some candidates ask how large the data set is, which is a good question; the really good candidates tell me why their question is important and the basis of it. Most candidates tell me very confidently that they would use the Student-t test, which isn't right but is a sort-of OK answer. I've happily accepted an answer along the lines of "I can't remember the exact name of the test, but it's not the Student-t. The test is for larger data sets and involves the calculation of confidence intervals". I don't expect candidates to have word-perfect answers, but I do expect them to tell me something about the methods they would use.

Often in business, we have problems where the data isn't normally distributed so I expect candidates to have some knowledge of non-normal distributions. I've asked candidates about non-normal distributions and what kind of processes give them. Sadly, a number of candidates just look blank. I've even had candidates who've just finished Master's degrees in statistics tell me they've only ever studied the normal distribution.

I'm looking for people who have a method toolkit but also have an understanding of the why underneath, for example, knowing why the Student-t test isn't a great answer for a large data set.

Data science != machine learning

I've found most candidates are happy to talk about the machine learning methods they've used, but when I've asked candidates why they used the methods they did, there's often no good answer. XGBoost and random forest are quite different and a candidate really should have some understanding of when and why one method might be better than another.

I like to ask candidates under what circumstances machine learning is the wrong solution and there are lots of good answers to this question. Sadly, many candidates can't give any answer, and some candidates can't even tell me when machine learning is a good approach.

Machine learning solves real-world problems but has real-world issues too. I expect candidates with postgraduate degrees, especially those with PhDs, to know something about real-world social issues. There have been terrible cases where machine learning-based systems have given racist results. An OK candidate should know that things like this have happened, a good candidate should know why, and a great candidate should know what that implies for business. 

Of course, not every data science problem is a machine learning problem and I expect all candidates at all levels to appreciate that and to know what that means.

Do you really have the skills you claim?

This is an issue for candidates with several years of business experience, but recent graduates have to watch out for it too. I've found that many people really don't have the skills they claim to have or have wildly overstated their skill level. I've had candidates claim extensive experience in running and analyzing A/B tests but have no idea about statistical power and very limited idea of statistical testing. I've also had candidates massively overstate their Python and SQL skills. Don't apply for technical positions claiming technical skills you really don't have; you should have a realistic assessment of your skill level.

Confidence != ability

This is more of a problem with experienced candidates but I've seen it with soon-to-be graduates too. I've interviewed people who come across as very slick and very confident but who've given me wildly wrong answers with total confidence. This is a dangerous combination for the workplace. It's one thing to be wrong, but it's far far worse to be wrong and be convincing.

To be fair, sometimes overconfidence comes from a lack of experience and a lack of exposure to real business problems. I've spoken to some recent (first-degree) graduates who seemed to believe they had all the skills needed to work on advanced machine learning problems, but they had woefully inadequate analysis skills and a total lack of experience. It's a forgivable reason for over-confidence, but I don't want to be the person paying for their naïveté.

Don't Google the answer

Don't tell me you would Google how to solve the problem. That approach won't work. A Google search won't tell you that not all count data is Poisson distributed or why variable importance is a useful concept. Even worse, don't try and Google the answer during the interview.

Don't be rude, naive, or stupid

A receptionist once told me that an interview candidate had been rude to her. It was an instant no for that candidate at that point. If you think other people in the workplace are beneath you, I don't want you on my team.

I remember sitting in lecture theaters at the start of the semester where students would ask question after question about the exams and what would be in them. I've seen that same behavior at job interviews where some candidates have asked an endless stream of questions about the process and what they'll be asked at what stage. They haven't realized that the world of work is not the world of education. None of the people who asked questions like this made it passed the first round or two, they just weren't ready for the work place. It's fine to ask one or two questions about the process, but any more than that shows naivete. 

I was recruiting data scientists fresh out of Ph.D. programs and one candidate, who had zero industrial experience, kept asking how quickly they could be promoted to a management position. They didn't ask what we were looking for in a manager, or the size of the team, or who was on it, but they were very keen on telling me they expected to be managing people within a few months. Guess who we didn't hire?

Be prepared

Be prepared for technical questions at any stage of the interview process. You might even get technical questions during HR screening. If someone non-technical asks you a technical question, be careful to answer in a way they can understand. 

Don't be surprised if questions are ambiguously worded. Most business problems are unclear and you'll be expected to deal with poorly defined problems. It's fine to ask for more details or for clarification.

If you forget the name of a method, don't worry about it. Talk through the principle and show you understand the underlying ideas. It's important to show you understand what needs to be done.

If you can, talk about what you've done and why you did it. For example, if you've done a machine learning project, use that as an example, making sure you explain why you made the project choices you did. Work your experience into your answers and give examples as much as you can.

Be prepared to talk about any technical skills you claim or any area of expertise. Almost all the people interviewing you will be more senior than you and will have done some of the same things you have. They may well be experts.

If you haven't interviewed in many places, or if you're out of practice, role-playing interviews should help. Don't choose people who'll go easy on you, instead chose people who'll give you a tough interview, ideally tougher than real-world interviews.

Most interviews are now done over Zoom. Make sure your camera is on and you're in a good place to do the interview.

Stand out in a good way

When you answer technical questions, be sure to explain why. If you can't answer the question, tell me what you do know. Don't just tell me you would use the Wilson score interval, tell me why you would use it. Don't just tell me you don't know how to calculate sample sizes, tell me how you would go about finding out and what you know about the process.

I love it when a candidate has a GitHub page showing their work. It's fine if it's mostly class projects, in fact, that's what I would expect. But if you do have a GitHub page, make sure there's something substantive there and that your work is high quality. I always check GitHub pages and I've turned down candidates if what I found was poor.

Very, very few candidates have done this, but if you have a writing sample to show, that would be great. A course essay would do, but of course, a great one. You could have your sample on your GitHub page. A writing sample shows you can communicate clearly in written English, which is an essential skill.

If you're a good speaker, try and get a video of you speaking up on YouTube and link it from your GitHub page (don't do this if you're bad or just average). Of course, this shows your ability to speak well.

For heaven's sake, ask good questions. Every interviewer should leave you time to ask questions and you should have great questions to ask. Remember, asking good questions is expected

What hiring managers want

We need people who:

  • have the necessary skills
  • will be productive quickly
  • need a minimal amount of hand-holding
  • know the limitations of their knowledge and who we can trust not to make stupid mistakes
  • can thrive in an environment where problems are poorly defined
  • can work with the rest of the team.

Good luck.

Monday, May 3, 2021

How to hire well

How I've learned to hire

I’ve done a lot of hiring and I’ve learned what works and what doesn’t work to make a good hire (someone who performs well and stays). I’ve come to trust my judgment but only within the confines of a hiring process that covers my blind spots. Here’s a description of what I typically like to do, but bear in mind this is an amalgamation of processes from different employers.

To be clear: what I say in this blog post might not reflect current or previous hiring processes at my current or former employers. I'm presenting a mix of processes with the goal of giving you insight into one amalgamated hiring process and how one hiring manager thinks.

Principles - caution, excitement, 'no', and decency

The hiring process is fraught for both parties. We're both trying to decide if we want to spend extended amounts of time with each other. The hiring manager wants someone who will fit in, perform well, and will stay. The applicant wants to work in an environment that suits them and rewards them appropriately. No one enjoys the interviewing process and everyone wants to get what they want quickly. This suggests the first principle: caution. It's easy to make a mistake when the pressure is on and the thing that will save you is having a good process.

Once the process starts, I try and follow an ‘excite and select’ approach. I want to excite candidates by meeting the team and by the whole interview process and I want them to feel energized by what they experience. I then select from enthusiastic and excited candidates.

My default position is always ‘no’ at all stages. If I’m in doubt, I sleep on it and say ‘no’ the next day. On occasions, I’ve been under a great deal of pressure to make a hire, but this attitude has saved me from hiring the wrong person. Even in the US, unwinding a bad hiring decision is extremely painful, and in Europe, it can be almost impossible. It’s far better to be sure than take a risk. I’ve only changed my mind after a ‘no’ once, and that turned out to be a good decision that I stand by.

My next principle is being humane. The interview process is stressful and I want to treat candidates well and with respect at every stage. Even if they’ve been rejected, I want them to feel good about the process. Let's be honest, sometimes there's just a mismatch of skills - I've said no to some really great people.

(Interviews should be friendly and humane, not an interrogation or a stress test. Image credit: Noh Mun Duek, license: Creative Commons, source: Wikimedia Commons.)

The hiring process

The job ad

I like to think very carefully about the wording of the job ad. It has to excite and attract candidates, but it also has to be honest and clear about the job.

I've had a few candidates who've misunderstood the job and that's become clear at the screening interview. To stop this from happening, I've sometimes created a longer form job description I've sent to candidates we've selected for screening. The longer form description describes more about the role and provides some background about the company. Some candidates have withdrawn from the process after seeing the longer form description and that's OK - better for everyone to stop the process sooner if there's no match.

Resume selection

This is an art. Here are some of the factors I consider for technical positions.

  • A Github page is real plus. I check out the content.
  • Blog posts (personal or company) or content for marketing is a plus.
  • Mention of methods and languages. Huge shopping lists of languages are a bad sign. I also want to know what they've done with languages and methods.
  • Clear descriptions of what they've done, with a focus on the technical piece. I prefer straightforward language.
  • Training courses. Huge shopping lists are again a no for me.

I don't tend to select on the college someone went to but I know lots of organizations that do.

The screening interview

The first interview is a screening interview with me as the hiring manager. I do this via video call so I can get a sense of the person’s responses and their ability to interact. I always have a script for these calls and always follow the same process. I work out the areas I want to talk about and create the best questions I can to differentiate between candidates. The script gives me a more consistent (and fairer) way to compare candidates and also enables me to learn what works and what doesn’t. For example, if candidates find a question confusing, or everyone answers a question well, I can change the question. For behavioral questions, I ask for examples of the behavior and my technical questions are usually about experience. Here are some examples:

  • Can you give me an example of how you dealt with conflicting demands?
  • Can you tell me about a time you managed an underperforming employee?
  • What’s the largest program you’ve written?
  • What are the biggest limitations of Python?

These questions are launch points for deeper discussions.

The technical screen

Next comes a technical screening. Again, this must be the same for all candidates. It must be fair and allow for nervousness. 

I'm very careful about the technical questions that my interview team asks. I make sure that people are asked relevant questions that reveal the extent of their knowledge and skills. For example, if my team were interviewing someone for a machine learning position, I would ask about their use of key libraries (e.g. caret), but I wouldn't ask them about building SVMs or random forest models from scratch unless that's something they'd be doing.

Cultural fit and add

Finally, there are in-person interviews. I like to use teams of two where I can so two people can get a read. Any more than two and it starts to feel like an interrogation. Each team has a brief for the areas they want to probe and a list of questions they want to ask. 

Team selection is something of an art; I’ve known interviewers who are unable to say ‘no’ to any candidate, no matter how bad. If I have to include someone like this on the interview team, I’ll balance them with someone who can say no.  

I’ve heard of companies doing all-day interviews, but this seems like overkill to me and it stresses the interviewee; there’s a balance here between thoroughness and being human. For in-person interviews, I ensure that every team offers the candidate a drink or time out to visit the restroom. 

Where I can, I have the very last interview as a discussion with the candidate, asking them what went well and what went badly in the process. Sometimes candidates answer a question badly and use the discussion opportunity to better answer the question. Everyone makes mistakes and interviews are stressful, it seems like a good opportunity to offer the candidate a pause for reflection and an opportunity to correct errors.

I always look for the ability to work well with others and I value that over technical skills. A good technical person can always learn new technical skills, but it's very difficult to train someone not to be a jerk.

Decision making

Before we go to a decision, I find people the candidate may have interacted with who are not on the interview team. Many times, I’ve asked the receptionist how the candidate treated them. On one occasion, a candidate upset the receptionist so badly, they came to me and told me what had happened. It was an instant ‘no’ from that point.

To decide hire or no hire, I gather the interview teams together and we have a discussion about the candidate. If consensus exists to hire, most of the time I go ahead and make an offer but only after probing to make sure this is a considered opinion of everyone in the room. On a few occasions, I’ve overruled the group and said no. This happens when I think some factor is very important but the group hasn’t considered it well enough. If the decision is a uniform no, I don’t hire. I reserve the right to overrule the group, but it’s almost inconceivable I’d overrule a uniform no. If the view of the group is mixed, I probe those in favor and those against. In almost all cases where views are mixed, I say no - this is part of my default ‘no’ position.

The benefits

I know this process sounds regimented, but there are important benefits. The first is fairness for candidates; everyone is treated the same and there’s a consistent set of filters. The second is learning; if the process is wrong or has failed in some respect, we can fix it. Thirdly, the process is inclusive - the team has a huge say in who gets hired and who doesn’t.

If hiring and retaining good staff is important, then it’s important to have a fair, decent, and thorough hiring process. Through years of experience, I’ve honed my process and I’ve been pleased that the companies I’ve worked for have all had similar underlying processes and similar principles.

Good luck

If you"re searching for a job, I hope this post has given you some insight into a hiring process and what you have to do to succeed. Good luck to you.