My Biggest Failure In Data Science, And What I Learned
There is nothing worse than feeling like a complete idiot. Throughout a data science career, there are going to be a few times where you cannot believe how bad something has gone. Whether it has to do with building a model that failed miserably, realizing you joined data incorrectly after presenting a project, accidentally dropping a table in production, or somewhere in between, feeling like you are supposed to be the smart one in the room who knows how to analyze data to a high degree has its bad days.
The toughest days happened after I finalized one of the best analysis projects in my early career. I was tasked to work with a client to build a dashboard analyzing their entire business revenue contracts. This wasn’t one of my standard clients, but I was brought onto the project because they wanted to have some “advanced” analytics sections included on the dashboard. I was really excited to play around with their data and show them something I could be proud of. I was eager, ambitious, and wanted to use this opportunity to show my company what I was really capable of with my new data science skills.
One part of the dashboard they wanted was called “contract clusters”. They wanted a visualization that would break out different contracts the business had, associate it with other similar contracts, and have a way to show if there was a pattern between the amount of money brought into the business and how it was generated. They also wanted the visualization to be broken out into large groups, meaning instead of seeing each contract, they wanted a large circle representing the entire cluster of contracts.
For me, this seemed like a great way to use a simple clustering machine learning model like K-Means to identify these clusters. I had multiple data points about each contract allowing me to find a way to identify what they had in common and group them into different categories I would label myself. I told them my plan and the client told me they loved the idea.
For a few weeks I spent a lot of time working through the data. Figuring out how to handle missing values, finding contracts that were outliers, and trying out different cluster values to determine what made the most sense. I tested out different feature reduction tools to take the dataset from multiple values to just two data points, allowing me to show it in a nice graph. Finally, I combed through each cluster to figure out what all the contracts had in common. I put together a dashboard showing multiple charts breaking out the details of each cluster, which was more than the client had asked for. I wanted to make sure I had every single basis covered on why my method made sense and how it could be utilized for their business needs.
The meeting was late on a Wednesday afternoon. Even more late for those on the east coast, as I was in Denver at the time. We all jumped on the call, excited to discuss what I had come up with. The client, more rushed than I was anticipating, wasted no time on the call to get right to the point. “What did you put together?”
I started off with a presentation going through the math of how everything worked. Starting with a large dataset of information and using data science techniques to find similar contracts to one another. Then disecting the math of how I could take multiple data points for each contract and compress it down to two representative values for different visualizations. Finally, I showed the dashboard, explaining what each section represented and how it could be utilized for business decision making. I finished with confidence, pride in my work, and excited to hear the positive reaction for all my hard work.
In return, I got silence. Actually, it started with silence, then followed with frustration. REAL, frustration. The kind of frustration you hear in the voice of someone who feel’s like they’ve been wronged.
“Why would you do any of this? This is not what we wanted. This is what you were working on this whole time? Where’s the dashboard we paid you for?”
I was perplexed. We had discussed my plans weeks ago, but clearly there was some serious disconnect on what my expectations were versus theirs. What I thought was half a joke actually turned out to become an extremely awkward conference call. My bosses had to step in and try to cool off the situation, while I sat in a panicked state wondering how this would conclude. The call ended on terrible note, with the client talking about how they were planning on presenting our results to their bosses the following day, and with it being so late in the day I would be on the hook to try and implement what they wanted.
End of call. Click.
My bosses told me not to worry about it, how they were a tough client to deal with, but I was pretty shaken up. I ran back to my desk, staying late to fix all of the imperfections so clear to them but hidden to me. I turned in my work to them and was taken off the project. I never got any feedback for all the work I did.
There have been bad days, but by far that was the worst.
It took a long time for me to decipher everything that happened on that call. Every presentation I’ve ever worked on since that day has completely been shaped by the result of that 20 minute torture session.
In reflection of a terrible situation, here were the main issues I ran into and how they’ve shaped every project I have worked on since.
Frequent Check Ins
The first problem I ran into was my expectations of the output versus theirs. There was about a two week gap between when I was put on the project to when I gave the final presentation and I didn’t have communication with them one other to tell them when I would be done with my results. I now know this is a huge no-no. Had I checked in with them at least once, someone would have stopped me earlier and told me my direction was going in the wrong way. Working on future projects, if there is going to be a decent chunk of time between the start and finish of a project, you have to show what you’re working on along the way. Projects aren’t a surprise birthday party. There is no reward to having a big reveal. Work with a client and show what you’ve come up with, receive feedback, and make tweaks. You dramatically reduce the risk of a presentation going off the rails if the client isn’t going to be blindsided with something they weren’t expecting.
Understand Their Schedule
When I started the project, the only timeline I was concerned about was when I needed to present. Little did I know they were planning on having a short turnaround after my presentation (the next day!). So the stress I heard in their voice when it wasn’t what they were expecting had baked in the idea where they felt like they would look like idiots to their bosses.
I felt like an idiot because these people were making me feel this way, because they were also afraid of looking like idiots to their bosses!
When you’re working on a project, it is not rude or invasive to ask how this dashboard will be presented to decision makers on the other end. If you have a better understanding of their schedule, you might want to adjust when you make a presentation. This allows a buffer to make any quick fixes after you present and you won’t have to stay up all night cramming in a project in high stress like I had to.
Should it really be their job to put themselves in a better position with their schedule? Sure, but everyone is a little different. You only can have control over yourself, so put yourself in the best position possible and set yourself up for the most success.
Presenting To A Non-Technical Audience
Why the hell did I feel the need to explain the math of clustering and feature reduction algorithms? I wasn’t presenting to a linear algebra class, I was presenting to people who want to utilize something for a business use case. Data science is one of those fields extremely intimidating to a lot of people. Nobody’s likes to sit down and listen to someone explain something extremely complex they know nothing about. These people hired my company to work on the hard problems, but giving an oral history of why an algorithm is useful is something they don’t care about. Whether I said some mathematician invented it 500 years ago or whether ChatGPT recommended it to me when I prompted it, what difference does it make to them? The only thing technical slides can do is open up more technical questions that are in the weeds that don’t drive the conversation forward.
When you talk about the steps you used for machine learning, understand who the audience is. In this case, I should have just said “After some data preprocessing…” and left it at that. The biggest restriction in meetings is TIME. Don’t use up the most valuable resource to explain something where you’re the only audience member who cares. If it is that important to the client, they will bring it up as a question. Another thing you can do is send a presentation or write up of your analysis and put it in an appendix. That way if they want to truly understand what’s going on under the hood, they will have that resource.
Some Clients Only Want What They Want
Sometimes clients just care about getting the exact output they desire. That is not a problem for a lot of computer science projects. Someone asks for an app user interface to look a specific way, that’s not difficult to do. For analytics, this is a much more difficult task. You don’t know exactly what the data will present to you. Some data is easy to analyze, some is a crippling headache. The projects where I don’t have a clear answer of how to solve it usually end up being the most fun to work on, but that doesn’t imply it is what someone is desiring.
Regardless of the situation, arguably the most important thing you can do is gauge the requirements of the client you’re working for. The client I worked with wanted one output, and one output only. If I had identified in the beginning there was no wiggle room, I would have saved time and effort with the dashboard, leading to happy results all around.
What I should have done is give them exactly what they were looking for, then presented them the next level up of analysis. Its like selling a car. Show someone what they came in wanting, and then introduce them to something they can’t resist.
If a client only wants one output, there is no amount of shiny objects you can throw their way to change their mind, and that’s fine. Not every project needs to be a grand slam. After this project, I made sure to find out the comfortability of being creative with the analysis, or if I needed to deliver an exact measurement of what they wanted.
Expect A Range Of Feedback
It sounds so intuitive to expect there is going to be good feedback and there is going to be negative feedback, but after spending a good amount of time and energy putting a project together, it becomes harder for you to see what people might not like about it. After this mix up, I have learned to be much more neutral about my expectations going into a project. I might even more on the pessimistic side. There are multiple benefits of understanding not to assume its going to be fireworks and standing ovations after you’re done presenting:
- You won’t be crushed by small amounts of negative feedback
- You will be less defensive when people ask a lot of questions
- You will be more excited about positive feedback
- Your approach aligns more with learning what the next steps are rather than believing you’ve solved all the problems
There is nothing wrong with being proud of your work and having confidence you did the best you can, but that a lot different than expecting nothing but excitement on the other end. Clients and customers are rarely completely satisfied and will most likely want something changed. Resetting expectations going into presentations and project checkpoints will prevent an emotional and stressful rollercoaster every week.
There will be some projects you work on ending in complete disasters. It’s going to happen. Not everyone is going to think what you put together is the best analysis for them, and that is fine. Just like with most arguments or problems in the world, it usually comes down to misunderstandings between two parties. It is not bad algorithms causing the biggest rifts on analytics projects, but expectations not being met. You can come back with an algorithm solving more problems than the client realized, but if it wasn’t done in the way they wanted without their prior knowledge, it might end in complete disaster.
It was a great learning lesson for me to go through this situation. Especially early in my career. There might have been nothing I could have done ending up with high praise, and now I know that’s okay. Growing as a data scientist doesn’t just mean learning how to solve different kinds of problems, but understanding how to communicate in the most effective way to minimize risk of project failures.
