90% Answers, and when they’re wrong.
November 10, 2010
I had a perfectly miserable experience yesterday, but in the end, I learned something valuable about myself, so it wasn’t a complete wash. You might notice the same pattern in yourself, so maybe this will be helpful to you, as well.
I’m here at LISA10, and I went out to dinner tonight with some friends. We got on the topic of clever interview questions (you know, things like “why are manhole covers round”, and “how would you move Mt Fuji”), and a couple of people at the table were from the company with the big G in the name, and talked about how they didn’t use questions like that anymore, but that they probed the interviewee in ways that exposed the underlying thought processes.
One of them turns to me, and says, “Suppose that you have a directory hierarchy consisting of directories named a to z, then under each of those directories, you have another set of directories named from a-z, and then under each of those directories, you have files. How do you move that from one place to another?” I said, “rsync”.
My friend then asked, “Alright, but are the limitations of rsync?”, and we went through questions back and forth. As it turned out, in his mind, the hierarchy was “arbitrarily large”, and “from one place to another” consisted of datacenters on different continents, and the data changed frequently. Obviously a simple rsync solution wouldn’t work, and once that became clear, I said so. The questioning continued while I tried to narrow down what my friend was looking for, all the while getting more frustrated and irritated at a line of questioning that seemingly had no end.
Eventually, I gave up and we left the restaurant, but I was sullen for the rest of the night. I wasn’t mad at my friend, I was mad at myself. I had not exactly passed with flying colors, and what made it worse was that, for the life of me, I couldn’t figure out where I had went wrong.
As an aside, if you only have a single point of datum, then you can not extrapolate from that with any confidence. If you put that point on a chart, there’s no line that you can draw that is statistically more likely to be right than any other. You don’t have enough data, and it’s extremely frustrating to try to form conjecture based on that. Just ask the people trying to decide if complex life is common in the universe. We’ve got a single point of datum, so we just don’t know. It leads to a lot of internal argument and debate.
That’s exactly what was going on in my head after the discussion at dinner tonight, too. What was my fallacy? Obviously, I should have asked for more information as soon as he asked me the question, but I didn’t. Why not? What other implications did this have for me? Of course, not having the answers to these questions didn’t put me in any better of a mood.
I went to bed still aggravated, and at some point in my sleep, a wrinkle in my memory turned over and I remembered something else that happened once.
I was on a job interview many years ago, and I felt like the interview had gone well. The interviewer and I had shaken hands, and I was getting ready to head out the door of his office, when he nonchalantly asked, as if he were asking the time of the day, “Oh hey, by the way, what would you use to back up a MySQL database?” Without batting an eyelash, I said, “Oh , mysqldump”. He repeated, “mysqldump?” I said, “Yeah, it comes with the database by default. It’s easy, you just run the command and give it the database that you want to dump, and it’ll spit it out. Pipe it to a file, and you’ve got the contents of the database”. He said “Alright, thanks.”
I didn’t get the job. I do, now, however, have another point of data.
I remember thinking, on the way to the car from his office, “Hrm, I should have asked how big of a database he was using that on”. I know that mysqldump has issues with large databases, particularly if they’re in heavy use while it’s running, but I didn’t mention it, because it’s almost never a problem.
Last night while I was sleeping, I remembered this mysqldump experience, and when I woke up, it hit me. The similarities were too close to ignore. If you have a tree of files and directories and you want to have a copy somewhere else, rsync is almost always the answer. There are lots of ways to manually do it, but those were typically what we’d do before we had rsync. That’s what rsync was meant to replace. Of course, there are times when you can’t, but they’re relatively rare. Just like the times when you can’t use mysqldump.
Drawing a line between these two experiences, I can say that when someone asks me a technical question, I give a 90% answer…meaning that I default to the assumption that the person asking me the question has one of the common environments and cases that the default answer applies to.
I’m still aggravated that I did poorly (both times), but at least now I’m aware of my bias, and so I can correct for it. I think having a default response of a question, rather than a 90% solution will be a good way to cover my inclination to immediately provide an answer. Maybe something like, “Tell me about your environment”. This should provide sufficient information to determine how to continue the discussion.
Is this something that you notice in yourself? Have you noticed other similar patterns in your life? I’d be interested in hearing about them (and self-evaluating to find out if I have them, too). Comment below!
Posted in General
Email me
content rss
November 10th, 2010 at 9:41 am
Thanks for this post. This is certainly something I’ve been guilty of, and I think it’s always good to have a reminder that you shouldn’t jump to conclusions about a client’s requirements.
November 10th, 2010 at 9:42 am
[...] This post was mentioned on Twitter by Stueccles Flipboard, steve sanderson and HN Firehose, Planet SysAd. Planet SysAd said: Standalone Sysadmin: 90% Answers, and when they’re wrong. http://bit.ly/bfVMSV [...]
November 10th, 2010 at 10:09 am
Thanks for the comment Scott. It’s definitely easy to assume that we’ve got all the right answers, and what’s worse is that it’s /appealing/ to assume that. Dangerous work.
November 10th, 2010 at 10:59 am
Matt, I’m guilty of this as well. I think you are right on point, too often we rush to provide an answer rather then sit back and consider if the problem lies in that other 10%. Most of the time you may be right but demonstrating you are cognizant of circumstances where the 90% answer would not work speaks volumes. I’ve done this on interviews as well and realized it afterwards kicking myself. I’m enjoying reading your posts from LISA however and looking forward to hearing more!
This makes me think though… how would you move the files in that case?? :)
November 10th, 2010 at 11:31 am
Excellent post! I linked to it from On Getting the Entire Question
November 10th, 2010 at 11:32 am
At least in the google case, I don’t think you did anything wrong. In my experience, the google interview process is to start with a simple statement and *collaboratively* define the question, thereby automatically adjusting to the technical level of the interviewee.
If you started by trying to better-define the problem, that might be fine, but it paints a picture of someone who needs to know all of the parameters of a problem before even beginning — not necessarily a desirable trait in an engineer at a fast-moving company.
As it is, after you started with rsync, your friend started to move through the iterative design process: what are the limitations of that process? OK, let’s go further and assume those limitations are significant in this situation. Then what? This is the kind of iterative problem-solving that *is* desirable! It shows that you can (a) throw an adequate solution at a small problem quickly; (b) anticipate when that solution will fail; and (c) design a more sophisticated solution at that point.
November 10th, 2010 at 11:34 am
Coming from a background of working for a consulting company that had to solve a lot of weird problems for new clients with unfamiliar environments, I’ve definitely gotten out of this habit. If I give the “standard answer” i usually try to qualify it with “X works for most scenarios. Some of the reasons it might not be the best answer are A, B, or Gamma.” Usually though, I will use it as an opportunity to get the relevant information from the client (or interviewer.) In my work history, if I was defining a solution, I was usually the one who would also be implementing it, so I learned that anything I missed up front would bite me later on.
November 10th, 2010 at 11:36 am
Dustin – that’s a good point as well. You can say “X works for most cases so, in the absence of a known reason not to use it, let’s start there and see if it works well enough for us.”
November 10th, 2010 at 11:54 am
This kind of crap pisses me off. I’ve been in exactly these sorts of interview situations too and they are baloney – the result is to reward people who are ‘clever’ over people who are solid thinkers.
Life is the 90% solution. I am currently part of a team that administers around 20,000 servers worldwide. Our emphasis is solving problems, not immediately finding edge cases. With that many servers there will bold be edge cases and weird scaling problems. At the same time, with that many servers there are also always an infinite number of projects to work on. You have to strike a balance. I don’t want my team to be slowed down by clever thinkers who treat every project as a beautiful and unique snowflake. In real life you do your best to think of all the likely failure modes and you design against them. Then you do some testing to try to uncover unlikely failure modes. Then you deploy to production, monitor, and move on to the next project.
This gets back to the saying, “the better is the enemy of the good enough”. If you’ve got it in production and it’s working ok, you’ve done your job. You cannot get hung up on your perfect solution. At the same time don’t think I’m advocating a sloppy or incomplete solution – you do have to do your due diligence and create a solid system. You just can’t go overboard on it.
November 10th, 2010 at 11:57 am
Matt, good insights here. However, I think you need to realize that both of these situations are contrived interview situations, and frequently these kinds of questions are designed by the questioner to vainly show how clever *they* are. They’re more concerned with fabricating a scenario that trips-up the interviewee more than they care about the answer, and enjoy seeing the interviewee squirm. This is not constructive and you shouldn’t beat yourself up about it.
As Dustin points out, this kind of questioning can be useful in the context he describes, but that’s all dependent upon the presentation of the question. If it’s presented as “here’s a scenario to explore”, then it becomes useful and creative. However, if they are putting you on the spot and every time you answer they find another reason why you’re wrong, you’re probably dealing with someone more interested in their own cleverness than yours.
November 10th, 2010 at 12:11 pm
You have not done poorly.
You might have as well found out that that directory structure is written on 9 thousend clay plates. Simultanieously modified by 200 woodpeckeres and constantly reaarranged by 100 monekys, and you should replicate it in rocks on moon of Gliese 581e.
When someone gives you a task and expects an answer you give him answer by assuming missing information according to what in your expeirience he might want. Otherwise you would have to answer all questions with “It depends. Tell me more.”
You got disturbed by being unable to solve riddle, but you didn’t have all the pieces. Right way to operate is to look for missing pieces only after it is appearent that you can make no sense of riddle without them. You don’t start solving puzzle by lifting all the furniture around because some piece of riddle might be missing.
About mysql job. Don’t beat your self up. It wasn’t about mysqldump. The just had someone cheaper or position was already filled at the time of interview by some managers cousin.
November 10th, 2010 at 12:14 pm
Don’t believe this line of thinking is unusual, 90% solutions serve us well in, well… 90% of cases. It is good to ask questions first to ensure the solution is suitable for the problem at hand, but if we required a full specification of the problem domain in advance of proffering a solution I suspect people would stop asking us questions.
November 10th, 2010 at 12:17 pm
Thanks for the post Matt. I used to do interviews on my own when I managed a service center. I have been part of some of the group interviews that we have used at my current employer. Being at my current position for nearly 14 years, the closest I’ve come recently to being interviewed for a job was a transfer between departments.
Our IT/SA industry keeps changing and there are always new things to try to keep up on. Obviously the “art” of interviewing is also changing over time. It seems this is yet another thing we need to stay up on, both when we are trying to select the best candidate and when we are trying to be that candidate selected.
Your posting has been quite thought-provoking.
November 10th, 2010 at 12:28 pm
I don’t think your argument is valid. Most ‘sane’ social people expect the 90% answer. Don’t over criticize yourself. It is always a good idea to say the 90% answer. If they come back and say that it did not work, then figure out why it did not work since you yourself may not know how to cover all the remaining 10%.
In your particular example, assume you have told something similar to “mysqldump is your answer; unless of course your db goes more than 8 GB or the number of tables is greater than 200″, this would sound ridiculously insane in 90% of cases. Not only you look like an self-imposed geek, people will be afraid to ask you questions. Also, imagine him replying ” my db is less than 8 GB with less than 100 tables. but it still does not work”. Now you will be stuck in the most awkward situation!!
November 10th, 2010 at 12:32 pm
Life is an edge case, and you’re never going to have all of the parameters. With that said, it’s never out-of-line to ask about the environment or stating “knowing nothing about your environment, I’d recommend …”
These questions shouldn’t piss you off. Not asking questions means that you’ve never been surprised by an incorrect assumption. Learning from poor assumptions is the hallmark of the experienced SysAd.
November 10th, 2010 at 3:03 pm
I want to point out that your friend’s requirements (as you reported them) are contradictory. First he says “how do you move that” and then “the data changes often.” Move implies that it’s not supposed to be at the source location anymore once it’s been copied to the destination. It’s incompatible with “it changes often.”
So if it indeed needs to be moved, rsync is indeed the best tool, period. Otherwise you’re talking about replication, and then rsync is good at that but obviously not adapted to all cases.
November 10th, 2010 at 5:11 pm
I don’t think I’ve ever been asked questions like either that you’ve provided in an interview. Its kind of a shame, really, since having something like those riddles to puzzle over would be a lot more fun to me than having to make up an answer to “Where do you see yourself in five years?”
If I remember correctly, the first examples of moving Mt. Fuji or some other contrived riddles were part of the first “Microsoft Interview”. The purpose being to demonstrate to the interview how the prospective employee went about problem solving. Its fairly innovative for its time, but eventually these became almost as trite as the ‘Where do you see yourself in five years’ types of questions.
Joel Spolsky has a great article on how he uses questions such as these in an interview process and the purpose he sees behind asking those questions. The problem is when this process is either misused to demonstrate the awesome intellect of the interviewer or followed in such a rote manner to be useless.
The short answer, like a few other commenters touched on, is that your friend’s question had no right or wrong answer. The purpose of asking it and throwing random pieces into the mix was to test how you responded to it. Its a thought exercise to see how you approach problem solving. Asking for more information is one avenue but so is attempting to work through the problem step by step (mirroring what happens IRL — rsync works as a solution for awhile, but then your needs grow to where its inadequate).
In the end, its more important for some teams/organizations that they’re on board with how people solve problems than it is looking for the “right” answer to a question and that’s what they’re looking for in a good hire.
November 10th, 2010 at 7:09 pm
Being where I am on ServerFault I’ve gotten out of this habit, much like Matt Finnigan up-thread. If I have interactivity (in a meeting, job interview, hallway being buttonholed by a co-worker) it’ll be a question-and-answer back and forth while I figure out the assumptions behind the question. If I don’t have interactivity (essay question, SF answer, writing a report for my boss) I end up mapping out at least the entire first step of the contingency tree, and sometimes deeper if the problem needs it. One of the harder parts has been learning to put in disclaimers that I don’t know how to handle certain cases, just the most common ones or ones I’ve run into before.
November 11th, 2010 at 2:55 am
I think it’s human nature – or at least our nature when we’re trying to help to immediately answer a question. And when the questioner doesn’t provide more information if he’s got it.. whose fault is it?
Assuming everyone’s friends and we want everything to be supergreen; it’s noone’s fault. But people ask what they think they want to know and we answer what we thing will help them.
Certainly I’m guilty of this too because “I want to help”, at the same time more information is more helpful. Should our knee-jerk reaction rather be “Can you tell me more?”
Or just like another suggests; “Without knowing more…” or “With just that to go on…”
November 11th, 2010 at 2:00 pm
The dialogue associated with defining a problem and find a solution *is* the solution. For me, “solve this riddle” type interview questions have been a valuable tool to assess the candidate’s skills with communication, troubleshooting methodology, and and patience. I wouldn’t want to work for somebody for whom it wasn’t expected that the response to a “solve this riddle” type question would be a dialogue to define the bounds of the problem. Likewise, if a candidate answered a “solve this riddle” question with anything other than a question for clarification I’d be disappointed.
Outside of interviews the “90% answer” is the reason that I frequently eschew using technical support resources. Too many support organizations are designed around getting tickets closed quickly, rather than solving problems. There’s not much more frustrating than a support technician nipping at my heels with 90% answers when I haven’t gotten done explaining the problem scenario yet. When I absolutely *have* to use telephone-based technical supoort I typically write a description of the entire problem scenario, stick it on a web site, and start the call with “Go to this web site and read this. I’ll be here, humming to myself until you’re done reading it. I won’t talk to you about it until you’ve read the whole thing.” Some technicians have probably hated me for that, but I’ve been thanked a number of times for making the technician’s job easier, too.
When I’m answering questions (like, say, on Server Fault) I do a lot of what Sysadmin1138 describes doing. I block out my assumptions and ask for clarifications where I know that a minute detail could make a big difference. With Server Fault, though, I do play a little fast and loose and invoke intuition a *lot* more than I would with my Customers or co-workers.
November 14th, 2010 at 6:20 am
[...] just isn’t good enough. If you’re a sysadmin and/or consultant you should read this and you will probably identify with what the writer is talking [...]
November 14th, 2010 at 11:57 am
I think it’s ridiculous that an interviewer would ask a such an open ended question with no details for the sole purpose of getting you the interviewee to push back. In our line of work I want someone who constantly has ideas rather than someone who constantly looks for specifics. The former is someone who can learn specifics and adapter a solution as they go, the latter is just a troll.
December 30th, 2010 at 6:50 am
I agree with the people above, no need to be over critical. I guess it depends on the context on weather a 90% answer is deserved or not.
The interview/mysqldump question we actually have a similar question during interviews and fail for “the 90% answer” and several others in that direction. It’s made clear at the start however that’s what is expected so to not be overly harsh on candidates.
The thinking behind it is that you have to had experienced the exceptions to understand when and where to differentiate from the norm.