Tomer Gabel's annoying spot on the 'net RSS 2.0
# Tuesday, September 16, 2008

(Cross-posted on Stack Overflow)

Update (17-Sep-08): Corrected the answer to an integer arithmetic question ("rotate to the right" was obviously incorrect - I was careless when I wrote this. Thanks, Kuperstein!) and some formatting adjustments.

Preparation

I never interview the applicant on the first phone call. This doesn't give me the time to go over the candidate's CV and consider if there are points I'd like to bring up on the interview, such as specific work experience or glaringly missing skills. Additionally, setting up a specific time for the phone interview allows the candidate time to mentally prepare, drink a coffee and settle down in a quiet spot somewhere. Just puts everyone at their ease.

When first calling the candidate I always take the time to introduce the company I work for, a synopsis of what we do and a general description of how the company operates. I then proceed to inquire if the applicant sees this as a potentially interesting place to work in and whether or not they have any questions; you'd be surprised at the time this can save, for example it's not always obvious whether or not the applicant is interested in working for a start-up company, or alternatively may not find the problem domain engaging.

Getting to Know The Candidate

To start the interview off, I usually skim over the candidate's CV and select two or three interesting projects that the candidate was involved in. I ask the candidates to describe their involvement (sometimes, but not often, going into a bit of detail if the domain is familiar to me), their specific contribution to the project, whether or not they had fun and why. This usually gives me a sense of what the candidates are looking for; are they heavily into design? Are they enthusiastic about a specific technology, and if so, do they have a sound reason for it? Were they frustrated by administrative issues, did they try to improve their working environment?

Technical Questions

Once those formalities and niceties are out of the way, I turn to my ever-growing collection of interview questions and select a small subset to present to the candidate. For example, a typical interview may include the following questions:

Integer Arithmetic

Does the candidate have a decent grasp of bits and bytes? I consider this a must-have; a candidate that fails this part has no chance in hell of tackling even the most trivial native code.

  • "Take an integer. How do you turn the 7th least significant bit off?" This question alone removes about half of the applicants from the equation. Some people tell you "you need to apply bitwise AND, but I can't remember the number you need to AND with" -- this isn't what you're looking for. A good answer would be something like "x &= ~(1 << 6)".
  • "How would you quickly divide an integer by eight (8)?" A good answer is, you shift right by three. A better answer would be "is the integer signed or unsigned?", with a bonus for Java developers who know the difference between >> and >>>.

Pointers and Pointer Arithmetic

This really depends on what your company does. Most developers today don't need to know in practice the particulars of pointers and pointer arithmetic optimizations, but if you're developing a highly scalable system and/or one with serious performance considerations, this is a very good measure of how likely the candidate will be able to tackle such challenges.

  • "How big is a pointer?" The only correct answer is, "depends on the architecture". 32-bit operating systems will have 32-bit pointers, 64-bit systems will have 64-bit pointers. Anything that doesn't fall into these two categories is not relevant for my purposes, and likely yours as well. If the candidate fails this question I usually mark this section as "failed" and move on.

Floating Point Numbers

  • "What is NaN?" A programmer who can't answer this question has never really worked with floating point numbers.
  • "How are floating point numbers represented in a typical modern architecture?" Failing this question is not a deal-breaker, but answering it correctly will score the candidate a lot of points. "sign, exponent, mantissa/fraction/significand" is a sufficient answer.

Essential Data Structures and Algorithms

This is the bread-and-butter of programming. A candidate should exhibit robust familiarity with commonplace data structures (hashtables, linked lists, trees etc.) and algorithms (sorting, graph traversal).

  • "Describe how quicksort works. Elaborate on its performance characteristics." Any professional developer should be able to explain quicksort in a few minutes, know its average- and worst-case complexity, and recognize pathological cases (typical school-level quicksort implementations exhibit horrible performance on pre-sorted data).
  • "How are hashtables commonly implemented?" A candidate should be able to describe the concept of a hashing function (uniform distribution) and how it relates to the internal data structure (normally an array of buckets).
  • "What backing data structure would you choose for a simple text editor?" The classic answer is arope, but it's unlikely that a candidate will be familiar with this data structure. A more likely response would be a "linked-list of strings", in which case you should ask about the complexity of various editing operations (deleting a line, inserting a line, deleting a character etc.) This question typically takes slightly longer to answer but I've found that it gives me a good measure of the candidate's intuition in choosing/analyzing data representations.

Threading and Sychronization

This is fast becoming the most important subject with which to distinguish the truly brilliant candidates from the merely competent; the ubiquity of multithreaded code nowadays also means that these questions can be used to quickly weed out the unworthy candidates.

  • "Describe one common way of synchronizing access to a shared resource." This is just a starter question, and if the candidate takes more than a few seconds to come up with an answer (mutex, semaphore, monitor or "synchronized" for Java developers, "lock" for C# developers) it's usually a good sign that they don't have any reasonable experience with multithreaded development.
  • "You have a shared cache with a very good hit ratio. How would you synchronize access to the cache with as little performance overhead as possible?" The answer is trivially a read-write lock which can accomodate multiple readers and a single, exclusive writer. Where appropriate, ask how the candidate would implement such a lock.
  • "Describe a nontrivial problem that you've had with threaded code." Responses usually fall into one of three categories: either (1) a reasonably experienced multithreaded developer would never make the same mistakes, in which case the candidate obviously isn't one; (2) a classic race-condition/deadlock/etc. scenario, which merely tells you that the candidate has some experience with multithreaded code and appears capable of tackling such challenges; or (3) rarely, a candidate may have a genuinely interesting "war story," in which case you'll probably want to hire them right away.

Peripheral Technologies

Approach this section with caution. A canditate that's familiar with a great deal of today's hot technologies may prove completely incompetent, whereas it's quite possible to find brilliant programmers that have never touched COM in their lives. I still like to get a sense of how "in touch" the developer is with contemporary technologies; familiarity with tools and technologies can definitely be a tie-breaker between two promising candidates.

  • "Are you familiar with COM? Describe an interface which any COM object must implement and its methods." Anyone who's even a bit familiar with Windows software development should be able to answer this question fully. For those unfamiliar with COM, describe it in a few words and ask the canditate to guess what the required methods of IUnknown are.
  • "What is a well-formed XML file? Give two examples of errors in an XML file which would render it non-well-formed." XML is prevalent in almost every software development domain. A candidate which cannot answer these questions (and doesn't have a very good excuse) will not go past this interview.
  • "What's XPath? Explain what a predicate is and how it's used." This is not most-have. I'd expect serious developers to at least have an idea of what XPath is. The second question is there to differentiate those who profess to know XPath from those who've actually done work with XPath and/or XSLT.
  • "If you had to verify the input of an e-mail address field, how would you go about it?" There are only two valid answers to this question: "I would use a regular expression," or "I'd like to use a regular expression, but I know that fully matching e-mails according to the RFC is insanely complex, which is why I'd get a proven library to do it for me." If the canditate is being a smart-ass you can always ask them about the performance characteristics of commonplace regex engines (which have pathological cases).
  • I also like to just toss buzzwords (Ruby, Boo, JSON, Struts, J2EE, WCF) around and examine the candidate's responses. It may also provide an interesting subject to ask about in a personal interview later on.

Concluding the Interview

The previous section usually takes between 10 and 15 minutes. At this point I normally ask the candidate if there are any questions they'd like answered, or anything I should know before we conclude the interview. Once that'd done, I thank them for their time and tell them (even if they've failed miserably) that I will call them back in a day or two with an answer.

Hope this helps, comments are welcome.

Tuesday, September 16, 2008 7:39:11 PM (Jerusalem Standard Time, UTC+02:00)  #    Comments [17] -
Development | Personal
Tuesday, September 16, 2008 7:50:52 PM (Jerusalem Standard Time, UTC+02:00)
Minor nitpicking - in the first integer question, you want to shift right by three, not roll. (I know that's what you meant, but it's an important distinction anyway).
Other than that, I rather like this set of the questions.
Wednesday, September 17, 2008 3:38:14 AM (Jerusalem Standard Time, UTC+02:00)
Tomer,
The int arithmetic questions only tell you if someone bothers to memorize such things.
I'll not be able to answer such questions of the top of my head. And I consider that a plus.

Dividing by 8, in particular, is something that I would strongly suggest to do by x/8. It is clear, simple, and I truly doubt that this is something that will give you any measurable perf.
Wednesday, September 17, 2008 10:31:56 AM (Jerusalem Standard Time, UTC+02:00)
Ayende: I disagree on all counts. To answer those questions you don't need to memorize squat, but you do need a good grasp of bits and bytes, bitwise logic and some integer arithmetic. A serious programmer should have this knowledge ingrained, and it's /always/ useful, whether you're developing a compiler, 3D engine or an enterprise back-end.

If I wanted to nitpick, I'd ask the applicant to recite the values of 2 to the power of 0-20. I actually /do/ consider this something any serious programmer should be able to know from memory, but I won't fail a candidate for not having good memory.

As for your final comment, it's completely irrelevant. The question wasn't about "how to write a division by 8 in the most readable manner"; the compiler will almost certainly replace it with a shift anyway. You would hardly expect a mathematician to use Newton-Raphson approximation instead of just writing sqrt(...), but you'd still expect a mathematician to know that a square root can be approximated and how.
Wednesday, September 17, 2008 10:33:28 AM (Jerusalem Standard Time, UTC+02:00)
It's not a matter of memorization. If you understand what the internal representation of integer is, and how bitwise operators work, this sort of thing is automatic. You don't even need to think about it.

As to whether you'll get a speedup by using ">> 3" as opposed to "/8"? Well, that depends.

Correct me if I'm wrong, Tomer, but I'm pretty sure that on modern x86 CPUs, integer division might actually be *faster* than a shift, due to weird architectural reasons. Nowdays, anyone doing ">> 3" when they really want a division in userspace code is probably an idiot - or is doing it out of habit. On the other hand, in a more demanding environment, there could be a considerable difference in the shift's favor.

So, why would you ask that question in the first place? Not because it's a useful skill (although it is), but because it's a good indicator the person you're interviewing has a basic understanding of how a computer works.
Wednesday, September 17, 2008 10:36:56 AM (Jerusalem Standard Time, UTC+02:00)
Divide by 8 is optimized into shift by any decent compiler in the last 10-20 years.
It's important to know such optimizations, and why it is better to write x/8 - because it's simpler code that is optimized.

I like asking trick questions, for example which of two equivalent pieces of code is "better", and leave the definition of "better" up to the candidate. I expect to get an answer like "the efficiency is the same, so I'd pick the simpler/more elegant solution, which is Foo". Of course, in cases where there's a significant perf boost to one of the solutions, I expect him to choose this solution, even if performance wasn't a part of the task definition.

BTW, more relevant to a meeting interview than a phone interview, I like asking the applicant to write code. Give him his favorite IDE & tools if it's possible and let him work for 15-30 minutes on actually coding something - you'd see right away what kind of code he produces, which is as important as his theoretical understanding.
Wednesday, September 17, 2008 11:13:02 AM (Jerusalem Standard Time, UTC+02:00)
What about the candidate's grasp of algorithms? And recursion? I'd expect at least some of your graph algorithms are recursive. I am not sure QuickSort counts, it is not all that complicated and is too well known. How about them graph algorithms?
Good implementers are important, but I still need them to be able to come up with original solutions to new problems, or at least modify an existing one to solve a similar-but-different problem.

Regarding the right shift thing:
a) it's true that most compilers will implement division using right shift, as it's usually more efficient.
b) there are many CPUs in major use today that simply do not have a divide instruction. ARM, for example. And many DSPs. On such machines, right shifting is a major win.
Mickey
Wednesday, September 17, 2008 11:45:09 AM (Jerusalem Standard Time, UTC+02:00)
Mickey: A couple of odd questions about algorithms are appropriate, but remember that this is an initial /phone/ screening. At any rate if you have specific examples feel free to mention them (or post them in Stack Overflow), it'll definitely be a benefit :-)

Michael: See Mickey's answer; some relevant CPU architectures simply do not have DIV. Never-the-less, any modern compiler will in practice optimize an "x/8" statement to a shift. The only case where it makes sense to use the shift operator in code is when you're actually handling bits (I think that's as far as agreeing with Ayende's earlier point as I will go).
Thursday, September 18, 2008 2:31:20 PM (Jerusalem Standard Time, UTC+02:00)
Oops...

* off to read QuickSort *
Ilya
Monday, September 29, 2008 7:17:30 AM (Jerusalem Standard Time, UTC+02:00)
Obviously you forgot some crucial external criterias, like candidate's hair styling? or her Milano-updated outfit.
Yoni
Tuesday, September 30, 2008 4:58:53 PM (Jerusalem Standard Time, UTC+02:00)
Printed and stashed aside for when the right time will come... :)
Wednesday, October 15, 2008 6:16:40 PM (Jerusalem Standard Time, UTC+02:00)
Eh...can't belive this true when was reading this. "Divide integers by 8 by shifting bits???" What year you are living in? 1950s?

First we invent high level, abstract languages to GET rid off this kind of stupidities like bit shifting. Because we "win" maybe two nanoseconds on that?

I wouldn't hire folk who suggest this kind of screening questions. They show clearly they know nothing about software engineering. How long code you once write *lives*, how much cheaper CPU cycles are compared to programmer hours and how expensive it is to sort out messes people like you left behind.

We have seen this so many times in 50's, 60', 70's and even 80's code. Please stop splitting hair, we live in 2008. CPU is cheap. Your time and my time is not.
Peter
Wednesday, October 15, 2008 7:45:21 PM (Jerusalem Standard Time, UTC+02:00)
I couldn't decide whether or not to merit your comment with a response; perhaps some good will come of it, so here goes.

1. I wasn't alive in the 1950s, so there's no real question there. I think you'll find that no matter how high-level the language you code in, when you need to handle binary data structures you will have to resort to integer arithmetic. And it can happen regardless of whether you're developing drivers or enterprise software which interoperates with legacy code.

2. Your comment on the "sort of messes people like me leave behind" is, beyond being completely childish, also patently false. If you're offended by "x>>3" statements in code, please try and read the comments above - I advise wholeheartedly not to be a coding smartass except for cases such as outlined in the previous paragraph.

3. CPU is cheap in the common case. When you're building highly scalable software, CPU time can quickly become an issue. No, shifting right by three probably won't help your Spring-based Hibernate-heavy code to function better, but it will help you knock anoter clock-tick off of that polygon rasterization inner loop, or maybe lower your interrupt code overhead. For that DAL-heavy code I mentioned above, a decent understanding of data structures will help you cooperate with your DBA to radically reduce indexing time and potential memory utilization with your RDBMS.

In short, I think you missed the point. Try re-reading the text before commenting again.
Thursday, October 16, 2008 1:38:28 PM (Jerusalem Standard Time, UTC+02:00)
I once knew a girl named Pandora, never got to see her box though.
Yoni
Sunday, November 30, 2008 10:54:02 PM (Jerusalem Standard Time, UTC+02:00)
Ok over two months after the post but I thought I'd comment anyway. Am I the only person who has been asked the kind of "war stories" question mentioned under the threading section, and simply doesn't have one, or is that the (poorly worded?) "scenario 1"?

I've been writing threaded code for many years and have honestly never had a "nontrivial problem"; all threading problems are typically pretty trivial. Nontrivial issues in threading usually aren't related to the threading itself but a design flaw elsewhere, sometimes even the decision to thread in the first place.

As an example I recently rewrote an application that the previous developer decided to thread (fork, actually) for increased performance. No thought was given to the performance impact this would have on other parts of our production environment, specifically the fact that rather than a single connection to the database for this application, there were now upwards of 50. This was a simple design flaw/oversight caused by threading, but at the same time not a problem with threading per se.

Short version: If you have no experience as a sysadmin, you have no business writing system (backend, n-tier) code.
Allen
Monday, December 01, 2008 11:35:17 AM (Jerusalem Standard Time, UTC+02:00)
Hi Allen, and welcome - your comment is certainly appreciated!

Your story in itself is a war story. Giving the interviewer the sense that you consider most threading problems trivial is not in itself a bad thing, and illustrating the point by telling a story where a multithreaded application requires careful design consideration elsewhere is definitely a way of telling the interviewer that you're smart without being smug about it. At least, it will get you through the phone interview (with me, anyway) and you can tackle more direct technical questions in the interview proper.

As for your final comment, I'm not sure I agree; while a good grasp of computing fundamentals is a definite boon to a developer (and certainly one of the things I'm looking for when I'm interviewing), there are many ways of obtaining it. Being a sysadmin is only one of those. Experience developing back-end systems as a junior developer will often times force you to learn these principles on-the-job; being interested in computer hardware in general (if only so you can build you own dream machine as a kid) also teaches a lot, and a smart individual will usually be able to extrapolate that knowledge to other domains.
Thursday, December 11, 2008 11:26:11 PM (Jerusalem Standard Time, UTC+02:00)
You are wrong. Offering the company name, what it does, etc, upfront to the candidate, on the telephone, puts the candidate in the drivers seat AND NOT YOU. It immediately sets the candidate on firm footing, because they are now thinking they have a position to accept or reject the job offer, which you haven't even offered yet. The best method is to interview your candidate on the telphone, then tell them IF they are considered for a face interview, that they will receive an email and FULL INSTRUCTIONS and detailed information about the company and the interview. THEN they can simply read the email and go to the company link you provide them in the email, and decide for themselves if they want to go to the interview or not. If they decide to go to the interview then they get back to you and select one of the available times you have provided for them. The point is, who has time to answer questions about the employer on the phone? Your job ad should tell them enough about the job and your link to the companies website tells them all they need to know about the company, and anything else they need to know they should do on their own time. All you end up doing is wasting your own time answering a bunch of mute questions about that employer. First things first, and that is if they are hungry for a job, they will get their butts out there and go to interviews, because getting them in the interview is the entire purpose. The telephone interview simply provides you a method to ask specific technical questions to filter out the dummies from the experience you are after. But I agree with what you said about how to scan the resume before conducting the telephone interview. Hope that helps!

Carr Executive Search
Friday, December 12, 2008 2:19:10 AM (Jerusalem Standard Time, UTC+02:00)
To say that I disagree with you on every count would be a gross understatement. The purpose of the phone interview is not, in any way, to assert my control over the candidate's fate, nor is there any point in trying to gain "control" over the interviewing process. Ultimately such an attitude will likely lose you many applicants, primarily the intelligent, confident sort that you're most after.

You waste two minutes telling them about what you do, and you gain their attention and honest interest. You should do this not only before you start the actual interview, but do it before you even set the interview up. Then, if they show interest in your company, you send them a detailed e-mail with links, relevant information and a reminder of the interview time. If you've done your job properly, they'll be excited when you do the interview. They'll be interested. They will have had time to research, find out a bit about your domain. It's not likely to help them get through a purely technical interview, but it gives a sense of common baseline and gives you a leg up in identifying candidates who either (a) know your domain or (b) are smart enough to extrapolate from the little they know, and are eager to learn more.

As my colleage Tal Kedar so excellently put it, a job offer should be a win-win situation.
OpenID
Please login with either your OpenID above, or your details below.
Name
E-mail
Home page

Comment (Some html is allowed: a@href@title, b, blockquote@cite, em, i, strike, strong, sub, super, u) where the @ means "attribute." For example, you can use <a href="" title=""> or <blockquote cite="Scott">.  

Live Comment Preview
Me!
Send mail to the author(s) Be afraid.
Archive
<February 2012>
SunMonTueWedThuFriSat
2930311234
567891011
12131415161718
19202122232425
26272829123
45678910
All Content © 2012, Tomer Gabel
Based on the Business theme for dasBlog created by Christoph De Baene (delarou)