InterviewStreet is looking for someone to build and maintain our relationships with universities.

CodeSprint 2 Statistics

The Participation Funnel

The numbers for the four main stages of participation:

Over 5000 people participated in CodeSprint 2, with 20% scoring well enough to apply to companies. Similar to CodeSprint 1, there is a noticeable gap between those who did well enough to be eligible to apply to a company, and those that actually did apply. Because we gathered additional profile data as compared to last CodeSprint, we’re able to further examine this gap. Here’s the breakdown of the 1003 eligible candidates, if they applied, and if they wished to stay hidden from outside companies.

We can say for sure that 5% of eligible programmers participated just for fun. The motivations behind the remaining 283 non-applied is slightly little more speculative, but a birds-eye view of the data tells us that most of these were in the 600-1000 ranking range, suggesting that it was a lack of confidence in their performance, more than anything else, that caused them to not bother applying. As we shall get to in the CodeSprint 3 details, we will try remedying the perceived non-competitiveness for these 600-1000, who, after all, still scored in the top 20% of all candidates.
One last note: Of the 18 who applied to companies, but did not want to be contacted by other companies, only 2 of them were non-students. The use case of “I’m secretly trying to escape my current employer” does not yet seem to be a popular one.

Participant Demographics

Demographic data is reliable only for participants that were eligible to apply, and, even then, there is some uncertainty. 6% of 1003 eligible programmers never filled in their country of origin, and 21% declined to share their current professional status.

Despite being now open to everyone, the participants of CodeSprint remained firmly university-centric. Over 65% of all eligible participants are currently attending university (or high school!). A quarter are looking for internships.

Geographically, CodeSprint 2 was nearly ⅓ US, ⅓ India (where InterviewStreet got started) and ⅓ everywhere else. We had eligible participants from 60 countries, slightly more than a quarter of all countries in the world, or alternately, the total number of countries in Africa.

The CodeSprint 2 Company Attractiveness Index

86 companies participated in CodeSprint 2. Below is a table of the 25 most attractive companies, sorted by number of weighted applications. (If programmer applied to 3 companies, each of those 3 companies receives 1/3 of a vote). Microsoft actually had the highest absolute number of applications by a very slim margin, but its applicants were slightly more promiscuous as compared to Facebook. On the whole, weighted applications followed absolute applications very closely, with the only other significantly selective companies being Groupon and Chachii (an Indian company). A histogram of all company applications can be found in the CodeSprint 2 Discussion section further below.

For CodeSprint 2, we limited applications to 10 companies per programmer. Roughly 30% of applicants maxed out, while the remaining 70% applied with a fairly even distribution with respect to the number of companies. In hindsight, a limit of 10 turned out to be a pretty good cap, and it doesn’t seem like it was particularly onerous.

Interview Invites

Compared to last CodeSprint, CodeSprint 2’s Invite Data has a greater possibility of systemic error: We explicitly exposed programmer email addresses to companies they applied to, and encouraged companies to reach out directly to these already-applied candidates. By reaching out directly, companies sometimes forgot to mark programmers in our system as being invited. (In fact, we here at InterviewStreet are guilty of this, we forgot to shortlist candidates, and instead reached out to them directly) Thus, this data is likely an underestimate of the true number of invites.

45 companies used our internal invite system to invite candidates to interview. 2 of them went overboard, inviting over 150+. We’ve made a note in our emails to programmers who they are. The company that sent out roughly 100 invites is large enough to make that quantity of invites reasonable.

In addition to this data being an underestimate, we’ve personally received word that 2 Very Large Companies were not able to look at candidates last week, and we’ve arranged for them to review their applied candidates late this week.

Removing the three biggest invite sources, candidates were invited in according to the following distribution. Roughly 300 people in this subset were invited to interview with at least one company.

Scoring and Contest Questions

Those that wish to view the contest questions, along with test cases and solutions, can do so at cs2.interviewstreet.com

The contest consisted of 15 questions: 3 Real World Problems, 4 Company Problems, and 8 Algorithmic Problems. Partial credit was given linearly to those who solved only a fraction of the test cases used to score a solution.

In terms of submission activity over the 24 hours, there was a noticeable uptick in the waking hours of the US and India, but overall, submission activity was more level than it wasn’t.

Once again, the C family was the clear plurality of language use at 43%, but did lose some ground as compared to last CodeSprint, with Python and Java taking up the majority of that loss. A graph of languages used per problem can be found here.

CodeSprint 2 Discussion

The Non-Effect of Company Profiles and Recruitment Videos on Applications

The inclusion of detailed company profiles for CodeSprint 2 was so that programmers would be better informed in choosing their applied companies. Even though, anecdotally, we have had great feedback on providing such great company information, it doesn’t seem to have had a substantial affect on aggregate application behavior:

There is no discernible trend between inclusion of additional company information and the number of applications the company received.

To be fair, this graph considers existence but not quality; a company may have filled out a profile, but badly. There were definitely some of these profiles, which I will tactfully call “unattractive”, but in the long tail we also had great, unknown startups; ones that I’ve personally talked to, who included a great company profile but still received <10 applicants. We here at InterviewStreet have a burden not only to find programmers great jobs, but also to help these more unknown-but-great companies find talent.

Clearly, profiles and video alone are not enough. Solving this problem was one of the main factors in us deciding to try out SpecialtySprints.

The CodeSprint Business Model

This was the first time we attempted to make money off CodeSprint. We used a pay-per-hire model, similar to technical recruiters: It’s completely free for companies to sign-up, look at candidates, and interview them. The only time they ever pay is if they actually hire someone. What’s more, unlike traditional technical recruiters, companies only pay a flat fee, not a percentage of salary.

We thought this business model was a good idea when we adopted it, and now that we’ve run it through one CodeSprint, we’re pretty convinced that it’s the best business model possible.

First off, getting companies to sign up has been an easy sell; there’s absolutely no risk whatsoever for companies to sign up and look at Codesprint Programmers. Secondly, by adopting a flat fee, there’s no incentive for us to place programmers at any particular company; all companies are monetarily the same. With pay-per-hire, CodeSprint is not optimizing for the number of companies, nor the number of programmers, nor the number of invites, nor the number of interviews. We’re designed to focus on one thing, and one thing only: effective hiring.

Needless to say, we’ll be keeping this model for the foreseeable future.

Test Cases and Problem Errata

In general, we’ve received nothing but great feedback on our problem statements. Everyone we’ve talked to has said that the problems, on the whole, are damn interesting. With that said, we’ve also received a lot of feedback, both in CodeSprint 1 and 2 about test case and problem statement quality. While we don’t want to give everything away with test cases and examples, the problems lacked polish. With the introduction of SpecialtySprints and CompanySprints, which have fewer problems, we’ll be able to give a lot more effort to each problem, and by CodeSprint 3 we will have hired enough additional problem creators to do the same.

The Development of Real World Problems

For CodeSprint 2, we had two main development goals: the Programmer Viewer (aka the Hackerboard) and Real World Problems. The Hackerboard was a decent sized undertaking; it took a fair amount of time to create and was released with a couple of rough edges, but it was still a very manageable project. Real World Problems, on the other hand, was a different beast entirely.

From its announcement up until CodeSprint 2, someone was working on Real World Problems every single day. Considering how much difficulty we have with just our regular CodeChecker, Real World Problems soaked up an insane amount of time and effort to get it to work correctly. Don’t get us wrong, we’re extremely proud to have created the Real World Problems infrastructure, but this project  was our equivalent of General Motors launching Saturn; a car with too many things to specially craft at one time. Real World Problems caused us to be underprepared for many other tasks, and is the main reason why our problem and UI polish was lacking.

Announcing CompanySprints and SpecialtySprints

In the past week, there’s been a lot of internal discussion at InterviewStreet on how often to hold CodeSprint, with possible frequencies from every month to once a year. It’s a decision with a ton of incomplete data and arguments from both sides. On one hand, we’d like to be able to cater to companies and programmers year round, and not have to make them wait a few months if they need a job / a job filled now. On the other hand, we don’t want to fatigue anyone, least of all ourselves: CodeSprint is crazy exhausting to host. As is all too common, the solution came to us in the eleventh hour, only after we took our minds off the problem and reframed it. The end result: Smaller Sprints, in the form of CompanySprints and SpecialtySprints. These will be more targeted, and most importantly for programmers, will be on a much smaller timescale: between 3 and 8 hours.

CompanySprints

CompanySprints are somewhat self-evident: They’re a CodeSprint for a single company. The 2-5 problems will be company-experienced problems, and exclusively appropriate to the positions that they’re hiring for.

The first CompanySprint will be CompanySprint: Quora, which will launch in roughly a month. It will likely include a relaunched, real-data version of the Quora Classifier problem. CompanySprint: Justin.tv will occur a month after, and we may be able to organize one or two additional CompanySprints even before that.

SpecialtySprints

For smaller companies, we’ll be holding SpecialtySprints, Codesprints designed for a single job type. The first one will be SystemSprint, which is most closely related to our area of expertise of algorithmic problems. It will contain many of the UI and Problem changes discussed below, and will likely take place on March 16th, the old date for CodeSprint 3. (We’re looking into if we can hold it even earlier, though) Following SystemSprint, and in preparation for its inclusion in CodeSprint 3 will be MobileSprint, focused on iOS and Android problems.

CodeSprint 3 Details and Changes

Less than 24 hours ago, CodeSprint 3 was originally schedule to take place March 16th, 2012. This has since been tentatively delayed to May 11th, though we may change this date depending on our experience with CompanySprints and SpecialtySprints. (To assure our loyal student user base: We will definitely have a CodeSprint take place in August / September, designed for the University Fall Recruiting Season) Whenever the next CodeSprint will be, however, here are some of the things that we’ve identified that will change.

More Company Problems

Our experiment with Company Problems was very well received by both programmers and companies. Programmers felt the problems were more interesting than algorithmic problems, and companies found Company Problems to be (somewhat self-evidently) better in assessing candidates. In marketingspeak: It’s a win-win situation for both sides of the equation, and so we’re going to ramp up Company Problems as hard as possible. We’re shooting for 25 problems for CodeSprint 3, more than CodeSprint 1 and 2 combined.

Introducing Specialty Weightings: AKA Problem Organizers

25 problems is a lot! It’s a lot not only for us to create, but also for you guys to do, and for companies to evaluate. In order to make sense of these problems, we will include Specialty Weightings. For example: a DBA Weighting, a Lead Developer Weighting, and an iOS Developer Weighting.

Weightings will determine the point values of a problem. The Quora Nearby problem, for example, is mainly a back-end programming problem, and so could be worth 25 points for the Back-End Programmer Weighting, but only 10 points for the iOS Developer Weighting. A Code Review problem would be useful for to evaluate both a Lead Developer and a QA Engineer, but with differing relevance.

Multiple Leaderboards

All problems will be evaluated on a simple 0-100% scale, and be multiplied by the weighting to obtain your total score and rank. We are eliminating bonus points. (which no one really understood in the first place) As you can imagine, it no longer makes sense to have just one Leaderboard, so each Specialty Weighting will have it’s own Leaderboard. For those of you that still want to compete for fun and bragging rights, we will still have a Generic Programmer Weighting which will become the canonical Leaderboard for CodeSprint Hall of Fame purposes.

We will release Specialty Weightings prior to the beginning of CodeSprint 3. Hopefully, programmers will enter the contest not planning to tackle a set of 25 problems, but only the dozen or so relevant to their interests.

Relaxing The All Languages Requirement

We’ve hesitated in including some kinds of problems in the past, because of their inability to be solved in all languages. Certain iOS problems, for example, can’t really be solved except in Objective-C. The same is true for many front-end issues, which is largely a JavaScript-only domain. Now that we have such a large problem space, we can include these languages as part of CodeSprint, and increase the breadth of our testing ability.

We be adding Problem-Creator-approved libraries for specific problems, such as Quora Classifier.

More International Companies

Our 86 companies were centered around our two major demographics, the US and India. Given that ⅓ of our participants are from elsewhere, CodeSprint 2 severely underserved some of our programmers. Of particular note is Europe, with 15% of our programmer base and only a handful of available positions. We’ll be making a more conscious effort to serve this region, along with the rest of the international programmer base.

General Sprint Improvements

More Interesting Algorithmic Problems

We thought our problems were really good, and our users overwhelmingly agree. But then someone showed us that we can do better. Not all of our problems will be that cool, but we’ll certainly try.

Better Test Case Feedback

Problem test cases will be baked into the design of the problem: Each test case will be included to test a specific part of your solution, whether it be an edge case or computational speed. Most importantly, test cases will have an associated message that, should your code fail, will let you know why the failed test case was included. It’ll be up to the discretion of the problem creator exactly how much information to reveal, but our goal is that you will always have an idea, at each stage of the problem, what you should be fixing in your solution. On rare occassions, when it doesn’t spoil the answer, we will also include more complicated sample cases (Quora Classifier would have benefited from this, Coin Tosses would not)

Small UI Fixes

Four things that we meant to include for CodeSprint 2, but didn’t get around to it:

• A countdown timer, so programmers don’t submit their final submission late
• A real-time, persistant information feed, so you are immediately notified of any announcements / errata, and can look back in it’s history to see if anything got changed.
• A very clear notification when you’re looking at the problems at interviewstreet.com, instead of codesprint.interviewstreet.com
• Country and Entry-Level filters for navigating company positions

Thanks and Congratulations

CodeSprint 2 was not quite as rushed as CodeSprint 1, but it was arguably more nerve wracking; this time, we had a reputation to keep.

We’d like to thank all of our companies for participating, but especially the brave ones that contributed a Company Problem: Groupon, Quora, and Newsle. We really appreciate the effort you went through to give us fully formed, completely drop-in problems.

On the student side, we’d like to thank the following people, who helped us immensely in IRC chat guiding people and helping them out: pedrosorio, aquatsr, as_df, sontek, Timothee, and TheCalled. If you see your name here (and your IRC handle was the same as your CodeSprint handle), expect an email from us sometime later today.

Finally, to the 665 applied candidates: Good luck.

Until next time,
The CodeSprint Team at InterviewStreet

If you are a company looking to participate in a Sprint, (Code, Specialty, or Company) please contact codesprint@interviewstreet.com

If you are a programmer and would like to receive Sprint Updates, please subscribe to our mailing list at codesprint.interviewstreet.com

Now that CodeSprint 2 is over (a full post-mortem can be found here), applicants are beginning to interview and receive offers from companies. Following up his very popular guest post, Assessing a Company: Questions you need to ask in an Interview, we asked Steve to share his expertise for the next step in the interview process: salary negotiation.

Starting Negotiations

In my most recent piece I covered the entire application process. What I want to touch on today is the part of the process where you, the applicant, have the most power: The job offer.

Negotiating a job offer is not as difficult as people think and companies generally tend to be a lot more flexible than commonly assumed. One simple fact that most people forget is that the negotiation process begins from the minute you apply for the job in the first place.

To put yourself in the strongest position possible to face a negotiation battle, you first need to lay some solid foundations and that all starts the first time you are asked to specify what salary you are looking for. If you are applying through a recruiter then chances are you will be asked that question within 5 seconds of expressing interest in any given job.

Before disclosing what salary you are looking for, do your research. Take a look at similar roles on various tech job boards and see what salaries are being offered, but keep in mind they can vary depending on geographic region so ensure the roles you are using for comparison are in the same general area that you want to work in. More and more recruitment agencies are compiling salary surveys free of charge for their clients. It never hurts to contact a tech recruitment agency and simply ask them if you can have a copy.

So now you have an idea of the market value for the position you are applying for, you now need to assess where you feel you sit in terms of the competition applying for this role. If you sincerely believe that you are above average then take a good look at your resume. If that fact isn’t instantly obvious by reading your resume alone then the employer won’t buy it. You either need to reign in your ego or more likely, you may need to adjust your resume. You can refer to my blog for more advice in this regard.

There is a growing trend, particularly amongst the tech community, to refuse to disclose your salary, so I better address that now to clear any confusion. First and foremost, I don’t agree with this approach in principal. I am in complete agreement with the ideology of not disclosing your salary to a recruiter until they have either disclosed the company name, or at the very least the companies budget for this particular role, but there is a common misconception that if you disclose your salary to an employer, it will limit the potential salary offered. It’s your experience that is indicative of your ability, not your current salary: Any decent employer worth their salt will judge your worth based on your potential and its value to their business.

If you follow the advice contained within this post, you will not be at a disadvantage by disclosing your salary when an employer asks.

If there are other factors at play aside from money,(for example, medical insurance is a significant issue in the US in particular) then ensure that you have full transparency on the benefits structure BEFORE applying for the job.

Stick to your Guns

Having done your research and gained a clear and accurate perspective of your market value, stick to your guns and don’t back down.

A clever employer will ask for clarification on your salary expectations during the interview and if they have already sold you the dream, at this point most people undersell themselves and suggest a salary range slightly lower than their initial expectations. If you mention a salary range during the interview, that is the one they will base your offer on.

There are ways of getting an insight into how an employer truly values your experience during an interview. The following questions will help you discern how your experience stacks up against others they may have seen.

What areas of expertise do you feel I may need to work on?

This is a tough and direct question but it will give you a brilliant insight into any potential weaknesses the employer may have identified. If they cite a specific language or framework that is integral to the role then you have identified a key issue that will affect their decision making process.

How does my salary expectation stack up against those currently working for the company with a similar level of experience?

This is a killer question and it’s more than reasonable to ask. You aren’t asking how much people are being paid, you are simply asking if the salary you requested correlates with what they are already paying their staff. The response here can be extremely revealing so make sure you pay attention!

If the interviewer confirms that it’s in the right ballpark then you have significantly strengthened your negotiating position. If they state it’s a bit higher than the average then the next question is crucial.

How does my experience in *insert relevant technology stack* compare to other people you have seen?

If they have interviewed a few people and you have stronger experience or simply more experience than the others they have seen then you are in the best possible position to negotiate a better package during the offer stage.

Negotiation != Compromise

Hopefully at this point you have set yourself up so that you are in the best possible negotiating position. The best case scenario is that the employer simply offers you the exact salary package you have asked for. This doesn’t necessarily mean you should have asked for more in the first place. One of the biggest issues in Software Development at the moment is keeping your best staff happy and away from the prying hands of the competition. If an employer truly values your skills and abilities then they will offer you a package that they feel will keep you happy as it’s counter-productive,(not to mention expensive) to bring an employee on board, only to lose them to the competition a year later because they offered 10% more than what you are currently getting paid. The employers who have a legitimate understanding of the market will offer you a salary they feel will keep you happy and away from the competition.

If you have followed all of the steps above and you are offered a package lower than your initial request, you simply need to ask why. Say that you’ve been offered $40k when you initially stated you wanted a minimum of $45k, remind them of your initial request, and politely ask why they aren’t matching that request. If they state the issue is because of budgets then refer to the second & third question listed in the previous section. If they had previously confirmed that your expectation was of a similar level to the team then you are being low-balled. Push back and stipulate that you would appreciate it if they could reconsider their offer and come back to you. If they state that they won’t budge, request 24 hours to consider your options and the next day call them back and politely decline the offer but ensure you stipulate that the reason you are turning it down is due to the salary package failing to meet your minimum expectations. More often than not, if they genuinely want you to work for their company they will offer more at this stage. If they don’t, move on.

Accepting an Offer

So you’ve nailed the interview and the company want to make you an offer you can’t refuse.

Never accept an offer on the spot. It’s really that simple. I don’t care if they have offered you a salary and package that far exceeds your expectations. Thank them and politely inform them that you would like to consider the offer before making a decision. Ideally, I recommend taking a maximum of 24 hours to thoroughly consider your options but it’s essential to set their expectations before doing so. When you state you would like some time to consider the offer, specify an exact time that you will come back to them with a definitive response.

If an employer puts you on the spot and insists on a decision there and then, I would sincerely question their motives. Never, ever accept an offer on the spot! At the very least, inform them you will call back within the hour with a response.

The reason I suggest you take time to consider your options is to enable you to put some of the benefits in perspective. Generally speaking, the person making you the offer has plenty of experience in negotiation and more often than not, benefits such as stock options and gym memberships can be oversold and made to sound significantly more attractive than necessary.

In reality, the only things you need to focus on away from the actual salary are specific details surrounding Intellectual Property Agreements and Non-Disclosure Agreements. More and more companies are stifling their developers creativity by restricting them from contributing to open source projects outside of their day job and the last thing any developer wants is to create an immensely brilliant project or tool in your own time only to learn that their employer owns your baby as you signed an IPA when you accepted the role without giving it a second glance.

What if things went a little too well and you have more than one offer of employment? Obviously this is the best case scenario but the point to remember here is to not get arrogant. First of all, weigh up your options. If one offer is significantly more attractive than the others, make sure it isn’t purely because they are paying more and that the company itself is one you genuinely want to work for. Honesty is key here. Before accepting the offer, phone the other companies that have made you an offer and let them know the situation. Give them a chance to respond before accepting anything.

This particular approach is completely normal and despite appearing like a difficult conversation, almost all employers appreciate open and honest feedback on your situation. If that means you have been offered more money elsewhere, then so be it. They would rather know in the beginning so that they have a chance to react than find out when it’s too late to change your mind.

Counter Offer

On a final note, if you are currently employed and you’ve been offered a job elsewhere, most employers will try and convince you to stay. After all, if you are good enough for the competition then maybe they aren’t truly appreciating your value right? Correct. Well, at least it’s correct for the first few weeks. The majority of people who decide to stick with their current company after being counter-offered leave within a year. The reason for this is simple: the motivation for you to want to look elsewhere eventually rears its ugly head after you have settled back in. If your motivation is money and you receive a raise, generally that raise will have been ahead of schedule and it will make it harder for you to negotiate a future raise. If the motivation is less tangible, for example, work/life balance or the general office culture, an increased salary very rarely, if ever, mitigates that problem.

The rules surrounding counter-offers are the same as the rules above. Sleep on it, and re-evaluate exactly why you are choosing to leave in the first place.

Receiving an offer of employment is an ego-boosting compliment. It’s also a crucial and strategic business decision. Be clear on your expectations and don’t settle for less than what you feel you are truly worth as you will end up resenting your job and be back to square one before you know it.

Here are top 4 apps we liked (yes, I know it’s biased )

• Hacker bkendall built http://bkendall.interviewstreet.info/tweetdensity/?handle=interviewstreet&count=50&type=html

• Hacker pedrodiogo built http://pedrodiogo.interviewstreet.info/tweetdensity/?handle=interviewstreet&count=50&type=html

• Hacker yazaddaruvala built http://yazaddaruvala.interviewstreet.info/tweetdensity/?handle=interviewstreet&count=50&type=html

• Hacker vijayrawatsan built http://vijayrawatsan.interviewstreet.info/tweetdensity/?handle=interviewstreet&count=50&type=html

Part 1: Wherein he finds us and gives us a lecture

Part 2: Wherein we decide to work on one project, and then change our minds

At this point, I give Stephen a call to discuss the particulars of the CodeSprint landing page.

Part 3: The First CodeSprint Landing Page

Part 4: Wherein I figure out what really bothers me

Epilogue:

As mentioned at the very, very beginning, Stephen was one of three designers that we worked with. They all turned out to have radically different modes of working, and yet were completely competent in their own way. If not for the fact that my interactions with the other two were mostly face-to-face, I would have also included their conversations, and this piece would have been much, much better.

As is self-evident, we went with the original landing page, but with the single page navigation that came out of this conversation, as opposed to a separate, second “Details” page. For that improvement alone, it was money well spent.

Stephen is currently working on the landing page for InterviewStreet’s Recruit product, which we hope to release later this month, after the end of CodeSprint.

One small note that should be mentioned: All our designer work is pre-paid up to agreed upon milestones. Spec work has it’s place, but not at this level.

Introduction

While delivering presents this Christmas, Santa’s sleigh ride inadvertently brought back a plot-convenient, elf-debilitating disease. (The specific origin of the disease is officially unknown, but seeing as how the first case was found in the stables, all rumors are pointing to Rudolph’s pit stop in the Amazonian jungle). Thanks to excellent corporate health benefits and an oddly specific epidemic clause in SantaCo’s disaster insurance plan, both the elves and the company will be well taken care of, but the fact remains that SantaCo needs to start hiring now to meet production quotas for next year’s sleigh run.

Luckily, SantaCo had already enrolled in the just-completed hiring contest FantasySprint, which describes itself as the best way to hire imaginary talent. With all the applicants having completed the contest and their scores graded, Santa hunkers down in his favorite lounge chair, sips a cup of Mrs. Claus’ special hot cocoa, and prepares to hire next year’s employees.

The FantasySprint Problem Set

The first thing that Santa does is familiarize himself with the FantasySprint problem set. Prior to the company review period, FantasySprint’s curators have kindly provided a summary and statistics for the problem set to all participating companies, including SantaCo. Unlike its less fictional counterpart, CodeSprint, FantasySprint has fewer questions and tests for a wider range of abilities. They are:

Hiring with the minimal Hackerboard feature set: Warehouse Elf

Having familiarized himself with the FantasySprint questions, SantaCo is finally ready to look at applicants. He fires up the FantasySprint Hackerboard, and in the dropdown window, chooses to hire for the Warehouse Elf position first.

Now, it’s fairly obvious that certain FantasySprint questions are more appropriate for a Warehouse Elf than others; Chess proficiency is pretty useless for moving boxes of toys around. Thus, the Hackerboard interface lets Santa adjust the point values of each question for this position.

Conveniently, there is a preset weighting for an extremely similar position: Easter Egg Courier. Santa selects the Easter Egg Courier preset, bumps up the importance of the Weightlifting question due to the heavier load requirements of a Warehouse Elf, and hits the Apply Weighting button.

Santa is presented with a ranked list of applicants, which would be considered the best according to his criteria. He clicks the first one, and is brought into the Profile Viewer.

The Profile Viewer is designed to quickly assess applicants as efficiently as possible. He scans each profile and shortlists the ones he likes. At the end of the FantasySprint Company Review Period, shortlisted applicants are automatically emailed SantaCo’s contact info to set up an interview.

Hiring for a more involved position: Marketing Strategy Elf

Santa decides to next hire for the Marketing Strategy Elf position. Unlike the Warehouse Elf position, Santa really needs to take his time and do an in-depth analysis of each applicant; a good Marketing Strategy Elf is worth his weight in gold, and with the growing cynicism of children on a global level, finding a good Marketing Strategy Elf is critical for the continued existence of SantaCo.

Santa starts with the Mad Scientist preset, adds a little weight to Sales, and gets his list of ranked candidates. He also applies a search filter for MASLab-X, the Christmas-themed finite-element marketing statistics tool that SantaCo uses.

With his more selective ranked list, Santa now clicks on a name to enter the Profile Viewer. This time, though, he scrolls down and views the entire job history. (Not enabled in the demo / too much fake data to create) In addition, he is really interested in the applicant’s performance in the Chessmaster 3000 problem. He clicks on the problem, and it shows a pop-out with the full moveset of the game. For especially interesting candidates, he also clicks through on the provided LinkedIn, Website, and Github links to learn more.

Finishing up

Santa continues selecting candidates for other positions, including Workshop Elf, Operations and Distribution Elf, and Time Dilation Elf. With a little help from that last elf, he manages to finish all his initial screening in a single night. Sure, he knows that FantasySprint isn’t perfect; it doesn’t measure cultural fit, and questions will neither be an exact match nor completely comprehensive to the positions he’s hiring for. Still it’s a lot better than his existing hiring methods, and this one happens to also have a pool of several thousand rankable applicants. Santa is confident that this year’s interviews will be, on average, a lot better than last year’s, and he feels pretty good about making enough toys for next year. In a very small, imaginary sense, FantasySprint saved next year’s Christmas.

Pretend to be Santa! Check out the demo of the FantasySprint Hackerboard

Are you a company interested in using this hackerboard to browse through (non-imaginary) programmers? Send an email to codesprint@interviewstreet.com this week

This is a guest post by Steve Buckley, who posts as Peroni on HN. Steve is one of the few recruiters that has earned the trust of the Hacker News community, and personally, has been a great resource of advice; He is directly responsible for the inclusion of several questions for our new Company Profile.  You probably want to check out his blog: voltsteve.blogspot.com.

How to find the right company for you

Applying for jobs is easy. Anyone with an email account can fire off a resume to a company. What I want to cover is how to find a company that offers more than just a job, a company that will offer you a career where you will feel comfortable and valued. Unfortunately every company thinks they are a great company to work for but we all know that the reality is very different indeed.

The first step is research. This may seem obvious but research goes far beyond reading their ‘About Us’ page on their generic corporate website. Job specs give a surprisingly accurate insight into how a company operates. Take a look at some of the positions they are advertising, regardless of how relevant they are to you and you will find that the language used in the job spec can tell a lot more beyond duties & responsibilities. Some people enjoying being a small cog in a massive corporate machine and if the job spec is littered with words and phrases like ‘Proactive listener’ and ‘blue-sky thinker’ then you are on to a winner. Few people fall into this bracket and if you are reading this post then there is a strong likelihood that phrases like ‘blue-sky thinker’ make you want to punch your monitor. If you are the hacker type or simply someone who is truly passionate about technology then you need to avoid generic sounding job specs. If a company hires intelligent Developers then the spec should and will be written with Developers in mind and not a generic audience. If the job spec appears quite formal but is heavy on technical detail then chances are you are looking at a spec written by a Technical Director or CTO that remained untouched by the grimy, generic hands of Human Resources.

There is a trend, primarily among start-up’s to make job descriptions fun & edgy. I’m personally not a fan of the approach but what you find, more often than not, is that in these circumstances the organisation are trying to reflect their office culture and attitude in the most obvious way possible. If, on the day of the interview, you are dodging nerf missiles and being interviewed over a game of foosball then chances are the original job spec had less ‘blue-sky thinking’ and more ‘we build cool shit because all the competition build terrible shit’.

Whilst the latter organisation may sound significantly more appealing, if you are incredibly focused, not particularly out-going and have a clear career path you want to follow then in the long run you may see more benefit from a more corporate structure.

Interview time: What are they hiding?

There is a single piece of advice that I tell people over and over again and I sincerely can’t emphasise the point enough. People naturally assume that an interview is an opportunity for an employer to assess the suitability of a candidate for a job opening they may have and they are right, well, they are 50% right. An interview is designed to also provide the candidate an opportunity to assess the companies’ suitability for them and what they want from an employer. Employers know this, human resource personnel know this, recruiters know this, but far too few job applicants know this. Employers both want and expect you to ask questions during an interview. They want to sit in front of an applicant who makes the effort to learn more about the company as they want to feel like this individual is making the effort to learn everything they need to know in order to make an informed and appropriate choice if they are offered the job. There are two clear motivations for this, firstly, an applicant who digs deep during an interview to find out the nitty gritty of the company is less likely to turn around in three months and quit as the role didn’t live up to their expectations and secondly, the person who asks lots of questions is someone showing a legitimate interest in the business and not simply someone just interested in getting a job.

During an interview, a seasoned employer will do his or her best to find out all the things you haven’t included on your resume. It’s their job to scrutinise every date and every detail to ensure your resume is an accurate and reasonable representation of your skills and experience. Beyond experience they are also trying to get an idea of what kind of person you are as more and more unsuccessful candidates are hearing ‘you weren’t the right culture fit’ as the office culture is proving to be a key driver in motivation and staff retention. They don’t want to spoil it by hiring a socially awkward penguin into a team of insanity wolves or vice versa.

Question time

When was the last time you sat in an interview and the employer said ‘the reason we are hiring is because we treat our staff like shit, they get fed up and they quit’?
Every employer will sit in an interview and try and sell you their company. They want to ensure that you feel like this is a company everyone wants to work for as that makes life significantly easier when it comes to salary negotiations should you get offered the job.
Whilst it’s important to focus on your suitability for the job, never ever shy away from asking what can appear to be difficult and direct questions. The following are some examples which will help you figure out what skeletons, if any, are hidden in their closet:

Why are you currently recruiting for this position?

The answer to this will open discussions about current projects or maybe staff that have jumped ship.

How long has the longest serving member of staff (not management) been working for you?

This is more of a bridge question but if the company is only 4 years old and half the team have been with them since day 1 then you are looking at a company who have developed a great culture from the word go.

What’s the average tenure for your staff?

This is much easier to ask if it follows the last question. Keep in mind that the average tenure with most companies is less than 5 years. The only time you should be concerned about the answer to this is if they are talking in months instead of years and again, that is on the assumption that the company have been around for more than a few years.

What are the biggest challenges your team are facing right now?

This is a broad question and it’s designed that way intentionally. Some employers will talk about key technical challenges that maybe are relevant to the position you applied for, others will talk about deadline issues or budgetary issues that are impacting the progress of a project or product. The latter is one you need to pay attention to. If the teams biggest issue is strict deadlines then it will be worth investigating that area a bit further.

What technologies/languages would you like to see your team adapt to that aren’t currently being utilised?

If you are passionate about technology and the employer is one who shares that passion then this is a killer question. They will start talking about new technology that you may not have even heard of. Make notes, do your research and should you get the job, you have a sneak peak at what language you should be learning next in order to impress those that pay your wages.

Few companies, if any, are 100% satisfied with the way their business is operating. If you could simply flick a switch to fix it, what one thing would you change?

Most companies are relatively happy with how they operate but I have yet to see a single company that is 100% satisfied with how they work. The change may only be slight but again, it gives you a direct insight into what annoys or worries the person you will be working for.

If you struggle to fill the position I have applied for, what impact would that have on the business?

That last question is one of my personal favourites. The answer will get you a direct insight into how crucial your role is in their company. You’ll find that some employers will be quite apathetic in their answer which could lead to the impression that maybe your role isn’t considered to be particularly important whereas some employers will wax lyrical about how the company/project will be doomed if they don’t find the right person soon. They are either exaggerating or else you are about to take on an incredibly pressurised and crucial role.

When you do ask a question in an interview, stop talking. Ask your question and simply stay silent. It can be all too tempting to ramble on to fill an awkward gap but if you have asked the question, the onus is on the employer to fill that gap.

Don’t be afraid to take notes in an interview (with a pen & paper, don’t break out your Macbook Pro or iPad) but make sure to ask at the beginning of the interview if they are ok with you taking notes throughout the interview. Most employers won’t object but permission is essential as otherwise it appears rude. If you are taking notes, make them short and snappy and don’t sit in silence composing a sonnet when you should be asking your next question or elaborating on a question asked of you.

Feedback

You got your foot in the door and you managed to get face time with people in a position to make a decision. That’s the hard part. From here, it’s almost entirely out of your hands. One point I need you to keep in mind; always, always ask for feedback. Even if you aren’t successful in being offered the job, pick up the phone and ask them why. Don’t be confrontational or defensive. Simply ask those responsible why you weren’t selected on this occasion and what areas you can work on to improve your chances further down the line. Regardless of whether you agree with the feedback or not is irrelevant. If you left them with an impression you don’t agree with, the fact of the matter is you did leave them with that impression so you simply need to figure out how to avoid making the same mistakes next time round.

If you are fortunate enough to be offered a job with a company that truly excites you, don’t let them take advantage of your relatively inexperienced situation. If you are being offered a salary far below your level of expectation, tell them. Tell them that as keen as you are to join their organisation, you are also as keen to feel like your experience and abilities are being valued and ask them if they can re-assess the salary offer. If the answer is a flat ‘no’ then inform them you will need 48 hours to consider your position. Give it serious thought. If you join on a low salary, are you going to be motivated enough to make an impact? If the honest answer is no, then tell them exactly that and inform them that at this point you would prefer to consider other options and thank them for considering you. Sounds crazy right? Never, ever take a job because you need the money, unless of course you and your family are facing eviction from your home and you aren’t sure where your next meal is coming from!

If you take a job simply because you need the money then you are compromising and that compromise will impact your resume, career choices further down the line and most importantly, your confidence. Would you marry someone simply because you’re lonely? Inevitably you will spend more than half your waking hours every week at work including commuting time. As uncomfortable as I am with the situation, the reality is that I currently spend more time with the people I work with than my family at home so why would you or I ever spend that much time in a job that doesn’t sit right with you. A career is an intimidating and complex animal. Be patient and think about what makes you happy. Good luck.

If you need any further help or advice in terms of finding the right job for you or even advice on how to secure said job, subscribe to my blog http://voltsteve.blogspot.com

Unlike everyone else at InterviewStreet, my background is not in Computer Science; instead, I was a Bioengineering and Materials Science Engineering double major at UC Berkeley. [1] As you can imagine, this piece of information doesn’t come up too often, and is rarely relevant to the conversations we have about CodeSprint 2. On occasion, however, it does give me a unique incredibly weird perspective on certain matters, and I’d like to share my take on some of the ways we’re trying to solve technical hiring.

The Universality of Kinetics, Thermodynamics, and Activation Energy

First, a very quick infodump to get centered. If you’re already familiar with activation energy and kinetics, feel free to skip to the next section.

In high school chemistry, you were probably introduced to this squiggly line. [2]

This picture depicts the defining distinction between two fields in chemistry: Kinetics and Thermodynamics. The latter, Thermodynamics, dictates the relative stability of the two states, and is determined by their difference in energy of the start and end components. The former, Kinetics, tells you the speed of the reaction, and is determined by the activation energy, Ea. The larger the activation energy, the fewer molecules there will be in an energy distribution to cross that hurdle, and the less you’ll see the reaction occurring. To summarize: Themodynamics tells you how badly a reaction wants to happen, and Kinetics tells you how quickly it happens, if at all. [3]

I find that I make use of the concept of activation energy unusually often, because it seems to be so universal. In corporate strategy, it’s called a barrier to entry. In physics, it’s the static coefficient of friction. In entrepreneurship, we call it a minimum viable product. In wiring, it’s the stress of the initial surge of electricity as opposed to constant load. [4] All deal with a one-time threshold unrelated to steady state or ideal circumstances.

In the business world, activation energy is most commonly known as Transaction Cost. When I was working at ZeroCater, our sales guy, Brooke, found it difficult to get companies to sign up, despite the clear ROI of buying lunch for employees (for a salaried programmer, saving 15 minutes of travel time for lunch is easily worth more than the cost of the meal). Even with on-site meetings (a hustle all in itself), Brooke found his conversion rate more than triple by making one simple change: An at-cost trial. At the time, our fee was relatively minor, but additional pricing required the approval of additional parties; an activation energy that killed many a deal. [5]

Perhaps the most salient and amusing example of Kinetics vs Thermodynamics: comparing it to relationships. Two people might be Thermodynamically compatible in the steady state, but the chemistry (ahem) won’t occur without sufficient initial intensity and a correct orientation of approach. [6] [7]

7 Seconds: The Microkinetics of Recruiting

In my undergrad days, I ran a freshmen engineering orientation class, which devoted 2 weeks to resumes and interviewing. [8] We brought in representatives from the university career center, who fielded a good number of incredibly specific, inane questions from paranoid freshmen. Still, enduring a litany of obscure high school activities and their appropriateness for resumes was (probably) worth the revelation of this data point:

On average, a recruiter spends about 7 seconds looking at your resume. [9]

It’s a shockingly small amount of time, considering the hours that people spend perfecting their resume. At that time scale, you can imagine that there’s a pretty loose correlation between good resume writing and good coding. [10] [11] [12]

As a 5 person company, we here at InterviewStreet can only work on a limited number of projects. For CodeSprint 2, we have 2 major projects. The first is Real World Problems. The second is the resume viewing interface. Internally, we call it the Hackerboard.

When you think about it, the resume is actually the driver of two distinct “reactions” in the hiring funnel: A Selection phase, which relies on a 7-second gut check (“Is he/she worth a 20 minute phone screen?”) and a Culling phase, when the shortlist is pared down to a manageable number of candidates. Engineers instinctively design towards the second reaction, towards the complete gestalt of themselves, and not so much the first, which is driven primarily by information flow (How succinct and atomic can you describe yourself, and how quickly).

Our goal in designing the Hackerboard is to make those 7 seconds as efficient as possible. The first and biggest advantage is the simple standardization of the profile; recruiters can expect to always find the same information in the same place. But through appropriate questions, we can also expect to coax out a better first-glance profile.

With that said, we have an even bigger trump card: Lengthening those 7 seconds. Why is it that the review time is so short in the first place? Consider the normal circumstances in which resume review takes place. A recruiter goes to a career fair or posts a job on some random jobsite, and gets a stack of a couple hundred resumes. The stack is unsorted, so the recruiter’s task is to blaze through it as quickly as possible to find the diamonds in a lot of rough.

By introducing a score and a ranking, the stack is now sorted. There will still be a lot of variance, but, on average, the quality of the applicant will be trending downward. The applicant that the recruiter is looking at right now is likely to be better than the following one.

I think there’s an even bigger effect than applicant sorting: applicant quality. Consider Bill Gates’ findings on human population growth:

In society after society, he saw, when the mortality rate falls—specifically, below 10 deaths per 1,000 people—the birth rate follows, and population growth stabilizes. “It goes against common sense,” Gates says. Most parents don’t choose to have eight children because they want to have big families, it turns out, but because they know many of their children will die. [13]

Recruiters go through resumes so fast not because they want to, but because so much of what they get is complete and utter crap. At some critical level of applicant quality, we hope to shift the recruiting attitude from that of a numbers game to one of serious consideration to each applicant from the beginning. In short, complete elimination of the first step through automation. And on the applicant side, we’re also trying to persuade more targeted company applications. The end result will be fewer but more thorough interviews.

Applicant Availability: The Macrokinetics of Recruiting

The picture above is a phase diagram. It’s used in Materials Science to determine the thermodynamically stable phase for a given set of parameters: In this case, temperature and the percent composition of 2 elements. [14] If you’re smelting a metal alloy with 30% of element A and cooling it to 400 K, it can tell you exactly what you’ve made.

The thing is, you never see phase diagrams in biology. The first reason is pretty simple: biology is complicated. There’s a lot more than 2 elements involved for even the simplest organic molecules. But, more fundamentally, even if we could somehow calculate the stable phases for the trillions of combinations of molecules, it would still be useless, because biology itself is not thermodynamically determined.

Phase diagrams are allowed to be the only consideration in only one narrow field, metallurgy, because it deals with incredibly high temperatures and slow cooling: The metal is allowed to slowly find its way to the lowest energy state.

Biology occurs at normal temperatures, where everything is determined not by if it wants to happen, but rather how can it get there. The world of recruiting is currently much the same.

I’ve wondered what would be the metallurgical equivalent of hiring would be. You’d need a high temperature equivalent: Everybody gets fired and finds a job at the same time. You’d also need slow cooling: Everybody takes a lot of time to find and consider their best options, to land in the best position for the system from a global perspective.

Of course, this could never happen in real life, but it’s a nice thought experiment. We have no control over firing everybody, but InterviewStreet can definitely help reduce the necessary slow cooling. We’re increasing both candidate and job opportunity availability. We’re guiding programmers towards better fitting jobs. We’re reducing the kinetic control of the job market, and hopefully coax it towards a lower energy equilibrium. [15]

Mike

This is easily the most absurd piece of prose I’ve ever written (and I’ve written a lot). Did I just really take a perfectly normal preview of an upcoming feature, badly fit it to an observation about biological vs metallurgical material creation (it really is a bad fit), and use economics to justify my title headers? Did I just really tie a bad resume with child mortality? Yes, apparently I did. That just happened.

To our programmers, companies, and especially our (still-secret) investors, I would like to assure you that I do not spend my days throwing darts at random scientific concepts and trying to tie them together. We’re hard at work designing the best contest for assessing programmers, helping programmers choose the right companies to apply for, and, I suppose, getting as many people to know about CodeSprint as possible.

[1] At the time, it was kind of an obscure dual specialization (insert appropriate hipster joke here), but had some seriously sexy research going on. In a parallel universe, I would have accepted my offer to be a PhD student at MIT’s DMSE, and work on some pretty cool stuff.

[2] I hesitate in calling this picture a proper graph, because the x-axis, the reaction coordinate, is kind of an abstract cheat.

[3] Certain friends from graduate school should be justifiably appalled at my anthropomorphization of scientific concepts. Unfortunately in this case, rigor takes a backseat to brevity, and I invoke xkcd Rule 397.

[4] The initial surge is what causes old-school lightbulbs to almost exclusively fail right after turning on the switch.

[5] Brooke has since upgraded himself to Director of HR at Justin.tv

[6] One of my professors referred to these highly universal ideas as Portable Concepts, but some quick Googling has led me to believe that it was more of a pet phrase of his as opposed to established nomenclature. Still, it’s good to identify these ideas with such a name. Some additional Portable Concepts:

• The viral coefficient in viral marketing comes from the biological reproductive ratio R0, mainly used in epidemiology. An identical concept, criticality, comes from nuclear fission, and is where we get the phrase critical mass, another fairly popularized Portable Concept.
• When journalists describe the sudden onset of revolutions and protests as “striking a match to dry tinder”, I think of the supersaturation of solutions. Everything has been built up past capacity, and a nucleation event occurs and causes a cascade. This is actually a specialized case of activation energy, but with a very specific context.

[7] I would strongly advise against bringing up this analogy on a first date.

[8] The class is still going strong since Tony, Heena, and I founded it six years ago.

[9] A quick attempt seeking independent confirmation of this data point was stymied by an incredible amount of blogspam. (which, I suppose, I’m only adding to) Still, the order of magnitude makes sense, and for our purposes only that level of accuracy is needed.

[10] See Steve Yegge’s million dollar typo.

[11] Even in the interview stage, which is order of magnitudes more accurate than the application stage, it’s hard to tell who will be good doing the real thing.

[12] The first of many Large Sweeping Generalizations.

[13] Forbes Magazine, Nov 2011: With Vaccines, Bil Gates Changes the World Again

[14] The derivation of this diagram is pretty simple, actually: Each phase has a free energy curve associated with it relative to temperature and composition, and this is just the projection of the minimum energy phase. Dual-phase states are a linear combination of the minimums of the two phases. If you’re at Berkeley, consider taking MSE 103 with Professor Glaeser; I still remember that off the top of my head, even after not doing anything materials related for 5 years. The guy has some immaculate boardwork.

[15] As I constantly have to tell myself, it’s very dangerous to think like this for extended periods of time.