The thing you're missing here is that these two statements are not contradictory:
-- CS grads are, on average, better programmers than non-CS-grads.
-- The average CS grad is not a very good programmer.
My company does a standardized interview with a straightforward coding problem that is only a few steps above fizzbuzz level. In just the past few weeks, we've had candidates who:
-- Had to look up basic syntax (defining a function) in the language they chose.
-- Duplicated a reference a few minutes in and could not figure out the problem in the remaining twenty minutes.
-- Couldn't extract the 2nd element of an array of five elements.
These same candidates, on other portions of the interview:
-- Knew nothing about common data structures (something that at the very least a CS degree should help with)
-- Couldn't handle an intermediate SQL query fix involving a simple normalized schema
-- Couldn't tell you anything about internals or security, and
-- Didn't seem able to describe how they'd build a CRUD app.
All of these candidates had CS degrees from respectable-to-good universities.
Anecdotes are not data, but high-level data says the same thing. Our coding problem is very similar to the practice problem shown at [1] (though slightly harder). It doesn't require any tricky lateral thinking, deep language features, esoteric algorithms, dynamic programming, weird race conditions, or any of the other usual trivia BS - it's as straightforward a task as we could design.
And if you look at how much progress people make (see [2]), it's not much. The vast majority of people we interview have CS degrees, and two-thirds of them still get ~nowhere.
Meanwhile, the best interview we've done this month was with a guy who has been the sole developer for an ag business in rural Arkansas for the past decade. He has no degree and his previous two jobs were a sandwich shop and running a website for his church.
(Specific details slightly fudged for the sake of candidate anonymity, but not in a way that materially affects the point.)
No wonder the PhD candidate who was supervising us was surprised I did that as a fun little side project in my first or second year at university.
Still, I'm not absolutely sure I'd have gotten to the end of step 5 (correctly working flood-fill) within 25 minutes despite having implemented the whole game in Java, REALBasic, and JavaScript over the years.
On the other hand, that's probably good. 25 minute coding challenge where you're not even expected to get to the end as per the instructions* is much better than "take this, do it over the next week, might take you 8 hours, we'll spend an hour talking about it" as I've been seeing.
* "Very few people will finish the full problem. You do not need to complete every step to pass our interview."
I have a degree in computer science, and I have been a professional full-stack developer for almost 9 years. I could probably not solve the problem in your first link in 25 minutes. I know exactly how I would solve it, conceptually and what data structures I would use, but I am not sure I'd get the adjacent mines part correct on the first attempt.
Then again, this is nothing like the type of problems I work on a daily basis.
(Not GP) After looking at the page and the problem description, I agree. The time limit of 25 minutes is way too short for this considering that the person may already be stressed and find it difficult to understand that big wall of requirements and implement it in this time span. It doesn’t matter if one is a self taught programmer or has a CS degree.
I don’t know if they’re looking for superheroes or people who can put in the work.
If the GP is reading this, I’d suggest increasing the timer a lot more (like one hour, what’s the big deal anyway?) and splitting the requirement and steps into smaller chunks. The “optimize for speed” remark in the instructions is also a bit confusing.
It's a bit off the main topic of this thread, but the problem isn't designed to be finished in time. From the page:
> Very few people will finish the full problem. You do not need to complete every step to pass our interview.
A problem that an average candidate completes doesn't have a lot of differentiating power on the high end, whereas a problem where an OK candidate finishes 2-3 steps and a great one finishes 4-5 carries a lot more information.
I don't think how far a candidate gets on this particular test determines if they're OK or if they're great. I posit that a great one could finish 2-3 steps and an OK one could finish 4-5.
Because like many people have stated, this is just a hand holding test. You're basically seeing if they can code. So you're testing how fast someone can take a list of 5 steps someone else gives you and translate it into code in under 30 minutes.
And I'll tell you, I've never, in my life, worked on code with a series of 5 requirements and submitted it for review after 30 minutes. That would be bonkers, because I'd probably spend at least an hour sussing out all the ambiguities and contradictions.
"Basically seeing if they can code" is the goal, yes.
> I don't think how far a candidate gets on this particular test determines if they're OK or if they're great.
It doesn't with certainty, but measurement of this kind is always subject to some degree of error. The point of interviewing (especially at the early stage where this interview is applied) is signal, not certainty, and we do seem to be picking up on signal:
-- # of steps completed on the coding task is the strongest single signal of success at later final interview rounds. Candidates who have gotten offers (not from us, to be clear, so this is uncorrelated error) average ~1.1 more steps completed on our coding problem than candidates who don't. That's after we filtered a lot of the weaker results out, so it's sampling biased in slower candidates' favor. This is a larger gap between offers and non-offers than any of the other nineteen individual internal scores we record for each interview, iirc.
-- # of steps completed correlates with results on the rest of the interview in a way that is aligned with reasonable psychometric measures of quality, same as most of the other parts of the interview (see e.g. [1]).
If your point here is just that interviews involve somewhat artificial work and are prone to error - well, yes. Everyone involved in testing as a field already knows that and is working around it.
To use your concrete example, creating a problem involving "sussing out all the ambiguities and contradictions" for an hour would be asking a lot more work from candidates. If we did that, I bet we'd have people complaining about the lengthy workload of doing an interview (or simply not doing it, which would be fatal to us as a business). I would also worry that that's a much fuzzier skill to measure, particularly in a cross-organizational way. It's not that, in isolation, you could never create an interview to test ambiguity-resolution, it's that (at least in my judgment), it would be impractical for us to do so given what we are trying to do.
And finally, to this:
> I posit that a great one could finish 2-3 steps and an OK one could finish 4-5.
So as an applicant, I would be left with the nagging feeling that I didn't finish the problem, and knowing myself and seeing the time constraint I choose to avoid those interview processes.
Yeah, I don't think this thread is about the interview process. It's just for the self-taught developers to feel superior. Or maybe I became too cynic, who knows.
I respectfully disagree that this problem needs anywhere close to an hour; and having the problem already broken down into rather simple steps is a significant amount of handholding.
As for being stressed -- if this is too much, how will you do when asked to solve even harder problems with tight schedules and/or demanding customers?
I agree with your assessment of the problem. It’s already been broken out into discrete chunks that are all simply solvable in short order. Even finding neighbor squares is reasonably straightforward (generating all 8 candidates is basically trivial, then pass them through a filter for each of the four edges).
But your take on the stress component of an interview is way off base. There’s a huge difference in having stress on you to deal with other people’s problems vs. your own. An interview candidate may be in financial trouble and may have been struggling for months to find employment in a terrible job market. This is existential stress on a completely different level than a customer wanting a feature yesterday.
I've solved plenty of hard problems at work, but none of them have ever looked like "count the grid squares adjacent to a given grid square while taking the grid's edges into account so the program doesn't blow up because you tried to access an invalid index."
And if I did have to do something like that at work, I promise my manager would give me more than 25 minutes to do it.
sum += (x > 0 && y > 0 && x < x_limit && y < y_limit) ? a[y][x] : 0;
That took less than a minute of thought; or you could use a common technique in image and video processing, which is to make sure that the edges are always present but filled with appropriate values by defining your grid to be slightly bigger than the active area.
I just concentrated on what would be the hardest part, the "edge" cases (literally.) Wrapping that in a loop that goes through the appropriate indices is not much more work. There is a sibling dead comment to mine which has the rest.
> if this is too much, how will you do when asked to solve even harder problems with tight schedules and/or demanding customers?
Isn't the whole reason you are joining a team is so that you have a support network when such situations arise? The period of acting alone to join that team is a temporary aberration and is to be recognized as such.
You may as well be the business selling to those demanding customers if you have what it takes to handle going at it alone. It will be far more profitable. But if you don't have it...
> As for being stressed -- if this is too much, how will you do when asked to solve even harder problems with tight schedules and/or demanding customers?
There's no such single "stress". It's all different. Being yelled at by mom is "stress", and so is being held at gunpoint. Not the same kind of stress.
Making the slightest mistake or wrong impression during an interview pretty much ends it right there and by extension kills your chances of employment. Your entire future career and financial stability for the next 5-10 years hangs in the balance at all times during an interview.
Not comparable in any way to "the customer is being annoying today".
If you just had "implement mine sweeper" sure, but the page does tell you exactly how to implement it so you don't even have to think to solve it. Just write the 5 functions they tell you to write and done, each of those are a few lines.
I just gave it a shot; I would definitely need more than 25 minutes.
> I am not sure I'd get the adjacent mines part correct on the first attempt
Yeah, getting to that point was easy. Solving that problem is where I got stuck. I'm sure I'd get it eventually, but getting to that point and figuring that out in 25 minutes is asking a bit too much.
Perhaps that is suited for candidates with image processing experience, because looking at surrounding pixels is a common pattern in image and video codecs. I suspect APL-family experience/thinking style will also help significantly (if you haven't seen it, I recommend watching this video: http://youtube.com/watch?v=a9xAKttWgP4 )
The floodfill part (the last one), on the other hand, is the one that stands out in difficulty compared to the previous parts.
Really? We had informal flood fill competitions in my high school... If you understand recursion, it's easy. If you don't... there's so many other things than flood fill you just can't do. And the generalization of saying "oh, that's just a work list" is easy and enables even more things.
> Then again, this is nothing like the type of problems I work on a daily basis.
I thought it'd be fun to take a stab at it in Python, which I haven't used in a while, but only barely still remembered that I could accept command-line input with the built in `input` function, something I don't think I ever used after writing my first lines of code 15 years ago. Then I figured I'd use just lists instead of a numpy array but had forgotten that [[0] * 4] * 4 would just create 4 references to the same list. And that pretty much derailed the whole thing for me, even though I was sure I'd get it done in 25 minutes or under :-)
It's results like this which put to rest the "myth" that "10x" programmers don't exist according to some people. In fact, it's been my experience that "100x" or even "1000x" programmers exist. (Fabrice Bellard is a prominent example of the latter.) For your given practice problem I'd probably spend more time reading it and writing the code than thinking, unless I'm asked to use something like an APL-family language in which case the majority of the time would be spent reading the problem description.
(Full-disclosure: mostly self-taught with several decades of experience.)
Compared to someone who's basically a zero in programming, any amount of knowledge pushes the ratio to infinity. The average CS graduate doesn't know shit because universities need volume to survive so basically once in, you are spitted out eventually. And faking projects and cheating on exams is rampant, with the complicit involvement of teaching stuff because they don't care and incentives are biased towards turning a blind eye.
So 90% of CS graduates are useless, which it why, thank God, the inflationary pressure towards programmer jobs is far lower than the droves of new arrivals would suggest.
Therefore the top 10% are 100x by default compared to the middle because the middle is really a liberal arts graduate, not a computer programmer.
> People who know the least amount universities talk the the most confidently about them.
Well, duh. Nobody is going to waste their time talking about something they know well. What would be the point? They already know about it. Anywhere you find people talking about a subject, you know they have limited understanding. That's why they are talking about it — seeking more information to expand their understanding.
Depends, maybe Its more a matter of description, but I would call the lowest quality engineer I've seen employed a "0.1x" engineer, before I would describe a good one as a 10x.
Just because it's a kind of visualisation of a distribution, the elite are not that far ahead of the competent, but there is a very long tail of shitters out there.
Now take that good one and compare them to a guy like Thibault Duplessis (Lichess author), look at his GitHub and commit history for Lichess and you will quickly realize that even with your 0.1x starting point 1000x engineers are out there.
The guy single handedly created the best online chess site out there outperforming whole commercial teams in every area (performance, interface, features, whatever you choose). His work ethic is legendary and he pays himself below entry level salary (for US standards).
Your statement is meaningless, as you did not define what the x in 100x is and what being a "programmer" entails.
Is x the average?
Is a programmer someone who just implements the requested features, or, like bluecalm commented below, someone like the Lichess creator who apparently does all of "performance, interface, features,", etc.?
Where can I get more of these small programming challenges?
I love small problems like this. I couldn't get all the steps in 25 minutes, but I think I was not that bad. (I was in stage 4 before time ended, got distracted about taking input with Rust, heh)
I think, working with these small problems would give my ADHD brain a boost before I start my daily works. If more detailed problem, I would spend the whole day on it, smaller would not give me much push.
That matches my experience: but it's to be expected as most CS degrees are applied Math not Software Engineering. You have to respect that.
The thing which makes SEs good is being smart and having passion. Self-taught selects for that. For young people, take a smart young person and put them through a CS degree whilst working. They'll be instantly productive and with a little guidance can be successful. Over time they become exceptional.
This was my first thought when i read the headline.
Many people were told to get CS degrees because they will get a job, not because they love it. The cost of quitting a degree or not using a "valuable" degree is relatively high, especially for people who are inclined to coast. Most people who start college also graduate, even if they aren't prepared or love their program, so we have lots of bad cs graduates.
On the other hand, most people I know have at least dabbled in learning to code. In this case the cost of quitting is very low and the cost of getting good is reasonably high, so it self-selects for people who excel at it and love it.
That said, if you only compare people with similar aptitudes, my experience is that people with formal training are better.
tl;dr: The most important thing to me is comparing past work, interview and personality. If you have a CS degree with no personal projects, you won't get a job offer. If you don't have a degree but have active projects with good code then I don't care if you have a degree. That said, I have been getting hundreds of applications for every entry level job I post so I can normally find someone who has both.
-- CS grads are, on average, better programmers than non-CS-grads.
-- The average CS grad is not a very good programmer.
My company does a standardized interview with a straightforward coding problem that is only a few steps above fizzbuzz level. In just the past few weeks, we've had candidates who:
-- Had to look up basic syntax (defining a function) in the language they chose.
-- Duplicated a reference a few minutes in and could not figure out the problem in the remaining twenty minutes.
-- Couldn't extract the 2nd element of an array of five elements.
These same candidates, on other portions of the interview:
-- Knew nothing about common data structures (something that at the very least a CS degree should help with)
-- Couldn't handle an intermediate SQL query fix involving a simple normalized schema
-- Couldn't tell you anything about internals or security, and
-- Didn't seem able to describe how they'd build a CRUD app.
All of these candidates had CS degrees from respectable-to-good universities.
Anecdotes are not data, but high-level data says the same thing. Our coding problem is very similar to the practice problem shown at [1] (though slightly harder). It doesn't require any tricky lateral thinking, deep language features, esoteric algorithms, dynamic programming, weird race conditions, or any of the other usual trivia BS - it's as straightforward a task as we could design.
And if you look at how much progress people make (see [2]), it's not much. The vast majority of people we interview have CS degrees, and two-thirds of them still get ~nowhere.
Meanwhile, the best interview we've done this month was with a guy who has been the sole developer for an ag business in rural Arkansas for the past decade. He has no degree and his previous two jobs were a sandwich shop and running a website for his church.
(Specific details slightly fudged for the sake of candidate anonymity, but not in a way that materially affects the point.)
[1] https://www.otherbranch.com/shared/practice-coding-problem
[2] https://ibb.co/CpznhK8m