Cody challenge hard coded answers

I have been using something similar to the Cody Challenge problem solving rig for many years (albeit more sophisticated) in teaching various programming languages, including Matlab. While browsing through some Cody problems and solutions, I noticed that some solutions consist of just hard coded responsese to the test input.

For instance, a solution to problem 63 (decimal to roman conversion) encodes the test cases, and appears as a valid solution.

Cody is just for fun and someone probably decided to play silly buggers, but still, should this appear as a correct solution when it is not?

function romStr = dec2rom(n)
  d = [1990, 2008, 1666, 49, 45, 0];
  r = {'MCMXC', 'MMVIII', 'MDCLXVI', 'XLIX', 'XLV', ''};

  romStr = r{d==n};
end

Answer 1

[John D'Errico]

Should it?

Not really, but what are you going to do? A good problem should include at least one random test input. But that also implies the problem author knows how to solve the problem themselves, not always the case. Are all of the Cody problems well written, with good test cases? No. Such is life.

Is it really worth worrying about?

[Shlomo Geva]

What to do? Apply a random test to detect hard coding. Elementary for most problems.

Is it worth worrying about? Probably not in the context of Cody which is just a bit of fun. Mind you, some people appear to derive much pleasure from writing the most concise solutions, albeit often qualifying as ingenius obfuscation masterpieces. Silly 'solutions' do stuff up the list. I only discovered these short hard coded solutions when curiosity made me look at significantly shorter solutions than mine...

[Rik]

So you know what to do when creating a test set yourself.

I'm curious to know how you can see solutions that are shorter than yours. That is one of my major gripes with Cody: it prevents learning from people who may come up with better solutions or other approaches.

[Shlomo Geva]

Is there any half-decent programmer who does not know how to create tests?

After you submit a successful solution to a problem, you are given the option to view where it sits overall. If you click to view all solutions, you can see a plot with the time of submission on the X axis, and the size of the solution on the Y axis. Each solution is represented with a marker in the corresponding position. If you click on a marker it shows the respective code below (you may need to scroll down the page a bit).

I don't know if this is new or had always been there. Perhaps it is a chnage in Cody?

Yes, it is interesting to look at other solutions - some people are absolute legends. Mind you, any minor change to specs is probably a head ache or even a 'back to square 1' proposition. Snakes and Ladders. A phenomena well understood by students of functional programming who are tasked with solving several problems, and then two months later are given a request for seemingly minor changes to the specs - that turn out to be real doozies.

[John D'Errico]

I said that myself, is that ALL Cody probblems should have a random test as one of the tests. Should any half decent programmer know how to do that? Well, yes. But the fact remains, Cody does not restrict anyone from submitting a problem. And that results in many problems with a terribly easy set of tests to game. Cody is what it is.

[Walter Roberson]

There are a lot of programmers who are terrible at writing tests. Programmers are quite prone to only thinking about something one way, and not realizing that other inputs are even possible let alone will cause a problem.

This is the reason that writing good test cases is an art: it takes skills to think of good ways to break a program.

But programmers tend to think that they have those skills, so it is common for programmers to look down on testers and refuse to cooperate with them. And then when the product ships with significant bugs because the testers could not get cooperation, it is the testers who get blamed. :(

(I know some professional testers.)