How to Solve Problems
February 4, 2014
I spend a lot of my time solving problems: I solve well-defined math problems in grad school, open-ended problems in statistical consulting, physical puzzles when I'm rock climbing, architectural problems when I build software systems, and mysteries when I debug them. Here are some strategies that have worked for me in solving the problems in these domains, presented in the order that I usually try them in. My hope is that they prove useful to you in solving your problems:
1. Change the Problem
For a problem to be worth working on, the ratio of how important it is to how hard it is must be sufficiently high. Sometimes, people make the mistake of trying to solve an impossible or near-impossible problem: I've made this mistake, for example, by trying to solve a math problem that was misstated. Other times, people waste much effort working on problems that aren't that important to solve in the first place. I've made this mistake before by painstakingly optimizing large parts of my code, only to discover that the real bottleneck was somewhere else.
So before committing too much time to your problem, think about it for a bit: Are you sure your problem is important to solve, and do you think you can feasibly solve it? Is there a similar problem you could solve that's much easier and almost as important? Could you perhaps solve an even more important problem with a just a little more effort? As John Dewey wisely asserted, "A problem well put is half solved."
2. Get Help
The world's economy has benefited immensely from the advent of specialization. Consequently, there are a lot of problems that somebody will immediately know how to solve, even if you don't. So after (or while) trying to improve your problem, it's also well worth it to ask for help. The trick to benefit from this specialization is, one, to be able to identify which problems are likely to have been solved before and by whom, and two, to not be ashamed or afraid to ask for help.
To deal with the first roadblock to effectively asking for help, try to keep abreast of what's happening in a variety of different fields. Also try cultivating the mindset that if you're not the first person to encounter your problem, then it's reasonably likely that somebody else has solved it before you and has documented how they did so, (Hello Stack Overflow!) To deal with pride or fear about asking for help, recognize that in today's society, you cannot and need not be great at everything, and it's hard to improve at something without asking for help a few times.
From personal experience, I can assure you that the benefits you can reap from specialization are huge. It took me a full day to figure out why I was getting poor scaling in my LazySorted project; when I (unfortunately later) described the problem to my friend Michael Lucy, he immediately and correctly guessed that it was a cache miss issue, something that wasn't even on my radar for most of the debugging.
When I later asked a group of my CS friends, who, like Michael, really, really understand computer systems, my friend Rafael Turner suggested that I look at GCC builtins to improve locality. At the time, I'd never even heard of them. So, I went and looked into it, and it turned out that adding a single line of code with the __builtin_prefetch function got me a 2x speedup for large lists. So, asking for help got me a 2x speedup with one line of code, and might have saved me an entire day of debugging had I asked for help earlier.
3. Explore the Problem
After trying to change your problem to a better one and trying to get help, it's time to sit down and get to it. If your problem is particularly tough, however, consider taking some time to "explore" it. In math, for example, instead of immediately trying to prove a hard theorem, I furiously scrawl down facts, observations, conjectures as fast as they pop into my head. I poke my problem from multiple angles--what happens if I do this, or that? What goes wrong if you try the obvious approaches? Is there a heuristic way to see that the statement is true?
Exploring the problem is a particularly useful strategy for open-ended sorts of problems as well. When I talk to clients for statistical consulting, for example, the first questions I ask them are always to understand their problem and their goals. This helps me to contextualize the work that I do, and often gives me good ideas.
4. Solve a Simplification
Some complicated problems have simplifications that are easier to solve. Oftentimes, solving these simplifications will provide insight into solving the original problem. In climbing, for example, if a dyno (jumping move) is too intimidating, I will try it with easier holds to get the gist of what body motions I'll need to make the move. When I want to prove a theorem in math or statistics, I will often start by proving a special case to get a flavor of the themes involved.
The reason these approaches work is that the simpler problems are easy enough that I can solve them quickly, but still capture enough of the spirit of the larger problem that solving them allows me to understand the general approach. Even for harder problem where the approach that works for the simplification does not work for the generalization, solving a simplification can often at least give you insights about the themes of the general problem.
5. Solve the Sub-Problems
Some types of problems can be broken up into relatively independent pieces. Instead of solving one large problem, you can then solve a few smaller ones. This is the underlying strategy behind the idea of writing modular software.
For example, to build a chess engine, you essentially need to implement three components: Firstly, you need to write a fast chess position data structure, with efficient operations to find pieces, check squares, make moves, and find all possible moves. Secondly, you need to design an accurate and efficient evaluation function, which should take a chess position and accurately estimate who's winning and by how much. And thirdly, you need to write a fast search routine, which searches through moves and positions to find the optimal move. Thinking about and working on each of these components separately is much easier than doing so for the entire original problem.
One downside to this strategy of solving the sub-problems, however, and the reason I list it last, is that in order to use it you often need to already have a pretty clear understanding of your problem. You might have been able to gain this insight, however, by exploring your problem, or, in the case of chess programming, by solving a simplification like tic tac toe or connect-four.
Conclusion
I use at least a few of these strategies every day, but I'm always looking for new ideas, tricks, and strategies. If you have problem-solving strategies that work for you in your domain, I'd love to hear about them!
