Mission-Centered Design · Part 4 of 9

The Estimate Phase

Why you can’t estimate until scope is defined, how to estimate projects, and the mistakes new designers make.

What Estimates Are Really For

How accurate are estimates assumed to be? In people’s minds, perfectly. An estimate carries unstated expectations about every project constraint:

  • Cost is what people openly ask for.
  • Scope: the customer has something in their head that they tried, and failed, to write down as requirements.
  • Schedule: can nine people make a baby in one month? Does the same math work for your team?
  • Quality: don’t aim for a million-dollar solution to a thousand-dollar need. In software you can spend infinite time on the smallest detail.

Estimates drive bidding and go/no-go decisions, and a too-high estimate kills a project before it starts. Asking “how many days will this take?” invites gold-plating. Asking “what can we do in X days?” invites lean thinking. In short, an estimate is a promise that you understand the project and can deliver it on time and on budget.

You Can’t Estimate Until the Scope Is Defined

The designer’s most important job in this phase is to make sure the scope is well defined and written down. Requirements alone aren’t enough. Use several techniques to clarify and communicate the vision, and share it in writing with the customer, the designer and the rest of the company.

A typical estimation loop:

  1. Review and collect all project information
  2. Interview the project stakeholders
  3. Clarify and bound the requirements
  4. Clarify and bound the design
  5. Clarify and bound the implementation
  6. Estimate the requirements with the team, going back to earlier steps as needed
  7. Deliver the estimate

Most estimates never become projects, so spend the minimum effort needed. But never skip this phase if you are committing to deliver on a fixed timeline. Flesh out just enough information to bound the project so it won’t go over budget. The estimate is really the set of constraints the future project will be bound by. Estimates get more accurate as the project becomes better defined: the “cone of uncertainty.”

Requirements, Design and Implementation

Bounding a project means working on three things at once, each feeding the next:

  • Requirements are a checklist. They are what the customer tests and accepts at the end, item by item.
  • Design is more subjective: how the product will meet those requirements. It will change as the project goes on.
  • Implementation is a high-level technical approach to building the design. It will be refined too.

Keep them separate, and estimate against the requirements. If the customer then asks for a cheaper version, you can drop requirements one by one instead of re-estimating a single giant design.

Start Broad, Then Go Deep

New designers tend to dive straight into individual requirements. Start broad instead: understand the customer’s need and the requirements as a whole, then go deep on the ones that matter. Going broad first:

  • Keeps you focused on the need, which is sometimes missing from the requirements entirely
  • Shows which requirements might be changed or dropped, so you don’t waste time on them
  • Reveals dependencies between requirements
  • Copes with inconsistent requirements. Customers who know your product write them in its terms; others describe the outcome they want, and several of their requirements may turn out to be one feature
  • Fits the time you have. An estimate rarely allows deep analysis of every requirement

Start with the biggest constraint: the budget. It’s like helping a friend choose a laptop: the first question is how much they want to spend, because it rules out most of the options. Even a rough range narrows the cone of uncertainty dramatically.

Expected Quality, Not Best Quality

Your price will be compared to competitors’. Aim for the quality the customer and other stakeholders expect, not the best quality you can imagine. And remember there are many stakeholders, not just the customer; sometimes the company wants higher quality in one area for its own reasons.

When the Budget Is Already Fixed

Sometimes the constraints are locked before you see the project: the price is agreed and work starts next sprint. The process is the same, but your job is to find how to deliver within those constraints first. Only raise the flag that it can’t be done once you have the evidence.

Estimating a Project Is Different from Estimating a Task

  • Estimate projects at the requirement level, not the task level. You can’t know how many tasks a requirement will take.
  • Account for unknowns. People underestimate because they forget them. Do they really know what the customer wants? How many bugs are in that existing code? Does that feature really work the way they think?
  • Use relative estimation against past projects (“t-shirt sizing”). Judging “A is twice as big as B” is more accurate than judging “A is 10 meters tall,” as long as everyone uses the same baseline.
  • Don’t estimate a delivery date yourself. It depends on too much outside your control: other projects, vacations, people joining and leaving. Leave that to a producer who can research it properly.

Preparing for the Estimation Meeting

  • Get familiar with the estimate spreadsheet or tool
  • Find the business context, including any internal reasons the company wants this work
  • Check the deadlines
  • Research what the product already does that’s relevant
  • Optionally, prepare artifacts: use cases, a storyboard, a system architecture diagram

In the meeting, record comments and assumptions against each requirement, initialed by whoever made them, so decisions can be traced later. These notes come out of discussion among several people with different skills; that discussion is the point.

Requirements Can Be Badly Written

Consider a vehicle simulation requirement: “The gearbox should be simulated as in the real vehicle.” It sounds reasonable, but it is vague and potentially dangerous. What does “as in the real vehicle” mean? How would anyone test it? How could you ever prove you met it? The way forward is to find out what the customer actually trains. Perhaps it is simply shifting gear at the proper speed. Then specify the requirement to match.

Good requirements are:

  • Unambiguous, clear and understandable
  • Testable, so you can verify them
  • Correct, feasible and necessary
  • Independent and atomic: one requirement, one thing
  • Implementation-free, describing what is needed rather than how to build it

Requirements Are Often Negotiable

Requirements are usually negotiable when you have a good relationship with the customer, when you wrote the requirements yourself, or on experimental and internal projects. They are less negotiable when the customer wrote them, or when the relationship is strained. Either way, when you spot a problem, write your assumption down next to the requirement, make it visible, and raise it with whoever owns the customer relationship. Most often you will simply specify or limit the requirement in writing.

For each requirement, ask: what assumptions would you make without further clarification? What could you assume if you had to? What questions do you have? What features would you need to build? What technical information would you need to estimate it?

Common Mistakes

  • Not taking notes. Even if it’s clear in your head, how will anyone else learn what you now know?
  • Asking “how important is this?” about everything. These questions come up constantly during development, and you can’t run them all past the customer.
  • Asking the customer to choose between designs too early. Ask why they want something, not which solution they prefer.
  • Over-thinking implementation in the meeting. You risk convincing yourself it’s simpler than it is.

Key Techniques

Business need (the mission), training goals (mission objectives), use cases, storyboards, features breakdown, system architecture diagram, requirements specification, product research and the estimation meeting. See the technique library.

Let’s Talk

I am open to senior product roles in defense tech. I also enjoy hearing from fellow Operation Flashpoint and VBS veterans, and from anyone writing or podcasting about simulation or war in games.

info [at] andrewbarron.net