How to Estimate a Software Project
Originally published on Scalable Path by Damien on February 19th, 2019.
Project Estimation. Two words guaranteed to make anyone in software development shift uncomfortably in their seat. Estimating is, by definition, a guess about the future. And considering 66% of development projects run over their allocated time, it’s safe to say that, as an industry, we’re not all that good at estimating. So it’s understandable to be nervous.
Yet there is no getting around the need for robust and accurate estimation because, ultimately, clients need to be confident they can fund a project before they commit to it.
Poorly estimated projects also have many other side effects:
- Depending on the contract in place, clients may have to pay more than they had budgeted.
- The product itself may suffer, as corners get cut trying to deliver within unrealistic time and budget constraints.
- The development team’s morale can be negatively affected, as they suffer from the stress and pressure of trying to meet unrealistic expectations.
- A project may never be completed if funding runs out.
Over the past 10 years, my team and I have planned hundreds of development projects. In the process, we developed a methodology that works and put it into a spreadsheet. Using this spreadsheet, we built an online tool and decided to make it freely available as part of our Product Owner Toolbox. This toolkit is designed to guide you through the most essential parts of the software development process: requirements gathering, estimation, recruiting and execution.
WHY ARE WE GIVING THIS AWAY FOR FREE?
Well, we’re not being entirely selfless here, the products we build are only as good as the planning that goes into them. The Project Estimator/Team Planner fits snuggly between our Product Requirements and Job Description tools. This suite of tools will help you prepare for a seamless handover to a staffing agency or dev shop — like us.
WHY IS ESTIMATING A PROJECT SO HARD?
Until only around 2011, the majority of development teams used the Waterfall methodology to plan projects. Waterfall requires all the specifications of a software project to be defined upfront, which is very helpful to the estimation process. But now, everyone is going agile. Products are shaped through ongoing stakeholder conversations, and what is delivered may not exactly resemble the initial concept. MVPs (Minimum Viable Products) are released quickly, feedback is obtained and improvements are made iteratively. While Agile has proved its value as a development framework, it has complicated things from a planning perspective.
So the question we wanted to answer was: “How can we accurately estimate within a framework that thrives on continuous unplanned change?”
THE SCALABLE METHODOLOGY
Our solution, which we have very modestly named the Scalable Methodology, advocates what Buddhists might call the ‘Middle Way’. You see, we’re not convinced the Agile/Waterfall dichotomy — that pervades the industry — is helpful. Things are rarely black and white in the real world. That doesn’t mean we’re rejecting either approach, the opposite in fact, we’re cherry-picking the best aspects from each. Agility is absolutely a good thing, it delivers the best products for our clients. But that doesn’t mean that Waterfall has nothing to offer. A healthy sprinkling of planning helps provide more accurate estimates when needed. The Scalable Methodology provides the best of both worlds. And it makes everybody feel more Zen.
WHO SHOULD DO AN ESTIMATE?
That’s a great question. The reason many people worry about the estimation process is that it requires a broad set of skills. From design and development through to strategy.
Estimates permeate the whole development process. From top-level project planning to version release planning to task planning. These all require estimates to be done.
In an ideal world, the person doing the estimate for a project should be someone who has built software and done estimates before. Estimating a project is itself a valuable skill. The more you do it the better you get. Because every estimate is shaped by the estimator’s personal heuristics and past experiences, the ideal person is someone who has also worked on a similar project.
Now, not every project will have someone who fits that mold. And that’s fine. We believe that with a little help (for example, from our tools and guides) anyone can take on the role of a product owner and create a solid set of requirements and an estimate. It will take some time, research and maybe some help, but it is achievable.
PROJECT ESTIMATOR AND TEAM PLANNER TOOL
With that said, let’s get into the nitty-gritty. Our estimator tool has two principal functions:
- Identify the team needed to deliver your project.
- Estimate the monthly and/or total cost for your project.
I’ve already mentioned how estimating is akin to predicting the future. The more information you consider, the better your model of the future will be. In other words, if you skip the requirements phase your estimate will have a lot of uncertainty. When I am asked to give an estimate on a project without proper requirements, I often refer to this as a “ballpark” or a “guess” because my margin of error will be very high. The more refined the requirements are, and the more thought and work are put into the estimate, the lower the margin of error on the estimate.
The first real step in a project is often the creation of a Product Requirements Document (or PRD). If this is the first time you’ve heard of a PRD, then I strongly suggest that you read our in-depth guide before continuing on with this. It’s by far our most popular blog post and has helped hundreds of product owners define their requirements.
For everyone else, let’s take a quick refresher course. A PRD should answer questions like:
- What is the purpose of this project (aka goals)?
- Who will use your product and how (aka audience, user personas)?
- What features will your product have (aka user stories)?
- How will this product be organized (aka sitemap, wireframes/screens)
- What risks could affect the success of this product?
The Project Estimator app works seamlessly with our Job Description app. So, once you’re happy with your team and estimate, we’ll help you craft a great job description and attract the right talent to your project. Finally, as both apps are plugged directly into our network of hand-picked freelancers, it’s very easy to see what talent is available for your project. If you get curious.
The time you spend on an estimate (or whether you need an estimate at all) will depend on the specific goals and requirements of your project. Some projects will require a Waterfall-style task-level breakdown before work can start, while others will skew towards the Agile end of the spectrum and will only need to use the team planning part of the tool (not the task breakdown).
We’ve designed the app to be flexible enough to help with all types of projects, budgets, and development approaches. To best demonstrate how this works, I’ve picked some common use cases. Let’s start with a simple use case.
USE CASE 1: STAFF AUGMENTATION
Your client is midway through building an online quiz app. They’ve decided they want to augment the project team with a Designer (part-time) and Senior Developer (full-time). The client has set aside a budget of $15k USD/month.
Sidenote: You’ll notice this Quiz App appearing throughout our Product Owner Toolbox series of articles. It’s based on a real application we built into our freelancer onboarding process. Using a consistent example gives us the opportunity to better show how the Scalable Path team conceives, plans, estimates, staffs and delivers a project.
REQUIREMENTS
The requirements stage for a staff augmentation contract is often straightforward. In this case, we’ll assume the client has already defined the skills, hours and rates they require. With that information to hand, head over to the Estimator Tool and create a new project. I’ll name my project Quiz App and click the START AN ESTIMATE button. You can ignore the template option for this use case (we’ll look at it later).
PROJECT DETAILS
In this section, you need to input the expected duration of your staff augmentation contract. You can enter this as a whole or decimal of a month. In this example, my project is a rolling 1-month contract. This section also includes an optional project description field.
TEAM DETAILS
This is where you build out your team. Click the +ADD TEAM MEMBER button and then choose your first role. I’ll create a Senior Developer role, but you can select whatever role is appropriate to your project. If none of the options seem to fit, just select Other and type in the name of your role.
Now input the desired hours you have agreed with your client. In my example, this will be a full-time role, so I’ll select 8 hours. You’ll notice the app will then pre-fill the rates. These aren’t set in stone though — you can edit them — they’re more of a guideline. In the case of my Senior Developer role, the app lists a $50/hour rate.
Next, you’ll see the CREATE POSITION option. If you’re looking for a remote developer to fill the role, we’ll be happy to help you. Simply click this link, create a job description and our team will contact you to discuss the details. If you already have someone in mind, you can skip this step. I also need a Designer for my example role, so I’ll repeat the above process.
Once I’m done, the Estimated Monthly Cost is calculated at the bottom. At close to $17,000, it’s slightly above the client’s budget. I have a few options though, I can chat with the client and explain they will need to up the budget to secure the right candidates. I could try to get lower rates (by negotiating with the contractors or finding cheaper ones, though cheaper sometimes means less productive) or I can reduce the hours/day for one or both of the team members.
As this is a Staff Augmentation project, you don’t need to fill out the Tasks section. So that’s it, you’re done!
But what if your project requires a detailed breakdown of the work involved?
Don’t worry, we got you.
USE CASE 2: PROJECT-BASED, WITH DEFINED REQUIREMENTS
Let’s stick with the same example — the online quiz app — but throw in a different type of client dynamic. In this case, your client has a fixed end date for the project. But they have provided you with a comprehensive Product Requirements Document to help with the estimate.
PROJECT TEMPLATES
This is a good time to introduce our project templates. We noticed that once grouped into categories, most projects shared similarities in the design, QA and hosting phases. The variation mostly happened in the development phase. For example, the number of screens an app will impact development time much more than hosting. With this insight, we were able to create templates for popular project categories.
While these templates can save a bit of time and help you think about tasks that will need to be done, it’s important to note that they are just a starting point. On their own, templates only provide a very generic estimate. It’s up to you to work with a requirements document and make your estimate as specific to the project as possible. Think of the templates as a framework to guide you — rather than a shortcut to an estimate.
PROJECT DETAILS
OK, let’s head back to the Estimator Tool, but this time we’ll scroll down to the list of templates and look for the best fit for our Quiz project. In this case, that is the NATIVE MOBILE APPLICATION WITH WEB BACKEND template.
Clicking on the template panel brings up a preview window with the project’s details, team size, list of tasks and of course project estimate. Click on the START FROM THIS TEMPLATE button, fill in your project name (Mobile Quiz App again) and you’ll find yourself back in the main part of the app — but this time everything has been pre-filled for you.
The template estimates that this project will take 1.5 months. The fact that this is a fixed end-date project (as opposed to a rolling monthly contract), means you need to consider the Estimated Monthly Cost and the Estimated Project Cost.
TEAM DETAILS
The template has started with a team of a Software Architect, Tech Lead, Senior Developer and Designer as the defaults. In this case, we’re assuming the Tech Lead is a mobile developer and the Senior Developer is a full-stack web developer. You can change the team to fit what you think you need for your project.
TASK DETAIL
This section also outputs an Estimated Budget, but unlike the above Team Details section, it does this by tallying tasks, not roles. That’s because we like to think through the team and task sections separately. A team estimate is a top-down approach. For example, “Who will I have on my team and for how long?” This is estimating from an Agile perspective. A task estimate is a bottom-up approach, that comes more from the Waterfall school of planning. Working through both these methods independently means we can then compare and contrast the two estimates and, if they differ, try and work out why. It’s essentially a way to look at your estimate from two different perspectives and validate your own estimate.
This can be a bit confusing, so let me dive into this concept a bit more. To give a real-world example, when we were creating our estimate for the Quiz App, we found that we accidentally overestimated the amount of time we would require a mobile developer (Tech Lead) when we started with the Team planning. That’s because we assumed since we were building a mobile app, that we’d mostly need a mobile developer. It was only while mapping out the tasks that we recognized that the bulk of the screens were actually on the web admin side and that more work was needed by our web developer (Senior Developer). Seeing a project through these two lenses can help identify and fix an issue like this, thus making the estimate more accurate.
Of course, you can only identify issues like this by having and referencing a PRD. The template we selected pre-filled tasks to give you a framework for this project, but remember it is generic. Combing through the project PRD is where you customize and potentially identify issues tailored to your project.
When curating your tasks list, be mindful of your audience. If this estimate will be delivered to a CEO (or another non-technical decision maker) I suggest you keep the number of tasks fairly low and high-level. If, on the other hand, you intend for these tasks to be viewed by a technical audience and/or converted into ‘to do’ items on a sprint, then, by all means, be as technical and specific as you can. For example, if you want your estimate to be more approachable and digestible for a non-technical audience, I suggest going only as granular as the page or screen as an estimate line item, like “Quiz List”. However, if your audience is more technical or you are thinking of using the estimate to seed development tasks in your project management tool, then, by all means, break your tasks down more granularly into things like “Quiz List Frontend” and “Quiz List API Endpoint”.
If that last sentence made you break out into a cold sweat — maybe because you don’t have much experience as a Product Owner — that’s OK, we’re here to help out. Just click on the Contact Us below and someone from our team can jump in. That’s what we’re here for!
Good luck with creating your estimate!
FAQ
Can I use this tool if I am working in an Agile fashion?
Yes. If you have the relationship in place with your client to run with a monthly budget and build in an Agile way, perfect. That is actually how we work with the majority of our clients.
SHOULD I PAD MY TASK HOURS TO ACCOUNT FOR UNKNOWNS?
No, you’ll notice at the bottom of the Task Details section there is a range. We’ve already built in the multipliers that should make the range fairly reasonable and account for a margin of error.
CAN YOU EXPLAIN THE MATH TO ME?
Sure. There are two main calculations going on. The monthly cost is calculated using a standard 174 hour work month, according to the US gov stats. The budget range is calculated using a x0.8 multiplier for the lower range and x1.4 for the upper range because projects are more likely to end up over the estimate than below it.
WHAT IS THE DIFFERENCE BETWEEN THE TEAM AND TASK PARTS AGAIN?
This might be best explained with an analogy. If you need a quote to get your kitchen remodeled, you will speak with a contractor. You’re asking this individual because they have experience in this area. Before you go to the contractor, you do some research: find photos of kitchens you like, draw some sketches, choose which floors, cabinets, fixtures, etc. you want. This is your Requirements Document for your Kitchen Project. Your contractor will use this information and, using their experience, give you an estimate. They remodel kitchens for a living, so they can guess how many people-hours will be needed to complete the project. This would be like your Team Estimate. But what if you want the contractor to really break down the tasks involved in refitting the kitchen. Maybe the estimate is close to your max budget, so you ask him to break down his estimate into tasks. This way you can decide if every aspect of the build is required, and find areas where you can save money. For example, the cabinets you chose might be expensive, so you opt for a cheaper option. This is your Task Estimate. This is a more involved process and as such, likely a more accurate estimate of the project.
We suggest you do them independently for the same reason you proofread an important email before you send it, or ask for a second opinion on something that matters. Validating an idea is important, and thinking about an idea from different perspectives is equally important.
To read more posts from the Scalable Path team, visit our blog.
