A practical guide for business owners who are about to hire one
By Steele Consulting
You’ve decided you need custom software built — or rebuilt, or modernized, or maintained — and you’re about to hire someone to do it. The proposals are stacking up. Everyone you’ve talked to sounds impressive. Everyone has case studies. Everyone says yes to your timeline and your budget.
And you don’t know how to tell who’s actually any good.
This is one of the more uncomfortable decisions a non-technical business owner has to make. With most vendors — accountants, lawyers, marketing agencies — you can at least skim the work and tell whether it’s any good. You can read a contract. You can judge a campaign. With software, you can’t open the hood. You’re hiring someone to build something whose quality you fundamentally cannot evaluate after the fact, in a domain where the cost of choosing badly is rarely a small thing.
In our 24 years building custom software at Steele Consulting, we’ve had the wider conversation more times than we can count — usually with a business owner who is on their second or third partner, trying to figure out what went wrong the last time. The patterns are surprisingly consistent. The good news is that you don’t need to be technical to spot them.
We use a simple framework with prospects who are evaluating us against other firms. We call it the 5 Tests. Each one is something you can actually run during your evaluation — questions you can ask, things you can request, behaviors you can watch for. If a partner clears all five, you’re probably in good hands. If they fail any one of them, that’s a real signal.
Why this decision is so hard when you’re not technical
Before the tests, here’s the underlying problem: when you can’t evaluate the work directly, you have to evaluate the people and the process. And most non-technical buyers default to the wrong proxies.
The most common proxies are: how impressive the case studies look, how big the team is, how slick the pitch deck is, and how confidently they answer your questions. None of those are reliable. Case studies can be embellished. Big teams can mean bigger bills and slower communication. Slick pitches reveal sales investment, not engineering quality. And confidence in answering questions is often a worse signal than thoughtful uncertainty.
What you actually need to evaluate is harder to see at first but easier to test for. You need to know whether they can translate, whether they’ve done something close to your problem before, whether they’ll listen to your business, whether they’ll push back when you’re wrong, and whether you’ll own what they build.
That’s the 5 Tests.
Test 1: The Translation Test
Ask a question about something technical they’ve mentioned in their pitch. Anything. “What’s an API?” “Why did you recommend that database?” “What does ‘refactor’ actually mean?”
If they respond with more jargon, you’ve learned something important. The best software partners can take any technical concept and explain it in language a smart, non-technical business owner can act on. Translation isn’t a courtesy — it’s a core competency. If they can’t do it with you in the sales conversation, they will not magically start doing it with your team three months in.
Watch especially for partners who answer your business question with a technical answer. “How will this affect my customer experience?” should not come back as “We’ll use a microservices architecture with…” If they’re talking about themselves instead of your customer, that pattern won’t change.
Test 2: The Track Record Test
Ask for a reference you can actually call — not a logo on a slide, but a real person at a real company who hired them to do something similar to what you’re asking. Then call.
Two specific questions to ask the reference:
- “What did the partner do when something went wrong?” Something will go wrong on any project. How a partner handles that moment matters more than what they do when things go right.
- “If you were doing this over, would you hire them again?” The honest answer is often more revealing than the yes itself — listen for the pause.
Be cautious about partners who can’t or won’t put you in touch with anyone. Confidentiality reasons are sometimes real, but usually there’s at least one client who would gladly take your call. If there isn’t, ask why.
Test 3: The Business Curiosity Test
How much do they ask about your business versus your technology?
A great software partner spends most of the early conversations trying to understand what you do, who your customers are, where the business is going, and what is genuinely keeping you up at night. The technology comes later — because the right technology is downstream of the business, not the other way around.
A partner who jumps straight to architecture diagrams, frameworks, and stack choices is solving a problem they understand instead of the problem you have. That’s flattering to them. It’s expensive for you.
The simplest way to run this test: at the end of the first or second conversation, ask yourself, “Did they ask me more about my business or tell me more about theirs?” The ratio matters.
Test 4: The Honesty Test
This is the test most non-technical buyers don’t think to run, and it’s often the most important one.
A good software partner will tell you what not to build. They will push back on features you want. They will say “we don’t think you should pay us to do that yet” or “this part of your idea is better solved with an off-the-shelf tool.” They will scope you down when they could just as easily take your money.
A partner who agrees with everything you suggest in the sales process — every feature, every timeline, every estimate — isn’t telling you the truth. They’re selling you.
The honesty catches up with you later, in the form of scope creep, missed deadlines, or a finished product that doesn’t actually solve the problem. Look for the partner who tells you something you don’t want to hear in the first three meetings. That’s the one you can trust with the work.
Test 5: The Exit Test
Imagine the relationship ends tomorrow — amicably or not. What do you walk away owning?
You should own the source code. You should have documentation that another engineer could pick up and run with. You should have access to every environment, account, and credential involved in running your own software. You should not be dependent on a single person at the partner firm who alone understands how it works.
Ask these questions before you sign anything:
- “Who owns the code we pay you to write?”
- “What documentation will we receive, and when?”
- “If we wanted to hire an internal team to take this over in two years, what would the handoff look like?”
A partner with nothing to hide will answer all three plainly. A partner who hedges, who talks about “their proprietary process,” or who wants the source code to stay with them is structuring the relationship to be hard to leave. That structure tends to be expensive over time.
A few red flags worth naming directly
Beyond the 5 Tests, a handful of patterns are worth flagging on their own:
- They quote a fixed price for a complex build without doing discovery first. That’s a guess, not an estimate, and the cost of being wrong will land on you.
- The team that sells you is not the team that builds for you. Find out who you’ll actually be working with day-to-day and meet them before signing.
- They oversell speed. “We can have this in three weeks” for something that should take three months means corners are being cut you can’t see.
- They use the word “agile” as a substitute for a plan. Agile is a method, not a strategy.
- They can’t tell you what success looks like, in plain English, at the end of the engagement.
What good actually looks like
The partners who consistently deliver good work tend to look the same in the early conversations. They ask more questions than they answer. They volunteer the trade-offs without being asked. They scope you down when they can. They name their assumptions out loud. They send you a written summary after each call. They tell you what they don’t know.
None of that is technical. All of it is observable, even when you can’t read a line of code.
And it’s worth saying directly: this isn’t only a software-partner problem. We covered the broader version of this question in How to Choose the Right Consulting Partner. The 5 Tests above are the software-specific layer on top of those general principles — the things that matter most when the thing being built is a system you can’t see.
How we approach this at Steele Consulting
We’ve been on the other side of every one of these tests. For 24 years, we’ve built custom web and mobile applications for clients who came to us after one or two previous attempts didn’t work out. The pattern in those conversations is almost always the same: the previous partner failed one of the 5 Tests, the warning signs were there from the first month, and the buyer didn’t have the framework to see them.
If you’re evaluating a software partner right now and you want a second set of eyes on the proposals or the questions to ask, that’s the kind of conversation we’re happy to have without expecting anything to come of it. Greg Steele’s line — “If you can dream it, we can build it” — comes with a quieter second half: only build what’s worth building, and only with people you can trust with the work.
Reach out and we’ll walk through what your evaluation looks like.