Test Automation for Managers: What to Look For

Those decisions are hard to make well without knowing what to look for. This page is meant to close that gap.

A common automation problem managers do not see

A common problem with test automation in real engineering organizations is not that teams have no automation. It is that they have automation they cannot trust.

A test suite that runs but does not find real problems. Tests that fail intermittently without any code change. A suite that passes reliably but missed the defect that made it to production. Coverage that looks broad on paper but is concentrated in areas of the application that rarely break.

These problems are hard to see in a weekly status report. They show up gradually: as release confidence drops, as engineers start re-running failed tests until they pass, as the word "flaky" starts appearing in conversations about the test suite. By the time it is obvious that something is wrong, the problem has usually been building for a long time.

The underlying cause is often the same: the automation was built and maintained by people whose expertise was limited to writing scripts, without the architectural and strategic knowledge needed to build a suite that actually holds up over time.

What expertise in automation actually looks like

When evaluating someone's automation expertise, the question to ask is not "can they write automated tests." Most people with any QA background can write tests. The question is what happens to those tests over time.

Genuine automation expertise shows up in these places:

How they think about what to automate. People with real automation knowledge make deliberate choices about what to include in a suite and what to leave out. They think about test return on investment: which tests catch real problems frequently enough to justify their maintenance cost, and which tests look good on paper but do not pull their weight. They can articulate why they made the choices they made.

How they design for maintainability. A well-designed automation suite handles application changes without requiring every test to be rewritten. This comes from architectural decisions: abstraction layers, modular components, careful separation of test logic from test data. Someone who understands this will design automation that can grow. Someone who does not will build a collection of scripts that gets more expensive to maintain with every passing month.

How they handle failures. Do they treat every failed test as potentially real and investigate it, or do they re-run until it passes? How do they distinguish between a test failure caused by a defect and a test failure caused by an environmental problem? How do they communicate failure information to stakeholders? These questions reveal a lot about how someone actually thinks about automation.

How they connect automation to quality goals. Good automation engineers can explain what the suite is designed to catch, where the coverage gaps are, and what risks remain unaddressed by the automated checks. They can describe what the suite does not cover as clearly as what it does cover.

The cost question managers often underestimate

Test automation has two categories of cost that are easy to conflate. Build cost is visible: someone writes the tests, that takes time, the time shows up somewhere. Maintenance cost is less visible but often larger in the long run.

As the application evolves, tests break. Not because of defects, but because the behavior they were checking has changed intentionally. Keeping the suite current as the product changes requires ongoing engineering effort. If that effort is not budgeted for, teams tend to handle it reactively: fixing broken tests in a hurry before a release, accumulating technical debt in the suite, or letting broken tests pile up until the suite becomes unreliable.

The ISTQB Test Automation Strategy certification covers the cost analysis side of automation directly, including setting up and maintaining test automation costs, identifying ways automation brings value to the project and organization, and defining metrics that help drive decision-making. This is precisely the knowledge that helps automation engineers make a credible business case for the program they are running.

When evaluating automation proposals from your team, asking "what is the expected ongoing maintenance cost of this" is as important as asking "what will this catch."

What ROI from automation actually means

Return on investment from test automation is real, but it is often described imprecisely in ways that make it hard to evaluate. A few clarifications are useful.

Automation saves time only if the tests it replaces were actually being run manually. If a test case existed in a manual test plan but was only run occasionally, automating it does not save the equivalent of a manual tester running it every day. It saves the amount of time that test was actually being run.

Automation delivers value only if it finds real problems. A suite with a 99% pass rate that never blocks a release because the failures are either false positives or low-priority issues is not delivering much value, regardless of how many tests it contains.

Coverage breadth matters, but so does coverage quality. A large test suite that covers many features shallowly may miss more problems than a smaller suite that covers fewer features with appropriate depth and rigor.

These distinctions matter for management decisions because they affect how you evaluate whether your automation investment is working. Metrics like number of tests, pass rate, and code coverage are easy to measure, but they do not tell you whether the automation is actually finding the defects that matter.

How the ASTQB Test Automation Pro designation helps managers

ASTQB Test Automation Pro is awarded, upon request, to professionals who earn ISTQB Foundation Level, ISTQB Test Automation Engineering, and ISTQB Test Automation Strategy through ASTQB & AT*SQA. ASTQB describes it as a way for employers to identify professionals who have proven knowledge across testing fundamentals, automation engineering, and automation strategy.

For managers, that combination signals something specific. It means the person has studied not just how to write tests, but how to build automation that is maintainable and scalable, and how to make the strategic decisions that determine whether an automation program delivers real value.

This matters because many automation job candidates can demonstrate some automation experience. Fewer have studied the full picture: the engineering architecture side and the strategy and governance side together. The designation makes that combination visible.

ASTQB also offers the Test Automation Accelerator, described as a cohort-based program for organizations moving teams through the certification path. Volume exam pricing is available through AT*SQA for teams with multiple candidates.

What to look for when building your team

When hiring for automation roles or developing people internally, a few practical markers are worth looking for beyond the basics.

Can the person explain an automation architecture decision and the reasoning behind it, not just describe what they built? Can they articulate what the current suite does not cover, not just what it does? Do they think about maintenance costs proactively, or only after something breaks? Can they translate automation results into terms that are useful to non-technical stakeholders?

These are not things that show up on a resume bullet for "automation experience." They are things you discover in conversation, in how someone describes past work, and in whether their thinking about automation goes deeper than which scripts they have written.

ISTQB certifications, especially ISTQB Test Automation Engineering and Test Automation Strategy, are one signal that someone has engaged with the body of knowledge behind these questions. They do not guarantee all of the above, but they are a meaningful indicator that the right concepts have been studied.

The next step

If you are building or evaluating an automation program, the ASTQB Test Automation Pro path is worth understanding even if you are not pursuing it yourself. Knowing what the designation covers helps you understand what expertise in this area actually looks like.

Start with the ASTQB Test Automation Pro designation overview or the FAQ for common questions. If you are building a team and want to explore volume pricing, AT*SQA has a contact page for those inquiries.