Test automation management tools

Test automation management tools are specific tools that provide a collaborative environment that is intended to make test automation efficient, traceable and clear for stakeholders. Test automation is becoming a cross-discipline (i.e. a mix of both testing and development practices.)

Motivation

Test automation systems usually need more reporting, analysis and meaningful information about project status. Test management systems target manual effort and do not give all the required information.

Test automation management systems leverage automation efforts towards efficient and continuous processes of delivering test execution and new working tests by:

• Making transparent, meaningful and traceable reporting for all project stakeholders.
• Easing test debugging through test results analysis workflow.
• Providing valuable metrics and key performance indicators – both technical and business-wise (trend analysis, benchmarking, gap analysis, root cause analysis and risk point analysis).
• Grid benchmarking and comparison of test execution days reduces analysis and review effort.
• Clean traceability with other testing artifacts (test cases, data, issues, etc.).
• Organizing historical data.
• Post-project analysis and automation performance assessment. (Progress of test coverage shows the group performance.)

Compliance with Agile

Test automation management tools fit Agile Systems Development Life Cycle methodologies. In most cases, test automation covers continuous changes to minimize manual regression testing. Changes are usually noted by monitoring test log diffs. For example, differences in the number of failures signal probable changes either in AUT or in test code (broken test code base, instabilities) or in both. Quick notice of changes and unified workflow of results analysis reduces testing costs and increases project quality.

TDD

Test-driven development utilizes test automation as the primary driver to rapid and high-quality software production. Concepts of green line and thoughtful design are supported with tests before actual coding, assuming there are special tools to track and analyze within TDD process.

Continuous Integration

Another test automation practice is continuous integration, which explicitly supposes automated test suites as a final stage upon building, deployment and distributing new versions of software. Based on acceptance of test results, a build is declared either as qualified for further testing or rejected. Dashboards provide relevant information on all stages of software development including test results. However, dashboards do not support comprehensive operations and views for an automation engineer. This is another reason for dedicated management tools that can supply high-level data to other project management tools such as test management, issue management and change management.