Test cases are multiplatform

Hundreds of user platforms are covered: real platforms used by real users. Platforms are chosen by the customer. Security aspects are dealt with according to the customer’s needs.

Test cases can cover a large set of conditions

• Interaction and navigation (like missing buttons and fields in a screen)
• data and text (like correctness of computed values or feedback messages)
• look & feel (like visibility of some elements, visual alignments and responsiveness).

A single test case can exercise the application under test in several ways, bringing efficiency and effectiveness to the test process.

Test cases are high level

• Test cases document and describe requirements: there is no need to describe the latter in detail. Tests are living documentation of the business logic.
• With 121T it is easy to decide which tests need to be executed or developed.
• With 121T test cases can be easily maintained compared to other lower-level software test automation approaches based on record & play, in which test cases are fast to develop, but turn out to be very expensive to maintain.
• 121T test cases can be easily understood by anybody for revising them, for identifying gaps in test coverage and overlaps with other tests.

Example

This is an actual executable test:

Given the page "http://www.gmail.com" is loaded
When logging in with username "USER-1" and password "something"
And creating a new message with recipient "USER-2@gmail.com"
And writing subject "121T example" and body message "Message content for the demo"
And sending it
When logging out and preparing to log in with a different account
When logging in with username "USER-2" and password "something-else"
And confirming identity through "city" with value "Udine"
When opening the "1" message
Then check that the sender is "USER-1"

Notice how:

• Steps and test conditions are easy to understand
• Steps and test conditions can be reused with other data
• The test is independent from the implementation of the user interface
• The test is independent from the technology used to simulate the browser

Test cases are automatized

Automation is necessary to carry out regression testing, especially for user acceptance testing: automating them is the only way to actually do regression testing. Tests can be run unsupervised at scheduled times and on demand; therefore can be run repeatedly on same platform or on different platforms.

Automatic reports are provided

Executive summary style of information: providing a quick overview of the test run and providing requirements coverage. It is easier to assess what has to be tested and what needs to be testes. Reports provide details of each single test case, including screenshots of tested application under test, therefore letting anybody to figure out what happened during the unsupervised execution.

Test code is versioned

In addition to tracking changes, with 121T we can simultaneuosly test one or more concurrent versions of the application under test. Test resources, such as databases and user data, are versioned as well. This provides a reliable test environment, which leads to repeatable results and a robust deployment process.

Test runs are managed through CUTE Frontend

CUTE (CUstomized Testing Environment) provides a web-based user interface that you can use to configure a test run. Test cases, the test environment to use and the user platforms can be easily selected.

CUTE Frontend lets you explore test results by viewing the automatically built test report. The report includes overall coverage metrics (screens and user actions that were covered by the test execution) and lets you access a Jira system that provides the test plan, definition of all test suites and test runs already completed. The specifically configured Jira system presents also reports with requirements coverage charts.

Test cases are related to use cases

Test cases developed with 121T are derived from conditions of satisfaction of features and therefore they can be easily related to business values and risks. Therefore impact of a test failure can be easily understood by a business analyst.

This is not usually the case with other approaches relying on lower level unit/integration test cases.

121T software is not technically coupled to the implementation of the application under test.

This means that the software is independent of technical choices made for development of the application under test. Coupling is achieved only with respect to external behavior of the application.

Investment on testing is protected against changes in the adopted implementation technologies. In other frameworks this is not possible.

121T test cases can be easily maintained against frequent changes in the user interface of the application under test

Because of the model-driven technology that is used to support them, test cases are isolated from the implementation of the user interface.

Therefore test cases become an asset that is protected from frequent changes in the user interface.

Our software testing automation complements other test automation strategies

121T complements unit and component testing. The risk-based planning in 121T helps identifying capabilities whose testing should migrate from end-to-end tests to lower level testing.

Our software test automation provides ways to address automation of acceptance testing, of critical end-to-end system tests done through the user interface, of testing the user interface component itself.

Another important use is to perform “semantic monitoring” of a distributed architecture. Automatically running a dozen of usage scenarios in a periodic way collects a lot of information of the health status of the underlying components, by simulating what real users do. This is complementary to using software monitoring tools, such as Nagios or Zabbix: the latter are normally not appropriate to simulate realistic, deep, usage scenarios.

Our test cases are manually crafted

They are optimized for reuse and mantainability. They rely on test conditions that are carefully choosen to exercise the system to address the chosen quality risk.

Our test harness is automatically derived from models

Models can be quickly developed and updated; therefore the test harness can be automatically updated to reflect changes to the user interface.

Test cases are protected from fragility due to changes in the implementation of the user interface.

Our test cases are at a use case level

As mentioned above, they document requirements, they make it easy to understand if a risk is properly covered, they can be easily understood. Test cases can be easily exported into html/pdf representations that make up what can be called a “living documentation“.