Types of tests

No matter if your organization is agile or waterfall oriented or follows one of the many hybrid variants. Sooner or later you’ll have an increment - an outcome from your software developers or customizers. You paid for it. You want it in production. But will it work? Will there be any unwanted side effects to existing functionality?

Increment testing

Usually an increment is tested manually by human testers who are not identical with the developers.

Depending on maturatiy of your organization and many other factors, the testers will be more or less clearly instructed, what to test. They might have written business requirements and deduct the test cases themselves. In ideal setups they were part of the development lifecycle, know the deviations from original requirements, pitfalls and workarounds and can adjust their test expectation accordingly.

Unless you’re in a greenfield situation where the whole system landscape needs to be tested and retested for months or years your Testers will focus on testing the increment - not so much the existing functionality, which used to work fine already.

Use baangt already in preparation of this test phase. Create all the test cases, that you plan to execute. Create all the data combinations, that you’ll want to have tested. Once the functionality is there, record the most complex scenario in the recorder. Instead of testing 100s of cases manually, you’ll need only one recording and the prepared dataset. Start the TestRunExecution, sit back and wait for the results. Simple like that.

Heartbeat and Alive-Testing

Alive-Testing is usually done with just one quick test case in all stages (Dev, Pre-Quality and Quality-System). It will show general availability of the landscape and applications running on it. Alive-Tests with some APIs could run for instance every 5 minutes.

Heartbeat tests are a smaller subset of regression tests. E.g. if you have 10.000 testcases in regression tests, you’d use a few hundred for heartbeat tests. They’d usually run a few times per day on Pre-Quality- and once per day on Quality-System) and of course in the build pipeline.

Regression testing

If you followed through on Increment testing imagine the joy of the next release! You’ll have the increment tested and run all test cases of previous increments as well. That’s called regression testing. If you did everything well use the results of regression tests and increment tests as rock-solid base for your decision whether to move on to production or not.

Performance testing

So you did regression and increment tests, moved to production and receive countless complaints from users, that the performance of the system is too slow. Additionally there are now bugs that appear due to timeout situations. Damn.

What happened?

You tested only for functionality, but not for load. With a few simple adoptions to your test cases you can simulate any number of users. To achieve realistic performance testing you’ll need more hardware for testing than for regression and increments. But you’ll use the same tool: baangt.

As of today (Jan 2020) baangt does not provide infrastructure monitoring. In order to analyze the results of your performance tests you’ll need additional tools, but baangt will give indications, which components or which functionalities need a closer look by your experts.

End to End (E2E) Testing

Whenever you have more than one system/microservice dealing with a process, you’ll need E2E-Testing. Of course E2E-Tests are more complex than just running test cases against one functionality and compare results to the expected values and behaviour. In larger organizations you’ll want to have E2E-Regression tests before you release increments to production. baangt follows a structure of TestCaseSequences where you combine multiple single Testcases into one Sequence, which is exactly tailored to run E2E Tests.

Lifecycle tests of business objects

Lifecycle tests come in basically two variations, but can be combined - depending on the requirements of the business. Many industries deal with objects, that follow a certain (long) life cycle. The life cycle can go over years or decades. These tests are complex and cost a lot of time and effort.

Time travel tests

Often companies have “Time travel” system landscapes, where they create copies of the whole system landscape (or large parts of the core systems), change the system time on all servers and run tests subsequently with different dates. baangt does not support this type of testing out of the box. But we provide a functionality to “Pause” Testcase and TestCaseSequence execution. You can easily subclass the corresponding master classes and create your own mechanism, when to pause a Testcase or TestCaseSequence.

Cradle to the grave

Another common form of lifecycle tests. In this case the system time remains basically the same, but the test cases are created in a sequence to follow the birth of an object until it’s deletion. This might be a material, which get’s created, production recipe created, work planned, sales contract and order created, produced, delivered, invoiced, paid and revenue calculated. In service industries C2G-Tests are designed around a customer. baangt fully supports complex testcaseSequences running on multiple technologies (Web, API, etc.) also in asynchronous scenarios, for instance if you need to wait for nightly batch processing of a mainframe.

No oversimplification

Please don’t get me wrong. Just because we have a great tool, it doesn’t mean that testing will happen by itself. There’s still a lot of expert work needed for Testdesign, Stagedesign, Creation and maintenance of Testsets, creation and maintenance of test data sets, deployment strategies. baangt provides efficient ways to work, but work still needs to be done.