Agile testing

A New Approach to Test Planning

Watch Tricentis Product Manager Elmar Pauwels introduce  Tricentis Tosca’s new Test Planning component. Elmar uses a real-world example to explain how the new test planning component can help you accelerate and simplify your planning process. You’ll learn why we developed this new functionality, what it involves, and how it can assist your team.

Here’s the full transcript

Welcome everybody. My name is Elmar Pauwels, I’m part of the product management team at Tricentis, and today I have the honor to introduce you to the new Tricentis Tosca Test Planning component. I’m going to talk about why we did what we did, what we actually did, and also a tiny bit about how we did it. Nevertheless I want to take this chance to show you what drove us to go in that direction.

So, let’s get started. Do you remember what test planning was like back in the old days, or maybe not even that long ago. Well, I do. Test processes were usually awfully complicated and convoluted, and although they were you know the roughly documented and each step had a different owner and everything. Let’s be honest, it was a mess because it was almost impossible to maintain, they were almost never up to date, and the result was that test miniatures were drowning in paperwork most of the time. With paperwork, I mean Excel sheets. You now, there were tons of Excels, one for the test plan, one for the test cases, one for the results, and there were multiple versions, and multiple different locations. The most awful thing was that they were also interconnected in some magic way that, if it all. Not even the test manager might remember what exactly he set up back then, and it was almost impossible to predict setting a result or entering a value at some point that was actually another point.

And, instead of focusing on testing, common questions I had when testing were, “Did they update the right Excel?,” or “Did I send the repost to the right bunch of people,” or “What is the test status actually?”

Most of the time, it might not be easy to say: “What is the test status right now?”

So thank God these times are over, and since an increasing number of companies are actually moving into the direction of agile and DevOps, it is quite obvious that the old way of testing and planning the tests just won’t work anymore in the future. So we need some new moral. We need something else, but the question is, what does this new model look like? How is test planning looking like? During this transformation towards agile and DevOps, how is test planning transformation? They are good questions to ask.

Actually, as an example, let’s have a look a the 12 principles of agile software development. You do not have to read all of it. I just want to highlight that there are some parts in it that are, or can at least be related to testing.

For example, early in continuous delivery, quite obvious. If you want to release early and continuous, we all, of course, need to test early and continuous. Working software, well I don’t need to talk about that I guess.

Maintaining a constant pace – indefinitely.

What do we need to do to keep a constant pace? We need to have a test portfolio that is actually executable within a certain amount of time within a sprint or iteration, whatever Agile methodology is in place. So it’s not only about keeping the developers up to speed, but of course also having the tests in the shape that is repeatedly executable.

Maximizing the amount of work that is not done.

This of course, does not only apply to development, but also to testing. This means thinking about what to test, what to test in which depth, where to focus on, where to start.

Self-organizing teams, that’s actually the most interesting, because that’s sometimes often, also an excuse, saying, “Okay the teams need to take care about testing themselves,” which is actually not a bad idea to begin with, but what I want to highlight here is that it’s just talking about the approach, meaning the “what.” What do we want to do? We want to do agile, in this example. We want to do scrum, we want to do whatever, but it’s not enough to just talk about “what,” but to also talk about the technique, the “how.” How should we be able to get there? Let’s say we want to do agile. Okay, that’s nice, but we also need to consider the technique of how we want to get there. So if we want to become agile, which test methodology do we use? How to we structure all this? How do we keep them maintainable? How do we keep them executable within a certain amount of time, and what’s happening most of the time that seems. I mentioned the self-organizing teams, or self-organized teams, teams need to take ownership of their product, of the product quality, and this means that leaving it up to the teams is actually not too bad. I’m not against this, but the point is it can result in the fact that each in the department team or project test slightly different.

So they all choose their approach that works for them, one do my CI, one do more API, one do more into service virtualization, but every team has a slight different approach. So this makes it just hard to come up with a consistent approach towards testing. So we need to also consider the technique, and that’s actually also the question that we had in mind when we were thinking about the test planning component; how to keep track of all those possible different tracks. Where can the projects go? Where are they? What’s the state? Which methodology do they use? So we wanted to come up with a simple, lightweight, but still powerful enough tool in assisting you in keeping track and overlooking the chaos of what’s happening out there.

So this is well-known as the tool of choice when it comes to dealing with testing activities, like this case design, this case specification, execution; you’ve heard of all of this in the last one and a half days.

So we’re good with activities, but sometimes it’s not enough to just plan, report, execute all of the activities. So test-management, and our understanding of test-management is actually also to be able to plan no only the activities, and not only monitor and execute the activities, but also have a rough plan and the structure around those activities. So managing the testing process means actually, activities combined with a plan. That’s our understanding of test management, and that’s actually also the reason why we went in this direction.

Let’s be honest, this was one of the weak spots and gaps of Tricentis Tosca 11.0, but this is our field. So with Tricentis Tosca 11, this gap is filled. We are there.

We want Tricentis Tosca to evolve from test execution monitoring, to test process monitoring, and from execution planning to process planning, and from execution management to process management.

The point is, covering the whole test process within one tool. You heard it, THE platform: Planning and executing, and reporting over the whole testing process. So that it doesn’t even need to be within Tricentis Tosca, it can also be an activity that is happening outside, like preparing some environment, or whatever. We’ll come to this point again, and as an example regarding those different approaches and testing activities as I was mentioning, let’s look at three development teams.

So let’s assume we have the three development teams. You might already be able to guess what kind of methodology they follow. So one is doing some kind of agile development, one is doing some waterfall approach, and the other one, let’s say kanban. It doesn’t really matter, but just to illustrate that there are different types of methodologies, and that’s actually what we observe out there with our customers in the early environment, that there are some teams that are already introduced to agile processes, to scrum processes, whatever, but there is still some things or some systems, systems of record most likely, and systems of innovation up there, that is still maintained or extended in a kind of waterfall way, and to make it even more real, let’s assume they’re spread all over the world.

They’re sitting in different locations, different time zones and so on. Just imagine too, come up with a consistent test plan that covers all of the potential activities of those different approaches and those different teams that are not even co-located, so they’re spread. I mentioned this jus at tough challenge, and why? Because for example, let’s look at the agile team. There are developers and team members that are talking about sprints. They have all the iterations. They ask for feedback. They get feedback. They start to shape the features, re-implement and so on. It’s an iterative approach. That’s what I want to say, and they mostly are doing user story-based testing, which is quite obvious because they have a back log full of user story, and they are working on these user stories. So it is quite obvious that they are using user story-based testing.

On the other hand, if you’re following a more waterfall approach, you will have phases – like dev and test phases, different types of test phases. It’s a non-iterative approach. You can have iteration, but it will be a long one actually, and it’s mostly specification-based testing. So there’s a document against which you test. So this is already a different approach, and if you have some other team, for example doing kanban, they have their swim lanes. There can be iterations, but it’s unnecessary to have them. The same for the estimates. In agile, they’re usually estimates. So every user story gets some kind of effective currency, which allows you to plan over a certain amount of sprints, which is not necessarily in kanban, for example, and those are the commitments to a certain delivery package for a certain amount of time, because usually in agile for example, or even in waterfall, you have a commitment to say “Okay,” over a certain timeframe in waterfall – it’s a bigger time frame, but in agile, it’s it’s usually a sprint, like one or two weeks is okay. The team commits to a certain package, and the product owner, or stakeholder, can inspect a certain product increment over a certain amount of time – which is not necessarily the case there.

So it’s already quite tough to cover this, and there are all the mixtures and if we mentioned this on an enterprise case, we can only have three teams, but imagine if we have 30 or 40 teams that are spread out sitting somewhere, some co-located, some not, different mixtures and different approaches. So this is quite tough, and that’s actually where Tricentis Tosca test planning comes in.

So what we wanted to do is to cover the “what,” because I was talking about the “why,” the evolution of Tricentis Tosca, the “what,” to actually be able to deal with those scenarios; we came up with a session-based test planning tool, so a session-based model. I will talk in a second about what session-based actually means.

We wanted this tool to be flexible enough to cover both classical, as well as modern (such as agile, DevOps approaches), and even mixtures.

We have already heard the term water scrum fall today. So there was certain mixtures and scrum-ban and whatever. So, we wanted to be flexible enough to cover as much of them, and also us already mentioned to cover the whole test planning process within one tool.

The best thing for you is if you have a Tricentis Tosca license – it’s for free!

No additional license is needed and it’s a part of the standard commander package. So whenever you purchase a Tricentis Tosca commander license, then it’s already in. No additional money needed for that.

So with 11, if you download and install it – it’s there.

Why did we choose the session-based concept?

Because we wanted to provide you with the framework, that is already mentioned, flexible enough to cover as many approaches as possible, but still sets some boundaries on the planning approaches and on how to plan, and which new planning artifacts to use, and so on.

So if you have a project that evolves over a certain amount of time, we want to set the boundaries in which your teams can move around and choose whatever approach is needed. This should be yellow, green, and blue.

So we want to se the boundaries to cover those approaches and mixtures, and the teams can actually revolve within those boundaries. They can choose approaches, they can have mixtures, and everything. Nevertheless, you’re able to cover and get a result about what’s the progress, where are you with your testing, what’s the status of your test process on a broader scope than just the execution.

To wrap up the “what,” you might wonder what the session-based concept actually is. The session is basically just a time box. Basically just a start and end date. It’s a time box enriched with all the necessary information: you can add a description, you can add attachments, you can describe in plain text what’s the scope of the session. Within this certain timeframe, you have a certain goal you want to achieve, or a certain scope you want to cover, and of course you can invite users to the session.

So you can add users and say, “Okay, you are participating within the session,” and of course, as already mentioned, you will have the status and the progress for the overall session.

You might say it’s quite similar to exploratory testing? That’s actually intentional – So yes, it’s the same structure, it’s also session paste, but we wanted to keep it as similar as possible. There are some deviations, but since it’s the same approach, was that okay? We don’t really need to re-invent the wheel, we just continue the well-known session-based approach, and it works pretty well.

So that’s a rough idea of a session. If we look at an individual user, John Doe in this example, he can get a user-specific description, and a user-specific attachment as well. So you can break down the scope of the session onto the level of the user and say, “Okay, within this time box, the user please focus on this do that run this and that test,” and such that the user knows what he has to do within the session, and the best thing that is also new that wasn’t there before, the user will get a notification.

So, whenever they log into a workspace, or they do an opt it all, they will get a notification, but there is something for them to do if they’re added to a session, or if they have some other work assigned, and quite likely the most interesting thing that is also deviating from the path that we took in the exploratory testing, you can break it down even further.

So from the session level, to the user level, and even further down to individual tasks. Within the session, you can say, “The user, please if you do not just want to describe, what session, or what he should do,” you kind of have to break it down into tasks, and say, “Here John Doe, please run the regression, build some tests for the new features, and after you’re done, send me a report,” for example, and the users will also get a notification about their task assignment.

Whenever a task is assigned to a user, he is informed, he gets a notification, and he immediately knows that there is something, and you can also assign the tasks to other users. In this way, you’re able to cover workflows or test cases that are involving different departments, different people. So, for example, the first guy starts. He’s doing his part of the test. For example, creates a customer, or whatever, and he gets back the number. Then he can assign the task to the next person, and say, “Okay, please I graded the customer for you, please do something with that. The next one will again get a notification for that, that he has a new task assignment, and so on and so forth.

So you can break the scope of the session to two individual activities or tasks, and they can be, if needed, also assigned to other users and tasks. So basically, again, time boxes. So within the session time box, the task can also, if needed, that’s more needed for the more classical approaches. You can also have a time box, and say we have two days to execute this task, for example, and again, the description and attachments to narrow it down in a certain amount of time that is actually doable, and the best thing, actually, is that on the task, you can also assign Tricentis Tosca artifacts. Why does it make sense?

In this way, you’re able to see how this activity, how this task within a session, actually affects your test portfolio. Which Tosca artifacts, which test cases, or requirements? Can be any Tosca artifact is actually can be affected by this activity. You immediately know if I run the regression, you need to consider the new requirement set, and the new test cases, and so on, and to still be able to deal with the old-school planning, you can also attach Excels, for example, or PDFs, but you know that’s a common functionality within Tosca. We can of course attach any file.

If you already have some kind of issue tracking tool for example, if the sprints are already there, for example, in an agile environment, that’s basically also just a time box that we can import. We are talking to the guys behind Tosca Connect and we will come up with something – although it’s not in 11, it will come. When all of this is set into motion, then you will see the progress of the activities, of the tasks, of the users, and of course, of the session, and in this way, with whatever approach you choose – since it’s just time boxes – they can be longer, they can be shorter, they can be chosen as like, and this way, we’re able to cover any approach that we were considering, and even mixtures of that.

So that’s the “what.” That’s what you can expect from Tricentis Tosca 11, but of course we’re not done yet. There is more to come with the next releases. We’ll enhance the artifact by redefining test phases, and sharing them in order to say, “Okay, we have this and that time frame,” and dear testing managers or your testing teams, or development teams, please consider this time schedule. It will further enhance the user area, because we want to make it as simple as possible for the session users. They should not need to set the status of each task, for example each and every time they want or need to drill down into the test planning section.

So they need to test all this within the user area. We will improve the session user management, you know, the whole invitation, and adding and so on of users, also a visual overview – to provide you with a visualization on how time boxes are set-up and how aligned they are with everything. Milestones – some are related to the phases, the notifications will be enhanced, and also we will come up with a responsibility to define your own view on that, because it can happen that your development teams plan the test cycles. They’ll still test activities, and you will say, but test engineering managers will mostly be interested in the rough overview.

So they can just pick certain sessions, and say, “This is of interest for me from team A. This is of interest from team B, and combine them to a customized view, allowing you to see exactly what you’re interested in, and of course, most importantly – your feedback.

So, as always, we’re relying on your feedback. We need to know what you’re thinking about this. Please let us know about our new Tricentis Tosca Test Planning tool – we are happy to incorporate any feedback we receive from you.

If you want to learn more about our Tricentis Tosca Test Planning tool, be sure to check out our webinar on ‘Reinventing Test Planning for Agile and DevOps.