Software testing

Building a Test Automation Strategy: Options for Client Application Testing | Whiteboard Friday

Welcome to episode 3 of our Whiteboard Friday series, Building a Test Automation Strategy: Client Application Testing. In last week’s episode, we covered what areas your test automation should be focused on by reviewing two test automation pyramids. This week, we’ll discuss your options for automating client application testing.

Watch the video

In this episode you will learn:

  • Options for programming languages
  • How to tie your programming languages to your test automation framework
  • The pros and cons of various automation frameworks

Additional Resources:

Using qTest APIs: How Tricentis Customers Scale Test Automation with qTest

Here’s the full transcript:

Hey everyone! Welcome back to “Whiteboard Friday,” where we continue our series, “How to Build Out a Test Automation Strategy,” and today we’re gonna be talking about options for client application testing. So the reason why we’re talking about client application testing is because predominantly when we talk to people, that’s primarily what they are testing against. So that just could be like a client application that’s on your server, let’s say an SAP application, or it’s an application that is something that you have bought from a third party. Or traditionally it’s more of like a web application that you operate through Chrome, or Safari, or Firefox, or whatever. Those are usually the applications that we’re talking about here.

And usually when it comes to that, there’s some options to think about when you’re going out and thinking about what type of test automation strategy you want to, kind of, tackle. So today, we’re gonna be talking about two things. One of the things is around the programming language options that you have. So, in test automation, everything is code. You have test automation that is being run through a certain language, and really, we wanna figure out is what language is best for your team, and it’s really dependent on some certain things that could drive that conversation.

So the first one is, looking at, kind of, what makes sense based off of what the programming code is actually written in. And what I mean by this is that, let’s say you are an IOS developer. Well, iOS has its own testing framework, kind of, built into that, I think it’s called XCUI test, X-C-U-I test, that’s probably best for you to use for that, right? If you’re using Angular, there’s a tool called Protractor, a framework called Protractor that’s using Selenium, right? So you probably wanna use Protractor based off that. Just an easy way for you to get into your automation, if you look at, kind of, what’s the application you’re testing under, what’s the code underneath it, and what’s the tool that by default really supports that.

The next thing is looking at, kind of, what makes sense for the test team. So what this means is, if you’ve got a test team that’s really gung-ho on using Ruby for their test automation, let them use Ruby, right? There’s Python, it’s another popular language that’s been, kind of, taken off recently. It’s really easy. It’s an easier coding language to use for testers. A lot of people use it, either in a Robot framework, or something else. So if you look at what the testers want to do, then maybe an option as well, to say, that they’re really enthusiastic about some programming language that’s easy to learn, go ahead and let them use that to drive your automation strategy.

The next one is really looking at what technology the devs know. And this is a pretty easy one. But, you know, when you’re looking at debugging your test automation scripts, and your test automation code, what you really wanna do is needing help from your development team, test automation engineers, your astests, right? And really, if they know…let’s say they know Java really well, well it may make sense that you do your automation in Java. So they can help out with that. It also just makes the coding that’s actually done in your application in the automation to be more uniform.

And the last one is really looking at, you know, the intelligent point and click type of recorder. That, kind of, gets into a little bit more of a linear way of doing your test automation, but there’s lots of tools out here that have more proprietary languages behind those, that are capturing the automation through point and click technology. You can probably think of a lot of these that have been done in the past, more at legacy, and there’s even some more in the future. People are trying to get away a lot from using this as a recorded playback, and we’ll talk a little bit about that in a minute. But that’s another option for you to take a look at.

And the next thing looking at, okay, so now let’s say you’ve got your test automation language you’re gonna use for your code, right? Well, the next question is, well, how do you then tie that into maybe a test automation framework, right? So a framework really is just looking at the glue, and the rules, and the guidelines that really help string together all your automation code into a way that’s reusable. And so lots of ways to do this, right? When we talk to customers at Tricentis, it ranges all across the board, right? They say, you know, “We have home-grown test automation framework we built on our own.” You know, it’s not something that is TestNG, or JUnit, or something like that, right? They’ve actually gone off and built something proprietary behind their own firewall, and it’s in their environment, and really is built by their developers and their automation engineers, and, you know, it’s in Java, right? So that’s kind of what we look at.

Real quick, just to run by everybody, this home-grown is really obviously something that people do all across the industry. And we’ve actually got five API case studies to talk about how people have used their home-grown solutions to integrate into one of our solutions called qTest Manager, and also a reporting engine called Insight. So we’re actually gonna tag this in the blog post at the end of this, because that’s really good knowledge for you to understand and it’s really good technical documentation around that, to learn from people in the industry.

The next one is looking at a modular object-oriented based framework. And that’s what we’re doing, is you’re trying to build an abstraction layer on top of your automation codes. That means you’re separating different logic in between your automated steps, so that you can more reuse it, and also to have single points of maintenance along the way. So that’s one way to do that.

The other way is looking at more of a model-based test automation approach. And a lot of teams are using model-based. They’re traditionally more vendor-locked. So you’re gonna have vendors that say, “Yeah, we do model-based testing.” And it’s really their own proprietary framework, not something that’s really open-sourced. And the reason why they do this is because one, they can specialize and certain applications they can test under. So, let’s say you’re testing an Oracle database, or you’re testing a third-party ERP system, right? They can really specialize in this model-based approach where, what you’re doing is, you’re able to tie all your test back to a particular model. And that model is something that will instantly create test automation scripts based at looking at the underlying code.

So again, it’s a good way to do that for…a lot of times we see this done more so on the packaged software applications that are out there. It’s an option for you, and again it’s for a lot of the vendors that are out there.

Now we’re not covering every single framework that’s out there. The last one is really looking at a BDD style framework. And really, it’s also a processed change of how you’re actually going about your test automation. So, BDD, of course, every time somebody thinks of BDD they think of Cucumber. However, Cucumber is not just a test automation tool, it’s more of a collaboration tool, that helps all the way from, being able to record what specification you’re going to release out into production, to document that in readable business logic, like the Gherkin Given-When-Then, to Type format. And then being able to automate those at the very end, to drive out your acceptance automations.

So that’s another way to look at that. Again, very big push towards this recently, accelerating, and a lot of companies we talk to, are really moving more towards this. Another thing I’ll say, is that we talked a lot about options for programming language, we talked about frameworks, but the underlying thing that you’re also gonna think about when you’re looking at your applications you need to test, whether it’s a web-based application, whether it’s an application that is more server-like, maybe it’s even APIs, which we haven’t really got into, and is really looking at some of my questions which is, are you going to use…and I’ll change my position up here, are you gonna use open-source? Or are you gonna use a proprietary vendor to do this, right?

And our thought is this, is that you’ve got different application you need to test under. Certain applications are gonna require a more specialized technology to do this. So, if you have let’s say, you’re testing a kitchen sink application, right? Or you’re testing a desktop, or set-top applications on a TV, right? You may want to use a solution like Eggplant, that can really tie into the image-based automation that’s on the screen, it’s really hard to pull back objects, right? If you’re using something like a web application testing the UI, maybe you wanna still use Selenium, right? So…And if you’re testing maybe even a ERP system, you may wanna look at using a proprietary tool like Worksoft, right? So it all depends, right? You can have a mix between open-source and proprietary or vendor-provided, right? It all just, kind of, depends on where you have to go. And our position it’s really around, “Do what strategy makes sense for the business applications you’re testing, and not just go all in on one that isn’t gonna be able to test across, you know, our applications.”

So we’re saying, test automation strategies are tough. They’re hard to do, and, yeah, we don’t wanna lock you into just one thing. So that kind of walks through our, third episode around building our test automation strategy. We hope to see you next time on “Whiteboard Friday” as we talk more about getting into design patterns, and also getting into reporting, and then kind of some take-aways you can get off and running, for this. So, glad to see you, and I hope to see you next time.