Apple Podcasts Google Podcasts Spotify

Have a look at our new "Complete Guide to Optimizing Slow Tests"! Discover →

Back to the list
Episode 66 · Jul 26, 2022 · 28:11

Janet Gregory on Holistic Testing: Where Testing and Coding Meet

Featuring Janet Gregory, Agile Testing Consultant
Apple Podcasts Google Podcasts Spotify

Testing plays a big role in software development; especially, in quality assurance. However, when these tasks are rigorously divided, testers can find problems getting along with engineers.

Still, to Agile Consultant Janet Gregory, testers and engineer teams only thrive when working together. In this episode, Janet will help us understand the nuances of these teams’ cultures and the benefits of collaboration for testing and software development in general.


Agile consultant and writer Janet Gregory started as a tester very early in her career. At her first job as a programmer during college, actually. While holding the developer title, Janet’s superiors made it clear to her that, while in charge of writing the code, she was responsible for testing it as well.

This culture of developing and testing as two sides of the same coin followed Janet once she left university and found a new job in the clearing system of the former Vancouver Stock Exchange. About the experience with code there, Janet recalls that “we were writing it in COBOL, so there were no unit tests.”

The lack of testing tools, however, favored a strong testing culture. A failure in the system could lead to potential losses, so she and her colleagues had to carefully scrutinize whatever code they wrote before deploying it.

What’s the right culture for testing?

Testing itself has been often portrayed as getting in the way of coding. In the past, Janet repeatedly came across the belief within the software development industry that developers were best left alone.

“Developers, don’t talk to them. They were the most important people: ‘Just slide the pizza under the door and let them do their coding’ ”, she ironizes.

Yet, as Agile methodologies began to take the foreground, the line diving and confronting testing and coding has diminished. As such, while Janet continues to see in a few companies the idea of providing complete autonomy to developers and their code, she also views that nowadays developers and companies are significantly more aware of the practice and importance of testing:

People are asking, ‘What problem are we trying to solve?’’ They’re looking at it and thinking together; It’s not two people who go off and figure it out and come and tell everybody else

-Janet Gregory, Agile Testing Consultant

Software engineering and testing cultures

When it comes to cultural differences between developers and testers, whether it is the case of Agile teams or not, Janet believes that it is difficult to come to reasonable terms if these keep a distance between their roles.

To describe this, Janet uses the analogy of developers sending their code to testers as if it “was thrown over the wall”. This illustrates that “when the testers are testing after code is built, it naturally becomes an adversarial relationship.”

In the dynamic of engineering and testing teams isolated from each other, every time testers find an error in code and report it, it becomes a burden to the developer, who can take it as a critique of their work and even take it personally.

Likewise, in case the developers fail to fix the code, the tester may take it the wrong way as well and regard the developers as unwilling to cooperate.

How to reconcile software engineering and testing cultures

Janet regards collaboration between engineers and testers as key from the moment a feature is introduced:

I think that magic happens when the team is on that journey all the way right from the beginning of a new feature, and then takes it right to production and understands what’s going on and how our customers are using it.”

-Janet Gregory, Agile Testing Consultant

Through collaboration, tester teams can learn from features as soon as these are introduced, and build tests for them learning what risks they could bring to the product, how to mitigate such hypothetical risks, and what’s the most convenient form of testing for these features.

In other words, depending on the nature of this feature and what could compromise, testers may opt for performance testing, load testing, recovery testing, etc.

However, Janet doesn’t ignore that achieving this level of collaboration isn’t easy, and even less when trying to meet product deadlines or with many products going on simultaneously. These should also come into the organization’s culture.

To Janet, companies should favor cross-functional and collaborative ways of working. She understands that “it takes the whole team to make a quality product; you can’t treat them as separate.”

As such, Janet proposes what she calls holistic testing as a way for developers and testers teams to work together.

As she defines in her webpage, holistic testing is a software development model that embodies testing and coding as part of the same feedback loop process.

Moreover, in holistic testing, testing goes beyond coding and is present in every instance of product design to maximize product quality. At the same time, holistic testing also encourages designing quality attributes. Quality attributes are arbitrarily chosen functional and nonfunctional requirements used to evaluate the product quality.

Janet recommends quality attributes to acknowledge the risks a failure in the system can bring, and plan the quality attributes and testing as aware of those indicators and avoid those scenarios.

Learn to prioritize

On the other hand, Janet believes that to create a culture that ensures a quality product, it is important for organizations and teams to learn what to prioritize. In this regard, while in favor of outside consultants, Janet proposes using from time to time team members to observe how the team is working.

In this way, team members can learn to take a step back and have fresh eyes to judge what they’re working on: “it could be as simple as how we are testing, which is not simple, but we could stop and think about it. Should we be testing differently? What has changed in our product? What has changed in our organization?

Maybe we grew from being a small team of 10 people to 60 people. Do we have to change how we think about dependencies? Do we have to change how we interact with other groups?”

Janet has a similar approach to quality assessment. Quality assessment is the practice of reviewing the code, architecture, and testing to learn if it complies with development and industry standards.

When it comes to quality assessments, Janet believes that it’s all about asking questions:

“We often do assessments based on the practices as outlined by, choose any methodology you want. Are we doing the practices? But again, it comes back to, are we looking at why we’re doing those practices? What is the reason behind it?”

-Janet Gregory, Agile Testing Consultant

She also found that teams shouldn’t be sorted out in a single quality level, since the depth of their different activities hardly translates into even results:

They might be at the beginning in chaos around some of their testing, but they’ve got really good planning, so teams are never on one level.

-Janet Gregory, Agile Testing Consultant

The bottom line

Janet is currently writing a book on quality assessment along with Agile Coach Selena Delesie. Check out Janet’s books on Leanpub and visit her webpage to learn more about her, her courses, and her publications.

Meet the host

Darko Fabijan

Darko, co-founder of Semaphore, enjoys breaking new ground and exploring tools and ideas that improve developer lives. He enjoys finding the best technical solutions with his engineering team at Semaphore. In his spare time, you’ll find him cooking, hiking and gardening indoors.

twitter logolinkedin logo

marko podcast host