How do people end up in test automation? Is it a skill people can learn at universities or it’s more of a vocation, really?
There is no one path that leads to QA. What’s more important is that testing is just a subset of quality assurance. There’re so many aspects to think about when people talk about signing anything off from a QA perspective.
In the most recent episode of Semaphore Uncut, we welcome Maryam Umar, Head of QA at Rezolve, international speaker. Maryam explains what the three Ps of testing are, how to advance your career as a testing engineer, and how to start a new position as a tester. Listen to the full episode or read the edited transcript.
What we talked about:
- Starting with test automation
- How Maryam ended up in test management
- How to grow a team from 20 to 500 engineers and keep the quality of the processes
- Three pillars of quality assurance
- How to start a new position: checklist
Like this episode? Be sure to leave a ⭐️⭐️⭐️⭐️⭐️ review on the podcast player of your choice and share it with your friends.
Darko Fabijan (00:02):
Hello, and welcome to Semaphore Uncut, the podcast for developers while building great products. Today, I’m excited to welcome Maryam Umar. Maryam, thank you so much for joining us. Can you please go ahead and introduce yourself?
Maryam Umar (00:18):
Thanks for having me! My name’s Maryam Umar. I work as head of QA in London at an Instant Salesware organization called Rezolve. I’ve been in the QA space for about 15 years, possibly since I was a child.
The reason I say that is because I was always very picky about things being correct or proper organized. If I was making a cake, I wanted it to taste a very specific way. Or if I was submitting an assignment for school, I wanted the formatting to be a specific way.
Maryam Umar (01:04):
I actually came into technology purely by accident. I kind of had no choice. I’m originally from Pakistan. When I was growing up in my school, we had either biology or computer science as an option. I’m very humbled, blessed and grateful that I found something which actually helps me a lot.
For anybody who’s done computer science, they know there’s such a variety of topics you read and do projects on. It becomes quite tricky to figure out where you want to land when you finish your courses or degree.
What I found was that my knack for having things being created in a certain way, so that the receiver is really happy, that kind of fell into testing a little bit and quality assurance. And this just solidified a lot when I did a course in software testing during my course in London.
Starting with test automation
Maryam Umar (02:14):
Then I ended up doing a thesis in mutation testing as well. And I really found it quite enjoyable. I didn’t see it as a drain, so I decided to pursue a career in software testing.
I think from now on the journey I’ve been on, I started on as a hands-on tester where I was doing a lot of things manually. But very quickly I realized that this is becoming really tedious. I want to find a way to automate the repetitive stuff. Because as humans, we are prone to missing things which are repetitive or they can become monotonous, right? And your eye just stops looking at things which are wrong.
As humans, we are prone to missing things which are repetitive or they can become monotonous.-Maryam Umar, Head of QA at Rezolve
Maryam Umar (02:56):
I stepped into test automation within the first three months of my first job, which was a small software house back on Liverpool Street. From then on the journey has been tremendous. I was a hands-on automation engineer for about 10 years. In those 10 years, I was fortunate to work in finance, mobile restaurant booking websites and a very large travel company.
Maryam Umar (03:30):
But what happened during that time was that initially my progression was more about, okay, I want to learn this framework and that framework. Okay, let’s try and stick to one language or another.
For example, BDD was coming up at the time and I actually learned Cucumber, and then SpecFlow got invented. And what I learned in those years was that it’s always best to create test automation in the language that is closer to the production code base. I’m sure we’ll talk about that in a bit.
How Maryam ended up in test management
Maryam Umar (04:01):
But what was missing was the guidance I needed to progress in my career. I’m sure a lot of developers or architects and people like that would feel the same.
After 10 years of writing code, you feel that you’re becoming stagnant and you want good leadership, and I wasn’t getting that at all. It wasn’t because I had bad managers, it’s just that they didn’t have the education.
So I decided to step into management by choice, not only to help myself, but also to help other testers or QA engineers as we could call them. And now this is, I think year six of me doing test management.
Maryam Umar (04:37):
I have managed teams in Capital One, Caplin, Thought Machine. And now I’m at Rezolve, mostly finance. And I have to say, management is not for everyone. But I’ve found that I really enjoy working with people and making sure they’re doing what they love to do.
Maryam Umar (05:15):
I think in terms of future aspirations, I’d like to stay in this space, but probably bring a hat of release management. And making sure we are getting what the people want is really interesting for me.
How to grow a team from 20 to 500 engineers and keep the quality of the processes
Darko Fabijan (06:03):
One of the things that’s been mentioned is that in just your previous role, when you joined the organization, it was an organization of 100 people. And as you’re just starting your position, that company has 500+ people.
One of the things that is connecting us with all of our listeners, except the topic of this test automation and testing, is that a wide majority of the companies are scaling in some way at their own pace.
Darko Fabijan (06:30):
It’s true for us, it’s true for your current company. And also for the majority of our clients. It’s just the question, are they scaling from 20 to 100 or 500?
You mentioned that you also think a lot about the quality of communication and the quality of processes that are present in the organization as a whole in engineering teams. Can you please explain to our listeners how you see that?
Testing is a subset of quality assurance
Maryam Umar (07:14):
I think the first thing to remember is that when people talk about QA in general, a lot of them are only talking about testing. And testing is a subset of quality assurance.
It’s extremely important to make sure you are creating that understanding in the organization. When you’re saying that this feature has got QA sign off, what is actually happening is that you’re only doing tests, manual tests, exploratory tests, whatever tests you’re doing.
Maryam Umar (07:50):
And then you’re saying, “Yes, I’m happy to sign it off.” Whereas you may not be doing things like, Okay, what happens if this feature’s flag is switched off? Or what happens when this feature interacts with another feature? It could be, “Have you updated the documentation that your customer’s using?”
There’s so many aspects to think about when you’re talking about signing anything off from a QA perspective.
Three pillars of quality assurance
Maryam Umar (08:17):
Secondly, as you mentioned, I’ve seen and experienced that quality assurance is kind of formed of three pillars, at least in my experience. And I talk about it as product, people and process, or the three Ps.
The three pillars of quality assurance – product, people and process.-Maryam Umar, Head of QA at Rezolve
First and foremost, you’re obviously thinking about the product’s quality. That is your first responsibility as someone in an organization, because I mean that’s what’s bringing in the money.
Maryam Umar (08:47):
And if the organization’s aren’t making money, hiring the right people will not happen. Creating an amazing estimation process will not work either. That’s the bread and butter of a company.
Your first responsibility as someone in an organization is to think about the product’s quality.-Maryam Umar, Head of QA at Rezolve
So it’s important to make sure that most of the strategy you create revolves around the product quality. And that can have things like test automation. That can have things like how are you doing CI/CD. That can have things like test coverage and things like that.
The quality of the team
Maryam Umar (09:14):
The second thing is the quality of people or the quality of the team. I’ll start with really talking about more from a QA team perspective. Lots of organizations have cross-functional teams. But lots of organizations also follow the Spotify model. Lots of organizations have QA sitting outside of their teams.
There’s no correct answer for this, “How do you want to structure your team?” Really, you need to understand how the business is flowing. But the type of people you want in the team is what is important here.
Maryam Umar (09:49):
I’ve had the fortune of hiring a team from scratch. But most often you come into a team which was hired already. And either they’ve not had a manager before, or they’ve had managers who were unsuccessful.
First, you need to understand, why the people in the team were hired in the first place.-Maryam Umar, Head of QA at Rezolve
So it’s really about understanding why those people were hired in the first place? What was the skillset they brought on board? And really thinking about what you need for the type of teams these people will support?
Maryam Umar (10:16):
And lastly, the third thing is process quality, which is really where we think about, “Okay, how is our agile process?” This is where you will see a lot of coaches or specialists come in and tell you where you can eliminate waste. And this is also why we talk about fast feedback. That’s what Agile’s about.
And really, are the processes you’re using catering to it or not? Is it taking too long to find out whether you’re developing something in one sprint and getting feedback in the third sprint? Obviously, a process like that will not scale.
And the reason process quality is so important as well, is because if you have the right in place, you’ll automatically become efficient as a wider engineering team.
How to start a new position: checklist
Darko Fabijan (11:07):
Great. And as you’re just starting a new position, this might be on top of your mind, you have to do some exploration. Once you join the exploration of the process, meet the people, see where the company’s going and all that. What’s your checklist?
Maryam Umar (11:28):
There is lots of material I’ve read. One of the books is called The First 90 Days by Michael Watkins. But it helps you understand how to set up yourself for success when you’re starting a new job.
t talks about how to have your first one to ones, and so on. Like some of the things I try to do is I intentionally ask people … Sometimes people try to sway your thinking, even before you’ve met a person in the organization.
Maryam Umar (12:00):
Let’s say, for example, I’m head of QA. I’m reporting to the CTO. And I have to meet the head of product, right? Or say head of infrastructure because most organizations have those roles.
And sometimes people will, instead of letting you form an opinion, they will say, “Oh, so and so is really bad with meetings.” Or they’ll say “You should have a catch up with the head of product. But don’t be upset if they seem preoccupied or something.”
Something about their personality or the way their team works, which already forms a bias in your head.
Maryam Umar (12:39):
I’m very, very mindful at the beginning of this job. I’ve been in this job five weeks now. There were some statements made by my team to me. And I was like, “No, we don’t talk about it like that. We’re all in this together.”
And then two weeks later, I think I was just really getting confused about some information and they started laughing. It’s like, “Maryam, would you like to change your opinion now?” But it’s very important for us to make sure you’re not letting bias creep in your head.
It’s important to make sure you’re not letting bias creep into your head.-Maryam Umar, Head of QA at Rezolve
Maryam Umar (13:07):
The second thing is I take the time to understand the product. So even if I’m not going to write code or ever going to look at a full request, it’s important to make sure you can set it up on your machine.
You can run the app or you have a basic test login to log into their programs. It’s so important because that then helps you understand what your team is talking about. It creates respect there.
Maryam Umar (13:34):
It’s okay. I’m not shy of saying, “I’m sorry, I still don’t understand this. Can you explain to me where the data’s coming from? Or I don’t have access to the database, whatever”
Because the more time you spend in understanding the actual product you’re going to be working on, the better it will be in the future.
It’s important to have one-on-one with the entire team
Maryam Umar (13:49):
Other things which form my checklist are I have a one to one with my entire team. And I have to say, this has been the most challenging piece in my current role. I have about 15 people and they’re all in different time zones. Nobody works in the UK time zone and that in itself is a challenge.
And then in addition to the time zone piece, if you have a truly global team, you’re also looking at global culture. Everybody comes from a different culture. So the way they express themselves in a conversation is completely different.
Maryam Umar (14:28):
Some cultures we can understand, some we don’t. So I have started giving everybody the benefit of doubt because maybe I don’t understand how they’re communicating to me with a problem.
But what’s important is when I do the one-to-ones, I ask everybody the same three or four questions, everybody. Like, What team are you in? What are your current challenges? How can I help you? And do you have feedback?
The power of feedback
Maryam Umar (14:53):
I am very, very, very much about feedback. Because feedback is what will really help you improve the quality of the work you do.
Feedback will really help you improve the quality of the work you do.-Maryam Umar, Head of QA at Rezolve
If you think about it, Agile is all about providing fast feedback. So it’s the same when it comes to people that want to have fast feedback.
It’s important to set time aside for troubleshooting
Maryam Umar (15:15):
And lastly, I think the other thing I do to try and set myself up right, is I try and have some core hours when I’m going to be available for troubleshooting or firefighting. But then you should also spend some time apart where you can actually do some thinking.
This is a mistake I’ve made in a lot of jobs. I would say it’s really important to set that time aside. And also tell your team that this is what you’re going to do.
Maryam Umar (15:44):
Let’s say, “The last two hours of the day, you’re probably going to be doing some documenting of a new strategy. Or creating some visibility, or creating some boards or something like that. Or reading some stuff, which you can apply to the team.”
So yeah, that’s how I try to set myself up in a new organization.
How to communicate issues throughout the organization
Darko Fabijan (16:34):
In terms of those three PS, of course, that’s too big of a thing to have in a single role in any organization, unless it’s a very tiny startup company just starting and there is someone taking care of all of those three things.
What’s your advice on communicating those issues throughout the organization? There might be some people who are more in charge of the process, as you are seeing and learning through the lens of a QA team and you as a QA lead. And you are noticing some things in the organization that might be tweaked.
Darko Fabijan (17:08):
It can be related to the engineering part of the organization, or a product focus, or a product direction and so on. What will be some of your advice, how to broadcast that throughout the company or to request a change or facilitate that change?
Maryam Umar (17:31):
This is quite tricky to do, as I’ve found in my last couple of roles, I think, especially when it comes to processes.
So for example, let’s just take an example. One thing I try to do when it comes to test automation is to try and use test automation as a quality gate. So most organizations, if you’re following git-flow or whatever process you’re using for CI/CD, we want to move from dev to staging, to pre-pro to production, right?
Usually, let’s just say those are the four environments. And we want to have quality gates in place saying, “Okay, these are the tests we want to run. If they’re green, then it will automatically progress to the next stage and so on.”
Maryam Umar (18:08):
Now, that’s a vision from a test automation perspective. If you think about it, that’s for product quality. But then if you’re going to try and enforce something like that in the pipeline, you will definitely have pushback from the engineering team saying, “Oh, let’s just say the end-to-end test takes too long to run. And it’s taking too long for us to go to staging, which means that we are down from 10 deployments a day to one deployment a day. Let’s just say, that’s the comment we are getting.”
Maryam Umar (18:40):
The problem then becomes that it’s not really about quality assurance. It’s more about the team being precious about their process.
The problem then becomes that it’s not really about quality assurance. It’s more about the team being precious about their process.-Maryam Umar, Head of QA at Rezolve
And that’s where my role then turns into, you have to articulate the value. And I use the word articulate rather than show here. Because if we are moving to staging quite quickly, let’s just say, but we are still finding one hot fix every three days, of course, that’s a lot of effort gone.
Maryam Umar (19:13):
That’s the number you need to articulate to the team that, “Look, if we are waiting for these tests to run, yes, they’re slow. Yes, we can make sure they can run in parallel. Or we can increase the number of pods that we are using to support these tests, et cetera, which is a technical implementation.”
But you really need to articulate the value that putting those all quality gates in place will bring.
“People in the QA space are highlighting…”
Maryam Umar (19:34):
And you cannot have the QA team implementing those gates everywhere. I see people in the QA space as people who are highlighting, where we can improve and helping us go into a preventative state rather than in a reactive state, as an organization when we find issues with the product.
Then, if you think about the people and the process piece, kind of all of it overlaps each other. Because one thing is creating a vision that okay, to improve product quality, we want automated quality gates.
Maryam Umar (20:07):
But then we need the right people who can implement them with that skillset and that mindset. But then also the implementation of it.
And if you want to do it over multiple teams, then you also want the right automated process in place. So, if you think about it, it’s just a way of presenting how we can improve something.
More on the process quality
Maryam Umar (20:30):
What I’ve seen, the most clashes you get is when you talk about process quality. It’s not even about people’s quality. Because at the end of the day, everybody has a team that they want to run in a specific way, which is fine.
But it’s something the team lead or the engineering manager should really have a vision of how they want to do, and they do.
Maryam Umar (20:48):
The process quality is something which all of us have a different experience of. Some of us have come from really small startups. And we think, “Okay, we can just deploy to production every day. Nobody’s using this API yet.”
Or you can have people coming from the finance sector and say, “Okay, we cannot do this until we have PIN testing done on this piece.”
When it comes to process and it really then becomes about, are you going to do what the people are saying that needs to be done? Or are you going to do what Agile tells us? Or are you going to do what the Scrum Master is saying?-Maryam Umar, Head of QA at Rezolve
So you can have those two extremes, right? When it comes to process and it really then becomes about, are you going to do what the people are saying that needs to be done? Or are you going to do what Agile tells us? Or are you going to do what the Scrum Master is saying?
Maryam Umar (21:23):
Again, it becomes more of an advice, and that’s where a lot of the metrics come into play. Dura-Metrics, which a lot of people now know of or use, they’re mentioned in the Accelerate book or the State of DevOps Report. They’re really useful to really help you understand.
Because sometimes you can have different processes being followed by different teams. And that’s also okay, as long as the output is the same. And that’s what we need to focus on rather than drill down on what one team is doing for estimation and one team is doing for a number of days. That doesn’t work. Well yes, we know it doesn’t work.
Maryam Umar (21:59):
But if they’re all producing the output at the same time, yes, forecasting becomes difficult. But at the end of their day, it’s about choosing your battles. And figuring out what is not giving you the desired output or what is creating more chaos and trying to fix that first.
How to proceed with one’s career as a QA
Darko Fabijan (22:15):
As you said, you entered that QA area from childhood, but you did enter it during university. That master that you did and other papers that you wrote. And you gave us the overview that you initially did some testing.
Three months later, you had those things automated. And then you spent a decade in that domain. What are some pieces of advice that you would give to people who are for sometime in the QA space and maybe are feeling stuck at this point in their career and they would want to break out of it?
There are various directions that people might want to take. What are some of the options that you have seen people taking, or you can maybe yourself consider those?
Maryam Umar (23:01):
So when you’re in the QA space, a question which you see, because that’s the behavior in your organization says, “Okay, the next step for me is to become a developer because I’ve done test automation for a while. I can write these frameworks.”
That’s probably a very senior tester. Let’s just go according to levels. And I actually, with my experience, resisted that move a lot. Because that is not what I know I’m good at.
In your heart you know what you’re good at.-Maryam Umar, Head of QA at Rezolve
In your heart you know what you’re good at. And if you’re a good QA who has made it to that level, then a lot of QAs, who become quite senior, start writing frameworks. Or they become test architects in the organization.
So they start working beyond BDD and all of that. They’re the ones who are looking into, “How can I do automated quality gates? Why don’t I wear that hat of adding a DevOps skillset to myself and really help move to testing left?” for example.
Maryam Umar (24:06):
Another one is, which I haven’t seen a lot of, but there’s a huge requirement for it now, is non-functional testing. There’s not a lot of people who understand it. I raise my hand. I still need to read a lot about it because it is difficult to understand.
People only think about non-functional testing as low testing and stress testing. But that’s not the only thing. There’s also accessibility. There’s reliability, there’s security. There’s so many variations of it.
Maryam Umar (24:34):
But then again, don’t start looking into all of them. Look into one, master it. It becomes difficult, I understand that. So it’s those areas, which I feel are still very much uncovered.
For the exploratory tester, who’s quite senior and there are lots and lots of people I’ve met whose books we’ve read and so on, there’s so much more you can bring to their table by devising ideas and visualizing where the gaps are.
The value stream thing technique
Maryam Umar (25:02):
For example, there’s a technique called value stream thing. You could do that and show how long it’s taking you to actually find out whether this new feature will work with another feature. Where is the waste?
Because again, it’s about being more efficient and trying to prevent issues rather than two teams completely finishing two features and then integrating them.
Maryam Umar (25:24):
Another way of doing this is, I’ve been very fortunate to work with a couple of very good exploratory testers. Who actually looked at things like impact mapping and so on. And actually trying to understand the architecture. Putting our hands up when there’s a technical task.
For example, I’ll give you an example. There was once a story where the team was going to add a new API. And whenever that API was triggered, there was supposed to be a new entry made into the database table or something like that. And that test engineer just held their hand up and said, “This task is too technical.”
Maryam Umar (26:00):
I disagree, because that task basically means you have to open up the database, run that task manually and see if they increase, that’s not difficult. If you can’t even do those things, you don’t work in tech. It’s not about using the application. It’s about making sure that the data flows in the application correctly as well. So you can expand to that. So this is probably for senior testers going back to mid-level or junior people.
Maryam Umar (26:26):
Again, there’s such a lot of opportunity there. I try to keep myself up-to-date with blogs published by the Ministry of Testing and InfoQ. There’s just so much information out there. Even Stack Overflow has some quite interesting ideas there, or people whose books you’ve read.
All of these people actually still do a lot of speaking. A lot of the conferences, content is now on YouTube. Thankfully, because of COVID, a lot of things have happened virtually. So a lot of this is now easily accessible.
Maryam Umar (27:00):
I would actually try and find their time to read and keep myself up-to-date with what is actually the new techniques. So for example, I’m trying to look for a front end automation engineer. And everybody I’m speaking to has come up with Cyprus. And my question is, “Why did you choose Cypress? You need to understand the why, rather than just following the trend, right?
Maryam Umar (27:25):
So everybody’s using Kubernetes today. Tomorrow it could be something else and everybody will start using it, not knowing why they need to use it. So always make the questioning of the why, which is kind of the core.
Always make the questioning of the why-Maryam Umar, Head of QA at Rezolve
I would say, the skill of a QA engineer is something you should always use when trying to learn more and go in a direction in your career.
Darko Fabijan (27:45):
Great. Great. You answered my last question before I asked it. I was about to ask what resources you suggest to your team members so that they can level themselves up.
From all of those valuable resources, in the end, it’s probably the most important one that is often overlooked. Why am I choosing this database? But everyone else is using it, so it must be right.
Maryam Umar (28:14):
It must be right. I’m sure it is right. But you also need to do your own diligence, right?
Darko Fabijan (28:19):
Yeah. Understand why is that right. And why is something else maybe not, right.
Maryam Umar (28:23):
Darko Fabijan (28:25):
Thank you so much for this great conversation. I’m sure that it’ll be very valuable for a lot of our listeners. Yeah, again, good luck with your new role. I’m sure that it will be very exciting setting up a new team.
Maryam Umar (28:38):
Yeah, exciting is a very small word. No, but thank you so much Darko for this. I really, really enjoyed speaking to you. And I hope I’ve managed to help some people through this conversation as well.
Darko Fabijan (28:53):
Thank you again.
Maryam Umar (28:54):
Thank you. Take care. Thanks everyone. Bye.