It’s been a month now since I attended Test Bash Manchester. I heard two very powerful talks at that conference which have been swishing around in my brain for a while now. Both talks came from speakers that shared a desire to advance the craft of testing.

The first talk was by Martin Hynie (@vds4), currently Director of Test Engineering at Medidata. The second talk was by Michael Bolton (@michaelbolton) a tester, collaborator, coach, consultant, author and Twitter super star.

Martin’s talk “The Lost Art of the Journeyman” and Michaels talk “Where do you want to go today? No more exploratory Testing” both invoked the same feeling in me. Change is still very much needed when we are talking about testing. Martin said that only by identifying entrenched beliefs can we find opportunities for change. He explained that one of these entrenched beliefs is what “testing” means. So to invoke change we need to approach from the same side as someone that doesn’t understand testing.

Both speakers talked about testing being a craft. Martin went a step further and said that testing is not a commodity.

I still frequently see testing treated as a commodity by people which do not work as testers. I get embarrassed when people believe smart self directed testing is of equal value to scripted testing. It’s also very hard being the person trying to explain that someone’s beliefs around testing are hindering and causing damage to a project. The belief that all testing is equal is one of those entrenched beliefs Martin told us to be mindful of.

Michael Bolton sums this up on his blog where he says scripted testing is expensive, time consuming, leads to inattentional blindness. Separating the designing of a test script from its execution in turn lengthens and weakens the feedback loop.

Michael told us that scripted testing makes testers incompetent as they are not empowered to think.

The Word ‘Empowered’ Matters.

As someone doing self-directed testing without a test script it can be very easy to criticise testers that write and work from test scripts and test cases. I have worked with financial institutions which rely heavily on scripts. I have met and spoken face to face with testers that work in this scripted way. Seeing things from their point of view I discovered some of the constraints they have to work within. They are not empowered to throw the scripts away. Management want them to work in this way as it is easy (yet foolish) to measure testing with numbers and stats.

When I worked in the UK games industry, I was lucky that I was able to do testing without scripts, but I was still not empowered. I was stuck behind a wall with many Devs throwing any code they wanted over the wall at a small group of testers. If bugs got missed, that was the testers fault - not the fault of a dysfunctional way of working.

Michael spoke about how the definition of testing had been stolen from testers. Now testing meant something completely different to people outside of testing. He said that the testing community needs to steal the definition of testing back.

What is Your Definition of Testing?

I have recently started asking some of my developer friends the following question: ‘What is your definition of testing?’ Some of the answers have shocked me!

The first Dev I asked said ‘testing is ensuring quality’. I had to try explain that this wasn’t entirely true. Testing is an activity that evaluates something (which could be anything) to find problems that matter. The discovery of those problems could have very little to do with ensuring quality if no action is taken once they are discovered!

My challenge to other testers would be start asking people you work with for their definition of testing. Start getting a feeling for how closely your ideas of testing are aligned. Just because you are using the same language does not mean that you are talking about the same things. Do not make the mistake of assuming everyone’s idea of testing is the same.

Michael wanted us to return to using the word testing (not exploratory testing - which he said was like calling a cauliflower a vegetarian cauliflower). Martin wanted us to change the language we use for describing testing and testers.

At an open space event on Saturday 28th October 2017 a diverse group of testers sat around a table and openly discussed the testing role. Specifically the language used to describe that role. One thing became very clear very quickly - The language and definition of testing are certainly not shared between testers and non-testers. Even some testers present had slightly conflicting ideas. We certainly have a lot more work to do in this area.

Patrick Prill (@TestPappy) said that he knows people called tester but what they are doing does not match the job ads. Recruiters are have a very hard time when it comes to describing job roles. Instead of hiring testers, maybe we should be hiring people with critical thinking skills. Maybe the best testers aren’t actually testers yet?

At the open space gathering it became clear that recruiters can be blind to what testers do. Both Neil Younger(@norry_twitting) and Martin Hynie shared their experiences of pairing with a recruiter. Essentially working together to identify good/bad candidates and the reasons why. Both had positive outcomes from the experience of a recruiter and a tester pairing up and working together.

From my own experiences, observations and conversations I am aware that some skilled testers are still not getting hired. ‘Manual tester’ has become a dirty word used to devalue testers. I have heard some pretty crazy things this year. I was asked recently by a recruiter if I knew anyone suitable for an ‘Automation Tester’ position. I also met a manager I met that told me ‘most of our testers are manual but they have been with us a long time so rather than replacing them we are going to to train them to be automated testers.’

The first thoughts that went through my head was what is an ‘Automated Tester’? Automation is a development task. There is no such thing as automatic testing. Automation is dumb, it can not direct itself, it can not explore or think. Further to that, automation in testing should be the responsibility of the whole team, not a single specialist. By putting the responsibility of an automation project on the shoulders of just one person you are heading for disaster (see the term bus factor).

A Keyword CV Search is Simply Not Enough.

When hiring testers, a keyword search on a CV is simply not enough. This comes back to a need to realign the language we use to talk about testing in the context of ‘that thing testers do’.

As well as starting conversations with people we work with about the definition of testing. I believe testers also need to start sharing information with recruiters. This was one of the reasons I was very keen to write and share an article with a recruitment blog. By sharing understanding and knowledge around testing skills and testing work with the very people that are trying to hire us, we not only make things easier for the people trying to hire, but we also make things better for people (like us testers) trying to get hired.

If my job suddenly switched from software tester to recruiter these are some of the things from my experience of testing and testers that I would take with me when trying to specifically recruit testers.

Stop filtering out testing candidates based on certifications.

ISEB/ISTQB really is not a good filter for testing candidates. When I surveyed 187 testers in 2016, only 48% had completed ISEB/ISTQB foundation certificate. I do not hold this ISEB/ISTQB qualification. Some of the brightest smartest testers I know also do not hold ISEB/ISTQB qualifications. There is a big difference between being able to learn answers to some multiple choice questions and test software. By demanding this qualification you will also probably alienate the kind of people you want to attract. Smart testers know these qualifications exist for profit to make money.

Everyone these days puts agile on a CV, this does not mean they are agile.

Better things to ask a candidate rather than looking for the ‘agile’ keyword:

• Ask them about a time they changed their mind or changed course.
• Ask about some experiments they have done within a team.
• Ask about a time they collaborated or paired with another team member.
• Ask how they eliminate wasteful testing documentation.

Acknowledge that there is no automated testing

There are no automated testers. Automation is its own development project and should be owned by the whole team. It is possible for someone that can automate (e.g. write code that checks things) to not understand what should be automated. Writing code is a different skill to being able to decide what that code should do.

Acknowledge that there is no manual testing.

There are no manual testers, there is only testing. Trying to divide testers into two groups of manual and automated is a big mistake. Please stop calling testers manual, we don’t like it and it damages our craft. If instead of labels we focused on hiring candidates with the ability to think critically and solve problems everyone would be in a much better place.

This post was originally published on my software testing blog Mega Ultra Happy Software Testing Fun time.

• conferences