Your Software Testing Dream Team
How to assemble and manage the best people
Do you know what to look for in your tester dream team? Good testing is affected by the quality of the team in place and you need to know how to go about assembling your team from scratch. So, what are the first few things you should consider?
I spoke to Christopher S Saleh, a QA software professional who has more than 25 years experience across a variety of software applications in various platforms, vertical markets and industries. Based in Austin, Texas, Christopher is also the Chief Knowledge Officer at SG Interactive, the R&D arm of Ritzio Entertainment Group, a large gaming conglomerate. He has been working constantly on research and evaluation of tools to improve processes within QA and software testing. He is currently in the initial phase of creating a QA and Software Testing school, with a fresh business model that addresses the shortage of skilled candidates and the degree to which it forces companies to outsource a few critical QA and testing responsibilities.
Aimed at the beginner to intermediate tester, our conversation takes a look at what factors to consider in bringing a team together, critical processes to put in place, considering personality vs experience, communication strategies and much more.
Become a CTFL qualified tester
To what extent, is the quality of the team in place, a major determinant of the success of testing activities?
Christopher : Let me start by expanding on how the success of testing activities is viewed. The effectiveness and optimal productivity of QA (Quality Assurance) are determined by assembling a team with the proper skills, knowledge as well as a few intangible attributes. The team assembled will effectively and methodically select the most appropriate tools and hardware/software resources needed to complete their testing objectives throughout the entire SDLC (Software Development Life Cycle) and at each iteration. This is a good starting point to creating an effective team which can successfully complete the testing tasks and activities. Having said that, it is equally pivotal that the QA team is supported by capable developers and software engineers. This completes the picture.
What, do you believe, are the five main qualities you should look for in creating your dream team?
Christopher : In creating an optimal and efficient team, regardless of the size of the team, you have to ensure your team is diverse, bringing together a range of skills, strengths and experiences needed, bearing in mind that what is needed depends on the project and the product/service. This is key.
Assuming the entire team is well versed in QA and standard practices in Testing, these are some of the major qualities I would look for:
- problem-solving skills;
- process-oriented skills;
- self-starters who have no need to be micro-managed;
- a mixture of creative members and members who are more logical and pragmatic. It is important to balance the team with both.
What are some critical things to consider when assembling a team from scratch – what should be the key focus areas?
Christopher : Firstly, the key is to evaluate and analyse the software product for the project at hand. Then, assemble a team that addresses all the QA and testing needs of the product from start to finish. You need to bring on the personnel, setting them up to succeed based on their skills, experience, education and their personalities.
If I create a template job description, and hire the needed members based on matching that one job description, I will not have a diversely skilled team. There will be areas that skill-wise, will be over-represented (an overkill) and other areas that will be lacking. However, it is common practice to hire an entire team based on the same set of skills but it usually is not by choice or oversight. At times, it is due to a shortage of certain skill-sets and expertise. Often, this presents a short-term problem and a long-term dilemma.
Testing tools should not be selected based purely on the fact that team members have experience with those tools.
Secondly, you should develop a process consistent with the methodology used by the development team, the company culture and its core values as it affects how priorities are set and milestones met.
Thirdly, it is critical that the right tools are selected for creating, assigning and executing all the pre-determined tasks. Testing tools should not be selected based purely on the fact that team members have experience with those tools. While this is a good thing to consider, you also need to evaluate whether these tools are the best fit for the application and type of testing to be conducted.
I have had more success working with an unfamiliar tool that meets the testing needs (and have even had the team ramp up on the tool) than using a familiar tool that is not a good fit for the given project.
When considering the right personality fit vs years of experience, in your opinion, which matters more and why?
Christopher: The important thing here is relevant experience as well as understanding the core function of QA and testing. Personality, however, should always be considered, to a certain degree. The goal is to build a cohesive team that can communicate with one another as well as across management and developers.
Regardless of the experience within, there comes a point in time wherein an individual will not be as productive and as efficient in one team as in another. When you consider these issues, it can lead to a win-win situation when properly evaluated during the hiring process.
What personality types, have you seen, as the best fit, for software tester roles?
Christopher: As mentioned earlier, the most functional team is a diverse one, with a variety of skills and experience that complement one another. It is also a team with many different types of personalities. A good team member would :
- have problem-solving skills;
- have pragmatic critical thinking capability;
- be imbued with a creative and rich imagination;
- be detail oriented with the ability to focus deeply and not be easily distracted; and
- be steady, calm and collected under pressure or in a crisis.
What are the most important personality requirements for a good test manager?
Christopher: I would say the following are needed:
- an ability to delegate and build trust and confidence within the team by allowing them to ‘own’ the project and be accountable;
- an ability to zoom in and out of daily tasks and issues that surface without losing sight of the big picture and objectives;
- possess the team’s trust and confidence;
- the ability to lead by example. Never have I been in a position where I did not work harder or just as hard as any member of my team. The person to always be held accountable first is the manager; and
- an ability to maintain a level of respect for all.
The success of a good manager is often evidenced by the success of those he manages.
What is the most basic level of domain knowledge that a tester should have? And what of that for a test manager?
Christopher: You’ll probably find my answer different from that of others but I believe that the most basic characteristic needed is curiosity. Thereafter, I would say they need to know the fundamentals of how products work and any variations thereof. That knowledge will continue to grow.
The same is required of a test manager but far more, from an experience perspective than from a theoretical one. The primary factor for a manager is that they must know how to respond to the unexpected because that is always a case of ‘when’, not ‘if’. They need to have a level of self-confidence and trust in their instincts, to calmly work through any crisis.
Every person in a testing team should have defined roles – do you agree? If you do, how do you go about defining what these various roles are?
Christopher: Yes, role definition is absolutely necessary. However, I believe in redundancy. Each member should have at least one primary role and one or two secondary roles with periodic rotation. This is part of my research into a new testing approach that is more effective and suited for the agile development environment.
Here are a few of the standard roles and how secondary roles could be added to it:
- Functional Testing – front end / back end;
- User Interface (UI) Testing;
- Database Testing;
- API (Application Programming Interfaces) Testing;
- Load/Stress/Endurance Testing;
- Test Automation – what to automate and what not to automate;
- Test environment / builds validation;
- White box / Black box / Grey box Testing;
- Security Testing / Negative Testing;
- Exploratory Testing;
- System Testing;
- Unit Testing / Integration Testing; and
- User acceptance testing (UAT).
Some of these testing activities overlap but may vary in duration or some other aspect.
Do you believe that there should be a balance of strengths and weaknesses in the individuals in the team? If you do, how would you suggest that this balance be achieved?
Christopher : There certainly should be strengths within a team. However, the approach that I eluded to earlier is that where a redundancy is created, the effect of any weakness is then minimised and strengths magnified. The way to achieve this is through the provision of secondary roles for team members who will spend time cross-training through OJT (On the Job Training).
In your experience, what do you believe makes for effective communication within the team? What apps or tools help in this regard? What are the absolute “no-go’s”‘ ?
Christopher : Communication is vital and an integral, causal part of a team’s success or failure. Things can very quickly get out of hand when there is a lack of effective communication which needs to take place not just among the QA / Testing team but with project managers, the development team, key executives and stakeholders.
I have broken down effective communication into three parts, nicknaming it CCC or 3C’s. They are communication, collaboration and cooperation.
Once formal and informal lines of communication are created via status documentation, there should be daily, weekly and start /end of a release formal meetings. This creates a form of live or near real-time broadcast and documentation of status, issues, completed tasks and analysis reports. From there, the team is then able to work out the collaboration and cooperation necessary, issuing deadlines and identifying ownership of issues as required.
What are things to avoid? You do not want to get into any situation where a critical part of the deliverable or requirements are changed after the QA has been signed off because any additional elements or changes made, at this point, would not be tested at the previously determined and agreed upon threshold and date.
As for useful apps, there are several good open source apps or a suite of applications that manages the different aspects of development, compiling, builds and regression testing. Although there are many good tools, what you consider the best one for you is dependent on your project, the platform used, programming languages etc.
This is a fluid situation where new tools come to market and older tools are revamped and packaged as entire suites of QA tools. However, some of the more well-established companies and tools on the market are the Atlassian products which include Jira, Confluence, Bamboo, SourceTree, and BitBucket, each serving a specific much needed functionally within the testing cycle/development cycle. Note that none of the above are meant for actual testing, they are largely used for organisation, collaboration, documentation and communication among many other functionalities. You can find out more about the Atlassian here.
Their top three products are :
- JIRA – to plan, track and release world-class software with the #1 software development tool used by agile teams;
- Confluence – spend more time getting things done. Organise your work, create documents and discuss everything in one place; and
- Bamboo – continuous integration, deployment and release management.
In terms of Testing tools, you could look at SoapUI for backend and API testing. You can find out more about SOAP, REST or any of the other protocols here.
SoapUI started as, and still is, an open source tool but a company, SmartBear, is utilising the open source functionality and adding additional features and functionalities for a price. It is one of the more popular API testing tools. They complement SoapUI with other tools, creating a testing suite which is called “TestComplete”.
HP has invested and bought software testing companies and created the HP ALM suite. The HP Application Lifecycle Management is another product that has been around for a while and is popular.
For those who want to avoid purchasing pricey licences for these tools, the trend seems to indicate the use of open source tools such as SoapUI, JMeter, Selenium etc. and getting it customised to meet their testing needs. Again, it is a critical judgment call for you to make. Sometimes, it is the best choice forward but sometimes, it is not based on the variables I mentioned earlier.
You will want to have a source code repository management system tied to an automated build system which deploys into a pristine test environment where testing can begin. You will also want a bug tracking system. The more that these various elements are integrated with each other (which is pretty much the norm nowadays), the smoother the cycle will be.
There are many stakeholders involved in the final product, each with differing priorities. These may include the development team, project managers, service team, product manager, customers, senior management, sales and the testing team itself. How do you decide on the who to pay more attention to, how much weight to place on their feedback and what each considers to be critical components of the product being tested?
Christopher : You want your QA / testing team to be involved from the start ie from your first meeting with the stakeholders, where you discuss their requirements and what they want. As the business requirements are converted into technical requirements, QA will study and work out how to test each requirement. Once these individual elements are broken down and then agreed upon, it will be documented and signed off by all parties involved.
The master QA plan should also refer to contingencies in that any request for a change or addition will need to be discussed and agreed upon for it to be included in the current release. If the change or addition is requested after testing is in full swing, then the requests can be added in future releases.
Having worked with US State governments, I have seen exceptions being made – in cases where there are changes in the state law – but these are the exception. In most other cases, the goal should be to identify and then clarify what can reasonably be delivered within a particular timeframe. This process, in my opinion, has been refined and therefore, much more manageable now than the longer cycles found in projects using Waterfall or V SDLC methodology.
In your experience, how do you begin to understand the testing responsibility of your team? How do you start to discover this? How do you deal specifically with the gaps within the team?
Christopher : It is essential that members of the QA / Testing team (not the entire team) are involved from the very beginning. This provides more time for QA to understand and itemise their responsibilities. The short answer is planning. When there is enough planning, QA is fully prepared. As they proceed, they are able to identify gaps using effective communication 3C’s and fill such gaps in situations where there are additional personnel or additional tools are required.
Testing tools should not be selected based purely on the fact that team members have experience with those tools.
Let’s say you’ve just assembled your dream team today. You’ve got a product to test tomorrow. What are 3 – 5 critical processes you would put in place to support your team?
Christopher : Hopefully, these critical processes are already in place and have been used for several weeks. In any case, the most critical processes, at the very start, are :
- Test plan, Test scenarios and Test cases are already in place and documented. All team members are familiar with it and update it as testing proceeds;
- All team members have received their assigned duties and responsibilities, either as individual members or as a small sub-group of the testing team;
- Test scripts are already being written. These are a component of the test case within a test scenario;
- A reporting system is in place. It has either been automated where tests performed get recorded, logs are created and a traceability trail for review is there, if needed; and
- A process has been designed and agreed upon that once a bug or defect is found and verified, it is then logged and reassigned to the original developer to fix. This is documented within the reporting system.
What are some typical mistakes you’ve seen made in how teams are formed? What would you have done differently in this regard?
Christopher: One of the biggest mistakes I have seen is making an assumption without verification and validation. An example would be your assuming a test engineer’s skill level to complete the assigned testing. On discovery, you then realise that their skills or experience have been over-estimated, thus requiring additional mentoring or training.
Another mistake I have seen is when people do not take maximum advantage of the learning opportunity provided by mistakes or inefficiencies discovered. The best move forward would be to analyse such mistakes or inefficiencies so that you can make improvements to the processes, communication or the execution of tasks.
Christopher Saleh received his Bachelor of Science in Social Psychology, followed by Master of Art in Psychology while working in Information Technology. After 10 years working as a systems analyst, he completed his Masters of Science in Computer Information Systems in 2000.
The start of his IT career coincided with the rapid emergence of new automated software testing tools. He has utilised early generation software testing tools. He has continued to research and evaluate tools and works on improving processes within the QA and software testing industry. His current challenge is to create a QA and software testing school to train/retrain candidates in having the skills and knowledge needed to fill the much-needed QA and software and system testing positions.
If you think this post is interesting, please share using the buttons below!
Team image courtesy Arte Ram of freeimages.com
#softwaretesting #tester #Tech #QA