Software testing is a crucial discipline within the development process, designed to identify bugs, verify functionality, and ensure the overall quality of software products. 

Inadequate testing can have far-reaching consequences – something the CrowdStrike mishap in the summer of 2024 clearly showed. Understanding what you can do to mitigate risk has to be a high priority. 

What does software testing entail?

Software testing is a process used to evaluate the functionality, performance, and reliability of a software application. Software testing is divided into two areas depending on the tester’s expertise: functional and non-functional. 

Within these forms of testing, whether it’s functional or non-functional, there are a variety of testing types one can do:

The primary goal is to ensure that the software meets the specified requirements and is free from defects that could impair its performance or usability. There are some key areas of interest, including: 

1. Requirement analysis: understanding the requirements of the software to create appropriate test cases where testers think thoroughly about all the negative scenarios first and then add some positive scenarios. 
2. Test planning: this is where the tester develops a detailed plan that outlines the testing strategy, scope, resources, schedule, and deliverables. Testers tend to use simple test plans and not the chunky documents like in the past.
3. Test design: this is the creation of detailed test cases and scenarios based on the software’s requirements and specifications.
4. Test execution: is the running of the test cases on the software to identify any defects or issues.
5. Defect reporting: testers will then document any issues found during testing and work with developers to resolve them. Testers and developers can also pair in this to triage the defects.
6. Test closure: This is when a tester can say they have completed all compulsory testing for a feature. It achieved all it must to reach the definition of done. Once this is met, the test is reported as complete.  

These steps aren’t universal, nor are they prescriptive. Some teams may opt to remove certain stages, or complete them in a different way.  It depends on the testing values of the team, but some version of this process should be present in engineering workflows. 

Why CrowdStrike would have benefitted from “shifting left”

"Shift left" means focusing on testing, quality checks, and performance evaluations early in the development process – sometimes even before any code is written. 

In July, CrowdStrike released a software update that caused a worldwide outage. By adopting a "shift-left" approach, security issues could have been identified as the code was being written.  

Using Crowdstrike as a case study, here are some other benefits earlier testing would have afforded:

• Early detection of vulnerabilities: early penetration testing, code reviews, and automated security scanning could have detected vulnerabilities. The earlier you catch an issue, the cheaper it is to fix. 
• Shift-left approach: by adopting a "shift-left" approach, security issues could have been identified as code was being written. This would involve security considerations being part of the design, development, and continuous testing phases, rather than being an afterthought.
• Improved code quality: continuous testing throughout development, including static code analysis and dynamic testing, could have ensured better code quality. Catching errors at the code level reduces the likelihood of significant issues arising later, which was the case with CrowdStrike's vulnerability.
• Reduced risk: early testing lowers the overall risk of vulnerabilities making it to production. For CrowdStrike, whose products are in the critical field of cybersecurity, such an oversight could have had serious implications for customer security. 

Testing can take on different shapes

Not all companies follow these testing shapes to the letter. Some firms have a pyramid, some an ice cream cone, and some a mix and match of all. The main goal here is to have testing as part of the development life cycle and have it planned early. 

When advocating for earlier or more involvement, be prepared to hear things like “There isn't much to test right now” or “We don't need you right now.”  However, you should stay firm and look through requirements regardless: ask questions, build some edge cases, look into environments if they are ready for testing, evaluate components at risk, have a risk register.

It’s understandable that teams may not want testers around their code, given that developers can review it themselves. But, having specialists on board introduces a new thought pattern and additional expertise.

Testers are also able to evaluate non-functional aspects such as security and accessibility. This ensures that code is adherent to relevant regulations and industry standards. Ultimately, these efforts help to improve the user experience, making it smoother over time.

Final thoughts

Software testing is a cornerstone of successful software development, playing a vital role in ensuring that applications are reliable, secure, and meet user expectations. By understanding the testing process and acknowledging the potential risks of inadequate testing, organizations can better appreciate the value of investing in a robust testing strategy. 

The CrowdStrike mishap serves as a poignant example of what can go wrong when testing is insufficient, reinforcing the need for diligence and thoroughness in the process.