How FAGMA Test and Maintain Their Software

Google has three main roles for their developers: Software Engineer (SWE), Software Engineer in Test (SET), Test Engineer (TE). Their involvement in the testing and maintenance procedures depends on their role. The SWE mainly works on the development of new features but they have to do unit testing. They also work closely with SET to write test cases relevant to the feature. The SET also writes codes but their focus is more on the testability of the code. They review design and refactor code to make it more testable. They also write testing frameworks and automation and are concerned with code quality, risk, and performance . The TE does the testing from the user’s point of view. They write codes for automated testing and simulate usage scenarios. TEs do more user-focused testing and ensure that the software meets performance and usability expectations. (Whittaker et al., 2012)

Other tests Google do are:

Tests are usually done manually and automated. They release software in “minimum useful product” version then do a quick iterative successive version. Their process goes through canary development, testing, beta, then to release channels. Simulations of different user environments are also part of the process.

Apple has a well documented process. They have a release time frame to layout the development efforts throughout the year and prepare for any changes that might happen. They have an Apple New Product Process (ANPP) that elaborates every detail of every stage of the design process. Their testing procedures are divided into sprints where they test and review, then iterate the process to improve the quality of their software.

Due to the issues of their iOS 13, they made a shift in their testing procedure. Since they do daily and weekly commits of new changes, which they call daily builds, by default, they’d disable unfinished and buggy features in test versions of future software updates. Only testers can enable those features to isolate the impact of each addition to the codebase.

Apple has their own “white glove test” where they measure and rank the quality of their software using a scale of 1 to 100.

Facebook does not invest in dedicated testers since they believe that social media is “non-essential” so reliability and performance doesn’t have to be high-quality like the other tech companies. They said that even if it goes down for a bit, people would still go back. Since they practice perpetual development, they do small and rapid deployments. The frequent deployments mean that each update only includes a few new lines of codes which also minimizes the risk of failure. Developers do the testing themselves. They do both manual and automated testing.

Tests done are:

Amazon does not have complex testing procedures. Their testing and maintenance rely heavily on their deployment process. They do automated build and release process and deployment. Each deployment goes through integration testing, browser and web-based testing, and load testing. Through these tests, they monitor and measure everything. Then they will roll out the changes to one box in one availability zone where each availability zone is independent and if a problem occurs there, other availability zones won’t be affected. If the deployment works, they roll out to the entire availability zone, then to more availability zones, and regions. In case of failure, they will immediately roll back to a known stable state.

Since their deployment process is mature, the roll outs and the roll backs are quite easy and fast. But before they deploy, their code is submitted for review and feedback before commit. Static analysis is also performed. The release pipeline is also being checked and there are canary monitors that do more checks before the code goes out to production. Their ultimate rule states that no deployment to every region at the same time.

The ratio of testers to developers in Microsoft is around 2:3 and the salary is the same (Dijkstra, 2020). This means that they are investing a lot in tests. Since their software is deployed into a user’s machine and not to websites and online services, it is much harder to update in case of failures and bugs. That’s why they make sure that before they deliver their product, it is of quality and high standard.

Microsoft has a test discipline with formal measurements in quality. They also have validation phases in their release cycles. Although at some point, testing procedures became a bottleneck that caused delays, with the availability of new tools and technology, they were able to speed up the process.

Some of the tests they do are: unit testing, integration testing, performance testing. Then they deploy and monitor. They also do peer reviews, design reviews, code reviews, and test reviews.

Whittaker, J., Arbon, J., & Carollo, J. (2012). How Google Tests Software (1st ed.). Addison-Wesley Professional.

Related posts:

The encrypted messaging company Signal has long been popular with activists, investigative journalists, politicians and assorted law enforcement officials because of its emphasis on privacy and…

A domain can be classified into a content category using Website Categorization API. It takes a URL as input and categorizes it according to its industry. There are many Categorizations APIs…

I have been considering starting my own publication for a few weeks now. And today, I finally did it! I love The Sound of Music. What’s not to love about that iconic movie! Every scene is a…