Early Evaluation
Why Early Evaluation is critical to QA release cycle.
In QA, it's good to not have any bad surprises. This is especially true around code freeze. Nobody wants to encounter blockers or critical issues that may impact the ability to release on time.
This is why many companies implement an Early Evaluation of releases. It's a chance to see where the release branch is, and if there are any issues.
Basic Definition of Early Evaluation
Early Evaluation is when QA builds the latest version of the release branch to a dev box a few hours before code freeze. If the build is successful, QA runs a suite of Acceptance Test to see if the release branch is stable. If there are any blocking issues, QA will notify developers of the issues.
Main Purposes
There are three main components of the Early Evaluation:
Is the Release Branch really Stable - This is the time to find out build issues. Not at midnight or off-hours when code freeze happens. Key developers may not be near their computer to help out.
Are there any Database Migrator issues? Some developers may not check for migrator conflicts when merging their code in. Checking for migrators errors can help determine of certain functionality are working as expected.
Acceptance Testing - The QA team should have a suite of manual acceptance tests. (These are specific tests that QA has defined are critical for sign off) QA should be checking for any Blocking Issues This is also a good time to run automated acceptance tests. If there are any automation failures, QA should check to make sure its not related to release intended changes. Fixing these now will help with overnight automation runs.
Worth the Time and Effort
Code Freeze day can be crazy, but it's important to take a break and test out the release as part of the early evaluation. In the past, there has been a lot of good bugs found before code freeze, saving a lot of time during code freeze.