QA Blog Posts
QA Release Checklist
Release time can be a crazy time for QA. There are always things going on, from last-minute issues discovered by QA to last-minute infrastructure changes for security purposes.
It’s really easy for QA team leads to get distracted and forget some of the essentials that need to happen for a release to be deployed successfully.
QA Checklist to the Rescue
The easy solution is to keep a release checklist. Something that will help you remember key things before a release gets deployed.
There’s no “one checklist fits all.” You should create a checklist that works best for your environment.
The nice thing about the checklist is that it acts as a release journal. If there are deployment issues, you can make notes on what to do in future releases.
Example Checklist Items
- All QA team members signed off on the release
- Automation Pass
- Automation total run time is acceptable
- All Critical Paths have been tested
- Release Branch matches the test branch
- Database migrators have no locking vulnerability
You can use any digital software to create the checklist: Evernote, Microsoft Word, Notepad ...
I would recommend a paper solution. That way you can focus your computer screens on the release.
The checklist can be next to your keyboard - reminding you of all critical tasks.
I would highly recommend getting the Emergent Task Planner and using that as your checklist. It’s really well put together. You can build a timeline of when sign-offs have to occur before the release is deployed.
Unhelpful Error Messages
Someone on Twitter asked:
What’s the most unhelpful error message you’ve ever seen?
I couldn't think of anything right away, so I search my collection on Google Photos, and I found two that I thought are worth sharing.
This is there that I am now getting while trying to install Big Sur on my iMac. There’s no indication of what I should do next.
There are plenty of websites that give some clues on what to try. Most of the advice is around some technical issues that Apple had during the original rollout. However, I am still encountering the issues - many days later.
Still working on finding what this error means. Hopefully, this gets fixed soon, and it doesn't involve formatting my computer!
Flavia Coffee Pot Error.
This is an error that appeared on a coffee machine at work. Multiple people have to get involved to fix this issue. Phone calls were made to the coffee company to report Error 328.
Turns out the problem is with a stuck freshpack. I found a website that explains how to fix the error.
It would have been so much better to say something like, "Clear the Stuck Freshpack."
Note: This isn't a problem anymore as the company has moved on to a machine that makes single-serve coffee with the need for K-cups or Freshpacks.
World Quality Day
On November 12, 2020 is World Quality Day.
World Quality Day is celebrated annually on the second Thursday in November.
World Quality Day
Each year the IRCA and CQI team put together a focus on World Quality Day.
This year the purpose of the day, is to put in the spotlight people that go above and beyond to improve customer value.
Quality at Drug Companies
I think there should be a focus on the various Quality Control teams in the drug testing field. These people are working overtime to make sure that the proper medication is being distributed. In addition, they are making sure that test results are accurate as possible.
Don't forget to thank your favorite QA person on World Quality day.
When Not to Automate
There are times when automation isn't practical. Here are four times when automation seemed like a good idea but in reality, it isn't.
When Automation is a feature that plays in minor significance of the product.
Don’t waste valuable Automation resources in testing minor issues. You should think If automation finds an issue how likely will it get fixed? While it’s great to automate everything in a project - it’s more fission to automate the most critical thing.
When the product or feature may get constant change.
If things are going to constantly get changed, why would you want to build automation around it? Why subject yourself to fixing known failures? I have found in the past to wait for a particular feature to be stable before committing any automation time and resources. Your best bet here is to talk to the product manager about when the product will be stable for automation.
If developers are still coding based on an integration branch, there's too much change to add it to automation.
When it's used for Metrics testing.
There are plenty of other tools available they can help with metric testing. Don’t use QA automation resources for metric testing.
When it takes a long time to run a test but has little value and slows down the ability for other critical test to run.
You should consider how long it will take to run a test and decide if it’s really worth using automation resources and time to run a particular test. Even in multithreaded environments, the QA test still takes time to run.
All week long we are celebrating Halloween here on cryan.com.
Here are some Halloween inspired Memes for those that work testing software.
Tester of EastWick Logo
I put these are part of the cryan.com's QA graphic gallery.
Make Test Steps Great Again
One of the things that I see over and over is when QA engineers fail to provide detailed testing steps to reproduce a bug. The tester assumes that the developer understands the vague steps to reproduce the bug.
There are usually three reasons why QA gets into a vague state.
- Rushing to get the bug into Jira. Perhaps QA found a bunch of similar issues and they are reporting as many issues as possible.
- Working on the project for a long time and knows the project very well. Why add details that no one cares about?
- Bug isn’t critical to get into the details. Usually, the title says it all - why add details to a low priority issue?
QA is in the Details
QA is in the details business. One of the rules of any QA Engineers:
QA should always be going out of the way to paying fantastic attention to the testing details.
This applies to everything from writing test cases to writing automation. The value QA brings is in the details.
Don’t Make Exceptions
The thing is if you start making some exceptions to the rule then it slowly becomes a habit.
This is similar to how when you make an excuse that it’s ok to eat one Twinkie when you're on a diet. Next thing you know you justify going to McDonald's for lunch.
At the End of the day
It’s important to keep providing detailed steps on how to reproduce an error. No matter how unimportant the issue is. You never know some future bug may encounter a similar issue and your detailed steps could help speed up the patching process.
Remember: Don’t take shortcuts.
What’s the best career advice you’ve ever had?
Over my career at various companies I have gotten great advice from managers and senior people within the organization. They have been helpful to where I am today.
Some of the advice has been specific to the needs of the business, but there’s been a few that I think about now and then.
The best career advice that I have gotten:
Alway maintain a positive image at work - no matter what’s going on in your personal life or what you know is going on behind the scenes at work.
The thing is that I have gotten similar advice from different people at various companies that I have worked at.
Things I have taken away from the advice
Every business has its up and downs. Bad times will happen just as often as good times. Stay positive and control what you can control.
Sometimes people will have bad days and being a cheerful person in the office can help calm things down.
Senior Team Members like people with optimism and will reward those that spread it around the company.
QA is a tough job because it’s important to communicate issues in a calm and understanding matter. Having a positive outlook can help communicate issues from saying its a critical issue to “customers will run into an issue if we did xyz…” They both communicate that there’s an issue just one is deliver in a more positive tone.
Any Good Career QA Advice?
Have you gotten any good QA advice?
QA Memes (October)
Here’s more collection of QA memes. Some of these you have to be doing QA for a while, but most of these you should understand.
Best Release Ever
Hello, QA Wants to go Home!
why can’t developers actually test their code?
QA up at Night
A common question that gets asked CEOs is:
What Keeps you Up at Night?
The response usually centers around new projects and uncontrolled risks to the company bottom line.
So, What about QA? What issues are keeping QA up at night? I can't speak for all of QA, but here are the top six things that keep QA Managers and Leaders up at night.
Six Things that keep QA up at Night
Ticket Scope Creek - Product making last-minute changes and not adding the change to the spec document. This can cause some unfortunate consequences later. For example, the product team didn’t realize that an extra line in a description field pushes the next button below the visible frame view.
Tight Deadlines- sometimes a feature has to go to Production with very minimal time for proper testing. In these rare cases, QA has very little time to fully test the feature. Example: Product wants to ship a new feature in time for the customer event, unfortunate delays with development means a shorter test cycle.
Dev Environment not Matching Production - Some testing can’t be accomplished because the testing environment doesn’t match production. Most of the time it’s load balancing and Cache that may cause issues. Example: New customer login path doesn’t take account of having different servers. QA passes the feature but the release is rolled back because users aren’t able to log in.
Developers that don’t test their code - some developers don’t test their code before handing it off to QA. They feel when it passes code review it’s good enough. Unfortunately, developers don’t check for how the change impacts the code. Most time code review is to test logic. Example: Developer submits code for QA and the build fails because the developer forgot to properly close a SQL insert statement.
Dealing with the Cash Register - Making the sale process smooth is critical to any business. QA needs to make sure that customers can buy and the sale occurs correctly for the customer and for the company. It’s important that customers are properly charged for the goods and services that they order. Example: At one of the companies that I worked at, a team of QA engineers was responsible for making sure that purchases were successful. They were trained to understand various tax rules and security regulations. Not all companies have the luxury of having a skilled team, so QA has to do their best to make sure that the sales process is good after every release.
Automation Failures due to UI changes - When a developer makes a UI change and doesn’t tell QA, it can cause some issues after the first automation run. The overnight tests may fail and QA will spend much of the morning fixing all the failed runs. This can cause other bugs to go undetected for a while. Example: A developer changes some of the IDs in the main navigation to keep with the new CSS standards. The change is considered minor and no ticket is created. On the first day of testing 90% of the automation fails because the old IDs can’t be found.
What do you think? Are there other cases that cause you to lose sleep?
Share your story in the comment section.
Recently I was doing some location testing using Google. The purpose of the test was to find a place in the United States where I could expand the radius targeting more than 50-miles.
When you use Google targeting you are allowed to expand that radius only when your location had a small population density.
If your range has a high population count Google will force you to have a smaller target.
I tried many remote areas of the United States and the only place that I could expand the target range to 100-miles was Winnemucca, Nevada.
I am sure that there are places in Alaska that would qualify - but I wanted to test for a place in the lower 48-states.
Hopefully this helps someone else doing targeting testing using Google Locations.
Code Freeze Meme
It’s been a while since I added QA graphics to the QA library. Here are some more images to add to my collection.
This week's theme is "Code Freeze"
Since today is Labor Day, I decided to go easy on today’s post and highlight some QA logos that might be fun for presentations or Notion headline.
I wish I knew more about...
Today’s blog post is all about the Ministry of Testing blog challenge for September.
Write a blog on the topic "I wish I knew more about..." before September 17th
My answer is simple:
I wish I knew more about Quality Chrome tools that can help me be a better tester.
I feel like there are a bunch of Chrome plugins that are out there that I should know about.
The next time that I have downtime, I’ll spend a few minutes searching the Google Play store for extensions that might be useful.
Some search queries that I can think of:
Fuzz testing (fuzzing) is a quality assurance technique used to discover coding errors and security loopholes in software, operating systems, or networks. It involves inputting massive amounts of random data, called fuzz, to the test subject in an attempt to make it crash.
This is usually a technique done with Automation to see how fields respond to random text and interactions.
Manual testers may want to use Bug Magnet, a popular Chrome Extenstion to add random data to fields. This is more exploratory testing than Fuzz Testing. Fuzz Testing is focused more on how the software reacts to a huge set of random data being entered.
Fuzzing is a critical part of testing as it checks for potential vulnerabilities with the software application and logic.
You can learn more about Fuzzing at the Open Web Application Security Project Foundation website.
In QA, it's good to not have any bad surprises. This is especially true around code freeze. Nobody wants to encounter blockers or critical issues that may impact the ability to release on time.
This is why many companies implement an Early Evaluation of releases. It's a chance to see where the release branch is, and if there are any issues.
Basic Definition of Early Evaluation
Early Evaluation is when QA builds the latest version of the release branch to a dev box a few hours before code freeze. If the build is successful, QA runs a suite of Acceptance Test to see if the release branch is stable. If there are any blocking issues, QA will notify developers of the issues.
There are three main components of the Early Evaluation:
Is the Release Branch really Stable - This is the time to find out build issues. Not at midnight or off-hours when code freeze happens. Key developers may not be near their computer to help out.
Are there any Database Migrator issues? Some developers may not check for migrator conflicts when merging their code in. Checking for migrators errors can help determine of certain functionality are working as expected.
Acceptance Testing - The QA team should have a suite of manual acceptance tests. (These are specific tests that QA has defined are critical for sign off) QA should be checking for any Blocking Issues This is also a good time to run automated acceptance tests. If there are any automation failures, QA should check to make sure its not related to release intended changes. Fixing these now will help with overnight automation runs.
Worth the Time and Effort
Code Freeze day can be crazy, but it's important to take a break and test out the release as part of the early evaluation. In the past, there has been a lot of good bugs found before code freeze, saving a lot of time during code freeze.