QA Blog Posts
QA Image Library
Check out the growing QA Image Library. This is my personal colleciton of Slack Images for that perfect QA Moment.
Automation is not testing it's checking.
What is Automation?
Automation makes hunmans more efficient, not less essentials
Automation is not a silver bullet - it won't immediately increase productivity, but if approached correctly, it will eventually support increase releases, greater test coverage, and overall quality of the product.
The Great Debate?
There's this debate in the testing community on the purpose of automation.
There's one group that talks about how automation is testing. They argue that automation tests to make sure that the build is stable and any changes doesn't break existing functionality.
There's a different group that says automation is checking. They argue that automation is only checking predefined paths and not testing situations that may be unique to the change being made. While critical paths may be working, some other functionality may be broken.
I am with the team checking. I believe that the purpose of automation is to check the stability of the build. Once automation passes, QA can perform manual testing.
Automation can be limited to the scope of checking/validating. In my experience, it can be tricky to automation 3rd party integration. Manual testing can be more effective in validating functionality.
Automation Has Value
Saying "Automation is Checking" doesn't change the value of automation. Automation plays a valuable role in the testing process. Don't put your eggs in one basket and solely rely on automation as your testing tool. Automation complements manual testing.
Automation test cases should be designed so that the build/branch is stable for manual testing. It should be checking that the critical paths are working.
There's no such thing as a manual or automated tester. Are you a manual or automated programmer?
June QA Images
It's been a while since I added some images to the QA library. Here are some new ones to my growing collection.
Be sure to check out all the QA Images in the QA Image Reference Library
February QA Images
Here are some QA graphics that I created for fun. I did some searching online and was surprised that there are good options to choose from.
Feel free to use these in any email or Wiki page.
QA Rule 1
The Art of QA
QA Reference Page
Check out all the QA Images that I have in my QA Library.
The Best 2020 QA Posts
It's time to look back at some of the best posts of 2020. (Check out the 2019 Post.)
- Slack Tips for QA - (February 11) Some useful tips and tricks to get the most out of Slack.
- Dynamic Bookmarklets - (June 24) Great way to build a Bookmark that is time based.
- Letter to the QA Manager - (September 9) An interesting letter that I found. Helpful in understanding the relationship of a QA Engineer and a QA Manager.
- Best QA Advice - (October 14 ) Some great QA advice that I have gotten over the years.
I'll keep posting useful QA Tips and Tricks that I learn. Most of the information is around the Software as a Service model.
I don't have any specific content targets. I'll keep the content going through February. I may take a break for some time to think of useful content to post.
QA Graphic Collection
Here’s some new graphics for the QA library. All month long, I have been collecting and using some QA memes.
Be sure to check out the entire QA graphic collection.
Yagni is an abbreviation for “You Aren’t Gonna Need It,” referring to code that does not add functionality to pass tests or meet requirements.
Four Reasons QA Should Be Aware of Yagni
Makes Code Hard to Read - Developers may leave unused code - perhaps for their testing or design process. When someone else picks up the code it could cause confusion as to what the functionality is doing. This is true for QA as sometimes they need to read the code review to understand how to unique exploratory testing around the functionality.
Removed QA-Only Code - Developers may add code to help QA validate certain functionality. This should get removed from the code, not commented out. This way the code doesn't accidentally get activated.
Adds Risk to the Code - If it's not part of the feature, then there's no reason it should go out in production. Keeping Yagni code in could open a back door to your application.
Applies to QA Test Plan - Check your test case repository. Are there tests that fit into the Yagni principle? Why are they are part of your test plan? Clear it out so that your tests don't look so overwhelming.
According to Google translate Yagni is Turkish for "That is."
Throw it over the wall
Some Developers use QA as their testing environment without testing their code first. In some situations, if the code compiled they thought it was safe for QA to test.
In software development, this is known as “throw it over the wall” testing. Developers feel that the Change is so small - they just give it to QA to test it.
Not a Good idea
Bad news - that’s not what QA is for. At least not in an agile environment.
Developers shouldn't be sending code to QA without having some sanity testing. They should at least know that the change they are making are working.
Yes they should be testing their code.
What Should QA Be Use For?
QA should be used to test security risks, performance issues, vulnerability, and usability. It shouldn’t be the front line of any testing.
Developers should always be testing their code. They should have accountability for how their code works.
Here are some more QA based graphics that I am adding to the QA Image Library.
If there's anything you like to see, let me know!
Jira Because - You Have Issues
Billboard - Don't Tell QA
Test Your Code on a street billboard - QA reminding you to test your code before handing it to them.
Quality, Stability through superior firepower.
QA Release Checklist
Release time can be a crazy time for QA. There are always things going on, from last-minute issues discovered by QA to last-minute infrastructure changes for security purposes.
It’s really easy for QA team leads to get distracted and forget some of the essentials that need to happen for a release to be deployed successfully.
QA Checklist to the Rescue
The easy solution is to keep a release checklist. Something that will help you remember key things before a release gets deployed.
There’s no “one checklist fits all.” You should create a checklist that works best for your environment.
The nice thing about the checklist is that it acts as a release journal. If there are deployment issues, you can make notes on what to do in future releases.
Example Checklist Items
- All QA team members signed off on the release
- Automation Pass
- Automation total run time is acceptable
- All Critical Paths have been tested
- Release Branch matches the test branch
- Database migrators have no locking vulnerability
You can use any digital software to create the checklist: Evernote, Microsoft Word, Notepad ...
I would recommend a paper solution. That way you can focus your computer screens on the release.
The checklist can be next to your keyboard - reminding you of all critical tasks.
I would highly recommend getting the Emergent Task Planner and using that as your checklist. It’s really well put together. You can build a timeline of when sign-offs have to occur before the release is deployed.
Unhelpful Error Messages
Someone on Twitter asked:
What’s the most unhelpful error message you’ve ever seen?
I couldn't think of anything right away, so I search my collection on Google Photos, and I found two that I thought are worth sharing.
This is there that I am now getting while trying to install Big Sur on my iMac. There’s no indication of what I should do next.
There are plenty of websites that give some clues on what to try. Most of the advice is around some technical issues that Apple had during the original rollout. However, I am still encountering the issues - many days later.
Still working on finding what this error means. Hopefully, this gets fixed soon, and it doesn't involve formatting my computer!
Flavia Coffee Pot Error.
This is an error that appeared on a coffee machine at work. Multiple people have to get involved to fix this issue. Phone calls were made to the coffee company to report Error 328.
Turns out the problem is with a stuck freshpack. I found a website that explains how to fix the error.
It would have been so much better to say something like, "Clear the Stuck Freshpack."
Note: This isn't a problem anymore as the company has moved on to a machine that makes single-serve coffee with the need for K-cups or Freshpacks.
World Quality Day
On November 12, 2020 is World Quality Day.
World Quality Day is celebrated annually on the second Thursday in November.
World Quality Day
Each year the IRCA and CQI team put together a focus on World Quality Day.
This year the purpose of the day, is to put in the spotlight people that go above and beyond to improve customer value.
Quality at Drug Companies
I think there should be a focus on the various Quality Control teams in the drug testing field. These people are working overtime to make sure that the proper medication is being distributed. In addition, they are making sure that test results are accurate as possible.
Don't forget to thank your favorite QA person on World Quality day.
When Not to Automate
There are times when automation isn't practical. Here are four times when automation seemed like a good idea but in reality, it isn't.
When Automation is a feature that plays in minor significance of the product.
Don’t waste valuable Automation resources in testing minor issues. You should think If automation finds an issue how likely will it get fixed? While it’s great to automate everything in a project - it’s more fission to automate the most critical thing.
When the product or feature may get constant change.
If things are going to constantly get changed, why would you want to build automation around it? Why subject yourself to fixing known failures? I have found in the past to wait for a particular feature to be stable before committing any automation time and resources. Your best bet here is to talk to the product manager about when the product will be stable for automation.
If developers are still coding based on an integration branch, there's too much change to add it to automation.
When it's used for Metrics testing.
There are plenty of other tools available they can help with metric testing. Don’t use QA automation resources for metric testing.
When it takes a long time to run a test but has little value and slows down the ability for other critical test to run.
You should consider how long it will take to run a test and decide if it’s really worth using automation resources and time to run a particular test. Even in multithreaded environments, the QA test still takes time to run.
All week long we are celebrating Halloween here on cryan.com.
Here are some Halloween inspired Memes for those that work testing software.
Tester of EastWick Logo
I put these are part of the cryan.com's QA graphic gallery.
Make Test Steps Great Again
One of the things that I see over and over is when QA engineers fail to provide detailed testing steps to reproduce a bug. The tester assumes that the developer understands the vague steps to reproduce the bug.
There are usually three reasons why QA gets into a vague state.
- Rushing to get the bug into Jira. Perhaps QA found a bunch of similar issues and they are reporting as many issues as possible.
- Working on the project for a long time and knows the project very well. Why add details that no one cares about?
- Bug isn’t critical to get into the details. Usually, the title says it all - why add details to a low priority issue?
QA is in the Details
QA is in the details business. One of the rules of any QA Engineers:
QA should always be going out of the way to paying fantastic attention to the testing details.
This applies to everything from writing test cases to writing automation. The value QA brings is in the details.
Don’t Make Exceptions
The thing is if you start making some exceptions to the rule then it slowly becomes a habit.
This is similar to how when you make an excuse that it’s ok to eat one Twinkie when you're on a diet. Next thing you know you justify going to McDonald's for lunch.
At the End of the day
It’s important to keep providing detailed steps on how to reproduce an error. No matter how unimportant the issue is. You never know some future bug may encounter a similar issue and your detailed steps could help speed up the patching process.
Remember: Don’t take shortcuts.
What’s the best career advice you’ve ever had?
Over my career at various companies I have gotten great advice from managers and senior people within the organization. They have been helpful to where I am today.
Some of the advice has been specific to the needs of the business, but there’s been a few that I think about now and then.
The best career advice that I have gotten:
Alway maintain a positive image at work - no matter what’s going on in your personal life or what you know is going on behind the scenes at work.
The thing is that I have gotten similar advice from different people at various companies that I have worked at.
Things I have taken away from the advice
Every business has its up and downs. Bad times will happen just as often as good times. Stay positive and control what you can control.
Sometimes people will have bad days and being a cheerful person in the office can help calm things down.
Senior Team Members like people with optimism and will reward those that spread it around the company.
QA is a tough job because it’s important to communicate issues in a calm and understanding matter. Having a positive outlook can help communicate issues from saying its a critical issue to “customers will run into an issue if we did xyz…” They both communicate that there’s an issue just one is deliver in a more positive tone.
Any Good Career QA Advice?
Have you gotten any good QA advice?