|Earliest: November 26, 2017||Latest: February 10, 2020||Total: 99|
Sometimes bugs are missed by QA because they are unique. They can be hard to find during normal regression testing.
Here are a couple of examples of bugs making it to Production due to complexity of testing:
There was a bug that only occurred for French users of a website that I was testing. The bug occurred because the translated text was too long and as a result, a dialog action button was not visible.
QA missed this in testing because the focus on testing was on English. QA wasn't notified that a particular dialog box was being translated to French - which wasn't an issue at the time since text content wasn't part of QA testing.
JBoss Installation Instructions
At another company that I worked at, I was responsible for software QA while at the same time doing some installs and training for customers. The company developers put together some documentation on how customers can install the product. The problem was that the document only handled a particular set of customers and several customers started complaining about the inaccuracy of the documentation.
QA missed this because the document was working for many customers. There were no major changes in the application that would have resulted in needing to test the installation process with the customer instructions. What happened was the sales team was selling the product to a different set of customers that required QA to check the documentation.
I then walked through the installation with a couple of customers while at the same time updating the document to make sure that it was clear for particular environments and make sure the terminology matched the audience that was doing the install.
Five Things I Learned to Handled Future situations
While the above is very specific examples, there are many more similar bug patterns that I have seen over my many years of QA testing. Here are some things that I have learned:
- It doesn't hurt to every once in a while to take a step back and manually go through the sales flow of the application. Are things working as they should be? How does the product look to new customers.
- Work with Developers to get some QA tools to help with testing. The French problem was being solved by having a special URL query for QA to force the page to load with a particular translation. This tool makes it easy to test the key languages when major changes happen in the application. Also, it makes it easy for automation to test the button visibility against various languages.
- Review the code changes. It doesn't hurt to check out the code review to see what has been changed. Many times I have found that a code change was made without thinking of other consequences - for example, what happens if customers use non triditional UTF-8 characters.
- Learn from the Bugs that Excape QA. One of my weekly tasks is to review the causes of customer reported issues and to see how it was missed.
- Learn new QA tools. There's always something new to learn in QA. There's always some new Chrome Extension, JQuery tip, database query and security lock down.
How Bugs Escape
There are many reasons why bugs are missed by QA and eventually make it to Production. Here are some of the reasons that I have encountered over the years in various QA roles.
Sometimes bugs occur because the bug occurs during a complex situation. These are hard for QA to detect or identify.
For example: If you have 99 items in your cart and your language is French, the shopping cart page crashes.
Realistically these aren't going to be found by QA. With a little help from Devs, QA can formulate test strategies based on code changes. Such as learning more about a 3rd Party library or support for Unicode.
QA Tasing time is finite. There's always too little time to test every possible situation.
This is why test strategies are so important. During off time, the QA team should get together and audit regression testing. The team should focus on what risk areas should QA be a focus on.
This is why developers' testing steps are important. QA should get a "heads up" on what to test.
Stale Test Case Repository
Bugs may escape QA because the manual test case repository tests might be state. How often are you looking at manual regression tickets? Are they update with the latest design changes? Are QA Testers actually following through the steps?
At previous companies that I have worked with, we review the manual regression steps with key stakeholders on a regular base. How often? Usually when there's a big design or code change. It's a good way to get some input in how QA is testing their product.
Don't take manual testing for granted. Make sure to review manual test cases on a regular base - especially if there's a lot of bugs making it in production.
A good QA Manager once told me, "that if an automation test case passes several releases, it should be audited. The test may not be challenging the code enough to be useful as part of regression."
In the month of February, I'll explore more into how Bugs escape QA testing. I'll talk about various strategies that I have found to work well to combat these bugs.
Jira Board Shortcuts for QA
Atlassian Jira is being used by more and more Engineering teams. It's flexibility and standard tools help make it easy for teams to track tickets and sprint progress.
During Sprint planning, it's very helpful to know some shortcuts to help make navigation easier.
Four Shortcuts to Know
There are four Sprint Board shortcuts to know. Simply type in the number when you're on the sprint board.
|Active sprints/Kanban board:||2|
|Dock/undock the filters panel:||[|
The last feature, "Dock/undock the filters panel" hides the left gray bar - which is not being used while you're looking at a particular sprint board view. By undocking the filter panel, you get the valuable real estate back.
New QA Memes
Here are some original QA memes that I came up with. These are just common occurrences that happen in QA. In particular, QA responding to conversations in Slack.
All of these images are Slack "ready" and will show up embedded in chat conversations.
These will also work as Jira comments too.
Check out the Library
These and all QA images are in the QA Graphic Library.
Human Testing better than Automation
Recently Ministry of Testing asked:
Based on many years of testing, here are my four things that humans testing is good at. Automation is a great tool, but for these items, they are no match.
Four Ways Human Testing is Better than Automation
Exploratory Testing - Looking for ways to break functionality is best done with human testing. People don?t always use the conventional path when using a website. Exploratory testing by humans can find unique bugs.
Debugging the route cause of a bug. Humans can use all sorts of methods to discover why a bug might happen. Humans can use the Chrome Console, log files and visual logic to better understand the root cause of a bug.
New Product/Feature Testing - It?s better to perform manual testing when a product/feature is new. The feature could go through numerous changes - so human testing would best to start before investing in automation time.
Third Party Tools Integration - Using third-party tools that require logins. There?s a chance that third-party companies could make changes that will break the automation flow. (Such as changing ids or layouts) Human testing can help bypass any complexity that third party websites have.
Apple Numbers (QA Fail)
Apple makes things easy - which is why I like using it as my computer platform. Just about everything is easier to do on Macintosh.
However, they seriously failed with making graphs in Numbers.
On Christmas Day, I was trying to create a chart of how early my daughter would wake up Christmas morning.
I was struggling with generating a chart from a bunch of data. Basically it was a spreadsheet that had years as one row and times in the second row.
Here's a sample shot of the data:
When I clicked on the "Insert Chart", this is what I got:
Looks like I'll have to do a lot of chart manipulation to make this to work. (I tried putting the data via column view and got the same results.)
I put in the exact same data in a Google SpreadSheet, select the fields and clicked "Insert Chart" and got a perfectly matched chart:
What I Learned
Apparently if I need to create a quick chart, the way to go is to use Google Spreadsheet (or Excel). Apple Numbers isn't all that user-friendly when it comes to creating charts.
Apple should make it simple to create charts - they do have some unique layouts/styles that aren't available in other applications. I shouldn't have to be a chart master to make it work - especially if the data is simple.
The Best 2019 QA Posts
This year I posted a lot of useful information about QA software testing. Along the way, I learned a lot more about new techniques and testing strategies.
- False Negative - Understanding how False Negatives can occur
- Massachusetts QA Fails - What happens when things do not go through QA
- Test Plan Guidlines - Fundamentals of what makes a good quality test plan
- Adobe Flash - Adobe Flash going away
- Test Strategy - Fundamentals of a Test Strategy document
- What Wikipedia Can Not Tell You About Test Link - Some TestLink Annoyances
- Letter to the QA Manager - Sample letter to the QA Manager
- Blue Button - Useful Chrome Extension to help debug a website.
- World Quality Report - Learn all about the World-Wide trends in qa testing in the Capgemini document
Very excited about some of the things I learned this year. I'll keep the QA Monday topic going until the end of February.
Three Specific QA Goals for the next set of posts:
- Discover some of the Basics of QA Testing.
- How can testers become better testers.
- What tools are available that people may not know exists.
I'll focus more time on the quality of content, not just to post because I have to post.
Add Search to Chrome
Did you know that you can search through many websites without actually navigating to them? This cool trick is beneficial if you wanted, for example, to go directly to the Wikipedia article on orangutans without visiting Google.com or Wikipedia's front page.
Setting up the Search
First, go to Settings > Search engine > Manage search engines. There, you'll see your default search engine (the one Chrome uses whenever you type a query into the search), other sites already available for quick searches, as well as the option to add other websites to the list.
To prep up for this, go to a site that you want a "quick search" and search for something using the search engine. Copy the URL that appears on the first page after you click on submit.
- Click Add
- Enter in a search engine name
- Enter a KeyWords - Usually just the search engine name
- Then enter the URL and replace the old search term with %s
Sample Search Entry
So, for quick Wikipedia searches, click "Add" under Manage search engines and add the site. Then, the next time you type "Wikipedia.org" in the search text, on the far right, you'll see a prompt telling you to press tab to search within the site. Once you press the tab button, you'll see "search Wikipedia" written in blue on the left side of the search field; type your query and Chrome will only search within Wikipedia.
This function isn't specific to reference or search sites. You can use the search field to search directly through nearly any site?even PCMag.com?as long as it's included in your managed list of search engines. Chrome will automatically add these "keyword searches" to any site you visit.
To do this manually, add "site:" to a query. Say, for example, you want to see everything PCMag has written about fitness trackers, you could type "fitness trackers site:pcmag.com" into the search field, and Google will return results from their site.
Incognito in Chrome
Google's Incognito Mode is a quick way to go "private" in Google Chrome. It's a way to browse the internet without keeping any history on your computer.
Simple description of Google's Incognito from Google:
If you don?t want Google Chrome to remember your activity, you can browse the web privately in Incognito mode.
Five Things I have learned while using Incognito
- You can only have one active Incognito session open at once. Which means that you can't open multiple Incognito windows with different Facebook logins.
- To Launch the Incognito mode simply Press ? + Shift + n.
- Did you know that Websites knows that you're in Incognito mode! This may impact you're experienced with the websites. For example, if you try to create a new Facebook account, Facebook will make your login experience a bit more adventurous.
- Google Extensions like Go Incognito let's you open the current page in Incognito mode.
- Some websites will set prices when browsing normal vs Incognito mode. Always good to see what the price of something is when your not logged in.
What happens when you browse privately
Chrome won't save your browsing history, cookies and site data, or information entered in forms. Files you download and bookmarks you create will be kept.
Your activity isn?t hidden from websites you visit, your employer or school, or your internet service provider.
Misconceptions that exist within software testing
Someone recently asked, "What are some of the misconceptions that exist within Quality Assurance Testing?"
Some QA misconceptions that I have seen/encountered over the years:
QA Can Be Done By Customers
Let's save money and time. Let's have the customers test this feature.
Big mistake to have paying customers test untestable code is very risky. They may encounter some bugs that may block them from performing crucial tasks.
In addition, you may introduce code into production that may not be reversible.
Let's Tell You How Long it Will Take
Some Developers and Product team members may say, " The feature shouldn?t take long to test, let the lead developers figure out how much time to test a feature. "
In reality, The only people that know how long to test something are the people doing the manual testing.
QA is Easy, Anyone can Do it
Some people think, Anyone can QA, that's an easy job.
Nice thinking, it's almost saying anyone can paint a wall. Sure anyone can. However, those that know what they are doing, actually do a better job and will produce a better quality product.
Don't trust your work to anyone, use someone that knows the technology and can better judge the work.