George Ukkuru is a seasoned technocrat and AVP of quality engineering, DevOps, and SRE @Marlabs Inc. Over the course of two decades, he has helped Fortune 500 companies implement Agile testing practices. He has also authored a number of books on quality engineering and test automation. 



In this QnA, Ukkuru lays out the scope and depth of his experience without even trying. He makes the points that non-functional testing should be higher up in your priority, that manual test data provisioning keeps you from truly reaping the rewards of automation, and that you must never miss an opportunity to celebrate good work. 



<div id="toc_container" class="no_bullets">Table Of Contents<ul class="toc_list"><li><a href="#Q1_What_changes_have_you_seen_in_SDLC_and_QAs_role_in_it_over_the_course_of_two_decades">1 Q1: What changes have you seen in SDLC and QA’s role in it over the course of two decades?</a></li><li><a href="#Q2_Whats_your_battle-tested_process_for_finding_and_resolving_bugs_in_Productionnbsp">2 Q2: What’s your battle-tested process for finding and resolving bugs in Production?&nbsp;</a></li><li><a href="#Q3_What_are_some_testing_anti-patterns_you_wish_never_existed">3 Q3: What are some testing anti-patterns you wish never existed?</a></li><li><a href="#Q4_If_you_had_to_restart_your_career_today_what_key_lessons_would_you_take_from_the_past">4 Q4: If you had to restart your career today, what key lessons would you take from the past?</a></li><li><a href="#Q5_To_aspiring_leaders_what_is_your_advice_on_how_to_be_a_good_manager">5 Q5: To aspiring leaders, what is your advice on how to be a good manager?</a></li></ul></div>
<h3 class="wp-block-heading" id="q1-what-changes-have-you-seen-in-sdlc-and-qa-s-role-in-it-over-the-course-of-two-decades">Q1: What changes have you seen in SDLC and QA’s role in it over the course of two decades?</h3>



In the last two decades, I have seen:



1. The emergence and dismantlement of the TCOE model, aka Independent Verification and Validation. It started in the early 2000s. Today, TCOEs have been converted to a hybrid model as teams adopt Agile &amp; DevOps. 2. Automation used to be a nice-to-have skill. When I started my career, automation was used for automating UI-based regression testing. Today, automation is a must-have skill, and the focus has moved from regression testing to complete test lifecycle automation.



The type of applications that are being tested has changed as well, from (now) legacy client server-based applications to microservice-based or cloud-based architectures. This changes how teams test for quality. 



The focus has shifted to non-functional testing, including running security tests, API &amp; workflow-based performance testing, Chaos Engineering, compatibility testing, etc. As technology advancement continues, we’ll see newer, more nuanced types of testing.



<h3 class="wp-block-heading" id="q2-what-s-your-battle-tested-process-for-finding-and-resolving-bugs-in-production">Q2: What’s your battle-tested process for finding and resolving bugs in Production?&nbsp;</h3>



Once a bug&#8217;s been found on production, I have several practices to fast-track its resolution:&nbsp;



1. Continuously review production tickets and identify defect trends, in collaboration with the Operations Team.2. Use an algorithm that identifies the best developers for determining the root cause and fixing the defect.3. Organize regression packs based on the type of fixes going into production and define when to use them.4. Monitor social media feedback, application logs, Mobil App Store feedback to identify the issues faced by the end users and take proactive actions to identify patterns.



<h3 class="wp-block-heading" id="q3-what-are-some-testing-anti-patterns-you-wish-never-existed">Q3: What are some testing anti-patterns you wish never existed?</h3>



I see several anti-patterns in software testing. But the ones that peeve me the most are:



1. Heavy focus on UI Automation. The effort and cost to maintain UI automation is disproportionately higher than the rewards, I think. UI changes frequently as the products evolve. Teams need to strike a fine balance between UI and API-based tests. API Tests are faster to design, execute and maintain.



2. Manual test data provisioning. The test data is ported to test environments after manually masking data from obfuscation. That is slow. You can&#8217;t truly automate for DevOps using this approach. Instead, teams should use synthetic data provisioning and self-service mechanisms to accelerate and automate data provisioning.



3. Late Identification of Non-Functional Defects. This is usually a result of a lack of clarity around non-functional testing requirements. Performance or security tests are seen as a part of &#8216;hardening&#8217; or release-level sprints. Unfortunately, performance and security bugs often need architectural changes to fix, and keeping non-functional testing for the end of the cycle can cause schedule and cost overruns.



I advise teams to look at behavior-driven testing techniques to document security and performance requirements. Engineers should also integrate and execute API-based performance tests and SAST/DAST scans to the DevOps pipeline.



<h3 class="wp-block-heading" id="q4-if-you-had-to-restart-your-career-today-what-key-lessons-would-you-take-from-the-past">Q4: If you had to restart your career today, what key lessons would you take from the past?</h3>



Master at least one programming language, preferably Java, Python, or C#, so that you can design and create scripted tests.



Focus on taking up courses around test automation, either with open source or commercial tools.&nbsp;Or both.



Spend time learning about practices and concepts in software testing. ISTQB Foundation syllabus should be a good starting point.&nbsp;



Learn about Cloud, Agile, and DevOps principles and practices. Learn the basics of tools like Jira, Azure DevOps, Jenkins, etc.



Your communication skills matter more than you think.&nbsp;Work towards improving them. 



<h3 class="wp-block-heading" id="q5-to-aspiring-leaders-what-is-your-advice-on-how-to-be-a-good-manager">Q5: To aspiring leaders, what is your advice on how to be a good manager?</h3>



Take up formal courses on Agile or Project Management. Refresh your skills regularly to stay relevant.



This is my style, but I advise new managers to get actively involved in the project. Remember that you are not a people manager. Understand what the project objectives are and how it is going to impact a business or end-user. Work from there.



Understand the technologies involved so you can grasp the nuances of the project. This will help you make decisions and manage risks.



Learn the art of remote management. Here are some practices that work for me:-



<ul class="list_style_disc wp-block-list"><li>Have frequent meetings with your direct reports, preferably over video.</li><li>Do NOT micro-manage and follow-up every hour. Trust your team. Move away from a task-based and adopt a Goal-based approach while assigning work.</li><li>Spend time with your team to understand their strengths, passions, and weaknesses. Assign work based on their strengths and provide training and mentoring to help employees overcome their weaknesses.</li><li>Keep your team in the loop about the happenings in your department and the larger organization. This will help take care of rumors and avoid perceptions.</li><li>Provide feedback on the work assigned without delays. Don&#8217;t be a bottleneck.</li><li>Never miss an opportunity to appreciate and celebrate good work.</li></ul>



For more experience-led insights from George Ukkuru, check his book on software testing estimation on Kindle.

Quality Engineering Discussions 5 Questions with George Ukkuru

Quality Engineering Discussions: 5 Questions with George Ukkuru

Rahul Parwal is a software engineer, speaker, mentor, and writer out of Jaipur, Rajasthan. He has dabbled in software development, testing, and automation, and often shares his learnings in <a href="https://testingtitbits.blogspot.com/" target="_blank" rel="noreferrer noopener nofollow">his blog</a>.



In this QnA, Rahul makes the point that a randomly failing test is always worth investigating and that assumptions are dangerous in testing.



<div id="toc_container" class="no_bullets">Table Of Contents<ul class="toc_list"><li><a href="#Q1_What_are_your_most_and_least_favorite_things_about_test_automationnbsp">1 Q1: What are your most and least favorite things about test automation?&nbsp;</a></li><li><a href="#Q2_It_sounds_great_on_paper_but_what_would_it_really_take_for_a_mid-sized_org_to_incorporate_continuous_testing_in_their_release_cyclenbspnbsp">2 Q2: It sounds great on paper, but what would it really take for a mid-sized org to incorporate continuous testing in their release cycle?&nbsp;&nbsp;</a></li><li><a href="#Q3_What_are_some_Severity_1_bugs_that_ALMOST_slipped_past_you_and_how_did_you_make_sure_they_never_did_so_again">3 Q3: What are some Severity 1 bugs that ALMOST slipped past you — and how did you make sure they never did so again?</a></li><li><a href="#Q4_How_do_you_manage_failed_tests_and_tests_that_fail_randomly">4 Q4: How do you manage failed tests and tests that fail randomly?</a></li><li><a href="#Q5_Whats_your_most_memorable_deployment_andor_release_day_Good_OR_bad_just_as_long_as_no_one_dies_at_the_end8230">5 Q5: What’s your most memorable deployment and/or release day? (Good OR bad, just as long as no one dies at the end&#8230;)</a></li><li><a href="#Bonus_If_you_could_give_just_ONE_piece_of_advice_to_fledgling_software_testers_what_would_it_be">6 Bonus: If you could give just ONE piece of advice to fledgling software testers, what would it be?</a></li></ul></div>
<h3 class="wp-block-heading" id="q1-what-are-your-most-and-least-favorite-things-about-test-automation">Q1: What are your most and least favorite things about test automation?&nbsp;</h3>



One of the things I like about test automation is that it can be scaled over multiple platforms, devices, interfaces, environments, etc. quickly and reliably. Test automation, if planned and executed correctly, can be a great service to testing and QA.



There is, however, The Silver Bullet Syndrome, where everyone wants to ascribe the responsibility of succeeding with test automation success to a tool. “Let’s find the one tool to solve all of our problems.” It doesn&#8217;t work.



<figure class="wp-block-image is-resized"><img decoding="async" src="https://s3.us-east-1.amazonaws.com/blog-assets-production.testsigma.com/blogs/wp-content/uploads/migrated_images/ghN79ATI0DLYh5CNL3-LxlBUzyTg_fCE011vqAikrZkvyJqUZWPufXB3-U_BQNUtUCnw_qjdmjD0g4OSQGM_kurFoNBWGnPFVwH-dpYhyrWL36RWBwbVm4P-ILV8z8KCIp6Q_--H_1748949287.png" alt="Definition" width="600"/></figure>



The concept of automation extends well beyond a tool. Teams attempting test automation should consider a tool’s capabilities and limitations. But first, they should establish realistic expectations, sort out people constraints, and have the support of management. Without all three, adopting test automation will just be an uphill battle.



<h3 class="wp-block-heading" id="q2-it-sounds-great-on-paper-but-what-would-it-really-take-for-a-mid-sized-org-to-incorporate-continuous-testing-in-their-release-cycle">Q2: It sounds great on paper, but what would it really take for a mid-sized org to incorporate continuous testing in their release cycle?&nbsp;&nbsp;</h3>



The best way to start is by evaluating and understanding your context, team, and environment.



&#8220;No matter how they phrase it, it&#8217;s always a people problem,&#8221; as Sherbie&#8217;s Law says.&nbsp;CI/CD and consequently, continuous testing, needs a shift in the team&#8217;s mindset. If you push a whole bunch of processes and tools on people without getting them on board, you are setting them all up for failure.&nbsp;



In practice, my advice is to start with small experiments and do one thing at a time. Don&#8217;t expect overnight results, and <ul class="list_style_disc"><li>Make it a team sport. Establish channels of collaboration. Hold knowledge-sharing sessions. Try Pair and mob testing to reduce knowledge silos within the team.</li><li>Plan before you execute. At each stage of the product life cycle, plan what and how you&#8217;ll test.</li></ul> 





<div class="wp-block-image"><figure class="aligncenter is-resized"><img decoding="async" src="https://s3.us-east-1.amazonaws.com/blog-assets-production.testsigma.com/blogs/wp-content/uploads/migrated_images/k81_BbjtH20lB1vpqnVsN9Sij_GDLcaZa6KtTE9P6hAF17sur2tq9Xj81j2CmxqWgPDEiMy3hrjeHj6x0ftLAlMia_XmvetYDw_WDirOJ3O5WbQPBHdPgkzVj4R9MG9rsfFRSw-M_1748949289.png" alt="Plan before you execute" width="600"/></figure></div>



<h3 class="wp-block-heading" id="q3-what-are-some-severity-1-bugs-that-almost-slipped-past-you-and-how-did-you-make-sure-they-never-did-so-again">Q3: What are some Severity 1 bugs that ALMOST slipped past you — and how did you make sure they never did so again?</h3>



I have had two such incidents. I’ll share both, along with the lessons I learned from them:



<ul class="list_style_disc wp-block-list"><li>Software loading was failing if a specific application is running in parallel. Lesson: Test your software in real user conditions. Remember to test how your app works with shared resources such as memory, space, network resources, etc.</li> <li>The application gave a timeout if the form submission exceeded 15 minutes. Lesson: Actual users are not as fast as an automation bot/script. Users will take will never perform a form submission as quickly as a script, so use waits judiciously.</li></ul>



<h3 class="wp-block-heading" id="q4-how-do-you-manage-failed-tests-and-tests-that-fail-randomly">Q4: How do you manage failed tests and tests that fail randomly?</h3>



A failing test establishes my confidence in a test I’ve designed. The amazing Angie Jones once tweeted something which sums up my feelings on failed tests.



<figure class="wp-block-image is-resized"><img decoding="async" src="https://s3.us-east-1.amazonaws.com/blog-assets-production.testsigma.com/blogs/wp-content/uploads/migrated_images/0SG95qwUYRPHWIFFupruMFOcQE4y1wgMeyTfDrUNOwAO7sHAERCa3PC7e_SEGVzhCbnRKyOxrHyizTYGGGv3cd5OpLg8LYL0E3yTlkVrTb5mQuJLLrXEWSrwhfRaC0ckYHeCUAy8_1748949288.png" alt="How do you manage failed tests and tests that fail randomly" width="600"/></figure>



Randomly failing tests are interesting. They give me a reason to go sleuthing, to find and uncover variables that might be affecting my product and its functionality. My bug analysis and exploration skills have greatly improved as a result of investigating such bugs. Whenever I see a randomly failing test, I triangulate failure conditions by varying:



<ul class="list_style_disc wp-block-list"><li>Data</li><li>Behavior</li><li>Options &amp; Settings</li><li>Software &amp; Hardware Environment</li><li>Configuration &amp; Dependencies, etc.</li></ul>



A randomly failing test is usually a symptom of a much larger problem. Sometimes, developers refer to such cases as corner cases, but I&#8217;ve learned to never ignore a random failure without investigating it. It might just be the tip of the iceberg of a whole load of flaky tests.



<h3 class="wp-block-heading" id="q5-what-s-your-most-memorable-deployment-and-or-release-day-good-or-bad-just-as-long-as-no-one-dies-at-the-end">Q5: What’s your most memorable deployment and/or release day? (Good OR bad, just as long as no one dies at the end&#8230;)</h3>



In my first project as a software tester, I was testing for a product release of a large enterprise financial application. The stakes were very high since this product was developed to sunset a legacy system that had been in place for decades.



During testing, we were going through all the flows on a mocked simulated data set that was providing data as per our testing layer. After months of rigorous testing, on the release day, we started getting errors from the monitoring system.&nbsp;



There were gaps in the mocked simulated data and the actual production data. We hadn’t questioned the authenticity of the test data during the testing cycle. We ended up with a release rollback.



Some lessons that I learned that day:



<ul class="list_style_disc wp-block-list"><li>Never assume anything &#8211; What if the data is wrong?</li><li>Learn how to create your own testing data. This way, you understand the sources, flows, and formats of the data.</li><li>If you use mock/simulated data for testing, be sure to run a significant number of tests using real data as well.</li></ul>



<h3 class="wp-block-heading" id="bonus-if-you-could-give-just-one-piece-of-advice-to-fledgling-software-testers-what-would-it-be">Bonus: If you could give just ONE piece of advice to fledgling software testers, what would it be?</h3>



Like Paul Muad’Dib of Dune, “Learn how to learn.” Testing is a continuous search for information: Both about your product and the tests that you run it through. None of us are born with knowledge. Trust that you can learn new ways to test. Be a continuous learner.

Quality Engineering Discussions: 5 Questions with Rahul Parwal

In this series, real (and really good) QA practitioners use their experience to support—or debunk what you might know about software quality.



James Espie is a test specialist, a quality engineering proponent, and a continuous learner from Auckland, New Zealand. He shares his insights and sporadic bursts of inspiration in a hilarious newsletter called Pie-mail. If you haven’t seen it, <a href="https://jpie.nz/newsletter/" target="_blank" rel="noreferrer noopener nofollow">you should check it out</a>.



<div class="wp-block-image"><figure class="aligncenter is-resized"><img decoding="async" src="https://s3.us-east-1.amazonaws.com/blog-assets-production.testsigma.com/blogs/wp-content/uploads/migrated_images/vrj-e9P8NUEQgfsq9LUdxmQ8ewzOaZ-ulDqcGjCq1n3YY6ZFTKlHCdoiDoFQ5mIBfDti4pDJkt8dKeg9_PheRwVMUTYlap00qW4kWixVFozXgywTSOAvrqU0jTeagyaiZjSAMiCb=s1600_1748949292.png" alt="Postman pie-mail" width="600"/><figcaption>&nbsp;Postman jokes. Do we need to say more?</figcaption></figure></div>



In this quick QnA, Espie makes the point that software quality begins with people, that more testing doesn’t necessarily mean better quality, and that context matters in metrics.



<div id="toc_container" class="no_bullets">Table Of Contents<ul class="toc_list"><li><a href="#Q1_What8217s_the_most_common_organizational_bug_that_you8217ve_seen">1 Q1: What&#8217;s the most common organizational bug that you&#8217;ve seen?</a></li><li><a href="#Q2_What_are_some_testing_anti-patterns_you_wish_never_existed">2 Q2: What are some testing anti-patterns you wish never existed?</a></li><li><a href="#Q3_Metrics_question_What_are_some_QA_or_testing_metrics_that_you_ALWAYS_want_to_track_Why">3 Q3: Metrics question. What are some QA or testing metrics that you ALWAYS want to track? Why?</a></li><li><a href="#Q4_What_are_some_Severity_1_bugs_that_ALMOST_slipped_past_you_and_how_did_you_make_sure_they_never_did_so_again">4 Q4: What are some Severity 1 bugs that ALMOST slipped past you — and how did you make sure they never did so again?</a></li><li><a href="#Q5_Whats_your_most_memorable_release_day_Good_OR_bad_just_as_long_as_no_one_dies_at_the_end8230">5 Q5: What’s your most memorable release day? (Good OR bad, just as long as no one dies at the end&#8230;)</a></li></ul></div>
<h3 class="wp-block-heading" id="q1-what-s-the-most-common-organizational-bug-that-you-ve-seen">Q1: What&#8217;s the most common organizational bug that you&#8217;ve seen?</h3>



An &#8220;organizational bug&#8221; I see often is teams that are ‘agile’ until it comes to testing.



Design, product, development functions are all collaborating and working together. But when it’s time to test, the team switches to ‘waterfall mode’. They pass the ticket to the tester and move on to something else.



Often agile teams don’t have a clear picture of where a test specialist should engage. The fix is to encourage teams to <a href="https://testsigma.com/automated-continuous-testing-platform">include testing at every stage of the development process</a>. But this is easier said than done.



One thing that’s worked for me is encouraging developers and testers to test together.



When a developer has a change ready to merge, they pair up with a tester. The developer demonstrates to the tester that it “works on my machine”. We call this process “WOMMing”. I’ve also called it “show and tell”.



(The term WOMMing was coined by one of my former colleagues at <a href="https://www.linkedin.com/company/pushpay-ltd/" target="_blank" rel="noreferrer noopener nofollow">Pushpay</a> &#8211; I don’t know who exactly to attribute it to though.)



During WOMMing, the role of the tester is to tease out edge cases, ask questions, and be critical of the change. It’s useful because it helps find bugs early. Additionally, it encourages the developer to learn some testing skills.



Of course, it’s not a silver bullet. Testers need to be pulled into other phases of the development life cycle. But if a team is at a point where testers aren’t included at all, pairing with developers could be a great way to start.



<h3 class="wp-block-heading" id="q2-what-are-some-testing-anti-patterns-you-wish-never-existed">Q2: What are some testing anti-patterns you wish never existed?</h3>



I wish massive end-to-end UI testing suites did not exist.



I see this often. Organizations use automation to test every screen, function, and feature in their product. In my experience, you only need a few E2E UI tests. As a test suite starts to get bigger, the value from each extra test starts to diminish.



Some things that have stood out to me include:



<ul class="list_style_disc wp-block-list"><li>Tests that check the same screen over and over, with only slight variations in data.</li><li>Tests that check things that are already covered by tests at other layers; Unit or API tests, for example.</li><li>Tests that check features that are of little to no business value.</li></ul>



These don’t seem like a good use of time. Also, each extra test will make the test suite take longer &#8211; and, potentially, more brittle.



A lot of organizations fall into the trap that you need to automate everything. Testers become test automation engineers and end up building more and more automated tests. I’m not convinced; I think a tester&#8217;s time could be better spent elsewhere.



<h3 class="wp-block-heading" id="q3-metrics-question-what-are-some-qa-or-testing-metrics-that-you-always-want-to-track-why">Q3: Metrics question. What are some QA or testing metrics that you ALWAYS want to track? Why?</h3>



First I’ll say, I am really careful with metrics.



If care is not taken, and teams don’t understand why metrics are important, they can be tempted to ‘game the system’. <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.theguardian.com/business/ng-interactive/2015/sep/23/volkswagen-emissions-scandal-explained-diesel-cars" target="_blank">There’s a really interesting (and shocking) example from Volkswagen from a few years back, where they ‘cheat’ to achieve emissions targets</a>.



That isn’t to say metrics aren’t important, but they do need to be treated with respect. Personally, I like <a href="https://www.cloudbees.com/blog/dora-devops-metrics-bandwagon" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">DORA metrics</a>:



<ul class="list_style_disc wp-block-list"><li>Deployment Frequency (how often does a deploy happen)</li><li>Mean Lead Time for changes (how long it takes to get a change to production)</li><li>Mean Time To Recovery (how long it takes to recover from an incident)</li><li>Change Failure Rate (how many changes fail in production)</li></ul>



My approach to metrics like these is simple: find out what the numbers are and then work to improve them. For example, it is useful to know the Mean Time To Recovery. If it’s half a day, then that’s a useful and interesting piece of information. Once we know that, then we can look at ways to improve that time and measure it again.



My suggestion with metrics though is:



<ul class="list_style_disc wp-block-list"><li>Be clear what action is being taken to improve a certain metric, and that it makes sense.</li><li>Measure, and measure again after improvements have been made. But, don’t set arbitrary values as targets. “Make our time to recovery faster” is far better than “make our time to recovery less than ten minutes”</li><li>Make sure everyone has an understanding of why you want to measure a specific thing. Context matters!</li></ul>



Hopefully, these suggestions will help to avoid becoming like the team at Volkswagen 🙂



<h3 class="wp-block-heading" id="q4-what-are-some-severity-1-bugs-that-almost-slipped-past-you-and-how-did-you-make-sure-they-never-did-so-again">Q4: What are some Severity 1 bugs that ALMOST slipped past you — and how did you make sure they never did so again?</h3>



This is a story about one that did slip past us. But, we got away with it.



My team worked on a billing product for a SaaS company. We had a seemingly simple task, replacing a couple of our products with new ones. The products also had new prices.



Now, the data structure was overcomplicated. There were two tables, product, and price. Each product had a start and an end date. It was related to a price, which also had its own start and end date. (You might see a problem here already!)



To make matters worse, the product dates were stored in NZT. But the price dates were stored in the local timezone for that region. We missed this in testing.



When we went to production, we ended up with a product that ‘ended’ at midnight NZT. Its associated price didn’t ‘start’ until midnight GMT. So, for a window of twelve hours, nobody in the UK could buy our product. Yikes!



We got lucky. Our product was a B2B product, and the 12 hour period fell on a Sunday. So it was very unlikely anyone was trying to make a purchase. After the fact, we checked the usage statistics &#8211; zero hits. Pretty thankful we made the call to change the price over a weekend!



We ran a post-mortem to figure out what could prevent this from happening again. We decided that the fix for it was to refactor our database so that everything used the same time zone. But, this was going to be a hefty piece of work &#8211; that one change had significant downstream effects.



There was other information to take into account:



<ul class="list_style_disc wp-block-list"><li>Price changes are a rare occurrence for this organization.</li><li>Product changes are even rarer.</li><li>There was an ongoing piece of work to replace this billing system &#8211; in a couple of years, we expected it to be gone.</li></ul>



We opted to do no mitigations. That’s kind of counter-cultural, but it was the right thing to do. I guess my point is, sometimes, doing nothing is entirely reasonable!



<h3 class="wp-block-heading" id="q5-what-s-your-most-memorable-release-day-good-or-bad-just-as-long-as-no-one-dies-at-the-end">Q5: What’s your most memorable release day? (Good OR bad, just as long as no one dies at the end&#8230;)</h3>



The world of CI/CD highlights the difference between deploying and releasing. We deploy software all the time. Silently behind feature flags or otherwise inaccessible by customers. Then, when it’s ready for customers, all we need to do is flip a switch. Then, it’s released.



My team did this recently for a new integration feature. All our code was sitting in production, behind a feature flag. Once we were confident everything was in place, all we had to do was switch it on. Very unglamorous, and no ceremony.



We returned to the office the next day. Our lead engineer went to check the logs first thing that morning,&nbsp; to see if anyone had used the newly ‘released’ feature. And sure enough, one person had!



Now I know, it doesn’t sound like much. But, for us, it was incredible. After months of work over that feature, we had our first real live user!



The joy comes not from releasing the feature but seeing people use it and get value from it.



This is one reason why having good monitoring and logging is so important. You can see what your users are doing, and celebrate the success!



<hr class="wp-block-separator"/>



For more test insights by James Espie, check out <a href="https://jpie.nz/" target="_blank" rel="noreferrer noopener">JPie.nz</a>

Quality Engineering Discussions: 5 Questions with James Espie

Why love software testing? Hear from the experts

Whenever a new methodology or a new system is introduced, with it &#8211; certain terms and guidelines are introduced too. These terms and guidelines are meant to be followed to ensure maximum output. But during implementation, some of these guidelines might be deemed not-so-important and might not be followed. 



Similar has been the case with Scrum Agile Methodology. Since it was introduced, it has been adopted by a majority of software development teams. But, the catch is that most of them are not following a basic guideline &#8211; the Definition of Done for a feature should also include testing and automation of the feature. 



And just that is the reason why we are laying special emphasis on ‘insprint test automation’. This blog is a special ‘round up’ blog where we asked experts some important questions around ‘insprint test automation and have compiled their answers in the form of a blog here. <gwmw style="display:none;">



We have a detailed ebook that discusses insprint test automation, why it is important in detail! Grab your free copy! <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://testsigma.com/ebook/in-sprint-test-automation-simplified" target="_blank">In-sprint test automation simplified! </a>



Now, let’s hear from the experts what they think about in-sprint testing.



<div id="toc_container" class="no_bullets">Table Of Contents<ul class="toc_list"><li><a href="#Brijesh_Deb">1 Brijesh Deb</a></li><li><a href="#Adebayo_aka_Tea_Pot">2 Adebayo aka Tea Pot</a></li><li><a href="#Ajay_Balamurugadas">3 Ajay Balamurugadas</a></li><li><a href="#Nikolay_Advolodkin">4 Nikolay Advolodkin</a></li><li><a href="#Conclusion">5 Conclusion</a></li></ul></div>
<h2 class="wp-block-heading" id="brijesh-deb">Brijesh Deb</h2>



<div class="wp-block-image is-style-circle-mask"><figure class="alignleft size-large is-resized"><img decoding="async" src="https://testsigma.com/blog/wp-content/uploads/Brijesh-Deb.png" alt="Brijesh deb" class="wp-image-16326" width="200"/></figure></div>



Here&#8217;s what ‘Brijesh Deb, Sr, Test Consultant at Sogeti’ explains: 



Q. How does insprint test automation fit in agile?



Let’s begin with the Agile principles. At any point during a project, working software is the primary measure of success. Now for a piece of software to be working, it is essential that the definition of done is met for all the user stories that have been worked on to build that working piece of software.&nbsp;



Now when we talk about the definition of done, it is incomplete to say that the stories will have development only and will be tested later. In order to meet the definition of done we need to build and test and only then the definition of done is met. So, if you are planning to implement a feature in a sprint, it must be ensured that the feature is built and tested in the same sprint to meet the definition of done and give a sense of in sprint testing.&nbsp;



Q. What would you say about teams that do testing in different sprints or wait till the end of the sprint to start testing?



Often it is seen that testing gets lesser time in a sprint or is done in a different sprint. Both situations are against the ethos of Agile and do not meet the idea of definition of done and give enough confidence to the product.



Testing playing catch up with development is not agile. The best way to deal with this is to involve testers early and ensure that the sprint planning is done keeping in mind the definition of done meaning if a story is a 5 pointer then it must be ensured that the 5 story points are assigned considering the effort needed to build and test the story.&nbsp;



Now let’s look at the other burning topic, automation. Often seen as a mechanism to reduce regression effort, Automation has far greater reach and can be of much more value than mere regression.&nbsp;



Smoke tests, sanity tests and regression tests are easy candidates for automation because they are repeated far too often. But let’s be honest, is that the only premise for automation? What about other tasks that can be automated like test data generation, test reporting? These are essential elements of every project and hence there’s immense value in automating these activities.&nbsp;



Q. Your suggestions on how to implement automation?



Now let’s look at automating the tests. It is of utmost importance to do a thorough feasibility analysis to choose the right tests that can be automated. Then the next important step is to choose the right layer in which the tests will be automated. Automating the tests in the right layer is always a wise choice and the Agile Test Pyramid provides a guideline for the same. With that as the guideline and collaboration with developers, fellow testers, business, PO and stakeholders, easily the right layer for automation can be established.&nbsp;



Once you’ve done that, it is also important to leverage the advantage of insprint automation which gives you feedback much faster and results in a quicker time to market. Note that, in sprint automation should also be a part of the established and agreed definition of done.&nbsp; 



<h2 class="wp-block-heading" id="adebayo-aka-tea-pot">Adebayo aka Tea Pot</h2>



<div class="wp-block-image is-style-circle-mask"><figure class="alignleft size-large is-resized"><img decoding="async" src="https://testsigma.com/blog/wp-content/uploads/Adebayo.png" alt="Adebayo" class="wp-image-16328" width="200"/></figure></div>



We asked Adebayo aka Tea Pot( Founder, Naija Test Crowd): 



Q. how to make Insprint Test Automation possible and this is what he said: 



“Insprint testing is quite important and should be the default way of working on any Agile methodology but there are a few things that must be done to make this possible:



Not in any particular order: The project must put all their ducks in a row: ensure that everyone understands the strategy, ensure that testers and developers attend scrum meeting where the sprint is the subject, ensure that the scope for each sprint is quite concise and achievable, test cases should be developed same time as the developers are developing the codes for the sprint and the codes should be delivered within time to allow the testers test, where the review of test cases is mandatory, this should be done during development so the test cases are reviewed and ready for execution as soon as the code drops. Also, and equally important, this can be automated as the sprint progresses into another sprint until the end of the project.”



Q. When we asked why he thinks insprint test automation is important he says:



In sprint testing will afford the tester to relate to the development more easily as each test is focussed on the latest release, equally focussed on the latest functionally. Conversation between developer and tester will flow so easily when both are discussing what they are currently doing, instead of what they did or shall be doing in the future. And the automation should probably be as per automation pyramid, with the right amount of overlapping and uniqueness of tests being automated on all the layers.&nbsp;



Q. When we asked him how much time can it take for an organization to fully achieve insprint automation, he says:



Automating the tests within a sprint is not impossible but it does take time to achieve, the process must be well defined and start from the earlier sprints, and everyone must have a very good understanding of the strategy, so all downtime is utilised rather wisely and efficiently too. Test cases that have been reviewed and executed in the previous sprint can be automated as the sprint ends and next sprint is being planned but there must be enough time for the tester to test otherwise, automating the tests is one futile effort that will simply result in a complete waste of time. Subsequent sprints can have the automated test included in execution while also making time to automate tests (reviewed and agreed) for the current sprint.&nbsp;



<h2 class="wp-block-heading" id="ajay-balamurugadas">Ajay Balamurugadas</h2>



<div class="wp-block-image is-style-circle-mask"><figure class="alignleft size-large is-resized"><img decoding="async" src="https://testsigma.com/blog/wp-content/uploads/Ajay.png" alt="Ajay Balamurugadas" class="wp-image-16330" width="200"/></figure></div>



<a href="https://testsigma.com/blog/qna-with-ajay-balamurugadas/" target="_blank" rel="noopener">Ajay Balamurugadas</a> points out 3 key principles in the successful execution of insprint testing. 



Effective communication, a common understanding of the goal and lots of collaboration are some of the key principles of in-sprint automation. Now, let’s break them down. 



i. Effective communication:



If you don&#8217;t communicate what you expect from each team, you are most likely not going to get it. 



ii. Common understanding:



All stakeholders should know what is targeted. There is no point testers trying to automate and designers changing the design or functionality changes introduced by the product managers 



iii. Collaboration:



Not an easy task, at the same time, not impossible either. Given the right teams, intent, tools and collaboration, in-sprint automation can be a reality.



<h2 class="wp-block-heading" id="nikolay-advolodkin">Nikolay Advolodkin</h2>



<div class="wp-block-image is-style-circle-mask"><figure class="alignleft size-large is-resized"><img decoding="async" src="https://testsigma.com/blog/wp-content/uploads/Nikolay.png" alt="Nikolay Advolodkin" class="wp-image-16333" width="200"/></figure></div>



Nikolay Advolodkin, Sr Solutions Architect at Sauce Labs also comments that he believes that testing within sprints is an absolute must. 



The product cannot be deemed finished until it has been tested and we are positive that we are releasing a quality product that works for our end users. Without in-sprint testing, we are risking the end-user experience by creating longer feedback and release cycles.



We agree with him and recommend that testers should keep up with the dev speed, leaving no gap between dev complete and test complete. And of course, when automation comes into the picture &#8211; it becomes crucial to adopt <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://testsigma.com/blog/in-sprint-automation/" target="_blank">a tool that enables insprint test automation</a>.&nbsp; 



<h2 class="wp-block-heading" id="conclusion">Conclusion</h2>



Of course, just blindly implementing automation won’t ever be enough. Intelligent automation is the way to go where the time and effort is only invested in the automation of tests that need and should be automated. Along with it, to keep the automation current, the automation for the tests corresponding to the development work in the current sprint should be targeted in the same sprint.



 Check out our latest guide on How to begin <a href="https://testsigma.com/blog/in-sprint-automation/">in sprint automation</a> for more.

Insprint Test Automation –  Round up

According to a <a href="https://www.softwaretestingnews.co.uk/new-survey-reveals-test-automation-trends" target="_blank" rel="noreferrer noopener nofollow">survey conducted on software test automation</a>, 51% of organisations prefer to retrain existing staff in test automation skills. While there are still 49% that would like to hire them specially for test automation related tasks.



Other findings from the same survey confirmed that organisations want to move to more of test automation and a minimum of manual testing.



To quote the findings, “Only 5% of survey respondents said they currently carry out 0:100 manual: automation testing. The majority (66%) are either at 75:25 or 50:50, and 9% said they are only doing manual testing.



When asked what they’d like to achieve in the next five years, the majority (73%) said they’d like to see 50:50 to 25:75 manual: automation testing.



And, 14% said they’d like to have no manual testing at all.” Similarly, with the working conditions changing during pandemic, automation testing is something that has seen a huge jump. Thus, it is clear that automated testing is the way to go for the software testing industry. 



We asked some of the industry experts how they thought was the best way for a manual tester to transition to automation testing.



Let’s see what they had to say:



1. Chaithra Rao (Chaithra is a Senior Manager of Test Engineering at Insider Inc.)



“I like to think of test automation as writing code with a tester’s mindset.



It’s not so much about a particular tool you use but choosing the one that best suits your needs, is simple to setup, use and maintain. 



For someone who is just getting started with test automation, I would recommend learning the basics of programming first. Once you know the fundamentals, you could learn about the different test frameworks and design principles that’s out there. Understanding how and why the tests are setup is more important than just writing automated tests.



Once you have this knowledge, it’s easier to write tests. When you are first attempting to write an automated test, start with a simple assertion and then build upon it. There are several online resources to get help like Udemy, Test Automation University etc.”



2. Samantha Connelly (Consultant &amp; Owner at Bug Hunter Sam)



“I think focusing on the manual to automation path is the wrong conversation. Technical skills are in demand, yes, but we don&#8217;t use manual/automation terms in any other profession. We don&#8217;t have manual doctors, lawyers or software developers. 



Also, the tools and technologies are always changing. I think focusing on software engineering skills are a better return on investment. 



The first skills I recommend improving for someone wanting to become more technical is git and the command line. Beyond that point, it really depends on what technology you want to learn. Understanding web protocols is also pretty useful.”



On her own website, Sam has compiled a testing career cheat sheet which we recommend you to go through.



3. Ajay Balamrugadas (AVP Delivery at Qapitol QA)



A manual tester making a shift towards automation should:



“1. Understand the terms. Ask questions and /atleast/ know just enough to get started. Know what is method, function, loops, repo, principles, and basics of using the tools.



2. While you spend time learning the programming concepts and the language, use your testing skills to identify what to automate and what not to automate



3. Start with simple projects, shadow people working on automation projects and with enough interest, persistence and hard work, you will be able to make significant progress.”



At the same time understand that functional testing or automation, solving problems is the key. Both are needed in different contexts.&nbsp;



Let’s try to summarize and create some steps for you:



1. When you want to begin your automation journey, it means you want to start automation of your test cases. It is here, that comes to the big question &#8211; how are you going to automate those test cases?



The first logical step should be to know what tool you should choose to begin with automation. It will be an easy jumpstart with the tool you have heard most about. But the best way will be to research and analyse a few others. Here, we have compiled some points that will help you <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://testsigma.com/blog/10-points-to-help-you-choose-the-right-test-automation-tool/" target="_blank">choose the right automation tool for you.</a>



Testsigma is a AI test automation tool that has all the features that will enable you to achieve insprint test automation easily and efficiently. To know more check the article here: <a aria-label=" (opens in a new tab)" href="https://testsigma.com/automated-testing" target="_blank" rel="noreferrer noopener">Why Choose Testsigma as your Automation Testing Tool?</a>



4. Start automation with simple projects and simple test cases and then build-up to the complex ones.



5. Along the way, there would be some basic tools for eg. git and command line. Try to have a hang of them. This will help you transition into automation faster.



<div class="cta-banner-v2">
<div class="title-with-description ">
Build and run tests for Web, Mobile, Desktop, and APIs &#8211; all under one platform of Testsigma. 
</div>
<a href="https://testsigma.com/signup" target="_blank" class="cta-btn-v1" rel="noopener">Try for free</a>
</div>



About tools, Ajay suggests, “I would like the testers to focus on the principle of&nbsp;



&#8220;Use tools where it helps you save time, effort and adds value&#8221; 



Also, do not wait for the need to hunt tools. Be prepared by exploring different tools across categories so that when you actually need them, you are ready.



Finally, I have realized that having at least 3 tools for every category helps especially when one of the tools doesn&#8217;t serve your purpose.



Two good sites to be familiar with are alternativeto.net and alternative.to”



We have compiled a detailed blog on <a rel="noreferrer noopener" aria-label="different types of tools that will help make the tester’s day-to-day life easier (opens in a new tab)" href="https://testsigma.com/tools/software-testing-tools/" target="_blank">different types of tools that will help make the tester’s day-to-day life easier</a>. Do check it out!



Even though you see yourself as an automation tester in the future, there are few things that you should always keep in mind:&nbsp;



1. Automation is an activity that should support testing but automation is not all. Sometimes you may want to automate some areas while you need to test some areas manually. Thus, you don’t have to focus on 100% automation but on quality automation. That means you should be careful to choose what to automate and take up automation with the aim of providing an ROI in future.



2. Even though automation tester might be a separate profile, there are some skills that it shares with manual testing which are: Inquisitive thinking, collaborating with the team and ensuring customer success.



3. Participate early on in the software development life cycle



4. Make sure to be well versed in the software development methodology being followed in your team. For e.g. Scrum or Kanban.



5. Understand the project toolset



6. Keep yourself updated with new developments in the testing industry. This can be done by attending testing conferences, meetups, following influencers in the field and talking to people working in the same industry.



We hope these tips will help you make your transition to automation in testing a smooth and delightful one! 



<div class="cta-banner-v2">
<div class="title-with-description ">
Create Automated Tests by simply recording your actions and test in Parallel on Cloud using Testsigma.
</div>
<a href="https://testsigma.com/signup" target="_blank" class="cta-btn-v1" rel="noopener">Try for free</a>
</div>

Skills of a tester- Manual to Automation testing journey