NFR Checklist - Cheatsheet

As a test engineer, our core responsibility to make sure we go through NFR checklist for each and every ticket we test and we ship any feature or change to production. Resiliency testing play a key role in microservice architecture. Let’s work towards that actively and build our system resilient.

Software resilience testing is a method of software testing that focuses on ensuring that applications will perform well in real-life or chaotic conditions. In other words, it tests an application’s resiliency, or ability to withstand stressful or challenging factors. Resilience testing is one part of non-functional software testing that also includes compliance, endurance, load and recovery testing. This form of testing is sometimes also referred to as software resilience engineering, application resilience testing or chaos engineering.

Since failures can never be avoided, resilience testing ensures that software can continue performing core functions and avoid data loss even when under stress. Especially as customer expectations are becoming higher and downtime can be detrimental to the success of an organization, it is crucial to minimize disruptions and be prepared for unwanted scenarios. Resilience testing can be considered one part of an organization’s business continuity plan.

 

Please follow below checklist as part of your regular testing

• Logging

  • Add appropriate logging towards any feature and any changes so we can debug any issue anytime later.

  • At the same time avoid adding unnecessary logging which are not needed for the changes.

  • Use INFO, WARNING, DEBUG and ERROR cautiously in the logging.

• Events in CT

  • Add CT events wherever it’s quite necessary to collect actions about users so we can take appropriate action based on user actions.

• SPOF (Single Point of Failure)

  • While building feature get an overall understanding of e2e flow and figure it out if this component or service fails, whole system or flow will go down along with it.

• Security (Credential Mgmt)

• Error Handling

  • Add appropriate logging for any kind of error which can happen towards feature or any change.

  • Test it out with diff set of data and how error handled at the API end apart from expected set of data.

• Timeouts

  • Timeouts will help you fail fast if any of your downstream services does not reply back within, say 1ms.

  • It helps to prevent Cascading failures.

• Retries

  • Retries can help reduce recovery time. They are very effective when
    dealing with intermittent failures.

  • Retries works well in conjunction with timeouts, when you timeout you
    retry the request.

• Fallbacks

  • When there are faults in your systems, choose to use alternative
    mechanisms to respond with a degraded response instead of failing
    completely.

• Circuit Breaker

  • Circuit breakers are used in households to prevent sudden surge in current
    preventing house from burning down. These trip the circuit and stop flow of current.

  • This same concept could be applied to our distributed systems wherein you stop making calls to downstream services when you know that the system is unhealthy and failing and allow it to recover.

  • Circuit breakers are required at integration points, help preventing cascading
    failures allowing the failing service to recover.

• Performance testing

  • Do perf testing of any new API we build to get benchmark of the same we can go as per the expectations and can track accordingly going forward.

  • Do perf testing of existing APIs if there are changes around it to make sure we didn’t impact existing benchmark results.

• Failure injection testing

  • Test your services via inject faults at integration points to verify how resilient is your service and entire system along with it.

• Health Check

  • Add health check for all the services and make sure it’s up all the time.