1
Quality Assurance in Data Science: Need, Challenges and Focus

Jasmine K.S.*, Ajay D. K. and Aditya Raj

Dept. of MCA, RV College of Engineering®, Bengaluru, Karnataka, India

Abstract

It is widely accepted that quality is assured for a product through the process of testing. With the rapid development in the area of data science, research is going on with proper management of data and with its right usage, test engineers can learn about their users. One can predict the associated risks and with a focus on data masking based on the data model. Prescriptive and predictive analysis can be more accurate if the techniques are developed and the accuracy is measured using metrics. Preparing data with required quality and identifying the possible resources are challenging tasks faced by a data scientist. The effective and systematic use of advanced technologies like high-speed hardware and network computing, cloud computing, cross platform tools, etc., continues to be an elusive goal for many organizations. In this context, the chapter investigates the feasibility of novel and practical solutions in this aspect.

Keywords: Quality assurance, testing, data science, data analysis, decision making

1.1 Introduction

1.1.1 Quality Assurance and Testing

In the traditional software development approach, Quality Assurance was done at the later stages of the development process and feedback was collected for improvement. In almost all organizations there exists a Quality Assurance team, responsible for identifying the product defects and resolving them before release of the product.

But in the Agile development approach, the Quality Assurance (QA) team works collaboratively with the development team with a shared responsibility in improving the quality of the product. Irrespective of the project and domain QA plays an important role in delivering high-quality products. It helps developers establish goals and define quality standards. The process also helps in identifying and resolving defects in the products before releasing them to market.

1.1.2 Data Science and Quality Assurance

Data is available in structured and unstructured format. There is a need of methods and algorithms to extract knowledge and inference so that available data can be converted into wisdom to apply in any application domain. In the context of data science, QA is a process of analyzing and modeling the available data to ensure high quality and meeting quality standards. Models have to be created with a variety of data sets and robustness tests have to be conducted for trained models similar to any use case for real-world scenarios [6].

1.1.3 Background

Data science basically puts efforts to verify the general consistency among relevant data and applies scientific methods to ensure the data quality. Domain knowledge is necessary to ensure not only the quality of data but also to avoid errors and inconsistencies in the data collected in the Quality assurance process [1]. After collecting the data, interpretation of data is also a huge challenge. So it is essential that quality assurance should be a continuous process throughout the product development phase, starting from data collection till the delivery of the product. Expert opinion will add value to it. In order to validate the results obtained after data analysis, domain experts play an important role. Finding and resolving quality-related issues once the product is delivered will result in unnecessary wastage of time, money and effort [2, 9]. In order to perform data analytics in an efficient manner, a variety of data sets is essential [3]. Frequency, time and many iterations are part of data analytics. So there is a need for systems which focus on data quality [4, 10]. Software quality and hardware quality also play an equal role [5].

1.2 Testing and Quality Assurance

Testing is a process which ensures the intended behavior of any product within the given time frame and also helps to avoid additional efforts, time and cost overruns. System testing ensures not only product quality but also process quality.

1.2.1 Key Terminologies Associated With Testing

1.2.1(a) Defect count: A defect count is one of the measurements of product quality. It shows the number of undetected errors. The defect rate can be computed by defective products observed divided by the number of units tested. For example, if 5 out of 10 tested units are defective, the defect rate is 5 divided by 10.

1.2.1(b) Test case execution: During the test execution process the developed code is executed with generated test cases and actual output obtained is compared with the expected output. After the test execution process, bugs and test execution status are maintained for further measures. Following are steps involved in test execution:

Step 1: Gathering testing requirements

Step 2: Test plan

Step 3: Test design

Step 4: Test execution

Step 5: Defect reporting and tracking

Step 6: Defect Mapping

1.2.1(c) Test execution classification: The type of testing can be classified based on the purpose for which it has to be executed: for example, performance testing, defects testing, regression testing, etc. During the test execution process, the main focus is to verify how much the actual results varied from the expected results.

1.2.1(d) Pre-requisites for test execution phase

1. Test Plan, test cases development and the test data should be ready
2. Test environment setup and its validation
3. Clear understanding of exit criteria

1.2.2(e) Post-requirements for test execution phase

1. Validation of actual results with expected results
2. Defects should be closed or deferred
3. Test execution and defects summary report
4. Prioritize the test plans based on the identified risks
5. Continue the process

1.2.2(f) Test velocity

Testing velocity shows the number of tests one is running per day/weekly/ monthly, etc. It also shows the difference between the planned and actual time.

1.3 Product Quality and Test Efforts

1.3.1 Testing Metrics

Testing metrics are required to measure and estimate process and product quality and to improve the efficiency of the overall testing process. Two major categories are defect metrics and productivity metrics.

Example of productivity metrics: Mileage of any vehicle compared to its ideal mileage recommended by the manufacturer. Figure 1.1 shows the test velocity and loop count also we can see the thread properties.

Figure 1.1 Test velocity, loop count.

Example of defect metrics: Number of defects found, accepted, rejected and deferred in the payment process using a credit card.

A few automated testing tools: Selenium, JMeter, Appium, Junit

The chapter demonstrates a few testing aspects using JMeter. It helps to test and analyze overall performance under different load types.

Description of Figure 1.1:

• Name: Provide a custom name or, if you prefer, simply leave it named the default "Thread Group".
• Thread count: The number of threads indicates the number of users under test.
• Ramp-up Period: How quickly (in seconds) you want to add users. For example, if set to zero, all users will begin immediately. If set to 10, one user will be added at a time until the full amount is reached at the end of the 10 seconds. Let's say 5 seconds.
• Loop Count: It indicates the number of times the test should repeat.

Figure 1.2 demonstrates thread delay of 300 milliseconds.

Figure 1.2 Thread delay.

In Figure 1.3, successful test results show in green icons. Figure 1.3 also shows load time, connect time, errors, the request data, the response data, etc. Figure 1.4 demonstrates the Test results in table and and Figure 1.5 shows the Test report summary.

Figure 1.3 Test results.

Figure 1.4 Test results in table.

CSV file is the source of data user for testing and one can see the dynamic results in the View Results Tree (Figure 1.6). In our example, it is no longer Boston and London, but Philadelphia and Berlin, Portland and Rome, etc.

Figure 1.5 Test report summary.

Figure 1.6 View results tree.

1.3.2 How to Improve the Business Value to Products Using Test Automation

With the advancements in the area of automation testing, in order to ensure software quality, not only the developed software has to meet the functional requirements specifications, it also has to meet the non-functional requirements specifications such as operational efficiency, reliability, security, maintainability, availability, code efficiency, etc., through which one can increase the business value.

With automation testing one can manage the performance of test activities there by giving conformance to test requirements. In the current scenario, automation testing became a crucial part of quality assurance [7].

The business value testers are delivering with their test efforts and can be evaluated by measuring the following attributes:

• Percentage of...