Regression Testing
Home Up Y2K Testing Article Regression Testing Data Aging PCs

 

Subject: Regression testing
Date: Sat, 23 Jan 1999 06:16:38 -0500
To: year2000-discuss@year2000.com

I wonder if someone could explain what the regression testing is? What steps do we take to do it?

Tevy,

You will get significantly different answers from different people on that one.

I define regression testing as executing your application before and after remediation and comparing the output to demonstrate the output only varies in ways you expect it to vary (for example, if you expanded, you expect the output files of the remediated program to have century digits where the older version may not have had them, if you have time stamps, you might expect those to vary). The goal of regression testing is to demonstrate you have not "broken" something in your application during the process of remediation. A regression test proves nothing about the success or failure of Y2K remediation efforts. In fact, any non-compliant application can always pass a regression test by not touching the application and it will still fail miserably when it starts to process Y2K data.

In a Y2K context, I differentiate that from time dimensional testing, where the remediated program is executed with date fields that vary by a "constant" amount of time (dates are "aged") and outputs are compared to demonstrate the output only varies in date fields the way you expect it to vary. The goal of time dimensional testing is to demonstrate that the application works the same "yesterday", "today" and "tomorrow" implying that Y2K will have no effect on it (when "tomorrow" extends into the next century).

If I broke it into steps, I suppose you would:

  1. Create your test data or use existing test bed/test cases.
  2. Run your application against the test data, saving the output files for later comparison
  3. Remediate the application
  4. Run the remediated application against the same input data used in (2), saving the output
  5. Compare the saved outputs from (2) and (4)
  6. if any unexpected differences show up, go back to step (3)

I put (1) and (2) first to emphasize that these two can be done before, in parallel with or even after remediation.

Another opinion is Don Estes, who I believe defines regression testing to include aging dates within this century and only considers time dimensional testing to include dates that extend past 1999.

If you have expanded your data, we have a file comparison tool that will help you in the regression comparison process. Without identifying the location or formats of any specific dates in the file, it will accurately compare the files, except for two well understood cases expressed on my WWW site, and provided your date formats are among the thousands we cover (or you've had us add your specific, unique format).  This identification characteristic also makes this tool ideal for comparing otherwise hard/impossible to compare files such as aged printer files.

When you get into testing with aged data, the same comparison tool can also tell you by how much the two files vary (in days or years).