Once you have completed your project, you need to demonstrate that it satisfies its requirements. There are several ways to do this. One is to develop an automatic test suite that covers all requirements. Once the test suite passes, the software can be shipped. This approach can be combined with an defect tracking system that ranks the severity of observed defects. In this case, the software can be shipped once all defects above a specific severity level have been fixed. However, creation of automatic test suites is a time-consuming process. Additionally, the test suite requires validation to ensure that is has sufficient coverage of project requirements, and the tests themselves are correct. Unfortunately, there isn't enough time in one quarter to develop your project and create a significant automatic test suite (though modestly scopes test suites are feasible, and could be quite useful).
Another approach is to manually perform an acceptance test. An acceptance test plan document details a series of tests performed to validate the software meets its requirements. An acceptance test is usually held during an official "sell-off" of the software to the customer. Representatives from the customer verify the test executes as specified in the test plan document. Deviations are noted, and result in either a correction to the software, or to the test plan, or both, depending on where the fault lies. After correction, the test is re-run until it passes completely. At this point, the customer accepts delivery of the software (and pays the remaining amount of the development contract costs).
During the last week of class, you will perform a sell-off of your project to the TA and the Professor (acting as the customer), using an acceptance test. During this sell-off, your group will execute the test procedure outlined in your acceptance test plan.
The test plan document describes a series of operations to perform on the software. This is a very detailed document. No key gets pressed, no menu item is selected, unless the acceptance test plan says so. Whenever there is a visible change in the state of the software (new window, change in the display, etc.) the test plan document asks the testers to observe the changes, and verify that they are as expected, with the expected behavior described precisely in the test plan. As the test is executed, testers check-off each operation performed, and write down observations made as the software executes.
For this class, the acceptance test is limited to an hour in length. It can be shorter than this, but no longer. Since the goal of the acceptance test is to demonstrate that the software meets its requirements, the hour limitation may mean there isn't enough time to cover all requirements. In this case, it is necessary to prioritize your test coverage, so the most important requirements are covered first.
Since the acceptance test is used by the customer to determine whether to accept delivery of the software, the customer needs to approve the acceptance test plan prior to its execution. So, in addition to receiving a grade on the initial submission of your acceptance test plan, you must revise and resubmit your test plan document until the TA formally approves it. Similarly, if you do not initially pass your acceptance test, your group must revise the software or the test (depending on the source of the observed problem) and then re-run the acceptance test. Failure to pass the acceptance test after two tries will have significant adverse affect on the "acceptance test plan performance" component of your class grade.
The acceptance test, along with the Software Inspection, comprise the only graded verification and validation (V&V) activities for your project. However, they should not be the only V&V actions you take. A successful acceptance test is usually preceded by significant unit testing and integration testing of the software, as well as a dry-run through the acceptance test itself. This implies that coding is completed far in advance of the acceptance test, so time and attention can be focused on testing. Your group should develop its own V&V strategy to ensure the software meets its requirements, and easily passes the acceptance test plan.
For more details, see the template, and the acceptance test plan execution description.
Last updated: