Delivered the first draft of the Testing Process (part 1) document.
Joel:
Working with Jonathan to configure and run the Agent Controller.
Code is written.
Jonathan just received legal approval for using freeSSH on Windows. Note, Cygwin cannot be used since it does not support custom environment variables on Windows.
Outstanding:
CVS check-out of all of the test plug-ins in the /test-results/<project>/* modules.
Running the correct test suite(s). For example, defining a root-level test suite for BVTs (possibly replacing the smoke tests). The test infrastructure will execute each test plug-in's AllBVTTest.testsuite, as referenced by the org.eclipse.hyades.tests/All<test type>BVTTests.testsuite. We do not need to check-in the execution results to CVS due to disk space limitations and polluting of our test pass results. Each developer can rerun the automated tests to reproduce a failure.
Automatically posting the Test Report to /tptp/test/reports/<release>.
May require a new report generator for generating detailed test reports from a root-level execution result.
Documenting the test infrastructure for use in the second testing process document, which will be required by TPTP adopters or extenders.
Email notifications to component leads for failing test suites.
Issues:
Defect 208616: Paul and Jerome are working on a fix.
When checking out a project using the CVS command, the name of the test plug-in is required to extract it to the correct directory. Otherwise, the test plug-in is extracted to the /test-results/<project>/<test plug-in> directory.
Paul:
Worked with Jerome on reproducing defect 208616 when deploying TPTP JUnit plug-in tests remotely.
No project leads have replied to a request to estimate the current testing costs.
Paul will remind the TPTP Project Leads to provide a sizing.
Action Items
All:
Review the Testing Process draft document by Wednesday, November 21.
Alan:
Complete the second draft of the Testing Process (part 1) document.
Complete the first draft of the Testing Process (part 2) document by Wednesday, November 21.
Includes the remaining sections of the existing TPTP Testing Strategy document, and Common Test Infrastructure.
Joel:
Prepare a small discussion on the architecture of the test infrastructure for next week's call.
Document this infrastructure for use in the second testing process document, which will be required by TPTP adopters or extenders.
Finish the PoC (integrate the ASF with the TPTP builds to check-out a 'Hello World' TPTP JUnit test suite from CVS, run the test suite on an Eclipse instance running on the target platform, and generate a BIRT report on the test execution using existing sample code).
