Only Paul and Jonathan reviewed/commented on the first draft of the Testing Process (part 1) document.
Alan will integrate review comments and post a second draft of the Testing Process (part 1) document for review by Wednesday, November 28.
Alan created a preliminary first draft of the Testing Process (part 2) containing the remaining sections of the existing TPTP Testing Strategy document.
Alan will add more detailed content to the first draft of the Testing Process (part 2) by Wednesday, November 28.
CVS check-out of all of the test plug-ins in the /test-results/<project>/* modules.
Outstanding:
Check for Intel build to complete (every x hours) before launching the BVTs.
Configure and run the Agent Controller on the reference platform.
Joel is working with Jonathan to integrate the existing code (target: today/tomorrow).
Create the root-level test suite for BVTs. The test infrastructure will execute each test plug-in's AllBVTTest.testsuite, as referenced by the org.eclipse.hyades.tests/All<test type>BVTTests.testsuite. We do not need to check-in the execution results to CVS due to disk space limitations and polluting of our test pass results. Each developer can rerun the automated tests to reproduce a failure.
Automatically generating the Test Report.
Until the blocking issue with remote test execution is resolved, we can start generating a 0% complete test report for the PMC to evaluate the TPTP test bucket for 4.5 iterations.
Automatically posting the Test Report to /tptp/test/reports/<release>.
Will require a new report generator for generating detailed test reports from a root-level execution result.
Cleanup of the reference platform.
Documenting the test infrastructure for use in the second testing process document, which will be required by TPTP adopters or extenders.
Email notifications to component leads for failing test suites.
Blocking Issues:
When generating reports, the report has no entries: Paul and Sheldon are investigating.
Defect 208616: Jerome are working on a fix. Paul/Kendric/Joel are verifying the fix.
Defect 200194: Symptoms have not been reproduced. Retargeting to i5.
Defect 190453: Joel has a work-around. Retargeting to i6.
Defect 210289: Joel has a work-around. Retargeting to i6.
Defect 210292: Duwayne triaged last week and provided a work-around for Joel. Retargeting to i6.
Add more detailed content to the first draft of the Testing Process (part 2) by Wednesday, November 28.
Joel:
Outstanding issues for the PoC (integrate the ASF with the TPTP builds to check-out a 'Hello World' TPTP JUnit test suite from CVS, run the test suite on an Eclipse instance running on the target platform, and generate a BIRT report on the test execution using existing sample code).
Paul:
Determine labor cost (PW) for a) full test pass and b) smoke test pass from project leads.
Encourage Project Leads to automate existing manual test suites.
Upcoming Meeting
We will meet next on Thursday, November 29 on the Agent Controller Status Call (9 AM ET).
