LSB DTK Manager Release Notes

From ISP_RAS

(Redirected from LSB TEF Release Notes)
Jump to: navigation, search

Contents

[edit]

Version 1.6.0 (2008-10-02)

[edit]

Changes Since Version 1.5.9

  1. Fixed the problem that autotest tries to download a package by itself (appeared in the Bug #2254).
  2. Fixed the Bug #2260 (Not enough of debug output from package manager).
  3. Fixed the Bug #2264 (Uninitialized value in report.pl).
  4. Fixed the Bug #2314 (DTK Manager failed to switch to the working directory).
  5. Fixed the Bug #2338 (Support for Cairo journal from Desktop tests).
  6. Changed path to qt3-azov and qt4-azov tests.
  7. Default verbose level was switched to 2 (high).
[edit]

Version 1.5.9 (2008-09-05)

[edit]

Changes Since Version 1.5.8

  1. Added support for Cpp-T2C tests.
  2. Updated support for qt3-azov, qt4-azov and Olver Core tests.
  3. Added 'LSB 4.0 (unofficial)' option at the Custom Tests page. (Certification for LSB 4.0 is still not allowed.)
    New manual Application Battery Functional Verification Tests for LSB 4.0 were added.
  4. Fixed searching files in a package (Bug #2257).
  5. Verbose output during package installation (Bug #2260).
  6. DTK start/stop scripts explicitly ask for root password now if 'su -c' command is used (Bug #2251).
  7. Other bugfixes. Among them is bug #2264 (uninitialized value).
[edit]

Changes Since Version 1.5.7

  1. Desktop-T2C was added as a certification test for LSB 3.2.
  2. Support for Qt4-Azov and Olver testsuites.
  3. Curl is supported now as an alternative download tool. It can operate through ftp proxy. (Many thanks to Shawn Zeng <shawn.zeng_at_oracle.com> for the patch.)
  4. The answer for "Does the implementation provide a C shell" in core test is "no" now. (Bug #2068)
  5. Warning messages at DTK Manager start are suppressed now. (Bug #2067)
[edit]

Changes Since Version 1.5.6

  1. Fixed incorrect profile name for vsx0 user in Core tests (bug #1978).
  2. Fixed cleanup in Xts5 test.
  3. Added one more sleep after killing httpd before starting Apache test.
  4. Avoid repeating "TC Start" comments in report.
  5. Minor fixes and improvements.
[edit]

Changes Since Version 1.5.5

  1. Fixed: Segfault in format_malloc() (bug #1863).
  2. Fixed: Wrong answer to XT_FONTPATH question in X11 tests (bug #1971).
  3. Fixed: parsing of `rpm -q` response.
  4. Running apache instance is stopped to release port 80 when starting the Apache test.
  5. Support for any install directory for Xts5 test suite (bug #1938).
  6. Now DTK Manager releases loopback devices engaged by core tests (bug #1947).
  7. Manual tests updated (added information about using XDG).
  8. Log file can be removed/edited while ATK Manager server is running.
  9. Updated logo in Web-UI.
  10. Copyright headers were updated.
  11. Minor fixes and improvements.
[edit]

Changes Since Version 1.5.4

  1. Apache test was fixed and improved. Automatic httpd starting/stoping was added.
  2. Installation of T2C-Desktop test from snapshots was fixed.
  3. Package version comparison was fixed.
  4. Packages reinstallation is forced now for tests from snapshots.
  5. Up-to-date waivers.
[edit]

Changes Since Version 1.5.3

  1. Printing tests are now included in the certification test set.
  2. Checks for fonts and xvfb were disabled for VSW4 tests in LSB 3.2.
  3. Size of downloaded files is verified now to avoid partial downloading.
  4. Refresh FTP listing before downloading snapshot packages.
  5. Fixed statistics calculation in report generator.
  6. Minor fixes and improvements.
[edit]

Changes Since Version 1.5.2

  1. Better detection of desktop environment.
  2. Do not check for fonts if lsb-xvfb is being used.
  3. The dependency lsb-test-perl on lsb-cc was removed.
  4. Downloading problem_db file (waivers) from server.
  5. Fixed some problems with output console in WebUI.
[edit]

Changes Since Version 1.5.1

  1. Workaround for locale issues in Core tests.
  2. Fixed tarball version for Tcl tests in LSB 3.2.
  3. Fixed some problems with console output log handling.
  4. Most WebUI elements are now measured in em's instead of pixels (fix for scaling in browsers).
  5. Minor fixes in WebUI.
[edit]

Changes Since Version 1.5

  1. Added xpdf Manual Application Battery test for LSB 3.0.
  2. Name of the Desktop-T2C test was fixed.
  3. Start/stop time for Desktop and Desktop-T2C was fixed.
  4. Error messages for download and installation failures were improved.
  5. Failures in 'unsorted' tests shouldn't affect the certification pass/fail status now (as well as Trial Use tests).
  6. Got rid of the non-standard URI::Escape Perl module.
  7. Fixed incompatibility with some browsers.
  8. Minor fixes and improvements.
[edit]

Changes Since Version 1.4

  1. LSB 3.2 is available:
    • New runtime tests are supported (Desktop-T2C, Perl, Printing, Python).
    • Qt4 tests are mandatory.
    • XTS5 test is used instead of VSW4.
    • Support for Trial Use Modules.
  2. Certification System at http://linux-foundation.org/lsb-cert/ is now used.
  3. AJAX is used also for:
    • refreshing the list of test modules;
    • displaying console output;
    • starting and stopping tests from the Execution page;
    • stopping tests and server from the Administration page.
  4. Minor fixes and improvements.
[edit]

Version 1.4 (2007-11-02)

[edit]

Changes Since Version 1.3

  1. Integration with the Certification System:
    • The Test Report page contains link to apply for certification. The distribution name, version, architecture will be filled in automatically when creating a new product in the Certification System.
    • Uploading test results from the DTK Manager can be performed now by means of the DTK Manager itself.
  2. AJAX is now used for work with profiles.
  3. VSW4 is removed form the "Core & C++" profile of LSB 3.1.
  4. Fixed: DTK scripts could crash when getting the list of test modules.
  5. Minor fixes and improvements.
[edit]

Version 1.3 (2007-10-01)

[edit]

Changes Since Version 1.2

  1. Support for desktop-t2c tests: libfontconfig (~150 interfaces), libatk (~220 interfaces), libglib (~830 interfaces), libgmodule (8 interfaces).
  2. Functional verification test for lynx fixed (app battery).
  3. Minor fixes and improvements.


[edit]

Version 1.2 (2007-07-18)

[edit]

Changes Since Version 1.1

  1. Support for snapshot tests (from 'snapshots' directory) - command line use only - use the --lsb=snapshot option.
  2. New command line options:
    • --ftp allows specifying an FTP mirror server which will be used as a source of test packages instead of ftp.linux-foundation.org.
    • --force-reinstall forces DTK Manager to reinstall all packages.
    • --force-dl forces DTK Manager to re-download all files. This option is useful for snapshot testing.
    • --comment appends a description to the test result.
    • --upload instructs DTK Manager to upload results via SSH to a remote directory. See LSB DTK Manager Nightly Run HOWTO for details.
  3. Support for the new version of lsb-test-core tests (in snapshots).
  4. Simple web-interface for browsing remotely uploaded results (possibly from different computers) at a central server through HTTP (auto_dtk_results.php).
  5. Minor fixes and improvements.


[edit]

Version 1.1 (2007-05-15)

[edit]

Changes Since Version 1.0.3

  1. Manual Application Battery tests are integrated into DTK Manager:
    • Manual Tests category added to the Custom Tests page.
    • New Manual Application Battery Tests page for executing the manual tests.
    • The results of the manual tests are saved in journals and packed together with journals of the automatic tests.
    • New status icons added (translucent) for the partially finished tests (when the manual tests were not performed yet).
  2. New Summary Report page: shows summary status for the automatic and manual tests as well as some additional information, and allows to continue with manual tests (if necessary).
  3. The Results page was improved and allows quick opening of the Report pages (both Summary and Detailed) and running the manual tests if they are not finished yet.
  4. LSB 3.0 tests do not allow selecting Core/C++/Desktop options anymore.
  5. Test suite names on the Custom Tests page are now shown as e.g. Command Check instead of just cmdchk.
  6. Session profiles management improved.
  7. Renamed packages (such as qm, libstdc++, core) are now supported (Bugzilla Bug 1590).
  8. Problem database has been renewed; it's format has been improved.
  9. Interruption of downloading won't lead now to the installation of the broken package.
  10. Package deinstallation was improved.
  11. Minor bugfixes (Bugzilla Bugs 1580, 1581).


[edit]

Version 1.0.3 (2007-03-26)

[edit]

Changes Since Version 1.0

  1. Some bash-related problems in lsb-tef-stop.sh fixed.
  2. New directories on the LF FTP used (e.g. released-3.1 instead of released-3.1.0).
  3. The fixed list of the certification test suites is used now instead of using all available stable ones.
  4. Total tests expected and Test skips unexpected (for Python) information added to the report.
  5. Minor bugfixes.
[edit]

Changes Since Version 0.6

  1. Certification results page was updated.
  2. New E-mail address for certification and problem reports are now used.
  3. The column titles Localhost, FTP, FTP (beta) on the Custom Tests page are clickable now (radio-buttons are selected in the corresponding column where possible).
  4. Some new waivers added to problem_db.
  5. The View test report link on the Execution page moved above the terminal window.
  6. Application data path was fixed to /var/opt/lsb/test/manager.


[edit]

Version 0.6 (2007-03-06)

[edit]

Changes Since Version 0.5

  1. WebUI improvements:
    • LSB Profiles option was added to the Get Certified page.
    • The cmdchk, libchk and desktop tests are now configurable (allow to turn on and off testing of Qt4 and Desktop modules).
    • Redesigned welcome screen on the LDTK Manager start page.
    • On the Execution page the type of the test run is shown ("certification" or "custom").
    • The Configure page was renamed to Custom Tests.
    • Profiles renamed to User Profiles to resolve ambiguity with LSB Profiles.
    • The session profile is now saved automatically on jumping by any link.
    • The tooltip and the drop-down list are hidden automatically when the user clicks anywhere on the page.
    • The get.pl script can be used now as a part of plain URL: http://localhost:8888/get.pl/var/opt/lsb/dist-testkit/somefile.ext (so that the browsers suggested the correct file name when saving).
  2. Command-line parameters added to turn on and off testing of Qt4 and Desktop modules.
  3. LDTK Manager now uses new paths: /opt/lsb/test/manager/ for executables and /var/opt/lsb/dist-testkit/ for program data.
  4. Font directory detection was improved.
  5. `localhost' was replaced with `127.0.0.1' in some places (so that it worked even when `localhost' is not mapped to any IPv4 address).
  6. Portland's xdg-open is used to start browser.
  7. Versions of the tests and their configuration information were added to report.
  8. Minor bugfixes and improvements.


[edit]

Version 0.5 (2007-02-14)

[edit]

Changes Since Version 0.4

  1. WebUI improvements:
    • New tooltip system on the Configure and Get Certified pages instead of redirecting to the Help page.
    • New interface for the Profile Management section implemented (editable drop-down list).
    • The layout on the Configure and Get Certified pages was redesigned.
    • Error/warning messages on the Configure and Get Certified pages were improved.
    • The cross-browser support was improved.
  2. Numerous fixes in English language (thanks to Jeff Licquia).
  3. The LSB versions 3.2 and 4.0 were removed from the list of the versions available.
  4. The --not-run parameter was not working.
  5. Significantly increased speed of getting the list of the local test suites.
  6. Test versions in the list should have more sense now.
  7. Some waivers were added to problem_db.
  8. Numerous minor bugfixes and improvements.


[edit]

Version 0.4 (2007-01-30)

[edit]

Changes Since Version 0.3

  1. WebUI improvements:
    • The interface was updated according to the new interface of http://linux-foundation.org.
    • The list of the test suites is now organized in categories and subcategories (both on the Configure page and on the report pages).
    • New design for error messages on the Configure and Get Certified pages (emulating dialog box).
    • The controls on the Configure page were regrouped slightly.
    • On the Configure and Get Certified pages the inline help marks were added which open the corresponding topic of the Help page.
    • The Help page was significantly improved:
      • added descriptions for the test suite categories;
      • added "Troubleshooting" section with the description of the commonly encountered problems.
    • The report page now contains short descriptions of the test suites.
    • Local files are now presented as links opening the corresponding file in the browser (via the Perl-script; only files from several specific directories are accessible!).
    • All external links are now opened in new pages.
  2. Some changes on the certification results page:
    • Integration with the new certification system implemented (mailing lists).
    • The list of the Application Battery tests is now dynamic (depending on the LSB version chosen).
    • Every Application Battery test is a link to the corresponding page with the step-by-step instructions on how to run the test.
  3. Pre-conditions for tests running are now checked only after pressing the Run tests button, not before. Besides, the error messages themselves were improved.
  4. All data files were moved into the subdirectories of the /var/opt/lsb/distr-testkit directory.
  5. Added support for the new (beta) lsb-runtime and desktop test suites.
  6. New records added into problem_db.
  7. Numerous minor bugfixes and improvements.


[edit]

Version 0.3 (2007-01-18)

[edit]

Changes Since Version 0.2

  1. The distribution comes now with the unified installator that works equally for RPM- and DEB-based distros.
  2. The distribution is provided for 4 architectures: i586, ia64, x86_64, ppc.
  3. Some improvements in WebUI:
    • New certification results page templates that contain information on how to get certified.
    • The Bug report button in the test report page is working now (it redirects to the Submit Problem Report page).
    • All errors are now shown with the same interface (no intermediate error pages).
    • Some errors are processed that earlier were just ignored.
  4. Some improvements in Web-server implementation:
    • The template of the error pages is now more informative.
    • HTTP HEAD requests from browsers are now processed, as required by the standard (rfc2616).
    • For unsupported HTTP requests the 501 Not Implemented responce is returned, as required by the standard.
  5. App-battery tests added: apache, groff, lynx, rsync, xpdf, raptor, samba.
  6. Stability: the tests are very unlikely to hang now.
  7. Many improvements for various Linux distros support.
  8. Minor bugfixes and improvements.


[edit]

Version 0.2 (2006-12-01)

[edit]

Changes Since Version 0.1

  1. New functionality for simplified process of getting the LSB certification:
    • Added Get Certified page to the web-interface.
    • After executing the tests an additional results page with general information is generated and automatically opened in the browser.
    • The certification results page provides possibility to send the journals to FSG (this function is in demo-stage yet).
    • The Results page now contains pictures indicating the status of every tests run.
  2. lsb-autotest is used now as an independent component.
  3. LSB Version can now be chosen (by default the LSB version of the current distro is suggested).
  4. Configuration data are now automatically saved in session profiles on any action (cookies should be enabled in the browser).
  5. Added progress bar when updating the information from FTP.
  6. Information from FTP servers is now cached, so the user does not need to call Refresh List every time.
  7. If any of the test suites fails, the LSB TEF proceeds with the next one.
  8. Report HTML files now contain the Reporg but in test buttons in the folded information blocks of failures. These buttons allow to report about possible bugs in test suites (this function is in demo-stage yet).
  9. Various minor fixes and improvements.


[edit]

Version 0.1 (2006-11-10)

The LSB Test Execution Framework (LSB TEF) is developed for simplifying the work with the lsb-autotest scripts written by Jiri Dluhos (SuSE Linux Products GmbH). The current prototype of LSB TEF consists of two main parts: web-based user interface and hacked lsb-autotest scripts themselves. In future versions of LSB TEF, lsb-autotest will be used as an independent component.

[edit]

Web-Based User Interface (WebUI)

WebUI consists of the web-server written in Perl and additional scripts that are used for maintaning the TEF via web-browser. These scripts allow to:

  1. View the list of all the available test suites:
    • local (both downloaded and installed);
    • located on the FSG's FTP server;
    • beta-versions located on the FSG's FTP server.
  2. Customize the tests to execute:
    • machine architecture;
    • possibility of keeping packages after the tests are finished;
    • additional comments for the report;
    • specifying which particular tests to download and execute;
    • etc.
  3. Save/load all the settings for further use by means of profiles.
  4. Watch the tests execution progress in a terminal emulation embedded in browser.
  5. Abort the tests execution.
  6. Visualize results of different test runs in a master table (usefull for tracking the progress as fails are fixed/added).
  7. For each line in the results master table, there is a link to corresponding detailed reports about the tests results.
[edit]

Lsb-Autotest Hacks

These scripts perform the actual tests execution, and are called automatically when the user runs the tests via WebUI. They are based on the lsb-autotest scripts by Jiri Dluhos and contain the following changes made by ISP RAS:

  1. Support for other distros:
    • Another implementation of `su_to_root backdoor' (in order to avoid entering root password many times).
       auth sufficient /lib/security/$ISA/pam_wheel.so trust group=vsxg0 use_uid 
    • `alien' and `dpkg' are used now for the LSB TEF to work on distros that have package manager other than RPM.
  2. LSB compatibility:
    • `sendmail' utility, which is specified in LSB, is used now instead of `mail' utility.
    • `Gzip' is used instead of `Bzip' (for packing results before sending them by E-mail).
    • `passwd' utility may not have `--stdin' option. Passwords assignment (for transient test users) is implemented via `expect'.
    • Section of LSB 16.4. `Path For System Administration Utilities' says applications should add some directories to their PATH to use system administration utilities (like `useradd', `userdel') as commands.
    • For the `rsync' test the `diff' command can be used (and is used now) without the option `-q' (not specified in LSB).
    • For the `python' test a command `tput reset' was used, but it isn't specified in LSB. This command isn't necessary and can be omitted.
    • The command `stty sane' used in the same test could fail when stdout is redirected to a file.
  3. Minor changes:
    • Verbose levels were added: 0 - no verbose, 1 - default, 2 - high (for debug).
    • Host architecture recognition procedure was improved.
    • `Wget' is now used with the `--timeout' option.
  4. Other changes:
    • All the code specific for handling each particular test suite is placed in its own file, and object-oriented model is used to handle this. Such an approach provides the following benefits:
      • It's easy to add a new test suite.
      • You have all the code responsible for certain test suite in one place.
      • It allows to carry out preparation, check, launch, cleanup and journal making for each test suite separately.
    • Better procedure of downloading and installation:
      • It gets the list of files from the FTP server, gets the list of the downloaded files and the list of the installed packages, and then it decides which files to download and install.
      • Different versions of LSB are supported.
    • Report is generated after parsing of all test journals. This allows to put a summary into the header (beginning) of the report.
    • Fixes in HTML generated code to become XHTML-valid now.
    • An option to replace the original test scenarios with shorter variants to speed up the test execution. It's helpful for debug and evaluation purposes of the TEF itself.
Retrieved from "http://ispras.linux-foundation.org/index.php/LSB_DTK_Manager_Release_Notes"
Personal tools