You are here: SE » ThesesHome » ThesisDPPXXV

Einführung eines Testprozesses (B,S,D,M)

worked on by: Lin Chen

Outline

Saros enthält einige Unit-Tests und seit neuestem auch ein Test-Framework, dass den Ablauf von verteilten Testfällen (die für realistische Tests unabdingbar sind) orchestrieren kann.

Im Rahmen der Arbeit soll die Ist-Situation bzgl. des Testens des Saros-Projektes evaluiert und anschließend Saros mit einem angemessenen Testprozess ausgestattet werden. Dabei soll untersucht werden, welche Arten von Tests sinnvoll eingesetzt werden können. Ein weiterer Schwerpunkt der Arbeit liegt darin, dass GUI basierte Test-Framework STF (Saros Test Framework) weiter zu entwickeln, das im Jahre 2010 vom Absolventen Sandor Szücs durch seine Diplomarbeit eingeführt wurde.

Schedule

Past days CW Goals target accomplished wrench
DONE 1 CWXX Goals accomplished

Weekly Reports

Week 1 (CW 29)

Activities

  • document and evaluate the current situation regarding the testing in Sarosprojekt
    • what tools are being used in what situation?
      • Junit-Test
        • Vor each releasechange the AllTestSuite.java im package Saros/test/src/de.fu_berlin.inf.dpp is run by Releasemanager.
        • As far as I know the junittest were no longer maintained, i mean, they are not adjusted when changes happen in the code.
        • which test cases cover the Junittest? (Add details later, ask phillip)
          • Test scenarios for the simultaneous editing with multidriver
          • Test Jingle/UDP connection Smack
      • EsayMock
        • It is used to isolate individual test objects from their test environment.
      • STF(Sandor Test Framework)
        • with STF different Sarosinstances in a test network can be remotely controlled, so that failures in the network can be measurable, testable and repeatable.
        • It is currently not used. STF is not mature to use
      • SWTBot 2.0.0.486-dev-e35
        • SWTBot framework is a eclipse-project and offers the possibility to test SWT and eclipse. It is used by STF.
      • Testlink 1.8.4
        • a software for organizing tests.It is running under machine ibadan.imp.fu-berlin.de.
        • vor each releasechange TM & ATM create suitable testcases according to the changelog created by RM in Testlink. after then these testcases are tested by developers.
      • Hudson 1.352
        • Hudson monitors executions of repeated jobs, such as building a software project or jobs run by cron.
        • It is currently not used
    • Where occur the highest quality problems? What they are? Why do they occur? How long they have existed? What has been done in QA measures? Why they were not likely to succeed?
      • It is difficult to say whether Saros in different network environment still works well. Within the handling of different work packages occurred always irregular error in the network layer. e.g. the service discovery in DiscoverManager.
      • The problem first, the repeatability and second is difficulty is the identification of the defect. There may be various reasons why this kind of defect occurs. It could be logical or temporal error in caching by using the Smack API, failure in the Smack API, or failure in answer to questions within its various server software, At worst it is a combination of the above errors.
      • Since Saros includes many aspects that are hard to test automatically we have created a test suite which we execute manually before each release.

  • try to run the STF in the buildserver builded by Florian Thiel( the buildserver had received three VMs in technology to run the STF on the central VSphere.)
    • With which login and PW i can access the machine Saros-build.imp.fu-berlin.de?

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

Week 2 (CW 30)

Activities

  • to run STF Tests locally you should configure the run-configuration of the saros-instance with the following infos:
    • -Xmx512m
    • -ea
    • -XX:MaxPermSize=128m
    • -Djava.rmi.server.codebase=file:$WORKSPACE/Saros/bin
    • -Djava.security.manager
    • -Djava.security.policy=file:/Users/Lin/Documents/workspace_saros/Saros/bin/all.policy
    • -Dde.fu_berlin.inf.dpp.testmode=12345
    • -Djava.rmi.server.hostname=localhost
  • with hudson-job "STFTestSarosRelase" run the Junit-test "RmiTest", which work well.
  • learn Ant4Eclipse http://www.ant4eclipse.org/node/81
  • i can edit java file direct under saros-build with sudo, then hudson compile the source code
  • problem by delegate.button("Next >").click(). It worked well locally, but in saros-eclipse* it fail.

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

Week 3 (CW 31)

Activities

  • Job "SarosTrunk" doesn't build,because the sarosproject use some plugins with higher version than the ones from eclipse in saros-build.
    • How to update eclipse?
      • Then i would like to update the plugins with eclipse, but there is no Xwindow system in machine "saros-build", so i installed a ubuntu-desktop for testreason with the command "sudo apt-get install ubuntu-desktop". Then i found some strange: the subclipse plugins in directory /home/build/eclipse/plugin have version 1.6.5, but what showed in the installations details in eclipse have version 1.6.12.
      • the target "build" in build.xml doesn't work because of the installation of ubuntu-desktop. so i have to delete it with comand "sudo aptitude remove ubuntu-desktop"
  • i can edit java file direct under saros-build with sudo, then hudson compile the source code
  • problem by delegate.button("Next >").click(). It worked well locally, but in saros-eclipse* it fail.

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

Week 4 (CW 31)

Activities

  • add shell script to delete de.fu_berlin.inf.dpp_*.DEVEL.jar
  • add shell script to delete the jar files in directory /home/rmiswtbot/sars_plugins
  • wenn you change files in sarosswtbot, you should restart eclipse in saros-eclipse* with run.sh, otherwise you can't see the change.
  • in run.sh you can define which de.fu_berlin.inf.dpp_*.DEVEL.jar to use.

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

Week 5 (CW 32)

Activities

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

  • i can't login Hudson with my uni_account. I try to update the saros-build with command "sudo aptitude update && sudo aptitude safe-upgrade", restart the eclipse-build with command "shutdown -r now", then it work again.
  • I've tried to run junit-test "shareProjectTest" with Hudson….. there will be started two saros instances named Bob and Alice which are running under 1) VM Saros-Eclipse2 and 2) VM Saros-Eclipse1 both on debian system. The junit-test "shareProjectTest" try to create a session using Bob as the host and Alice as the guest. Invitation didn't succeed and hangs forever. It be possible that saros doesn't work on debian system. To run the STF i need to install two other VMs with e.g. Ubuntu OS or someone of us should fix the Exception.

Week 6 (CW 33)

Activities

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

Week 7 (CW 34)

Activities

  • Lern SWTBot.
  • Repair STF Tests and write some new Test with STF and SWTBot.
  • I've test all of junit-test in package "de.fu_berlin.inf.dpp.stf.test" locally. Locally all works very well.
  • Try to run the tests with Hudson under the virtual test environment builded by Florian.
  • Install VMs (Debian, Ubuntu, FreeBSD) locally to test if saris work well on these systems.
    • Mac -- Ubuntu : work wen use IBB
    • window -- Ubuntu: work when use IBB
    • Mac -- Debian : not work
    • Window -- Debian: not work

Literature

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

  • STF junit-tests don't work well on all the different OS.
    • Mac OS X ( 10.6.4) :
      • The main menu bar of eclipse is not built directly into this program, but at the top of the screen.The menu bar will changes when you change the program. i.e. you should get the WidgetNotFoundException if the eclipse's menu bar isn't active.
      • Actions as "connect", "share project " on MAC OS take more time than on Windows OS. It's possible that you get the TimeoutException or WidgetNotFoundException.

  • Sebastian tell me that SWTBot doesn't support native Dialog.

Week 8 (CW 35)

Activities

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

Week 8 (CW 36)

Activities

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

Week 8 (CW 37)

Activities

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

Week 8 (CW 38)

Activities

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

Week 8 (CW 39)

Activities

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

Week 8 (CW 40)

Activities

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems

Week 8 (CW 41)

Activities

Results

  • achieved XYZ

Next Steps:

  • planning …

Problems