Weekly Reports (DPP XVI)

You will find a report on the progress of the Bachelor thesis 'Supporting Various Programming Languages and Plug-Ins Within a Tool For Collaborative Software Development' for every week here.

Week 1 (CW 49/50)

Activities:

• installed and set up Eclipse 3.5.1 & Saros (latest release)
• made a list of Eclipse plug-ins that fit for testing
• defined a general testing plan
• did some initial testing
• joined pre-release tests of Saros and got some valuable clues about testing in general and some special issues of Saros in particular
• read through some literature on testing to be able to define more detailed test scenarios
• started testing CDT & PyDev

Results:

• there are lots of plug-ins that have commercial licences and thus do not fit for testing
• picked a few plug-ins to start some general testing with (basically those from the existing compatibility list)
• both CDT and PyDev passed first simple tests (covering a simple connection setup & editing with two participants)

Next Steps:

• run further tests for some plug-ins to analyze what the crucial points are and what the testing needs to focus on
• decide what are the basic features of Saros that need to work to use Saros properly
• define more detailed test scenarios
• evaluate the test results to define further actions

Week 2 (CW 50/51)

Activities:

• installed most of the plug-ins I picked for testing (all except FlexBuilder & Aptana)
• read a lot of documentation on the different plug-ins
• defined first test cases
• started testing JDT, CDT, PDT, PyDev, Dev3
• picked some features of Saros that need to work properly if a plug-in wants to be marked as supported

Results:

• many test cases can be applied to all of the plug-ins, but there are also lots of things that are specific for a certain plug-in, so I decided to make two categories of test cases: general ones that every plug-in has to pass and specific ones for each plug-in.
• so far I did not experience any difficulties during the tests (apart from issues with Eclipse and some funny connection behaviour, see 'problems'-section below)

Problems:

• it is difficult to evaluate which functionality of a plug-in is usually used by developers if you are not familiar with a plug-in or worse: not even the programming language it was designed to support
• testing a plug-in is far more time consuming than I expected because of all the pre-cautions you have to take (run configurations...) and in the first place because three instances of Eclipse really challenge my notebook

Next Steps:

• get more detailed information on the characteristics of each plug-in
• proceed testing with the rest of the plug-ins
• evaluate test cases to see if every important aspect is covered
• evaluate the test results to define further actions

Week 3 and 4 (CW 51/53)

Activities:

• evaluating the test cases showed that I forgot about re-factoring and inconsistency handling, so I adapted them and added new test cases to cover these aspects
• browsed through TestLink to get familiar with it
• proceeded testing, starting with re-testing some plug-ins to apply the new test cases
• unfortunately I got ill plus there was Christmas, so I did not manage to get as many things done as I had planned to

Results:

• more detailed list of test cases (available in SVN)
• JDT, CDT and PDT also passed the new test cases

Next Steps:

• define spcial test cases for some plug-ins
• proceed testing with the rest of the plug-ins
• evaluate the test results to define further actions

Week 5 (CW 53/1)

Activities:

• tested all plug-ins except FlexBuilder (as I have some questions on licensing here) and Aptana Adobe AIR (I started but got weird errors and messages)
• made a detailed list of all test results
• started to prepare a presentation for my talk in "Spezielle Themen der Softwaretechnik"

Results:

• Dev3 seems to be compatible although it was marked as 'not supported' on the website
• I seem to have found a bug of Saros regarding the handling of inconsistencies in combination with changing user roles
• …

Problems:

• after installing Aptana Studio I had serious memory problems that could finally be solved by setting MaxPermSize to 265m
• when testing a plug-in I keep getting an 'ICMP port not reachable' Error (this does not occur when testing with only the JDT active) - however, this does not result in any problems during further testing
• sometimes (I am still trying to reproduce this intentionally), although both participants use the same Saros version, the host gets notified that the user they want to invite does not support Saros

Next Steps:

• finish presentation for talk in "Spezielle Themen der Softwaretechnik" on Thursday, Jan 14th
• transfer test cases to TestLink section 'Language Plug-Ins'
• add test cases for multi-user tests
• evaluate the bug mentioned above
• pick some more plug-ins to test

Week 6 (CW 1/2)

Activities:

• tested more editors of Aptana Studio
• transferred first test cases to TestLink section 'Language Plug-ins'
• continued to prepare the presentation for my talk in "Spezielle Themen der Softwaretechnik"
• decided to contact plug-in communities (user & developer) to find some relevant use cases for testing plug-in features
• chose more plug-ins for testing

Results:

• HTML-, CSS- and ERB-Editors did not pass the tests
• highlighting does not work correctly, CSS editor crashed completely
• the bug I mentioned above really is a bug

Next Steps:

• finish presentation for talk in "Beiträge zum Software Engineering" on Thursday, Jan 14th
• add test cases for multi-user tests
• write to plug-in community mailing-lists to ask for support
• report bugs to Saros' bug-tracker
• test more plug-ins

Week 7 (CW 2/3)

Activities:

• finished presentation and had a talk in "Beiträge zum Software Engineering"
• started writing my thesis
• did some research on Eclipse user data collection
• tested Adobe AIR & FlexBuilder

Results:

• Eclipse's user data collection turned out to be not very useful
• Adobe AIR also uses Aptana's HTML & CSS editors, so there's the same bug as before - the JS editor works fine
• FlexBuilder is not compatible (conflict while trying to install) since it uses out-dated versions of Eclipse bundles

Problems:

• I don't really find any bug that I could fix, apart from the highlighting issue and time's running out

Next Steps:

• test more plug-ins
• add test cases for plug-in features
• write to plug-in community mailing-lists to ask for support
• report bugs to Saros' bug-tracker

Week 8 (CW 3/4)

Activities:

• tested plug-ins on separate machines
• continued writing my thesis
• added test cases for some common plug-in features
• posted to some forums & mailing lists of plug-in communities
• added an overview of the plug-ins I've tested so far to my thesis' page
• tested EPIC and some EasyEclipse distributions

Results:

• different machines didn't make a difference
• EPIC works well, EasyEclipse is all buggy (again, this doesn't seem to be Saros' fault)
• some finished chapters of my thesis

Problems:

• so far none of the communities answered
• everything I find that's not working looks like I can't fix it (as it's not caused by Saros)

Next Steps:

• fix highlighting bug
• find more features of plug-ins to test
• keep writing the thesis

Comments