Bug writing guidelines

Revision as of 12:53, 27 June 2009 by Mheiland (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Bug reporting guidelines

Why report a bug?

Please help us to improve our source code quality. Bug reports in the forums are a good start, but we can make a lot more use of entries in the Bugzilla Bug Tracking System, because we have a solid and tested workflow for Bugzilla which assures that bugs are confirmed, assigned, fixed and verified by our QA. We do not have anything like that for the forum, as you might guess.

Open-Xchange depends on issues reported by the community. Open-Xchange as a company is dedicated to find and fix all bugs using advanced test automation, user testing, error prevention concepts and much more - but we certainly don't catch every single bug. You, the community users have different points of views, other use-cases for a component and various cultural and linguistic background. This is a invaluable advantage for the whole project when it comes to issue tracking.

How to report a bug?

If all those switches and forms at Bugzilla irritate you, just ignore them. As long as a report gets into Bugzilla instead of the forums, the most important step is done. Else the bug will probably be overlooked.

It helps a lot if you can bring as much information as possible about the problem you encounter. If you provide all the information, it is much easier for us to act! Otherwise bugs tend to become some kind quiz and most of the time is spent on getting all the information we need instead of getting to work and fix the issue.

We track, prioritize, fix and verify the reported bugs in a dedicated way. So if you find multiple issues, please open different bugs, each with a precise description.

Bugs are also valid if they cannot be reproduced, but usually it is much easier to find the cause, write tests and resolve the issue if the bug report contains steps to reproduce. Please make sure that the reported steps can be reproduced by another person. Double-check that the issue is really part of the Open-Xchange Server, not a bug at a mail system, build environment, missing configuration or missing credentials. If you don't know how to reproduce this exactly, please note it in the bug. Simply posting a stacktrace without steps to reproduce is better than reporting nothing but this bug report will take way longer to reproduce than a report that holds detail information how the issue can be reproduced.

Severities and Priorities

Bugs can be classified for different severities. They contain information how critical this bug is. All bugs are bad, but to setting a priority and severity helps to classify the bug and set a fixing order. Before setting a exorbitant high severity because the bug seems to be very evil to you, please think twice how this influences the whole product. For example: A text inputbox is misaligned and look not nice. This may be a major bug to you because it annoys you, but for the product at all it is trivial or minor because no functionality gets lost. Please note that bug severities are reviewed before fixing them - reporting everything as a blocker will not automatically lead to a faster fixing if the severity is overcasted. A severity is usually derived from the impact and the frequency of a specific issue.

Severity Description Example
Blocker Blocks development, testing, working, security issues. Login not possible after creating a new user.
Critical Crashes, partly unavailability, severe resource usage problems Saving a contact leads to a session termination.
Major Major loss of function Saving a reminder to a appointment does not work
Normal Loss of function, bad usability experience When sending a E-Mail, the mail will only be sent when clicking the send button twice
Minor Minor loss of function, or other problem where easy workaround is present Sorting tasks by flag does not work in list view, for split view it works
Trivial Cosmetic problem like misspelled words or misaligned text A textbox is slightly misaligned, trivial translation bugs
Enhancement Request for enhancement Implement a YouTube OXtender

Table taken from Bugzilla documentation and modified

To fine-tune the bug a priority can be set. These are predefined from P1 to P5 where P1 is the highest priority and P5 is the lowest. From the personal point of view every bug is very important to be fixed, but think about how development resources can be used in a effective way. Reporting mostly "P1" bugs will block fixing other bugs. As a matter of fact those priorities will also be reviewed before actually fixing the issue.

Priority Description
P1 Most important
P2 More important
P3 important
P4 Less important
P5 Least important

Table taken from Bugzilla documentation and gcc.gnu.org

How to report a bug in a wrong way

These are fictionary negative examples how a bug can be reported "wrong" which means that it will take much longer to grab all information about the issue, to fix and verify it.

"Webmail does not work."<eof>

  • OK, but what did you do to reproduce this? What OX Version, what Browser, what configuration did you use?

"Translation is wrong at <here>, <here>, <here> and <there>, ah and btw. <this> does not work, too."

  • The several development teams, translation team, product management and so on work on their dedicated part of the Open-Xchange Project and use predefined, solid processes to keep up and optimize their workflow. Please help us to avoid overhead and regressions by reassigning bugs. The Quality Assurance of Open-Xchange verifies that bugs are fixed by reproducing them.

Lets take this example and see what will happen "behind the scenes":

  • The issue was reported to the team-leader of the GUI team. He reassigns the bug to a GUI developer who fixes his part of the whole Bug. After that the bug gets reassigned to another GUI developer because there is another issue reported in another GUI module.
  • The GUI team-leader reassigns the bug to the team-leader of the Server team who cares that his part of the bug is fixed.
  • And finaly the bug is reassigned to the translation team that fixes their part of the reported Bug.
  • QA does a retest of a bunch of bugs that are marked by the product management e.g. for a bugfix release
  • If QA notices that one part of the bug (e.g. a single translation issue) has not been fixed, or contains a follow-up bug, the whole bug will be reopened even if all other issues are fixed.
  • The bug gets reassigned to the developer who is in charge for the issue that leads to reopening and the whole process starts again.

You may guess it - this leads to a massive amount of overhead. Additionally, reviewing bugs with 30+ Comments may lead to misunderstandings between the reporter, the developer, the product management and QA team. This is not about laziness but about effective work. In the end, all participants of that issue tracking process will win if bugs are reported precise and dedicated.

Bug reporting checkup

A small check list of information you should collect for your bug:

  • A detailed bug description for us and not just "Mail does not work ..."
  • Report only one issue with one bug report. Open several bugs in necessary.
  • Make sure the issue is reproduceable
  • Provide steps to reproduce the issue on another system
  • Log files! Depending on your problem you should check these files:
    • Open-Xchange log file
    • Open-Xchange admin log file
    • Apache error.log
    • Mod_JK log file
    • IMAP and SMTP log file
  • If you have problems with the GUI, take some screenshots
  • If you already worked on a bugfix, attach it to the bugreport