In 2001 I started my first and perhaps most successful Open Source project I've ever made: IssueTrackerProduct. After nearly a decade of maintaining it I have now officially abandoned it.

It all started when I needed a way to track feedback on my personal website. That's why it was originally called "SiteTrackerProduct". I needed something where I could collect bug reports and any other pieces of feedback and then process it in some structured fashion. It was therefore very important that it would be possible to run the application open for anonymous access. People should be able to submit bugs and issues without having to create an account. You see, kids, back in that day it was actually very common that sites would force users to register and create accounts even just because the content owner wanted it. These days, it's common knowledge that to get people to open up and share anything for others to benefit you make it absolutely trivial to jump straight in without having to see a registration page that looks like a tax return form.

Now, since I long ago abandoned the Zope2 application server technology stack and I no longer use IssueTrackerProduct for anything real it's no longer feasible to maintain this project. In the last five years or so we were actually using it actively to track all projects at Fry-IT where I used to work. I have to say, even though we did grow out of it, it was actually successful. It handled the load (after some much needed patches towards optimization) and it was easy for people to actually use since unlike many other bug trackers, it focused on the non-technical end user first and foremost. As much as possible was done to make it trivial to type in your bug or issue and it automatically took care of all notifications and access rights.

Being a personal Open Source project, over the years, it became a melting pot for experimenting and perfecting various new ideas. Many of them we take for granted today but back then it was quite novel if I may say so. This includes:

• ability to auto-save unfinished form inputs (added before Gmail had it)
• automatic updates of the content without reload made it possible to see other people participating as you're typing
• ability to reply directly to email notifications without having to open a web browser
• an advanced via-the-web programmable interface for adding and modifying custom fields (e.g. "Customer reference code")
• full-text search combined with ability to search on specific fields by key
• file attachments that are images automatically appear as little thumbnails
• file attachments that have text become searchable (e.g Word documents)
• advanced filtering where you can easily decide to search inclusive or exclusive on certain fields
• persistent filtering automatically saved and share'able between different users
• programmable search filters that are coded in Python which made it possible to create very specific reports
• ability to export and import bugs from and to Excel for offline processing

Writing all of this, I can not resist to get a bit nostalgic. I did sink A LOT of time into this project. Today when I look back at the code and almost feel sick seeing all the mistakes that I made. Much of the ugliness of the code can be attributed partially to the fact that I often used and abused the code to add new features. Also, because we often needed some features (since it was used to manage all of our projects) "yesterday" and then it was hard to justify doing things "properly". For example, the main .py file is over 14,000 lines of code!

I did called it "perhaps most successful Open Source project I've ever made" in the first sentence. The reason for that is that over the years many many people have downloaded it and installed and let it be used by thousands of users. That's something to be proud of.

Anyway! It's time to move on. So long and thank you for all the fish!

The code is still available at github.com/peterbe/IssueTrackerProduct

Today I finally got around to launching a new version of the IssueTrackerProduct. After a years worth of bug fixes and new features added the most exciting one to me is the Monthly summary feature.

Gosh! I really regret I've never properly collected email addresses for people who actually use the IssueTrackerProduct. I was using a SourceForge mailing list for a while but that got to spam ridden I gave up on it a long time ago.

Check out how long the list of change is on version 0.11

Feedback mucho appreciatado.

IssueTrackerMassContainer is a simple Zope product that is used to put a bunch of IssueTrackerProduct instances into. It doesn't add much apart from a nice looking dashboard that lists all recent issues and then with an AJAX poll it keeps updating automatically.

But what it was doing was it recursively put together all issues across all issue trackers, sorting them and then returning only the first 20. Fine, but once the numbers start to add up it can become a vast sort operation to deal with.

In my local development copy of 814 issues, by the use of pympler and time() I was able to go from 7 Mb taking 2 seconds down to using only 8 Kb and taking 0.05 seconds.

Here's the initial naive version of the code:

    def getRecentIssues(self, since=None, recursive=True, batch_size=20, batch_start=0):
       """ return a list of all the most recent issues """
       issues = self._getAllIssues(self.getRoot())
       if since is not None:
           ... # checking variable since
           issues = [x for x in issues
                     if float(x.getModifyDate()) > since]

       issues.sort(lambda x,y: cmp(y.getModifyDate(), x.getModifyDate()))
       return issues[int(batch_start):int(batch_size)]

So, instead of making a fat list of issue objects, just turn it into a list of the things we really need. The second version:

    def getRecentIssues(self, since=None, recursive=True, batch_size=20, batch_start=0):
       """ return a list of all the most recent issues """
       root = self.getRoot()
       issues = self._getAllIssues(root)
       issues = [(x.getModifyDate(), '/'.join(x.getPhysicalPath())) for x in issues]
       if since is not None:
           ... # checking variable since
           issues = [(t,i) for (t,i) in issues
                     if float(t) > since]

       issues.sort()
       issues.reverse()
       issue_paths = [x[1] for x in issues[int(batch_start):int(batch_size)]]
       return [root.unrestrictedTraverse(p) for p in issue_paths]

The issue method getModifyDate() returns a Zope DateTime instance which is ridiculously nifty datetime implementation but it sucks in memory use and performance. See this blog about how it sucks compared to mxDateTime and standard lib datetime. So, this time, turn it into a float and then sort. Final version:

    def getRecentIssues(self, since=None, recursive=True, batch_size=20, batch_start=0):
       """ return a list of all the most recent issues """
       root = self.getRoot()
       issues = self._getAllIssues(root)
       issues = [(float(x.getModifyDate()), '/'.join(x.getPhysicalPath())) 
                 for x in issues]
       if since is not None:
           ... # checking variable since
           issues = [(t,i) for (t,i) in issues
                     if t > since]

       issues.sort()
       issues.reverse()
       issue_paths = [x[1] for x in issues[int(batch_start):int(batch_size)]]
       return [root.unrestrictedTraverse(p) for p in issue_paths]

And the results for my local copy of 818 issues?:

Version 1:
   7842736 bytes (7.5 Mb)
   2.1547999382 seconds

Version 2:
   834880 bytes (0.79 Mb)
   0.210245847702 seconds

Version 3:
   87448 bytes (85 Kb)
   0.0538010597229 seconds

Granted, Zope will release this memory by the garbage collector but why even let it get that big if you have concurrent hits or if anything gets stuck for longer than necessary. Python 2.4 can free memory used but not return it to the operating system to reuse unless the process dies (this was fixed in Python 2.5).

That's a memory usage improvement of about 90 fold and a speed improvement of about 40 fold.

Fry-IT has huge issuetracker instances and at the time of writing keeps 6668 "active" issues across all projects.

The Custom Fields feature started as a consultancy job in which we agreed the work can be open sourced as part of IssueTrackerProduct so I never got around to write an sensible high level documentation for it. Now I have! From the news piece about it:

"Custom Fields was a feature that was released almost a year ago but didn't have much documentation. Especially easy documentation that describes what it is and how it can be used. That has changed now.

In Custom Fields it is now described what they are and how they can become useful to you. It's such a powerful tool that very few "competing" issue/bug tracking systems can offer."

The written documentation is here: Custom Fields

Feedback appreciated.

A new (development) version of the IssueTrackerProduct has just been released. It contains some crucial bug fixes (to some people) that I really couldn't delay much more.

Sadly I never really had the time to fully implement the TinyMCE support. It's going to come in the next release. It actually works already but you need to know which buttons to press since there is no simple one button to press for it to work.

I just got back from the Helpdesk & IT Support Show in Olympia (Kensington, London). My main impression is: there are many, big players in this industry.

My pet project, the IssueTrackerProduct is very basic in comparison to some of these companies products. Although it's often used in help desk situations the kind of help desk solutions I've seen today are way different. For many of them, it's all about integrating various systems such as asset management, call logging, configuration management, knowledge management, etc. It seems that the actual help desk apps seems to have to be low priority compared to getting all pieces to fit together.

I walked one lap around the perimeter of the big hall so first I saw the little stands. There they called it "Help desk". As I moved in towards the centre of the hall where the really big players had really big stands, big plasma screens and lightly dressed young ladies a different word was used: "Service management". I talked to a really friendly chap from ICCM Solutions and asked him honestly: what is Service Management? He gave me two answers. One short, one longer. The short version was: the smaller less advanced providers haven't evolved and still call it help desk; the bigger players who have evolved more call it service management. The longer version was that the expression "help desk" is more associated with IT and problems that need immediate solutions. Service management is more about monitoring and controlling: Configs, Incidents, Problems and Change. In that order. It's all business/marketing lingo that don't mean much but at the time he explained it it actually started to make sense. In simplicity, they just want to get away from the word help desk because it's got ingrained annotations to it and they want to do something else.

Very few of the screenshots and demos I saw impressed me in the detail work. I saw endless horrible help desk issue/incident entering screens where some poor user has to select the right fields to fill in amongst a screen packed of different input fields of various sorts. Many systems had issue/incident reports and lists again packed with far too much screen-noise. Some of them had quite a slick and neat design but a lot of them had windows GUI apps that look like a video-rent-shop-epos system from 1995.

Although I never saw it in action, HelpSTAR had (supposedly) a clever solution. You can apparently drag and drop emails from your MS Outlook into their help desk system. That email is then logged, archived and can now be responded to and become a email correspondence ticket. They also had (supposedly) an interesting templating system. That makes it easy to reply to frequently asked questions that people send in to a help desk.

One promising company that I talked to was Logicalware and their product MailManager. Their brochure-work was, unlike all others, really down to earth and easy to read and understand. MailManager is, like the IssueTrackerProduct, written in Zope and is Open Source too. The IssueTrackerProduct has email ticketing capabilities but that's not its core functionality. With MailManager, I think email ticketing is the only thing it does and does it well. Keep up the good work guys!

In conclusion, my IssueTrackerProduct does do much but the few simple things it does it does very well. If you don't have £10,000 to spend on licenses, consultants and headaches and just need a simple system for logging issues, pick my system

The IssueTrackerProduct was on tuesday this week featured on Ajaxian.com which is a really good looking site that focuses on nice usage of AJAX in websites.

They could have chosen a better screenshot I guess but other than that I'm quite happy to be included.

Email replies and AJAX is the highlight of the latest IssueTrackerProduct release.

If you set up your POP3 for email replies you can simply hit reply on the notifications that goes out and continue an issue discussing without leaving the comfort of Outlook/Thunderbird/Gmail/Hotmail.

Another underestimated important feature is the automatically refreshing issue where the content is periodically refreshed (if the content changes) whilst you're on the page so that you know you're always looking at the very latest copy before you take any action.

For the people who've dared to use the CVS, they've probably already seen the new keyboard shortcuts that is inspired by Gmail. It's very addictive to not have to scroll up and down to reach the navigation or to be able to quickly jump to another issue without having to click on List Issue.

And for the Gettings Things Done people you might like how you now can get a list of "You next action issues" on the Home page. This is a clever list that attempts to figure out what you need to do next in that order.

I've finally put together a neat little list of some of the most important features of the IssueTrackerProduct on one page. It's not quite finished yet because I want to expand it with some screenshots embedded as little thumbnails. (hint hint to the community if they want to contribute)

Check it out: www.issuetrackerproduct.com/Documentation/Features

An interesting thing about this page is also the javascript used to enhance it. If you look at the HTML source of the page you'll see that there's no markup for the table of contents, the #top links or the importance voting links. All of those things are added after the page is loaded with some javascript loops. The advantage with this is that the markup (aka. structure) is kept very slim and simple. In actuality, some browsers such as "Lynx"n:/plog/your-webpage-in-lynx, "Googlebot"n:/plog/google-is-blind and IE4 don't need these extra effects. It's nice if they work but the core thing here is the content, the text about the issuetracker features.

The voting links are ajaxy but at the time of writing it don't work in Internet Explorer because I can't attach a decent onclick function to the links when the links are created with document.createElement("a"). Don't matter so much because it still works even if AJAX fails. Please don't click the voting links just to test the AJAX, click to actually vote what's important to you.

A week ago I ran some ad hoc benchmarks on various suspect functions in the IssueTrackerProduct and came to a clear and simple conclusion: searching is the bottleneck and within the search it's the searching for file attachments that take all of the time.

If you're interested and open minded, here's the results of that benchmark This sparked some thoughts. First I wrote a filename splitter which isn't rocket science but I'm proud to say that it's use is brilliant. Before, the find-by-file function in the IssueTrackerProduct used a plain old find() test like this:

filename.find('foo') > -1

This is very fast but not very intelligent. For example it'll with match on foobar.txt and plainfooter.gif. So, what I did instead was to create a KeywordIndex and index all the splitted filenames in that index.

What you have to remember is that KeywordIndexes are case sensitive so when I populate the KeywordIndex I have to lowercase everything. But, now let's get to the performance fix.

The problem was that before, I used a ZopeFind() to find files by filename and ZopeFind() is slooooowww compared to a catalog search. Before I switched on the new code that searches the filename splitted keyword index I did a coule of searches on an issuetracker with about 200 issues and a couple of searches on an issuetracker with more than 10,000 issues. The results can be seen here: itp-benchmark-medium-before.log and itp-benchmark-huge-before.log Quite terrible isn't it. Then I switched on the new code that is much more intelligent and hopefully faster. The results can be seen here: itp-benchmark-medium-after.log and itp-benchmark-huge-after.log

Averages:

itp-search-Real-before.log
0.692274004221
itp-search-new-tracker-before.log
7.29415249824
itp-search-Real-after.log
0.0011158519321
itp-search-new-tracker-after.log
0.00395481482796

As you can see, that's an enourmous increase in speed. For the medium sized issuetracker, that's a 600% increase and for the huge issuetracker that's a 2000% speed increase. Oops! This shouldn't be called optimization maybe. Perhaps bug fixing is a better word :)