Monday, May 19, 2008

Slow progress in this blog?

Folks, I started this blog off well over a month ago with an initial posting, and it would appear that I haven't been doing anything since!

Well, on the surface of it that's true. But I can assure you that I have indeed been working on things in the background. Progress is slow because (a) I have other blogs to work on, such as itWire and (b) because I have hundreds of gigabytes of files on my desktop system. it takes days to re-index them and this alone is slowing down my progress.

One of the products that I've been testing is Copernic Desktop Search, or CDS for short. Late in 2007 they released a corporate edition (they still have a free edition, which is what I've been using for years). As a commercial business -- Asia/Pacific Computer Services -- I decided to be honest, and so purchased a license for the CDS corporate edition which is what I've been testing since late 2007.

Unfortunately, the corporate edition has, in my case at least, so far proved to be no more stable or reliable than the free edition. The problem that I originally encountered, starting several years ago, is that the indexing engine proved incapable of scanning every type of file and would "stall" on certain files: that is, it would come across a certain file (such as a Eudora mailbox file, or some PDF document or other) and just sit there chewing up CPU cycles and not moving on to other files. The one advantage that I've found as a paying subscriber to the corporate edition is that (at last) the Copernic technical support team has been prompt in trying to assist me (for which I thank them).

Earlier in 2008 an update to CDS was released which seems to have resolved the indexer "stalling" issue, and I was able (after about four days of indexing activity) to use the search function knowing that all the underlying files had been indexed -- at least, I presume so!

The a month or two ago, quite out of the blue, something happened and the entire CDS index got reset to zero indexed documents (when it should have remained well over 300,000 documents). I don't know what it was that caused this catastrophic failure: I suspect that it was occurred when coming out of Windows XP stand by mode, but am not at all sure.

After having spent at least three or four days at a time rebuilding the index during the last year or two, for similar reasons -- quite a few days in total, I'm sure you will agree -- I got so discouraged/frustrated with CDS that I left gave up on it for a couple of months and put some time into studying both X1 Desktop Search and dtSearch Desktop search (more about them in other postings).

Then last week along came a newsletter alerting the release of CDS version 2.3 (build 23). After some prevarication, I installed this new version late last week and started it off building an index from scratch. Am I a glutton for punishment or am I not?

After running for 10-12 hours per day for over three and a half days, the indexer seems to have nearly reached the final stages. What a relief! Will there be another catastrophic index failure again? I suppose only time will tell. Whatever happens one way or the other, I'll report it in this blog.

During the indexing process you can put up an Indexing Status panel like the following:

(Click to view a larger image)
(Click to view a larger image)

Compared with all the other desktop search products, this is a well-presented way to follow the progress of indexing. Some people will not be in the least interested in doing so, but others certainly will, especially when a problem document is encountered or the indexing progress takes days and days. A nice feature of this panel is that you can drag the bottom right corner in order to see more files names (vertically) at one time and/or longer file names (horizontally). From what I remember, none of the other products allow you to do this.

What's wrong with this progress display? Several things, the first of which is the fact that the title bar simply says "Index Status" (circled in red) when surely it should say "Copernic Desktop Search - Index Status" to differentiate it from all the other windows that you have open. What a silly omission! And in my case it becomes even sillier, since I use the excellent free WinRoll tool to collapse windows like this when I want to free up valuable screen real estate, like so:
(Click to view a larger image)
(Click to view a larger image)

Patently, this collapsed window "Index Status" might be referring to other running tasks, not just this one from Copernic.

Then there's the fact that indexed file names annoyingly roll off the top of the panel, and you have to keep sliding the vertical scroll bar downwards (circled in green) to see the names of the most recently indexed files. For some reason that's beyond me, after you have done this for a while it seems to lock into position and you don't have to keep scrolling down any more. I have recommended to Copernic that they alter this behavior so that the most recently indexed files are kept in view at all times. (The dtSearch indexer does this, so why not CDS too?)

A much more important deficiency in the CDS indexer is the fact that you are totally in the dark about how long the indexing job will take. You wonder if it the job will take minutes, hours, or days; then you wait, and wait, and wait... As I said, in my case it was over 3 days! In comparison, the dtSearch indexer gives a rough estimate of the hours and minutes it will take, and the X1 indexer gives indicates the percentage of files indexed so far. Either or both of these should be added to the CDS window, say in the area at bottom circled in brown.
Finally, when you have the window open it would be extremely useful to have a Pause/Resume toggle button, for example in the area at bottom circled in blue. As it stands, you have to shift focus and right-click on the CDS icon in the system tray then select the "Pause Indexing" option, which is not very convenient.
Anyhow, it seems that the CDS indexer has finished its main run, and it's now showing a status of "Idle" -- until a file is added, deleted, renamed, etc, whereupon the indexer will update the index to take account of this within about 15 seconds, because I have configured the extremely nice On the fly indexing feature. (The others force you to run batch indexing tasks, which is not anything like as convenient as Copernic's on the fly indexing.)