Opened 13 years ago

Closed 11 years ago

#25 closed enhancement (fixed)

Move long scans to secondary threads

Reported by: Gregg Young Owned by: Gregg Young
Priority: major Milestone: Release_3.14
Component: fm/2 base Version:
Keywords: performance Cc:

Description

The freeze occurred when FM/2 tried to scan a network drive that is setup using the netdrive samba plugin. Normally if it has problems it returns an error 65. However, the system the drive was on had locked up and when fm/2 scanned it the WPS froze and I had to kill FM/2. The only way I could reopen FM/2 was to exclude the 2 drives. After I rebooted the other machine all works fine. The WPS drives object opened without problem it showed the 2 netdrive drives as empty but no freeze occurred. The same error message as below occurred again but I think the message was generated when I tried to kill FM/2 not when the freeze occurred.

Most of the scans that can take a long time are done on separate threads, so this kind of failure will not lock up fm/2. However, there are a few that are still done on the PM thread and this can cause apparent Desktop lockups. If this is the case, pressing Ctrl-Esc several times should cause PM to take control away from fm/2.

If we can identify what fm/2 was trying to do, we should be able to move this logic to a thread.

The trap should not occur, even with slow access. Unfortunately, the trap is not within fm/2 code, so it's hard to analyze without a process dump. I will be adding exceptq support to fm/2, once I get some in-progress exceptq rework completed. This will provide enhanced trap logging, without the need to resort to trap dumps.

The trap appears to somewhere in DOSREADASYNC or DOSWRITEASYNC, but that does not tell us much about where and how fm/2 managed to pass bad arguments to the API.

Change History (15)

comment:1 Changed 13 years ago by Steven Levine

Summary: Move long scans to secondary treadsMove long scans to secondary threads

comment:2 Changed 13 years ago by Gregg Young

The netdrives problem is fixed if multiple drives are mounted as subdirectories (ie if laptopC and laptopD are mounted as U:\laptopC and U:\laptopD) the initial scan problem goes away as the drive and root subdirectories are returned by netdrives and dbl clicking on the directories quickly returns an error it the remote computer is shut down. What is different about this case?

comment:3 Changed 13 years ago by Steven Levine

Owner: StevenHL deleted

comment:4 Changed 13 years ago by Steven Levine

Keywords: performance added
Milestone: Release_3.7

comment:5 Changed 13 years ago by Gregg Young

Version: 3.07

comment:6 Changed 13 years ago by Steven Levine

Milestone: Release_3.7Release_3.8
Version: 3.073.08

comment:7 Changed 13 years ago by Steven Levine

Type: defectenhancement

comment:8 Changed 12 years ago by Gregg Young

Milestone: Release_3.8
Version: 3.08

comment:9 Changed 12 years ago by Gregg Young

Milestone: Release_3.9

comment:10 Changed 12 years ago by Gregg Young

Milestone: Release_3.9Release_3.10

comment:11 Changed 12 years ago by Gregg Young

Milestone: Release_3.10Release_3.11

Well one relatively simple solution that does not require massive rework would to put a loop around the drive tree fill loop. The result would be

for (pass = 0; pass < maxpass; pass++) {

for (drvndx = 0; drvndx < maxdrv; drvndx++) {

if (pass == 1 and drive flags day do in pass 1)

fill

else if

...

}

}

Pass 1 could be local drives that are not marked slow.

Pass 2 could be locall drives that are marked slow.

Pass 3 could be remove drives

etc.

This might not even require any tweaks to the container insert logic.

comment:12 Changed 12 years ago by Steven Levine

Milestone: Release_3.12Release_3.13

comment:13 Changed 12 years ago by Gregg Young

Milestone: Release_3.13Release_3.14

comment:14 Changed 11 years ago by Gregg Young

Owner: set to Gregg Young
Status: newassigned

comment:15 Changed 11 years ago by Gregg Young

Resolution: fixed
Status: assignedclosed

While I didn't add additional threads, it appears that by looping through multiple times with fastest drives first that scan times are significantly improved (here at least). I am marking this fixed.

Note: See TracTickets for help on using tickets.