Opened 17 years ago

Closed 17 years ago

#139 closed enhancement (duplicate)

Increase findbuf to speed file loading

Reported by: Gregg Young Owned by: Gregg Young
Priority: major Milestone:
Component: fm/2 base Version: 3.07
Keywords: Cc:

Description

The findbuf had been limited to 128 files in many place. Increasing it appears to be a trade off between loading speed and memory use/temporary poor WPS response. Memory use should be trivial with modern computers and appropriate use of threading should protect responsiveness as well or better than limiting the findbuf. Load times are increased fro 7 min 20 sec to 36 sec by changing findbuf from 128 to 40960 for loading 33,000 files. I see know reason to limit the findbuf (other than expedience of programming) The problem is making end users understand how to optimize it for their usage patterns. I think a default around 10240 would probably be best the time performance per load is fast enough so that any WPS slowing should be trivial while it will load most directories in a single pass maximizing performance. The 36 sec load above was less than one third drive scanning so we need to work on optimizing the container loading code.

Change History (1)

comment:1 by Gregg Young, 17 years ago

Resolution: duplicate
Status: newclosed
Note: See TracTickets for help on using tickets.