Assist: Improve scanning of MLC array

[Andreas Schnellbacher]

Assist behavior is about 100 times too slow. Tests show that parts were executed multiple times.

Improvements are:

1. In locateMLC, the search can be improved: Instead of simply starting at item 1 and advancing the counter by 1, it should start at the middle and then either decrease or increase it be the half value. (This is the largest improvement and a safe method has to be created.)

2. In defproc locateMLC, fix multiple calls of the loop starting with 'do j = 1 to MLCListCount'. It apparently depends on the amount of MLCs in between a bracket start and end area. (This would be a large improvement and the culprit is currently not found.)

3. locateMLC itself is called two times, for the opening and for the closing token (e.g. bracket). The second call is caused by a bracket-matching command. The corresponding token is highlighted then. (A minor improvement would be to start the search for the matching token at the pos. of the starting token instead of at the middle.)

4. Improve other code, like using faster search options (g instead of x) if possible and optimize nested IF conditions in locateMLC. (This saves only about 20 % of the time.)

[Andreas Schnellbacher]

Large improvements in sight:

0. Avoid rebuilding the multi-line comment cache at every passist call.

• This was done before creating this ticket and is about 10 times faster.

1. A prototype is done and improves the scan time for about 50 times. Now it takes 4 times longer to build the cache than to scan the entire file for TagScan, but see 5.

• Almost done.

2. Testing with bracket matching: Multiple calls have to be made, one call for each opening or closing bracket of the same type.

• Nothing to do.

3. No improvement.

• Nothing to do.

4. Minor changes.

• Almost done.

5. New: For unnested multi-line comments (all languages except REXX and E), building the cache can be avoided completely. It should be much faster to simply search backward for the next opening or closing comment token. If an opening token comes next, then the given position must be within a comment.

• To do.

[Andreas Schnellbacher]

Useful function are:

• InString (exists already)
• InSingleLineComment
• InMultiLineComment (automatically uses MLC cache method for a nested mode)
• ValidToken
• NextValidToken (with option for the direction), like next_nonblank_noncomment_nonliteral

[Andreas Schnellbacher]

• Not faster, because e.g. for C mode TagScan, multiple matching brackets have to be found. Additionally, for each of that call of the passist procedure, many checks have to be made if a bracket area contains multi-line comment chars. Scanning an index is much faster. Just the build time could be reduced.

That is were another ticket could start. Some early ideas:

• Building the index for the entire file is only required for modes that allow nested multi-line comments (only REXX and E).

• On editing a file, the index should be rebuild only for a part of it.

• For bracket matching for modes except REXX and E, build the index only for a part of the file. That should avoid the "hangs" on scrolling while the cursor does bracket matching. (Matching happens also on typing the opening bracket and doesn't require a rebuild.)

• For rebuilding of just a part of the index, the changed line(s) have to be determined. Finding the nearest start point in the index could be optimized (like in locateMLC). Adding or deleting comment entries to the lines with index entries should move all remaining items.

• On leaving a modified line (use defmodify and isadirtyline() to store the current line number), the index for that line should be rebuild.

[Andreas Schnellbacher]

The main improvement was implemented by r3029 and r3032: The scanning of the multi-line comment array was extremely improved:

Instead of iterating through the items from 1 to the found item, the new method takes the available range of items and chooses to check the item in the middle of that area. That check gives the info if the searched item comes before or after the position to be checked. With each processed check, the remaining amount of array items is reduced by its half.

The next improvement could be made by reducing the build time by scanning just a part of a file. Also the amount of events that cause a rebuild of the array could be reduced. But therefore another ticket has to be created.