Version 0.10 r213/r238 does not work

[erdmann]

Tested:
1) fat32-0.10-r213-os2.wpi
2) fat32-0.10-r238-os2.wpi

None of these versions work: I cannot access FAT32 formatted USB media via the WPS. I cannot run chkdsk against any FAT32 formatted drive. It got completed messed up where Windows showed that the media did not have a chkdsk problem. What did work was to list the directory contents from a commandline. I had to revert back to version 9.13 from Ko Mjung-Hun.

[Valery V. Sedletski]

2Lars: You provided too little info. The same versions work just fine for me. I tried partitions on hard disk, as well as USB flash disks and floppies, and even USB-attached CDRW disk with FAT16 filesystem on it. Everything works just fine. CHKDSK/FORMAT/SYS work too. Access to all disks from WPS works on my machines too.

Maybe, your problem is with something else, not with fat32.ifs? Did you tried standard IBM's version of USB drivers? -- Maybe, the problem is with USBMSD or other USB drivers from you, or some other components? So, 0.9.13 from Ko Myung Hun works for you, but mine not? Very strange...

What OS/2 version do you use? What are symptoms when you open the USB disk in WPS? Does opening disks from FC/2 work? Do non-USB disks work? Any other details on your system?

PS: Can you attach logs from f32mon.exe you got when trying to open the USB drive?

PPS: My CHKDSK and FORMAT use DSK_READTRACK ioctl, whereas 0.9.13, uses sector mode + calling fsctl and ioctl functions from IFS. Mine version of CHKDSK is self-contained and does not call these functions from the IFS. So, they work differently, so something not work in your circumstances (weird ones, which I didn't encountered).

[Valery V. Sedletski]

Hm, tried USBMSD.ADD from usbdrv216.zip, everything is working for me, too. So, I wonder, what could be wrong on your side :(

[erdmann]

Before I do that:

can you tell me which switches for FAT32.IFS and CACHEF32.EXE work ok and which ones I better not use ?

[Valery V. Sedletski]

They all seem to work ok -- they should not cause bad consequences. There is /largefiles switch which adds possibility to use files > 2 gb. I didn't saw any glitches with it. Also, there are /fat and /exfat switches -- they add support for FAT and exFAT, correspondingly. The 1st one works ok too, but I had a single glitch with FAT support, which I cannot reproduce anymore: when copying a big directory to FAT16 disk, I got trash in the directory entries. But I cannot reproduce it anymore. With /exfat switch, you enable exFAT support. exFAT and FAT support theoretically, can not hurt FAT32 disks, so. I always enable all three switches. The only unstable now is exFAT support. FAT works reasonably well. exFAT is readonly atm, because I need to fix ModifyDirectory() for exFAT. Theoretically, readonly exFAT support should work ok, but it tested ok so far with cluster of 128 KB -- It can read and copy out some files, but it can cause files disappear from the root directory, for yet unknown reason (for winxp exFAT dirver. fat32.ifs still sees them ok). Then winxp CHKDSK gets back them again, and some of them become lost chains. But it is unstable now. If you fear you loose some data, you can enable /fat and /exfat only on test media. On critical media, you may disable them. With /largefiles, and with media with 64 KB per cluster, I didn't saw any glitches so far. -- These switches are for fat32.ifs. No other switches for other programs added and they seem to be reasonably stable. And you cannot lose any data by only trying to open a disk and take a log.

[erdmann]

Are you using the regular OS/2 kernel or the OS/4 kernel ?

[Valery V. Sedletski]

no matter, it works with both (I chеcked, to be sure)

[Valery V. Sedletski]

Lars, maybe, your USB disk is dirty? Currently, fat32.ifs denies access to disk if it is not clean -- maybe, it is the case? Did you tried

f32stat d: /clean

? It should show disk contents after that. (Or, run CHKDSK).

[erdmann]

I now removed the /ac: and /largefiles switch from FAT32.IFS. Now it works.
I very much suspect it's the /largefiles switch that blows the system.
I will now try and readd /ac:* and see what happens.

[Valery V. Sedletski]

You still didn't said what exactly did not work for you. I doubt that /ac:* or /largefiles can hurt something.

[Valery V. Sedletski]

There was a problem on Merlin kernel, if you specify /largefiles then FS_INIT could fail. (It checks for KEE library presence and if not found, disables the large file support). But I doubt you have Merlin or don't have KEE :) Or, you need to have older builds of fat32.ifs to have such problem. With newer builds, there should be no such problem.

[Valery V. Sedletski]

Maybe, this is autocheck failed, and disk is still dirty? -- Then you should force disk to be clean with f32stat, or run windows CHKDSK. Does CHKDSK write something on fat32.ifs init? (when /ac:* is specified)

PS: You can return the old behaviour back, so the disk will remain visible, but became readonly, if you add the "/readonly" switch. Will it help?

[erdmann]

Tested it:

/largefiles is the problem. If I have that switch enabled I cannot open one single FAT32 formatted drive under the WPS.
Note: I DO have a kernel with KEE and large file support.
Maybe this problem goes together with EA support. I have added /eas switch because I have used EAs in the past. When I have /eas added (with /largefiles still removed) I can run chkdsk on a FAT32 formatted sick but it complains that it cannot find 2 EA files for files I have long deleted:

[d:\]chkdsk c: /F
The current hard disk drive is: C:
The type of file system for the disk is FAT32.
The FAT32 file system program has been started.
FAT32: Unicode translate table for CP 437 loaded.
FAT32 version 0.10.r238 compiled on Apr 12 2017
The volume label is WHITESTICK.
The Volume Serial Number is DCFE-A836.
The type of file system for the disk is FAT32.

CHKDSK is checking fats :Ok.
CHKDSK is checking files and directories...
A lost Extended attribute was found (for C:\Pack-SPM.rar)
Cannot convert C:\Pack-SPM.rar EA. SF. SYS0001
A lost Extended attribute was found (for C:\R3-LEAGO.rar)
Cannot convert C:\R3-LEAGO.rar EA. SF. SYS0001
CHKDSK is searching for lost data.
CHKDSK has searched 100% of the disk.
The correct free space is set to 852617 allocation units

 16013819904 bytes total disk space.
     2703360 bytes in 36 hidden files.
     1368064 bytes in 155 directories.
     2367488 bytes in extended attributes.
  9022742528 bytes in 4567 user files.
  6984638464 bytes available on disk.

        8192 bytes in each allocation unit.
     1954812 total allocation units.
      852617 available allocation units on disk.

0% of the files and directories are fragmented.

Errors may still exist on this volume.
Recommended action: Run CHKDSK under Windows

If I try to follow the advice and do a chkdsk under Windows, Windows cannot find anything wrong with the FAT32 formatted USB stick and therefore I cannot get rid of this problem (I guess reformatting would do ...).

For /largefiles: I see that FAT32.IFS does NOT export "FS_MPSAFEFLAGS2" (goes into a 16-bit data segment) and "FS32_ATTRIBUTE" data dwords (the latter has to go into a 32-bit data segment).
As far as I understand it is essential to export "FS32_ATTRIBUTE" data word for the kernel to properly recognize the IFS as one with 32-bit entrypoints and "FS_MPSAFEFLAGS2" for its further capabilities (needs kernel spinlock when accessing underlying .ADD or not etc.).
OpenJFS code will reveal to you what needs to be set in "FS32_ATTRIBUTE" and "FS_MPSAFEFLAGS2". Unfortunately this is all mostly undocumented (as so many other things in OS/2).

[Valery V. Sedletski]

Windows doesn't understand EA's created by fat32.ifs. You can try deleting the orphaned EA files. But strange, in my case if CHKDSK finds lost EA's, it properly writes the EA mark byte, so it becomes not lost. Obviously, there is some problem CHKDSK cannot fix atm, so I need to reproduce it on my machine to fix it. So, atm, I only can suggest try deleting the lost EA files.

/largefiles switch does nothing with CHKDSK and EA's. I am using /eas switch on my machine too, and it does not cause any problems. CHKDSK cannot fix the problem, so it remains dirty. You can also set disk to clean state by running f32stat d: /clean.

/largefiles does not need neither FS_MPSAFEFLAGS2 nor FS32_ATTRIBUTE because it is a 16-bit IFS and does not export FS32_*. It uses FS_CHGFILEPTRL/FS_NEWSIZEL only. FS32_ATTRIBUTE is needed for 32-bit IFS drivers only. The only 16-bit IFS which supports large files is DUMPFS. I only exported the same *L-functions and added FSA_LARGEFILE bit to FS_ATTRIBUTE. Now 64-bit file size and offset are working, but the problem remained, that FS_CHGFILEPTRL is not called, but FS_CHGFILEPTR is called instead and file pointer is truncated. But I am able to copy in/out files of size up to 4 GB. Probably, FS_CHGFILEPTRL is not called by kernel at all, maybe, I'll need to export FS32_* in the future too (I am already trying to create FS32_READ/FS32_WRITE/etc as 32-bit wrappers for FS_READ/FS_WRITE/etc for 64-bit seek to be working, but this is unfinished).

[Valery V. Sedletski]

I looked into CHKDSK source, the error "A lost Extended attribute was found" is shown when there is the EA file, but the file these EA's belong to, is not found. So, I suspect that this error should be fixed if you manually delete the orphaned EA file.

[erdmann]

You are not exporting the FS32_ ... entrypoints ? The following is directly from your driver. If you don't want FS32_ ... used at the moment then it might be a good idea to not export them yet:

Operating System/2 Executable File Header Utility
Version 4.01.003 Dec 11 2003
Copyright (C) IBM Corporation 1988-2003
Copyright (C) Microsoft Corp. 1988-1992.
All rights reserved.

Module:                         fat32
Description:                    @#Netlabs:0.10.r238#@##1## 12 Apr 2017 23:15:01     dtp       ::::0::@@  "Fat32 Installable filesystem, Henk Kelder & Netlabs"

Module type:                    Physical device driver
                                NO internal fixups in executable image
Number of memory pages:         0000002d (45)
Initial CS:EIP:                 object 0 offset 00000000
Initial SS:ESP:                 object 0 offset 00000000
Automatic data object:          4

 no. virtual  virtual  map      map      flags
     address   size    index    size
0001 00010000 0000e1f6 00000001 0000000f EXECUTABLE, READABLE, SHARED, PRELOAD,
                                         16:16 ALIAS
0002 00020000 0000fc03 00000010 00000010 EXECUTABLE, READABLE, SHARED, PRELOAD,
                                         16:16 ALIAS
0003 00030000 000055de 00000020 00000006 EXECUTABLE, READABLE, SHARED, PRELOAD,
                                         16:16 ALIAS
0004 00040000 00008fb0 00000026 00000007 READABLE, WRITEABLE, SHARED, PRELOAD,
                                         16:16 ALIAS
0005 00050000 0000007f 0000002d 00000001 EXECUTABLE, READABLE, SHARED, PRELOAD,
                                         32-bit
0006 00060000 00010000 0000002e 00000000 READABLE, WRITEABLE, SHARED, PRELOAD,
                                         16:16 ALIAS
0007 00070000 00010000 0000002e 00000000 READABLE, WRITEABLE, SHARED, PRELOAD,
                                         16:16 ALIAS
0008 00080000 00002000 0000002e 00000000 READABLE, WRITEABLE, SHARED, PRELOAD,
                                         16:16 ALIAS


Exports:
 ord  obj   offset    name
   1    3  00000165  FS_ALLOCATEPAGESPACE	exported
   2    1  00000000  FS_ATTACH		exported
   3    2  0000b2b2  FS_CANCELLOCKREQUEST	exported
   4    2  0000b279  FS_CANCELLOCKREQUESTL	exported
   5    2  0000491e  FS_CHDIR		exported
   6    2  0000b5dd  FS_CHGFILEPTR	exported
   7    2  0000b2ff  FS_CHGFILEPTRL	exported
   8    2  00007e59  FS_CLOSE		exported
   9    2  0000b6ab  FS_COMMIT		exported
  10    1  00000039  FS_COPY		exported
  11    1  000006d9  FS_DELETE		exported
  12    3  00000231  FS_DOPAGEIO	exported
  13    1  00000a48  FS_EXIT		exported
  14    2  0000559d  FS_FILEATTRIBUTE	exported
  15    2  0000c363  FS_FILEINFO	exported
  16    2  0000d851  FS_FILEIO		exported
  17    2  0000bab6  FS_FILELOCKS	exported
  18    2  0000ba7d  FS_FILELOCKSL	exported
  19    2  00000000  FS_FINDCLOSE	exported
  20    2  000000eb  FS_FINDFIRST	exported
  21    2  000009a7  FS_FINDFROMNAME	exported
  22    2  00000a3d  FS_FINDNEXT	exported
  23    2  00004397  FS_FINDNOTIFYCLOSE	exported
  24    2  000043d0  FS_FINDNOTIFYFIRST	exported
  25    2  00004409  FS_FINDNOTIFYNEXT	exported
  26    1  00000c43  FS_FLUSHBUF	exported
  27    1  00000d46  FS_FSCTL		exported
  28    1  00001592  FS_FSINFO		exported
  29    1  00002332  FS_INIT		exported
  30    1  00002f95  FS_IOCTL		exported
  31    2  00004c7d  FS_MKDIR		exported
  32    1  0000c638  FS_MOUNT		exported
  33    1  00004111  FS_MOVE		exported
  34    2  0000bc59  FS_NEWSIZE	exported
  35    2  0000bb15  FS_NEWSIZEL	exported
  36    2  0000d88a  FS_NMPIPE		exported
  37    2  0000693a  FS_OPENCREATE	exported
  38    3  00000000  FS_OPENPAGEFILE	exported
  39    2  000057e4  FS_PATHINFO	exported
  40    1  00004919  FS_PROCESSNAME	exported
  41    2  00007ffb  FS_READ		exported
  42    2  0000511c  FS_RMDIR		exported
  43    3  0000051a  FS_SETSWAP	exported
  44    1  000049c9  FS_SHUTDOWN	exported
  45    2  00009547  FS_WRITE		exported
  46    4  00002b4c  FS_ATTRIBUTE	exported
  47    4  00002b46  FS_NAME		exported
  48    5  00000000  FS32_CHGFILEPTR	exported
  49    5  0000002a  FS32_CHGFILEPTRL	exported
  50    5  00000034  FS32_READ		exported
  51    5  00000069  FS32_READFILEATCACHE	exported
  52    5  0000006f  FS32_RETURNFILECACHE	exported
  53    5  00000075  FS32_WRITE	exported

[erdmann]

There is no orphaned EA file that I could find. That's exactly the odd thing. The original file no longer exists, the associated EA file does not exist either. At least it is not visible with the current FAT32.IFS. Unfortunately I had to reformat the disk, therefore I cannot double check.

[erdmann]

I vaguely remember that you will get a different parameter block in FS_OPENCREATE when DosOpenL is used to open a file. I'll need to find the discussion in the internet ...

[Valery V. Sedletski]

Yes, it still exports FS32_READ/FS32_WRITE/etc. but I commented out FS32_ATTRIBUTE atm, so they are disabled this time. As I said, I made the 32-bit wrappers, but not finished them.

There should be an orphaned EA file, otherwise there will be no traces of EA at all. Please, try disabling the /eas switch, then it will show it, and you delete it (When /eas is enabled, the EA files are not shown on disk).

I vaguely remember that you will get a different parameter block in FS_OPENCREATE when DosOpenL is used to open a file. I'll need to find the discussion in the internet ...

Not in FS_OPENCREATE, but in FS_FIND[FIRST/NEXT/CLOSE], FS_FILEINFO, FS_PATHINFO, they are called with FIL_STANDARDL parameter, instead of FIL_STANDARD, for example. This is working atm too.

[erdmann]

How can you be sure that the kernel will not try to query the proc address of the FS32_ entry points ? You likely don't know.
I think it is a very bad idea to export these entry points until they are actually required.

[Valery V. Sedletski]

It does not call them. They are dummies and don't work atm. But it calls FS_* instead and everything is working. Otherwise, it would not work. And I know for sure that if FS32_ATTRIBUTE is not exported, other FS32_* are not called.

[erdmann]

The only thing I can say is that the /largefiles switch does not work here.
When I specify it, I cannot access my FAT32 partitions any more (no chkdsk possible, no formatting possible).

[Valery V. Sedletski]

Are you sure? May be it is /ac:* indeed? Please, double check. And please, take a com port log by adding the /monitor switch. Never had problems with it, and nobody complained so far.

PS: Any details on your configuration? Do you have DaniS506 or os2ahci? I never had os2ahci, only use danis506. No other hypotheses yet.

[Valery V. Sedletski]

Hm, strange, I have this, if I comment fat32.ifs out:

[d:\]chkdsk w: /f
The current hard disk drive is: W:
SYS0026: Указанный диск или дискета недоступны.

Do you have the same? So, if fat32.ifs isn't loaded, access to the FAT32 disk in DASD mode does not work. Strange, how then CHKDSK at init time works?

So, probably, fat32.ifs isn't initted in your case, and then CHKDSK and FORMAT does not work too. But I don't know why. So far, I've only seen this on Merlin case. So, please take a log to clarify this.

PS: and the following questions: 1) What kernel version are you using? 2) Does fat32.ifs write "WARNING: Large files support will be disabled." on init?

[Valery V. Sedletski]

PPS: With which version you have non-working FORMAT/CHKDSK? r213 or r238? There was Merlin-related fix in r220, between them (after that fix, no more unsuccessful init should be possible). So, if it is r213, please update and retest!

[erdmann]

1) FORMAT / CHKDSK neither worked for r213 nor for r238.
2) I am using this kernel (which comes with eCS 2.2 beta):

[d:\]bldlevel os2krnl
Build Level Display Facility Version 6.12.675 Sep 25 2001
(C) Copyright IBM Corporation 1993-2001
Signature:       @#IBM:14.106#@_SMP  IBM OS/2 Kernel
Vendor:          IBM
Revision:        14.106
File Version:    14.106
Description:     _SMP  IBM OS/2 Kernel

3) I am using OS2AHCI.ADD for hard disks. I somewhat doubt that this is relevant as I am talking about USB media which uses USBMSD.ADD exclusively.
4) I'll need to recheck if there is a warning "WARNING: Large files support will be disabled" if I remove the /largefiles switch.
5) I have an 8-core AMD system and I am using all 8 cores. That will make it necessary to make FAT32.IFS SMP safe. Whatever it takes the IFS to do that.

How do I take a log ? I don't have a COM port on this system. I could do the monitoring with f32mon. Please advise on how to set up the system in order to do that.

[erdmann]

I mentioned some internet discussion, here it is (in case you do not know already):

​https://groups.google.com/forum/#!topic/comp.os.os2.misc/E_SNsuNT-uo[51-75]

It centers around the change in call parameters for: FS_CHGFILEPTRL, FS_NEWSIZEL, FS_CANCELLOCKREQUESTL, and FS_FILELOCKSL.

It also shows some example code. However, I think the way to go is to write routines that are able to deal with 64-bit offset and do some "parameter reformatting" for say FS_CHGFILEPTR so that FS_CHGFILEPTR and FS_CHGFILEPTRL can then call the very same handling routine. Likewise for FS_NEWSIZE and FS_NEWSIZEL.

[erdmann]

I just looked at the FAT32 code:

example "FS_CHGFILEPTRL": I can see that parameter "offset" has now become a "long long" which makes perfect sense. However, "struct sffsi" has not changed.
Are you sure that you do not need to change the structure variables "sfi_size" and "sfi_position" to also become "unsigned long long" instead of "unsigned long" ?
Same question goes to FS_NEWSIZEL, FS_FILELOCKSL (for "struct sffsi" as well as for "struct filelock" for its variables "FileOffset" and "RangeLength"), FS_CANCELLOCKREQUESTL (for "struct sffsi" as well as for "struct filelock").

I am just guessing as I have not tried out but I would think that for files larger > 2 GB, all these structure variables would need to be 64-bit

I am guessing this way as for example FS_CANCELLOCKREQUEST does not have an offset parameter but still, a new FS_CANCELLLOCKREQUESTL entry point exists.
A good indication of why this might be necessary if FS_NEWSIZEL: you increased the "len" parameter to become "unsigned long long". But then the "sfi_size" parameter in "struct sffsi" should also be 64-bits.

It might even be necessary to change these structures even for the legacy ("non L") entrypoints once you specified the FSA_LARGEFILE flag in FS_ATTRIBUTE.

[Valery V. Sedletski]

How do I take a log ? I don't have a COM port on this system. I could do the monitoring with f32mon. Please advise on how to set up the system in order to do that.

Monitoring with f32mon will not help, because you need to switch log on early on boot. So, you need the "/monitor" switch on the command line.

Try another system with a COM port. Just add a "/monitor" switch to fat32.ifs and boot. Or, install OS/4 kernel, boot to desktop and take a log via a command

copy kernlog$ kernlog.txt

But this will not help, if you cannot boot the system successfully.

I did "parameter reformatting" for FS_CHGFILEPTRL/FS_NEWSIZEL/FS_CANCELLOCKREQUESTL/FS_FILELOCKSL, otherwise files > 2 gb would not work. Parameters "pos" and "size" are long long, but not "sfi_size" and "sfi_position", but "sfi_sizel" and "sfi_positionl", just look better at the end of "struct sffsi".

Aurora adds two new "sfi_sizel" and "sfi_positionl" 64-bit members to "struct sffsi" at the end (because "sfi_size" and "sfi_position" cannot be extended, as they are located in the middle of the structure). I need to update both 32-bit and 64-bit structure members in parallel.

I need /largefiles switch to enable 64-bit size/offset support, because on Merlin and older kernels, there are no 64-bit members, so their offsets are invalid and could lead to a trap, if accessed. So, on Merlin and older kernels, I check for /largefiles specified, and if occassionally specified, I additionally check for KEE module presence. And if missing, the large file support gets disabled. Prior to r220, if /largefiles is specified and no KEE module found, FS_INIT returned a non-zero return code, so fat32.ifs did not installed. And because it was not installed, CHKDSK and FORMAT did not worked, because it cannot open the disk in DASD mode. Since r220, it is corrected, and it should not fail to init, if /largefiles is specified, but no KEE module is found. The KEE module is searched for by trying to DosLoadModule for "KEE" module. This works fine on all my machines, with either, 14.104 or 14.106 IBM's kernels, or OS/4 kernels, but fails on Merlin kernels. So, I'm not sure why it could fail on your machine.

FORMAT / CHKDSK neither worked for r213 nor for r238.

I mean, does fat32.ifs init successfully on r213, or r238? As I see no possibitity for r238 to init unsuccessfully, as it always returning 0, as I look into the code.

It should write "WARNING: Large files support will be disabled" to screen, if it did not find the KEE module in the system, so r213 may to not boot.

[Valery V. Sedletski]

copy kernlog$ kernlog.txt

If no OS/4 kernel is installed, you can try QSINIT:

copy oemhlp$ oemhlp.txt

If you have ACPI.PSD installed, it changes oemhlp$ to its own, so you need to type

copy ___hlp$ oemhlp.txt

to get log.

PS: Actually, I repeat the log to three places: 1) fat32.ifs log buffer 2) com port 3) OS/4 or QSINIT log buffer, if available. Logging is enabled if f32mon.exe is started, or /monitor switch for fat32.ifs is specified. The latter enables logging permanently, and the former enables it temporarily.

PPS: Also, you could try running f32mon.exe, when system is started (if you specified the /monitor switch), and see f32mon.exe log, but fat32.ifs log buffer is very small and may be overwrapped, so older messages will be erased. So, it is better to use QSINIT or OS/4 kernel log buffers (they could be up to 32 MB in size, quite big. fat32.ifs log buffer is only 8192 bytes long).

[erdmann]

I can boot the system successfully, so this is not the problem.
The problem is that then, accessing a FAT32 large floppy stick (or a FAT32 partition on a USB stick) will not work: the access does not properly work, chkdsk does not work reliably (sometimes it works, most of the time it does not).

I have the lastest OS/2 kernel, I have KEE support and large file support.

[Valery V. Sedletski]

I don't see any FAT32 messages in the attached log. Did you added the "/monitor" switch? Did fat32.ifs initted properly? Please, check its output. Did it wrote, that large file support was disabled? (Please add PAUSEONERROR=YES to config.sys and some syntax error after ifs=fat32.ifs line, say, device=aaa.sys. It should pause booting after fat32.ifs loading, and show the IFS output).

PS: It seems, the log is empty because fat32.ifs did not initted successfully, and so, there were no messages. So, please, check fat32.ifs output, as I asked above. And, please take a boot screen screenshot, if possible.

[erdmann]

Oops, sorry I had deinstalled your FAT32.IFS. Now I reinstalled, added the /monitor switch to FAT32.IFS.

I definitely have to REMOVE the /largefiles switch. If I have it enabled my whole system if screwed up, I even cannot access any partitions that are not FAT32. I have not tried commandline, but the WPS will completely screw up (which could be an inidication that something goes beserk with EAs maybe).
Or maybe an alignment problem in the sffsi structure ? Where did you get the updated structure definition from ?

[erdmann]

Question since I have /largefiles switch removed:

I saw that the oemhlp.txt log contains a call to FS_CHGFILEPTRL since that is "routed through" by the call to FS_CHGFILEPTR. Isn't it dangerous to call FS_CHGFILEPTRL when /largefiles is not active ? I would think that you are expecting a different "struct sffsi" definition (along with a 64-bit signed offset instead of a 32-bit signed offset) when the /largefiles switch is specified, or not ?
My gut feeling is that you have to expect the legacy structure definition for as long as you do not specify FSA_LARGEFILE in FS_ATTRIBUTE (I haven't looked through the whole FAT32 code so I might be missing something here).

[erdmann]

Another one: are you sure that you can modify FS_ATTRIBUTE dynamically during FS_INIT ?
Are you sure that this will work in all cases ? My gut feeling is, it won't.
If there are some simple instructions, maybe I can build FAT32.IFS and as a test set FS_ATTRIBUTE statically to FSA_LVL7 | FSA_LARGEFILE.
Or would you be able to provide me with a test driver ?

By the way: what you can always do according to IFS spec is to return from FS_INIT with a return value != NO_ERROR in which case the system will unload the driver.
It might be a better idea to check within FS_INIT if large file support is available and if it is not available, then return with error.
Of course that blocks out older kernels but then you might be able to build 2 versions of FAT32.IFS ...

If you check for KEE, I think it is also a good idea to use "DosQueryModuleHandle" instead of "DosLoadModule". If KEE exists "DosQueryModuleHandle" will return successfully but without attempting a further load of the module.
Neither "DosLoadModule" nor "DosQueryModuleHandle" are listed as supported Ring-3 API functions supported on FS_INIT. But if "DosLoadModule" actually works then "DosQueryModuleHandle" will also work.

[erdmann]

Yet another suggestion:

always set FSA_LVL7 | FSA_LARGEFILE in FS_ATTRIBUTE, older kernels will ignore the FSA_LARGEFILE flag as they don't know about it. Then in FS_INIT keep checking for KEE module. If the check fails you can safely assume that only the legacy FS_ entrypoints are called (but not the "L" type entry points) and that the sffsi structure will have the OLD form (without the additional 64-bit fields in sffsi). Of course, if you only have legacy support then it is strictly forbidden to call an "L" entrypoint from a "non L" entry point as you are currently doing for FS_CHGFILEPTR.

Going this way even eliminates the need for the /largefiles switch alltogether.

[erdmann]

Ok, I now patched FAT32.IFS to always set the FSA_LARGEFILE attribute in FS_ATTRIBUTE.Additionally I still specified the /largefiles switch as the program logic in FAT32.IFS currently requires that.

The result was (unfortunately) that it still does not work (it works as bad as it did before). When I used Theseus to make sure that FS_ATTRIBUTE is properly set I also had a look at the device driver list. What I observed with the /largefiles switch active was that the device driver chain was broken in the Theseus view, not all drivers were listed and there was an error message at the bottom of the view.
With /largefiles not specified the device driver chain in Theseus looks just fine.

That gives me even more an indication that something is severly broken with /largefiles support which affects the whole system. Maybe as a test we should remove "DosLoadModule" test on KEE ...

[Valery V. Sedletski]

Where is your screen photo? I asked you to take a screenshot of FAT32 boot screen...

I saw that the oemhlp.txt log contains a call to FS_CHGFILEPTRL since that is "routed through" by the call to FS_CHGFILEPTR. Isn't it dangerous to call FS_CHGFILEPTRL when /largefiles is not active

No, FS_CHGFILEPTRL is always called by FS_CHGFILEPTR. If /largefiles is not set, or large file support is disabled, 64-bit fields are not accessed.

I would think that you are expecting a different "struct sffsi" definition (along with a 64-bit signed offset instead of a 32-bit signed offset) when the /largefiles switch is specified, or not ?

struct sffsi is always the same, but without two 64-bit entries at the end.

Another one: are you sure that you can modify FS_ATTRIBUTE dynamically during FS_INIT ?

Are you sure that this will work in all cases ? My gut feeling is, it won't. If there are some simple instructions, maybe I can build FAT32.IFS and as a test set FS_ATTRIBUTE statically to FSA_LVL7 | FSA_LARGEFILE.

I am sure, it works fine. No need for any experiments.

Or would you be able to provide me with a test driver ?

it is freely available to download from Netlabs' svn.

If you check for KEE, I think it is also a good idea to use "DosQueryModuleHandle?" instead of "DosLoadModule?". If KEE exists "DosQueryModuleHandle?" will return successfully but without attempting a further load of the module.

Neither "DosLoadModule?" nor "DosQueryModuleHandle?" are listed as supported Ring-3 API functions supported on FS_INIT. But if "DosLoadModule?" actually works then "DosQueryModuleHandle?" will also work.

DosLoadModule works fine even on Merlin. No difference between DosLoadModule and DosQueryModuleHandle, as KEE module is always loaded. And no indications so far that I must change this. I need to see what fat32.ifs writes on boot first, otherwise I cannot decide, what's happening and what I need to change.

oemhlp.txt is useless, since you disabled /largefiles and now I see the messages. But I need to check this with /largefiles, not without it.

[Valery V. Sedletski]

What I observed with the /largefiles switch active was that the device driver chain was broken in the Theseus view, not all drivers were listed and there was an error message at the bottom of the view.

fat32.ifs does nothing with device driver chain, it is not a PDD

[erdmann]

1) /largefiles breaks my System. Since it becomes unusable, I cannot enable that Switch
2) I know that FAT32.IFS is not a PDD. Still, it breaks the device driver chain when /largefiles Switch is enabled. Presumably, it messes up something in some kernel segment.

I don't think this discussion is leading us anywhere. I think I'll stick to the original driver.

[Valery V. Sedletski]

So, you don't want to enable it even temporarily, for testing purposes? Then I cannot help you.

If you don't want to test with /largefiles enabled, it is useless. So, don't complain that it's not working on your system, I'm closing the ticket then. If you cannot help me, I cannot help you.

PS: I hope, someone else will encounter the same problem and I'll fix it still

[Valery V. Sedletski]

PPS: We can speculate about that setting FSA_LARGEFILE dynamically should not work (I know for sure that it's working. I tried to set FSA_REMOTE for a FS driver on/off at init, and this worked since 2009, when I made my universal MiniFSD, which can work with RAMFS.IFS, as well as with FAT32.IFS and HPFS/JFS), or something else, for very long. But I need to know for sure, what's happening on your system. For that, I need fat32.ifs output on init. Without that, only speculations are possible, which is useless. I still hope you will help me, then I'll help you.

[Valery V. Sedletski]

I definitely have to REMOVE the /largefiles switch. If I have it enabled my whole system if screwed up, I even cannot access any partitions that are not FAT32. I have not tried commandline, but the WPS will completely screw up (which could be an inidication that something goes beserk with EAs maybe).

You even cannot access partitions which are non-FAT32? Are you sure? How did you booted the system, otherwise? The boot disk should be accessible anyway. Maybe, you have EA problems and Drives object just does not show disk contents? Or, it is possible that some disks are locked, and this breaks the WPS. Did you tried FC/2? If only WPS doesn't work, this doesn't mean the access to disks isn't working. You must check this, to be sure. And please, take screenshot of a boot screen.

Also, I didn't understood, does access to FAT32 disks work for you if you didn't specify /largefiles ? And, please check this in FC/2, not only in WPS.

[Valery V. Sedletski]

FORMAT and CHKDSK are not working for you because fat32.ifs is not loaded. I've got the very same symptoms when trying to do CHKDSK (error trying to BEGINFORMAT with "f32chk d: /f" and ERROR_NON_DOS_DISK == 26 with "chkdsk d:", the latter becuse DosQueryFSAttach fails to determine the IFS name, if fat32.ifs is not loaded) when I commented the fat32.ifs out in config.sys. That is because fat32.ifs is not loaded. So, this is the problem. So, we need to know, why it is not loaded. In theory, FS_INIT may only return a non-zero result only if DosGetInfoSeg is failed, but it is almost impossible. It even returns zero if DosLoadModule is failed, regardless of /largefiles switch. So, I wonder, what could be wrong.

[erdmann]

oemhlp with /largefiles enabled

[erdmann]

Ok, did enable /largefiles and attached "oemhlp.2.txt"

When I say I could not even access non-FAT32 drives, I was referring to the WPS, that is, I have problems opening a non-FAT32 drive object if /largefiles switch is in effect. No, the WPS is not broken: when I remove the /largefiles switch or revert back to the old FAT32 driver I have no problems with the WPS whatsoever.

Yes, if I remove the /largefiles switch, I can access files on a FAT32 volume. I also have no problems with the WPS then (that is, I can open files via the WPS).

I never have problems booting as such, therefore I cannot take a boot screen picture (there is no error reported by FAT32.IFS).

[erdmann]

oemhlp with /largefiles enabled and vfdisk.sys removed (drive X:)

[erdmann]

with /largefiles enabled, never ending disk/file access

[erdmann]

trap in UFAT32.DLL on chkdsk on a FAT32 volume

[erdmann]

A few more things:

1) after looking at "oemhlp.2.txt" I decided to remove VFDISK.SYS. VFDISK.SYS is an "old-fashioned" block device driver (predating the ADD block device drivers) that emulates FAT12 floppies of various floppy sizes:
​http://hobbes.nmsu.edu/h-search.php?key=vfdisk&pushbutton=Search
As a test, can you install VFDISK.SYS and see if it has any repercussion on the problem ?
I was using this commandline in config.sys which should be enough for a test:

DEVICE=D:\OS2\BOOT\VFDISK.SYS X: 1

2) another "oemhlp.3.txt" after VFDISK.SYS was removed. Scrambled filenames show up on a FS_PROCESSNAME call (FS_PROCESSNAME for G:\A.+,;=[].B).

3) "f32mon.txt" shows the monitor output where a FAT32 is accessed indefinitely. I eventually had to reboot.

4) "POPUPLOG.OS2" shows traps in UFAT32.DLL on a chkdsk attempt (with /largefiles active)

[Valery V. Sedletski]

Hm, tried vfdisk.sys. If I add it after FAT32/HPFS/JFS, everything is working. But if I add it to config.sys before all IFS-es, vfdisk.sys mounts on U: drive letter (though, I specified it x:). U: is CDROM drive letter in my system. Then when fat32.ifs tries to mount on this U: drive letter, I got a trap in fat32.ifs somewhere in FS_MOUNT. Need to determine, where exactly. But I see that some pointer is NULL and it tries to dereference it. Also I see that when vfdisk.sys is loaded after fat32.ifs, VFDSIK floppy is mounted with in-kernel fat driver. Trying to remount this drive letter with remount.exe from ext2-os2.ifs, I see that fat32.ifs doesn't mount it, though it recognises all other FAT drives, including physical floppy, and even a CDRW disk with FAT filesystem. Why it is not mounted? Is it the usual .add driver, or what principle it is based on?

So, looking at the logs, I see that on your machine, vfdisk.sys prevents fat32.ifs (and other drives IFS-es) to mount and it is initted ok indeed. But I cannot reproduce that on my machine, yet. Also, I don't understand what it does with /largefiles ? So, if I correctly understood, everything is working on your machine, if you disable vfdisk.sys, but leave /largefiles? Disk access works? And CHKDSK works, but you got a trap in CHKDSK?

[Valery V. Sedletski]

Such filenames: G:\A.+,;=[].B I see in fat32.ifs, as well as in vboxsf.ifs (VBox shared folders IFS I am developing too). It is the kernel supplies such filenames. The symbols "A.+,;=[].B" seem to be the list of invalid filesystem symbols.

[Valery V. Sedletski]

This: http://trac.netlabs.org/fat32/attachment/ticket/40/f32mon.txt looks like an infinite directory entry listing returned by DosFindNext. Maybe, corrupted filesystem? Did you tried to run CHKDSK? (maybe, on windows). What is in this directory? Does it look ok when accessed from windows? How could I create a testcase?

[Valery V. Sedletski]

I never have problems booting as such, therefore I cannot take a boot screen picture (there is no error reported by FAT32.IFS).

You can try adding PAUSEONERROR=YES to config.sys, and some syntax error right after ifs=fat32.ifs line, for example, "device=aaa.sys", where aaa.sys is a non-existing driver. Then booting will stop on this syntax error and you will see the fat32.ifs boot screen. I hope to see some unusual messages, like "Large files support is disabled", and also, to see if the fat32 version loaded is indeed r242/r238, not r213.

[Valery V. Sedletski]

We tested fat32.ifs and vfdisk.sys together with AlexT and got some results. It appears that when vfdisk.sys is active, fat32.ifs fails to mount the virtual floppy, because the kernel calls FS_MOUNT with wrong bootsector (BPB is empty). The boot sector begins with bytes: f0 ff ff 00 00 ... and subsequent bytes are zero, so fat32.ifs fails to mount. It appears that in such case, the kernel supplies the FAT starting sector, instead of the BPB. (DOS versions prior to 4.0 had no BPB, instead, they had only predefined media types f0 for floppies and f8 for hard disks. f8 is the 1st FAT byte for hard disks, and f0 for floppies. So, this is a standard pre-BPB media handling, which is the same in OS/2.). To prevent this, you need to specify the DEV_NON_IBM flag in DD_Attr in device driver header. It appears that both VDISK.SYS from IBM and SVDISK.SYS from Hobbes have this bit set, but VFDISK.SYS has it missing. I patched my copy of vfdisk.sys, and since that, fat32.ifs is now supplied a valid BPB and it now recognizes the virtual floppy. I just changed bytes 0880 to 2880 at 0xa8 offset from the binary start. I can attach the patched binary, or you can patch it yourself. I suspect that the cause of non-mounted FAT32 (and not only FAT32 on your machine) may be the same (FAT starting bytes, instead of a boot sector dump).

So, after patching the vfdisk.sys driver, fat32.ifs now mounts the virtual floppy. But some problems still remained.

If I enable autocheck: /ac:* in the fat32.ifs command line, and the order of drivers is the following: first vfdisk.sys, then HPFS.IFS, then JFS.IFS, then FAT32.IFS (all these HPFS/JFS/FAT32 have autocheck enabled), I have vfdisk.sys taking the 1st drive letter after RESERVEDRIVELETTER. Then after autocheck, the kernel FAT is mounting the virtual floppy. FAT32.IFS' FS_MOUNT then is not called by the kernel, because it is already mounted by the in-kernel FAT driver. Then, when cachef32.exe is starting, I force remount of all FAT12/FAT16 drives (for FAT drives to remount from the in-kernel FAT to fat32.ifs). At this point, the FS_MOUNT routine in fat32.ifs gets called, and it traps here. This is because FS_MOUNT finds a dup hVPB from the previous mount, and tries to use it, but a VOLINFO pointer in vpfsd appears to be invalid (For the very 1st mount of the volume, fat32.ifs should store VOLINFO pointer in vpfsd, and it should restore it from there on subsequent FS_MOUNT calls (see ifs.inf, the section regarding FSH_FINDDUPHVPB)). So, I suspect that the VOLINFO pointer is invalid because the dup hVPB was allocated on autocheck, so it did not store the VOLINFO pointer, so, this pointer is invalid. But this problem appears only with vfdisk virtual floppy. With two another FAT12 and FAT16 partitions on the hard disk, there's no dup hVPB appeared and I see no trap. Still investigating this problem. AlexT thinks also that my FAT32 autocheck hurts something in vfdisk, hence a trap. But I don't see anything dangerous in autocheck. It just reads the disk, checks for the dirty status, and if it is clean, it just exits. It should not even write there, only read. And all disks were clean, so it should not modify anything.

If /ac:* is not set, everything is fine, the disk is mounted and all is working.

PS: Tried the different floppy formats with vfdisk.sys. If I specify

device=d:\os2\boot\vfdisk.sys 4

it creates a 2.88 MB floppy, for example. But

vfdctrl x: <X>

with any <X> makes vfdctrl.exe hang, and trying to kill it, I got the whole system hanging. vfdctrlpm.exe also hangs in this case.

Trying to take a floppy image with dd.exe (I use the old EMX version of dd.exe, the kLibc version from the yum repo appears to be broken. I could share it, if needed), I got broken image. The boot sector repeats seeral times, and I don't see FAt's in the image. Some trash. Dd.exe takes fine the images of any other disks, including the physical floppies. Only virtual floppy image gets corrupted.

[Valery V. Sedletski]

Also, tried to do

savedskf x: vfdisk.bin

-- I got a 970-byte file filled with complete trash. It would be good to get possibility to take floppy disk images somehow, otherwise the use of vfdisk.sys may be limited.

And, I managed to make IBM's VDISK.SYS working too. It creates the < 32 MB virtual hard disks. BTW, it is an interesting case. It creates the BPB, but there's no filesystem name and a volume label after the BPB. I needed to check if the JMP command, jumping over the BPB has offset < 0x29 (the BPB size), otherwise it did not mounted. But now it mounts fine. I hope that any other FS than FAT should have the FS name and volume label fields after the BPB.

[erdmann]

No need to release a binary patched VFDISK, the code is right here:

​http://trac.netlabs.org/vfdisk

I'll update the driver and rerelease it.

What I do not understand: you are talking about load sequence of VFDISK.SYS and the various .IFS drivers. But VFDISK.SYS (being a device driver after all) should ALWAYS load before any .IFS driver even if the sequence in config.sys is different.

Next thing I don't understand: why did you add FAT12/FAT16 support to FAT32.IFS ? Can I turn that off in FAT32.IFS (or is it disabled by default) ?

I don't remember having a problem with VFDISK.SYS with the kernel built-in FAT12/FAT16 support.

Lars

[erdmann]

Updated VFDISK to version 5.0

​http://hobbes.nmsu.edu/h-search.php?sh=1&button=Search&key=vfdisk&stype=all&sort=type_name&dir=%2F

currently in the "incoming" directory.

I tested vfctrl.exe as well as vfctrlpm.exe: both work perfectly well with the original FAT32 driver version 9.13 in place. I can change media type and I can eject the media.

[Valery V. Sedletski]

What I do not understand: you are talking about load sequence of VFDISK.SYS and the various .IFS drivers. But VFDISK.SYS (being a device driver after all) should ALWAYS load before any .IFS driver even if the sequence in config.sys is different.

VFDisk don't need to be loaded after all IFS'es. I tried to load it both ways and it works fine. IFS FS_MOUNT occurs at a very late stage indeed, when run=... and call=... statements are processes, so all device=... and ifs=... already loaded. So, no problem if they are loaded in other order.

Next thing I don't understand: why did you add FAT12/FAT16 support to FAT32.IFS ? Can I turn that off in FAT32.IFS (or is it disabled by default) ?

FAT12/FAT16 is a part of generic FAT support which is common for all FAT versions. FAT32.IFS now supports FAT12/FAT16/FAT32/exFAT. All these are not very different from FAT32, so why not support all them in a single driver? And, as a bonus, FAT support in fat32.ifs adds VFAT support in FAT12/FAT16, which is missing in IBM's FAT driver.

Indeed, I fixed fat32.ifs to work both with vfdisk.sys and vdisk.sys, now it works with both. svdisk.sys is in the queue. Also, I added support for floppies without a BPB (they have only media type byte in 1st FAT entry). So, even a non-patched vfdisk should work too. What did you changed in VFDISK if the version is 4.0->5.0? Only the bit in Device Attributes, or the hang when using vfctrl and reading sectors from virtual floppy are fixed too?

You can disable FAT12/FAT16 support, of course. For that, just don't add the /fat switch. And likewise for exFAT, if you don't add the /exfat switch, exFAT support will be disabled by default. Additionally, you can specify the mask of driveletters to be mounted for each filesystem: /fat:<drive letter list> or /fat32:<drive letter list> or /exfat:<drive letter list>. The <drive letter list> has the same format like in /ac:<...>.

I don't remember having a problem with VFDISK.SYS with the kernel built-in FAT12/FAT16 support.

Because IBM's FAT driver is well-tested with any of all these virtual disk drivers from the beginning. Kernel FAT driver supports passing the FAT start in place of the BPB (now fat32.ifs should support it too, though). So, this was not documented anywhere and hence, not supported previously. VDISK.SYS has no FS name after the BPB, so it needed special support to be successfully mounted, and so on. So both are interesting border cases which were not to be intended to work initially.

PS: And I'll release the new fat32.ifs version soon, which should work with VFDISK.SYS, please test it when released.

[Valery V. Sedletski]

Just tested the new VFDISK.SYS 5.0. It is mounted well with fat32.ifs. And I confirm that VFCTRL/VFCTRLPM are now working. But taking an image with dd.exe still returns a 0 sector instead of 1, 2, 3, 4, ... ones. (Need I attach dd.exe here?) Also, I still see the "VDISK 4.0 (c) 2003 D.Engert (2010 L. Erdmann), Drive A:" (the version shown is still 4.0 not 5.0), but it's the minor problem.

[Valery V. Sedletski]

But VFDISK.SYS (being a device driver after all) should ALWAYS load before any .IFS driver even if the sequence in config.sys is different.

No, IFS=... and DEVICE=... load as they specified, not DEVICE=... first, IFS=... second. IFS and DEVICE init routines are executed at one pass. But IFS'es are mounted later, at call=.../run=... load time (except for the boot disk IFS for which FS_MOUNT is called by a minifsd, so, it executes at early stage).

[Valery V. Sedletski]

The version r248 is released, look at http://trac.netlabs.org/fat32/. This version has some enhancements regarding FAT12/FAT16/exFAT, so that VFDISK.SYS and VDISK.SYS now should work. Also, added support for formatting into the exFAT filesystem. Please test if VFDISK.SYS will work ok for you.

PS: The exFAT filesystem, created by our FORMAT version, is recognized by fat32.ifs, but winxp still rejects it. So, the work is still in progress.

[erdmann]

Tested fat32-0.10-r248-os2.wpi:

[d:\os2\boot]bldlevel fat32.ifs
Build Level Display Facility Version 6.12.675 Sep 25 2001
(C) Copyright IBM Corporation 1993-2001
Signature:       @#Netlabs:0.10.r248#@##1## 2 May 2017 17:54:37      dtp       :
:::0::@@  "Fat32 Installable filesystem, Henk Kelder & Netlabs"
Vendor:          Netlabs
Revision:        0.10.r248
Date/Time:       2 May 2017 17:54:37
Build Machine:   dtp
File Version:    0.10
Description:     "Fat32 Installable filesystem, Henk Kelder & Netlabs"

I still have the very same problem with the /largefiles switch as I had before. I do have an update vfdisk.sys 5.0. Are you testing on a multi-core system or only on a single core system ? I have 8 cores.

[Valery V. Sedletski]

My main machine is single-core (Athlon 64 processor). But I test it also on two Core2Duo's with both cores enabled -- I see no problems, though, I did nothing special yet for SMP support. Could you please, show me the screenshot of what fat32.ifs writes on the boot screen, as I asked before? (with /largefiles enabled). And the COM port log would be desirable, too, if possible (maybe, on another machine). And also, please, attach your config.sys here too.

[erdmann]

This is the screenshot with /largefiles switch enabled.

I don't have a COM port.

Config.sys also attached.

[Valery V. Sedletski]

Where are fat32.ifs messages? Please, remove the /q switch and take the snapshot again.

I don't have a COM port.

Too bad. How are you debugging you drivers without a COM port?

I see nothing unusual in your config.sys. I only don't know what is it:

BASEDEV=MCHNCHK.SYS

?

[erdmann]

I rechecked: if the /largefiles switch is active I can access files (on a FAT32 medium) via the command line, I can also open those files with an application.

However, accessing the files via the WPS does not work.

For me this has now become plain obvious: the EA support does not work if /largefiles is active because the WPS heavily relies on EA support. Since the EAs are files by themselves my suspicion is that the access to the EA files goes wrong once the /largefiles switch is active.

MCHNCHK.SYS is a BASEDEV driver written by myself (a machine check exception handler) and has been in use for at least the last 1,5 months. No it has no relation to the problem.

I debug my drivers by using ICAT from a remote machine (in which case I have to limit the number of CPUs on the target machine to 1 which I do via a separate installation and setting the /MAXCPU switch in ACPI.PSD).

[erdmann]

New screenshot with /q switch removed and /largefiles switch still in place.

[Valery V. Sedletski]

I rechecked: if the /largefiles switch is active I can access files (on a FAT32 medium) via the command line, I can also open those files with an application.

However, accessing the files via the WPS does not work.

This is completely another problem. If only WPS file access is not working, then these are minor problems. Need to understand, which exactly, of course. I would suggest to take a f32mon log when opening a WPS folder. As WPS is not the primary way of working with files, I would check first if it works with FC/2, and only as the next attempt, would test WPS. As with WPS things could be more difficult.

For me this has now become plain obvious: the EA support does not work if /largefiles is active because the WPS heavily relies on EA support. Since the EAs are files by themselves my suspicion is that the access to the EA files goes wrong once the /largefiles switch is active

Quite not obvious. I have both /largefiles and /eas enabled too, and most drives are opened fine with the WPS. Only exFAT drives give an error: "No objects have found that match the specified find criteria.", then "An error has occured and the message is not accessible. Contact the supplier of the application." Just opened the FAT32 drive and copied there the Desktop from the boot drive (with FC/2, of course). Opened the Drive in a WPS folder (via Ctrl-W), then the Settings notebook of Desktop, and on the 2nd page of File bookmark see the list of available EA's, can open all of them. So, EA's are working. But there may be problems with creating the wp root. sf files or similiar. So, the log could help very much here.

Also, checked one FAT16 drive, EA's are working here the same as on a FAT32 drive. But still not on an exFAT drive.

[erdmann]

1) As I said repeatedly, I can open files from the commandline. I can also open a file via FC (the textmode version of FC/2).

2) I now tried a partitioned stick with a FAT32 partition. I still have the very same problem. I then deleted "WP ROOT. SF" on that partition. That did not help either.

If it does not properly work with the WPS, it is unusable for me. I also have tried with QSINIT as well as the original OS2LDR which did not make a difference (but that was not expected anyway).

[erdmann]

monitor output from a FAT32 partition with /largefiles switch on

[erdmann]

Attached monitoring result. The WPS view never properly refreshes but there should be 3 files on it: "WP ROOT. SF" (hidden), "CHKDSK.LOG" and "CHKDSK.OLD". Looks like it enters an endless loop.
Looks like FS_FINDNEXT always gets the very same 3 files returned, that is, it does not seem to remember which files it returned on the preceding request.

[Valery V. Sedletski]

That's FS_FINDNEXT returns three dir entries. The next call to FS_FINDNEXT should return ERROR_NO_MORE_FILES, but it returns the same three entries. Hence, the endless loop. Are you sure only CHKDSK.LOG and CHKDSK.OLD are there? There should be the 3rd file too. I suspect that is WP ROOT. SF. If large files support is enabled, FS_FINDFIRST/NEXT is called with FIL_QUERYEASFROMLISTL flag, instead of FIL_QUERYEASFROMLIST. So, it seems to be a bug in FS_FINDFIRST/NEXT, in FIL_QUERYEASFROMLISTL-related section. Need to look at the code.

[Valery V. Sedletski]

2Lars: Please test this version: ​ftp://osfree.org/upload/fat32/test/fat32-0.10-r248-os2.zip. (I added some new debug messages). I need the f32mon log again.

[Valery V. Sedletski]

Hm, ran CHKDSK on an empty FAT32 disk two times, it created two files, CHKDSK.LOG and CHKDSK.OLD. Opened it with the WPS, it created WP ROOT. SF. Now I have a situation like you have. But in my case, WPS calls DosFindFirst with FIL_STANDARDL, not FIL_QUERYEASFROMLISTL and I see no dead loop, every thing ok. So, the testcase is not reproduced. What did you do that it called FIL_QUERYEASFROMLISTL? I tried both, 1) open WPS folder by Ctrl-W in FC/2 2) Open the disk from Drives object. Your situation is still not reproduced.

PS: FIL_STANDARDL == 11 and FIL_QUERYEASFROMLISTL == 13

[Valery V. Sedletski]

Tried to open a FAT32 drive root directory in all four available views, including new xview (I have XWP 1.0.11, Dec 3, 2016). Everything is working fine here. Which version do you have and how are you trying to open it?

[erdmann]

I don't know what you are trying to tell me. Why don't you just write a simple test case that issues DosFindFirst/DosFindNext with FIL_QUERYEASFROMLISTL set ?

I have XWP 1.0.11. As far as WPS is concerned, I am opening drives from the drives object.

By the way: a long time ago I fixed an error with querying EAs. If a file has no EAs you still have to return the EA names from the GEA list in the FEA list (in other words: it's not enough to just do nothing.) I hope that code is still there but I think it is. Maybe it is not when /largefiles is in use ...

[erdmann]

I tried the version you suggested in comment #68. It completely hung my system, with or without the /largefiles switch.

[Valery V. Sedletski]

By the way: a long time ago I fixed an error with querying EAs. If a file has no EAs you still have to return the EA names from the GEA list in the FEA list (in other words: it's not enough to just do nothing.) I hope that code is still there but I think it is. Maybe it is not when /largefiles is in use ...

The code with FIL_QUERYEASFROMLISTL is almost the same as with FIL_QUERYEASFROMLIST. The difference is only in returned file size: it is 64-bit. Maybe, there is more differences -- need to research. Yes, the code should be there as I did not removed anything.

I tried the version you suggested in comment #68. It completely hung my system, with or without the /largefiles switch.

Where are the logs? "It completely hung my system" contains almost no information.

It would be better if you could help me to reproduce the problem on my machine.

Why don't you just write a simple test case that issues DosFindFirst/DosFindNext? with FIL_QUERYEASFROMLISTL set ?

That's also possible, but it's easier to get ready-to-use testcase, i.e., use some already written program, like some XWP use case.

Maybe, some other program calls DosFindFirst with FIL_QUERYEASFROMLISTL? Like XCommander/Larsen Commander/FM/2/etc?

[Valery V. Sedletski]

XCommander, DosNavigator/2, Connect/2, FM/2 work fine... Larsen Commander too.

[erdmann]

"It completely hung my system" means I could not use mouse nor keyboard.
Therefore I could also not save anything.

There is no other program that I used. I just accessed the drive object via the WPS (double click on the drive icon).

So, is Larsen Commander etc. using the FIL_QUERYEASFROMLISTL subcommand ? If not then your test is no proof.

I still think that writing a simple test case and using subcommand FIL_QUERYEASFROMLISTL to query the EAs for a couple of files is the easiest thing to do.

[erdmann]

in "http://trac.netlabs.org/fat32/browser/trunk/src/ifs/ifsea.c", routine "usGetEAS" I can see that you are handling FIL_QUERYEASFROMLISTL differently from what I had implemented for FIL_QUERYEASFROMLIST.

I would look at that place. Again: you HAVE to fill in FEA list with what GEA requests. If EAS do not exist, you have to return 0 for cbValue for that FEA and of course in this case you won't append any EA data in the return structure.

If "usReadEAS" returns an empty list (pSrcFeal) then you currently do nothing which is WRONG.

[erdmann]

I think you can just do this in "usGetEAS":

from:
   if (usLevel == FIL_QUERYEASFROMLIST)
change to:
   if ((usLevel == FIL_QUERYEASFROMLIST) || (usLevel == FIL_QUERYEASFROMLISTL)))

[erdmann]

In "http://trac.netlabs.org/fat32/browser/trunk/src/ifs/ifsfile.c", routine "FS_FILEINFO" you are not handling FIL_QUERYEASFROMLISTL (but you handle FIL_QUERYEASFROMLIST and also 4, whatever that is ...)

Either 4 is FIL_QUERYEASFROMLISTL (which is normally 13 ...) or I am missing something ...

Again, you should be following what is done for FIL_QUERYEASFROMLIST and call "usGetEmptyEAs" when EAS are disabled because of missing /EAS switch.

[Valery V. Sedletski]

"It completely hung my system" means I could not use mouse nor keyboard. Therefore I could also not save anything.

But for such cases, there is a COM port and a second machine. I suspect that a kernel developer should have the ones. If not this machine, then any other one. (also, I suspect that the network-attached ICAT is of a no help if your driver hanged or trapped the machine, so I see no alternative to a COM port. And it's cheap to buy a PCI, or PCI-e COM port board, if not integrated. I have a COM port even on my ThinkPad's docking station. Also, I can use any laptop under any OS as a terminal, as I have an USB2Serial adapter (Prolific pl2303-compatible), and I can attach a null-modem cable for it. I cannot type the kernel debugger commands, though, as I use a 9-to-25 pin null-modem cable, then 25-to-9 converter, and then an USB2Serial adapter, so TX signal is lost somewhere, and I cannot type commands. But I can receive COM port output, and this is sufficient. If I attach to a physical COM port, I can both TX and RX working, so the kernel debugger is working)

There is no other program that I used. I just accessed the drive object via the WPS (double click on the drive icon).

You use WPS for copying files? And don't have FC/2? This is very strange for me. As WPS is not very useful as a file manager and is not quite reliable for that. All advanced users I know use FC/2 or DN/2 or any similar program.

So, is Larsen Commander etc. using the FIL_QUERYEASFROMLISTL subcommand ? If not then your test is no proof.

I don't see other programs using FIL_QUERYEASFROMLISTL so far. My test is not a proof, but a constatation of a fact that I cannot reproduce your case so far.

I still think that writing a simple test case and using subcommand FIL_QUERYEASFROMLISTL to query the EAs for a couple of files is the easiest thing to do.

Probably, I'll do, as there is no other way remained, so, I'll write such a testcase. I just did not wrote r3 programs using EA's so far, So I'm not sure I'll quickly create a correct testcase, that's why I was looking for a ready to use testcase.

I think you can just do this in "usGetEAS":

from:
   if (usLevel == FIL_QUERYEASFROMLIST)
change to:
   if ((usLevel == FIL_QUERYEASFROMLIST) || (usLevel == FIL_QUERYEASFROMLISTL)))

Yes, already noticed that. And, I forgot to pass FIL_QUERYEASFROMLISTL to usGetEAS, I'm still passing FIL_QUERYEASFROMLIST to it. I'll look closer at these places, of course.

In "http://trac.netlabs.org/fat32/browser/trunk/src/ifs/ifsfile.c", routine "FS_FILEINFO" you are not handling FIL_QUERYEASFROMLISTL (but you handle FIL_QUERYEASFROMLIST and also 4, whatever that is ...)

Yes, indeed

Either 4 is FIL_QUERYEASFROMLISTL (which is normally 13 ...) or I am missing something ...

No, FIL_QUERYEASFROMLISTL is 13, but 4 exists too as a separate case. 4 was here before my changes, probably, since Henk's original version (or was it KO Myung Hun?)

Again, you should be following what is done for FIL_QUERYEASFROMLIST and call "usGetEmptyEAs" when EAS are disabled because of missing /EAS switch.

ok, will consider it too

[erdmann]

I am looking forward to a new version. I am fairly confident that with the above changes, everything will work. As to "4", my gut feeling is that very early kernels used this value for FIL_QUERYEASFROMLISTL, before it was changed to 13. My gut feeling is that this value is not used anymore. But that would require some testing ...

I can also build the driver myself. But for that I need file "version.h" which is not in the SVN repo. I assume you create that file on-the-fly. Could you send me a usable version of that file and tell me where to place it in the source tree ?

Lars

[Valery V. Sedletski]

Ok, I'll try to fix the above issues in usGetEAS, FS_FILEINFO etc, then will upload the fixed version. The ver.h file is generated automatically by the build system. (Please look here for the hints for building the 0.10 sources: http://trac.netlabs.org/fat32/wiki/BuildingFAT32-0.10). It should be like this:

#define FAT32_VERSION "0.10.r248"

(the file is created as include\ver.h)

[erdmann]

Found another one:

"http://trac.netlabs.org/fat32/browser/trunk/src/ifs/ifsattr.c", routine "FS_PATHINFO"

Same applies as to "FS_FILEINFO".

Lars

[erdmann]

Ok, found value "4" here:

http://trac.netlabs.org/fat32/browser/trunk/src/include/fat32def.h:

#define FIL_QUERYALLEAS 4

I think it would be a good idea to use the define name in the code
(in "FS_PATHINFO" and "FS_FILEINFO").
Then it becomes clear what that subcommand is supposed to do ...

About "usGetEAs": you have to be careful that usLevel = FIL_QUERYALLEAS needs a different behaviour as levels FIL_QUERYEASFROMLIST and FIL_QUERYEASFROMLISTL !

That's because for FIL_QUERYALLEAS the GEA list is not set as no specific EAs are requested. And therefore "usGetEAs" needs to follow the "else" branch ...

[Valery V. Sedletski]

Added missing edits regarding FIL_QUERYEASFROMLIST/FIL_QUERYEASFROMLISTL/FIL_QUERYALLEAS. But it looks that FIL_QUERYEASFROMLIST and FIL_QUERYEASFROMLISTL processing is the same and passing FIL_QUERYEASFROMLISTL instead of FIL_QUERYEASFROMLIST to usGetEAS changes nothing. So, I doubt that something will better than the previous build. I uploaded r249 at usual place and committed the sources to the repository. Please, look at my changes, if I overlooked something. Also, this commit contains some other changes, like FS_MOUNT enhancements regarding the Volume Serial Number and Volume label, and also, as an experiment, it changes the FS name to UNIFAT, in case /fat or /exfat are set. This is to avoid confusion when FAT16/FAT12/exFAT drives are reported as being FAT32.

[erdmann]

fat32-0.10-r249-os2.wpi:

still does not work. There is still an error in ifsattr.c, "FS_PATHINFO":

around line 288 need to be:

case FIL_QUERYEASFROMLIST:
case FIL_QUERYEASFROMLISTL:
case FIL_QUERYALLEAS:

As to your analysis: there IS a difference in updating usGetEAS even though processing is the same in usGetEAS. The places from where usGetEAS is called also includes calls from FIL_QUERYALLEAS which requires a different processing.

I wonder if the GEA or FEA list are DIFFERENT for FIL_QUERYEASFROMLIST and FIL_QUERYEASFROMLISTL. You would need to debug the GEA list with the kernel debugger and check its internal structure.

By the way: why are all entry points marked with "_loadds" and then use "_asm push es" at the beginning and "_asm pop es" at the end ? I would at least refrain from saving and restoring the ES register. The ES register will be loaded when it is needed. It never needs to be preserved (interrupt and timer routines being the only 2 exceptions).

I'll try and enable the monitor.

[erdmann]

FIL_QUERYEASFROMLIST vs. FIL_QUERYEASFROMLISTL:

I had a look into the DDK file "ddk\base\h\bsedos.h":

typedef struct _GEA {
   BYTE cbName;
   CHAR szName[1];
   // CHAR szName[cbName] follows implicitely, exclude zero terminator in length
} GEA;

typedef struct _FEA {
   BYTE fEA;
   BYTE cbName;
   USHORT cbValue;
   // CHAR szName[cbName+1] follows implicitely, include zero terminator in length
   // CHAR szValue[cbValue] follows implicitely
} FEA;

typedef struct _GEA2 {
   ULONG oNextEntryOffset;
   BYTE  cbName;
   CHAR  szName[1];
   // CHAR szName[cbName] follows implicitely, exclude zero terminator in length
} GEA2;

typedef struct _FEA2 {
   ULONG oNextEntryOffset;
   BYTE  fEA;
   BYTE  cbName;
   USHORT cbValue;
   CHAR   szName[1];
   // CHAR szName[cbName] follows implicitely, exclude zero terminator in length
   // CHAR szValue[cbValue] follows implicitely
}

I think it is possible that for FIL_QUERYEASFROMLISTL you will need to use the FEA2 and GEA2 structures instead of using FEA and GEA structures.
All of these structures are variable length structures that are tightly packed in the corresponding lists and work like this for FEA and GEA:

gea1.cbName = 6; // length without zero terminator
gea1.szName[7] = {".TITLE"}
gea2.cbName = 8;
gea2.szName[9] = {".SUBJECT"};
etc.

returns:
fea1.fEA    = 0x00; // non critical EA
fea1.cbName = 6;
fea1.cbValue = 20; // example value, is variable
fea1.szName[7] = {".TITLE"};
fea1.szValue[20] = {20 bytes of string EA, title}
fea2.fEA    = 0x00; // non critical EA
fea2.cbName = 8;
fea2.cbValue = 27; // example value, is variable
fea2.szName[8] = {".SUBJECT"};
fea2.szValue[27] = {27 bytes of string EA, subject}
etc.

GEA2 and FEA2 are not that much different, however they have the "oNextEntryOffset" field added and FEA2 now explicitely lists the "szName" field. Maybe these structures are used for FILE_QUERYEASFROMLISTL. The "oNextEntryOffset" field gives the byte offset from one EA to the next. In the above example:

gea1.oNextEntryOffset = sizeof(GEA2) + gea1.cbName;
gea2.oNextEntryOffset = // offset to next GEA2 entry or 0 if last GEA2 in list

likewise:

fea1.oNextEntryOffset = sizeof(FEA2) + fea1.cbName + fea1.cbValue;
fea2.oNextEntryOffset = // offset to next FEA2 entry of 0 if last FEA2 in list

That would then also apply to routine "usGetEmptyEAS".

I hope, any EA names are still ASCII (1 byte per character) and not UNICODE, correct ?

[erdmann]

I fear that for FS_FINDFIRST/FS_FINDNEXT things are much worse and complicated:
see the "Control Programming Guide and Reference" and also the "Programming Guide and Reference Addendum" if Level is 2,12,3 or 13 for routines "DosFindFirst" and "DosFindNext".

In short: it's not only an FEA list that is returned. Instead you need to intermingle file information in FILEFINDBUF4,FILEFINDBUF4L,FILEFINDBUF3,FILEFINDBUF3L structures with FEA entries (I think I was wrong about the FSD needing to support GEA2 or FEA2: this is converted to/from GEA/FEA transparently by the kernel).

Also need to read info for "DosQueryFileInfo" (FS_FILEINFO) and "DosQueryPathInfo" (FS_PATHINFO) to really understand what needs to be returned in what format.

The IFS specification is just an incomplete mess ...

[Valery V. Sedletski]

There is still an error in ifsattr.c, "FS_PATHINFO":

around line 288 need to be:

case FIL_QUERYEASFROMLIST:
case FIL_QUERYEASFROMLISTL:
case FIL_QUERYALLEAS:

Yes, right. Added.

I hope, any EA names are still ASCII (1 byte per character) and not UNICODE, correct ?

No, not Unicode. Why could they be Unicode? I did not changed the EA processing for FAT12/16/32. In exFAT it is almost the same, except for the EA mark byte, which has different offset into the directory entry. And no EA's on exFAT on windows at all, So I had nothing to copy from windows.

I fear that for FS_FINDFIRST/FS_FINDNEXT things are much worse and complicated: see the "Control Programming Guide and Reference" and also the "Programming Guide and Reference Addendum" if Level is 2,12,3 or 13 for routines "DosFindFirst?" and "DosFindNext?".

Thanks for the hints. I also think, I need to look into JFS sources in the 1st place and look how they did that there. And I still need a concrete place where it fails. And fix this place specifically. So, I need logs. Also, I'll need to understand how EA's returning works in cpref.inf. FEA's, GEA's -- I still have no clue about them. Did not studied how EA's work yet. Just left the previous EA processing intact.

[Valery V. Sedletski]

By the way: why are all entry points marked with "_loadds" and then use "_asm push es" at the beginning and "_asm pop es" at the end ? I would at least refrain from saving and restoring the ES register. The ES register will be loaded when it is needed. It never needs to be preserved (interrupt and timer routines being the only 2 exceptions).

Because DS and ES registers need to be preserved. By default, they are not. I needed to add push es/pop es too, because it did trapped previously when the kernel restored ES.

I'll try and enable the monitor.

Yes. But monitor is useless if the system is hanged or trapped. The log will have zero size. So, you still need a COM port for that.

[Valery V. Sedletski]

The oNextEntryOffset field is shown in DosFindFirst/Next parameters, but it is removed by kernel and IFS does not see it in the received structures. I think, it is the same with FEA/GEA vs. FEA2/GEA2 structures.

[erdmann]

I am beginning to understand what goes wrong:

for FS_FINDFIRST/FS_FINDNEXT there is a remark in the Control Prog Guide and Ref" that if you return EAs (via FIL_QUERYEASFROMLIST / FIL_QUERYEASFROMLISTL) then then the last two structure elements are NOT used (cchName and achName) or something like that, I'll need to reread the section ...

Then there is some confusion about the used structures because there should be a match FILEFNDBUF3 / FILEFNDBUF3L and FILEFNDBUF4 / FILEFNDBUF4L but instead there is a match FILEFINDBUF2 / FILEFNDBUF4L in the code, look at Routine "FillDirEntry0" ...

And then FILEFNDBUF3 / FILEFNDBUF3L are not logically equivalent, they should only deviate in the length of the "cbFile" and "cbFileAlloc" and "attrFile" fields, same should then hold true for FILDFNDBUF4 / FILEFNDBUF4L.

It's easy to write a test case to trigger the problem. I'll write one with DosFindFirst / DosFindNext use and levels FIL_QUERYEASFROMLIST / FIL_QUERYEASFROMLISTL set. I can then apply this test case against a JFS Drive (subdirectory) and see what it returns exactly. Then I can apply the same test case against a FAT32 drive (subdirectory) and see what it returns. Then I can compare ...

By the way: the IFS spec clearly states that you HAVE TO support EAs in OS/2. If your filesystem has no EAs then you have to FAKE the responses correspondingly. Therefore you will also need to do that for the exFAT filesystem.

[erdmann]

Replying to valerius:

The oNextEntryOffset field is shown in DosFindFirst/Next parameters, but it is removed by kernel and IFS does not see it in the received structures. I think, it is the same with FEA/GEA vs. FEA2/GEA2 structures.

Yes, that is most likely correct and I was hoping that it would be that way.
At least I never bothered about FEA2 and GEA2 in the initial EA implementation.
I think IBM enhanced the structures for applications from OS/2 1.3 to OS/2 2.x so that the kernel can faster iterate through the EA list internally. Internally it changes the structure on calling the IFS/returning from the IFS to match the initial format. But then IBM added user code requirements for DosFindFirst/DosFindNext to align GEAs on a 4-byte boundary ...

[erdmann]

I have a test application in place that uses FIL_QUERYEASFROMLIST and also FIL_QUERYEASFROMLISTL on a DosFindFirst call.

That should allow to figure out what is wrong with FAT32 and FIL_QUERYEASFROMLISTL.

I can send that to you (including the source code). It allows to specify wildcards from the commandline. It's hardcoded to request the .TITLE and .LONGNAME EAs.

Lars

[Valery V. Sedletski]

2Lars: Yes, please send it to me. I created a test with DosQueryPathInfo too, but something is wrong with it. Can you look at it? It is like this:

#define  INCL_BASE
#include <os2.h>

#include <stdio.h>
#include <string.h>

int main(void)
{
    EAOP2 eaop2;
    GEA2LIST  gea2list;
    char  szEAName[12] = {0};
    FEA2LIST  fea2list;
    char  buf[256] = {0};
    APIRET rc;

    memset(&eaop2, 0, sizeof(EAOP2));
    memset(&gea2list, 0, sizeof(GEA2LIST));
    memset(&fea2list, 0, sizeof(FEA2LIST));

    gea2list.cbList = sizeof(GEA2LIST) + sizeof(szEAName);
    gea2list.list[0].oNextEntryOffset = 0;
    strcpy(gea2list.list[0].szName, ".SUBJECT");
    gea2list.list[0].cbName = strlen(gea2list.list[0].szName);

    eaop2.fpGEA2List = &gea2list;

    fea2list.cbList = sizeof(FEA2LIST) + sizeof(buf);

    eaop2.fpFEA2List = &fea2list;

    rc = DosQueryPathInfo("z:\\dll.txt", FIL_QUERYEASFROMLIST,
                          &eaop2, sizeof(EAOP2));

    if (rc)
    {
        printf("Error in DosQueryPathInfo, rc=%lu\n", rc);
        return rc;
    }

    printf("%s\n", fea2list.list[0].szName + fea2list.list[0].cbName + 1);
    return 0;
}

Now it returns ERROR_BAD_FORMAT == 11. It tries to get the .SUBJECT EA from a file.

[erdmann]

Find attached.

I now have the final proof: run this test prog against a FAT32 drive WITHOUT the /largefiles switch active. The test prog will successfully execute.

Now run this very same prog against a FAT32 drive WITH the /largefile switch active. The test prog will trap on execution of FIL_QUERYEASFROMLISTL (but work fine for FIL_QUERYEASFROMLIST). Execution under a debugger reveals that the embedded FEA2LIST structure (it's after the FILEFINDBUF3L structure truncated by the last 2 FILEFINDBUF3L structure elements and followed by the last 2 FILEFINDBUF3L structure elements) is broken which either means the field lengths of the preceding FILEFINDBUF3L structure are incorrect or the FEA2LIST is missing or there is some alignment issue.

If I would have to guess I would think that the FSD internally used FILEFNDBUF3L structure needs the "attrFile" field to be a USHORT rather than a ULONG (same for FILEFNDBUF4L of course). But that will now be easy to "track back" in the debugger.

Note: the problem is in DosFindFirst/DosFindNext. It's well possible that DosQueryPathInfo works perfectly well !

Call the test prog like this: testea c:\*.* (if C: is a FAT32 drive).

[erdmann]

As to your code Fragment:

1) the doc for "DosQueryPathInfo" says that each GEA2 element must be DWORD aligned.I don't know if that is really true. I don't think so.
2) the FEA2List and GEA2List are TIGHTLY PACKED. That means you cannot allocate a separate buffer for the list and the FEA2 / GEA2. Likewise the FEA2LIST / GEA2LIST structure does not give you room for a complete FEA2/GEA2 structure. That's because the EA Name has to DIRECTLY follow the FEA2/GEA2 structure and it is of variable length, in Addition for FEA2 the value of the EA follows the EA Name:

char gea2list[512];
char fea2list[4096];

eaop2.fpGEA2List = (PGEA2LIST)gea2list;
eaop2.fpFEA2List = (PFEA2LIST)fea2list;

strcpy(eaop2.fpGEA2List->list[0].szName,".SUBJECT");
eaop2.fpGEA2List->list[0].cbName = strlen(".SUBJECT");
eaop2.fpGEA2List->cbList = sizeof(eaop2.fpGEA2List->cbList) + sizeof(GEA2) + eaop2.fpGEA2List->list[0].cbName;

// you can go from one GEA2 to the next:
gea2 = eaop2.fpGEA2List->list;
// go to next GEA2 structure
gea2 = (PGEA2)((PUCHAR)gea2 + gea2->oNextEntryOffset);

// analogous for FEA2 ...
fea2 = eaop2.fpFEA2List->list;
// go to next FEA2 structure
fea2 = (PFEA2)((PUCHAR)fea2 + fea2->oNextEntryOffset);

[erdmann]

You should also thoroughly read this, to be found right here:

http://trac.netlabs.org/fat32/browser/trunk/src/ifsinf/addenda

It gives some very valuable clarification of how FS_FINDFIRST and FS_FINDNEXT are REALLY supposed to work for the various Levels.

[erdmann]

Ok, I now checked:

the corruption happens beginning with field "cbFileAlloc" in structure FILEFINDBUF3L (that is: field "cbFile" is still OK).
That leads me to believe that you need to change the FSD internal structures FILEFNDBUF3L and also FILEFNDBUF4L and make field "attrFile" (which follows the "cbFileAlloc" field) a USHORT instead of a ULONG:
(http://trac.netlabs.org/fat32/browser/trunk/src/include/fat32ifs.h)

292	typedef struct _FILEFNDBUF3L {
293	    FDATE    fdateCreation;
294	    FTIME    ftimeCreation;
295	    FDATE    fdateLastAccess;
296	    FTIME    ftimeLastAccess;
297	    FDATE    fdateLastWrite;
298	    FTIME    ftimeLastWrite;
299	    LONGLONG cbFile;
300	    LONGLONG cbFileAlloc;
301	    USHORT   attrFile;     <------ change here
302	    UCHAR    cchName;
303	    CHAR     achName[CCHMAXPATHCOMP];
304	} FILEFNDBUF3L;

308	typedef struct _FILEFNDBUF4L {
309	    FDATE    fdateCreation;
310	    FTIME    ftimeCreation;
311	    FDATE    fdateLastAccess;
312	    FTIME    ftimeLastAccess;
313	    FDATE    fdateLastWrite;
314	    FTIME    ftimeLastWrite;
315	    LONGLONG cbFile;
316	    LONGLONG cbFileAlloc;
317	    USHORT   attrFile;     <------ change here
318	    ULONG    cbList;
319	    UCHAR    cchName;
320	    CHAR     achName[CCHMAXPATHCOMP];
321	} FILEFNDBUF4L;

If that works out then with a very high likelyhood you will also need to change structures "FILESTATUS3L" and "FILESTATUS4L" for the very same variable "attrFile"
(http://trac.netlabs.org/fat32/browser/trunk/src/include/fat32def.h)

Can you build a new test version with these changes ?

Lars

[erdmann]

Updated, fixed "cbList" computation

[erdmann]

I don't think it's the problem listed above, at least not for FILESTATUS3L and FILESTATUS4L. I am going to look into "FillDirEntry0" routine. For the time being, I am not going to touch "FillDirEntry1" (exFAT specific version).

[erdmann]

I fixed the problem, see my private email.

Lars

[erdmann]

Handling of FIL_QUERYEASFROMLISTL level for FS_FINDFIRST, FS_FINDNEXT, FS_FINDFROMNAME has been fixed. The problems of displaying files in the WPS when /largefiles switch is set is now gone.