Finish fat32format port from Win32 to OS/2 and fix some fat32.ifs traps caused by this port

[Valery V. Sedletski]

Now fat32format is mostly ported to OS/2 (now not yet finished, present in 0.10 branch of fat32.ifs). Also, fat32.ifs is modified accordingly (added missing MOUNT_ACCEPT and modified MOUNT_VOL_RELEASED, MOUNT_RELEASED flags for FS_MOUNT, etc.). FORMAT entry is added to ufat32.dll, and "format x: /fs:fat32" is mostly working. But when OS/2 booted from fat32 (OS/2 bootable flash), and trying to format another partition to fat32, a trap occurs.

[abwillis]

I just updated my svn to 101 and built it. I am booted from JFS HDD and find that formating fat32 causes a trap d. The format was successful even on a 160G drive (149G free after format which is consistent with when it is formated on windows). I also found that when I booted with the new fat32 files in place that I got a trap d on boot in dumpfs.ifs at the point that the desktop should have been about to come up. I ended up just remarking out the dumpfs so that I could boot and test the new fat32. dumpfs did not have any issues with the previous fat32.

[Lewis Rosenthal]

Just to add that we would like to have an updated FAT32 with format capability available to ship with Blue Lion. If there's anything Arca Noae can do to help, Valerius, please advise.

[erdmann]

I have now tried this: ​ftp://osfree.org/upload/fat32/fat32-0.10-alpha2.zip

When running "format" from a commandline with this version, the format completely hangs the system on the last step (seem to remember it says something about writing some structure or so). I have JFS as my boot partition. I tried to format a partitioned USB stick.

[Lewis Rosenthal]

I have not seen this, yet, Lars.

What shell were you running? I typically use 4OS2. My UFAT32.DLL is currently:

3-21-15 20:02 87,617 124 UFAT32.DLL

which is I believe the same one in the package you referenced, above. What are your FAT32-related statements in CONFIG.SYS?

[Lewis Rosenthal]

I should add that my own statements are now the same as Valerius discussed in ticket #18.

[erdmann]

I just deduced from ticket #18 that I need to specify the /monitor switch for fat32.ifs which I had forgotten ...

Will retest and report results.

[erdmann]

Specifying the /monitor swith on the FAT32.IFS config.sys line just hangs my system when CACHEF32.EXE loads.
As described in ticket #18 I have been using this:
IFS=D:\OS2\BOOT\FAT32.IFS /h /monitor
CALL=D:\OS2\CACHEF32.EXE /l:off /p:2 /m:50000 /b:250 /d:5000

I have also double checked that the FAT32 version 9.13 from Ko Myung-Hun that I am normally using is not getting in the way.

[Lewis Rosenthal]

Lars, to be clear, do you:

1. See this on only one stick?
2. See this on unpartitioned media, as well?
3. Have any trouble formatting the same stick with HPFS?
4. Have any trouble formatting the same stick with FAT16?
5. Have any ability to try this with non-USB media (hard drive partition, perhaps)?

I'm trying to figure out what's different between us.

FWIW, I am using the 11.14 Arca Noae stack, with

 6-21-16   7:52          50,972    124  usbmsd.add

Signature:       @#Lars Erdmann:10.206#@OS/2 USB MSD Class Adapter Driver
Vendor:          Lars Erdmann
Revision:        10.206
File Version:    10.206
Description:     OS/2 USB MSD Class Adapter Driver

Testing again, right now...

[Lewis Rosenthal]

Hmmm... Attempting to reformat a FAT16 partitioned stick (60MB - really, MB), I get:

The new type of file system is FAT32.
The disk has no volume label.
Warning!  All data on hard disk n: will be lost!
Proceed with FORMAT (Y/N)? y
ERROR: Failed to get device geometry
SYS0528: The specified disk did not finish formatting.
SYS0026: The specified disk or diskette cannot be accessed.

EXPLANATION: The disk or diskette is not properly formatted
for the operating system.

ACTION: Format the disk or diskette for the operating system.

Error code: 26

In all fairness, DFSee did warn that the geo was off. Fixing that... Wiped the stick, wiped the partition table, created new 0x0B partition, ejected stick, reinserted, and still I get the same error on this one.

I used DFSee to create a working FAT32 format. Once that was done:

[c:\var\log]fat32fmt n:
The new type of file system is FAT32.
Enter the current volume label for drive n: DFSeeFAT32
Warning!  All data on hard disk n: will be lost!
Proceed with FORMAT (Y/N)? y
Enter up to 11 characters for the volume label
or press Enter for no volume label. FAT32IFS
Size: 0 MB 112392 sectors
512 Bytes Per Sector, Cluster size 512 bytes
Volume Serial No. is 171b:292d
Volume label is FAT32IFS
32 Reserved Sectors, 865 Sectors per FAT, 2 fats
110630 Total clusters
110629 Free Clusters
Formatting drive n:
100% percent of disk formatted ...
Wrote 902656 bytes in 4.60 seconds, 0.19 Megabytes/sec
Clearing out 1763 sectors for
Reserved sectors, fats and root cluster...
Initialising reserved sectors and FATs...
Done.

I returned to the prompt. I'll need to wipe this stick and damage it a bit to see if I can get a decent FAT32 format out of it when the data is messy on it. However, I do not get the hang you describe, Lars.

Can you try wiping the disk with DFSee first? I'm thinking that we're just not handling unexpected conditions all that well (just a WAG).

[erdmann]

1) when I do not specify /monitor for FAT32.IFS I get the hang on the formatting attempt (as already mentioned by Valerius in ticket #18). In this case I get the hang when the "Initialising reserved sectors and FATs..." message appears. I assume this is the point in time when the REDETERMINE MEDIA ioctl is issued.
2) when I do specify /monitor for FAT32.IFS I already get the hang on booting up the system (in the instance where CACHEF32.EXE loads).

So, in the second case, wiping the stick with DFSee will make no difference as I cannot boot my system.
I have no problems whatsoever formatting a partition on the stick with FAT32 by using DFSee (which I did now).
I also have no problems formatting any stick with HPFS or JFS or FAT16 (all of which I did multiple times when I worked on USBMSD.ADD).
I have not yet tried with a non-removable partition.
I also have not yet tried with "large floppy" USB media.

[erdmann]

A couple of additions to the attempt to format a 60 MB as FAT32:

the MS spec mandates that a partition has to have >= 65525 clusters in order to be formatted as FAT32. Strictly speaking, a partition that does not meet this requirement cannot be a FAT32 partition and MS OSes will refuse to mount such partition.

With a cluster size of 512 bytes (which is the minimum) you would end up with a minimum of about 30 MB which your 60 MB partition size would easily meet. HOWEVER, if the cluster size is allocated to a bigger value (say 1024 bytes) for whatever reason (error in the formatting code) then the limit would increase to > 60 MB and then a 60 MB partition would not meet the requirements.

I guess DFSee is doing the right thing and computing everything correctly (cluster size etc.) whereas the FAT32 formatting code does not. Maybe the FAT32 formatting code reuses a wrong cluster size from the BPB if it finds an already existing FAT16/FAT32 BPB but I don't know. The latter would explain why you can format correctly with FAT32 after wiping the first sectors to zero.

There is absolutely no relation to what the disk driver does.

[Lewis Rosenthal]

Okay, so I wiped the stick completely with DFSee, exited DFSee, ejected, and then re-inserted the stick.

Starting DFSee again and selecting the stick as the object to examine, I see:

DFSee OS/2   13.3 : Executing: disk 5                                          
                                                                               
 Disk 5 L-Geo from: Device info, empty or garbage sectors!                     
 L-Geo Disk 5 Cyl :       7 H:255 S:63  Bps:512   Size:0x0001B747 =    54.9 MiB

which seems entirely normal to me.

Without a partition on it, using:

6-21-16 7:52 50,972 124 usbmsd.add

and attempting a format, I get:

The new type of file system is FAT32.
The disk has no volume label.
Warning!  All data on hard disk g: will be lost!
Proceed with FORMAT (Y/N)? y
ERROR: Failed to get device geometry
SYS0528: The specified disk did not finish formatting.
SYS0026: The specified disk or diskette cannot be accessed.

EXPLANATION: The disk or diskette is not properly formatted
for the operating system.

ACTION: Format the disk or diskette for the operating system.

Error code: 26

which I guess isn't too unexpected. Adding a partition (and LVM) via minilvm (deleting the fake one, first), ejecting, and re-inserting, gets me:

[c:\]fat32fmt n:
The new type of file system is FAT32.
The disk has no volume label.
Warning!  All data on hard disk n: will be lost!
Proceed with FORMAT (Y/N)? y
Enter up to 11 characters for the volume label
or press Enter for no volume label. FAT32
Size: 0 MB 112392 sectors
512 Bytes Per Sector, Cluster size 512 bytes
Volume Serial No. is 16ed:312c
Volume label is FAT32
32 Reserved Sectors, 865 Sectors per FAT, 2 fats
110630 Total clusters
110629 Free Clusters
Formatting drive n:
100% percent of disk formatted ...
Wrote 902656 bytes in 4.26 seconds, 0.20 Megabytes/sec
Clearing out 1763 sectors for
Reserved sectors, fats and root cluster...
Initialising reserved sectors and FATs...
Done.

No problem.

I think in your case, we should look at what is causing you the difficulty with CACHEF32 and the /monitor option for the IFS, first, Lars. There is obviously something different between us, as I have no problem using Valerius' CONFIG.SYS settings. FWIW, on my system, CACHEF32.EXE is:

3-18-15 19:11 42,850 124 CACHEF32.EXE

and FAT32.IFS is:

3-21-15 18:16 203,440 124 fat32.ifs

[erdmann]

I had been using all of ​ftp://osfree.org/upload/fat32/fat32-0.10-alpha2.zip.

I will need to check the file dates. I seem to remember all of the files had the same file date. Do you have a newer fat32.ifs by any chance (newer than what's included in the zip stated above) ?

[Lewis Rosenthal]

I just did a binary compare on all of the files I have, and they are the same as the ones in that package. I am not using the compressed version of the IFS, of course.

[erdmann]

I retried with the above zip:
1) if I set /monitor for FAT32.IFS but comment out CACHEF32.EXE from config.sys the system still hangs. In this case it hangs at loading NETBIND.EXE but that's just two RUN statements later from the original CACHEF32.EXE CALL statement (in config.sys).

2) if I remove /monitor from FAT32.IFS, the system will boot (with or without also loading CACHEF32.EXE). However, if I then run "monitor.exe" from the \OS2 directory (just as a test), the system hangs hard. At that point I can only turn it off.

3) I remmed a couple of IFS statements (in my case: DUMPFS.IFS and NDFS.IFS) in the hope it would make a difference but it did not.

Funny enough, if i now DO NOT specify the /monitor switch for FAT32.IFS I am now finally able to format a partition and it also won't hang. Is it now just the opposite: NOT specifying the /monitor switch will work ok whereas specifying the /monitor switch will lead to the RETERMINE MEDIA ioctl hang problem ?

[Valery V. Sedletski]

2Lewis: so yes, wiping the partition, ejecting/inserting, creating it again and then formatting helps? I always do ejecting/reinserting after wiping, so that it correctly invalidated the old BPB, then partition it and format. And partition size matters here too. I didn't tried to format partitions lesser, than 800 MB. 60 MB may be below 65535 clusters limit.

2Lars: can say nothing -- very strange -- as I tried, formatting without /monitor always gives me a trap 000d at the time REDETERMINEMEDIA ioctl is done. It can look like a hang if a debug kernel is installed. BTW, have you a debug cable? Maybe, you can see something useful in the trap screen, if you attach it? Dumpfs and NDFS should not influence if it does trap or not, so it is funny. So, you disabled just dumpfs and ndfs and all?

PS: /monitor adds not only the possibility to use f32mon.exe utility for displaying the debug messages, but repeats them on the COM port too, so you can see some additional messages here.

[Valery V. Sedletski]

PPS: Lars, your 3) problem seems to be the same as in ticket #17 -- people noticed traps in dumpfs.ifs when booting with new fat32.ifs. It is another problem and should be fixed too. But I haven't dumpfs partitions and have no place for creating them ATM. :( You could try also changing the order of IFS'es dumpfs.ifs and fat32.ifs -- maybe, this should help.

[Valery V. Sedletski]

Now uploaded the new build of fat32.ifs, version 0.10a4: ​ftp://osfree.org/upload/fat32/fat32-0.10-alpha4.zip. Fixed fat32.ifs trap at completing the format (FS_MOUNT deallocates the selector, and it is tried to restore by "pop es" after exiting FS_MOUNT, hence, trap 000d). Now this is gone, and it does not trap regardless of specifying "/monitor" in the command line.

PS: To people, having dumpfs.ifs installed: please check if the trap on boot still exists, when "/monitor" is specified. This is somehow related to specifying the "/monitor" switch, so it needs to be checked, if it was fixed too.

[Valery V. Sedletski]

PPS: Just installed dumpfs.ifs on my machine -- I see no trap. But I have no dumpfs partitions.

[Valery V. Sedletski]

Uploaded the new version of fat32.ifs: ​ftp://osfree.org/upload/fat32/fat32-0.10-alpha5.zip. Fixed a trap in FS_DELETE (a side effect of format bigfix) when disk is dirty and we're trying to overwrite files. Also, new is CHKDSK code -- I got rid of DosFSCtl/DosDevIOCtl calls to fat32.ifs from CHKDSK, they are changed to pieces of code, ported from fat32.ifs which implement these calls locally. So, now we can start CHKDSK, when IFS is not yet initted. Also, in this build, I added /autocheck: or /ac: parameter to fat32.ifs. If it is present, CHKDSK is called on init for specified disks, and they are checked. So, now CHKDSK can be called on IFS init, instead of cachef32.exe. This can be used when booting from fat32. Also, I modified cachef32.exe to not call CHKDSK.

Note: a problem is present in CHKDSK code, so it does not fix errors when /f is specified -- I'm investigating the problem.

[Valery V. Sedletski]

UPD: Reuploaded ​ftp://osfree.org/upload/fat32/fat32-0.10-alpha5.zip under the same name: Add making disk clean and issuing REDETERMINEMEDIA ioctl to take new disk status in effect. Now CHKDSK marks volume clean.

[diver]

is this now fixed or what's the exact state?

[Valery V. Sedletski]

Fat32Format is mostly working. I don't see any traps, at least. But these are some minor problems. It fails to set the volume label sometimes. But for the next attempt, it succeeds, for example.

[Valery V. Sedletski]

The bug with failing to set the volume label (this was because the disk was corrupted when trying to reformat the fat32-formatted drive again) is now fixed. The fixed version can be downloaded at ​ftp://osfree.org/upload/fat32/fat32-0.10-alpha5.zip. It fixes also several others problems.

[Valery V. Sedletski]

PS: The binaries are now updated to ​ftp://osfree.org/upload/fat32/fat32-0.10-alpha6.zip

[Valery V. Sedletski]

Updated the binaries again, the link is at http://trac.netlabs.org/fat32/ticket/24#comment:11

[Valery V. Sedletski]

The format code seems to not cause ring0 traps anymore. So, closing it.