Opened 15 years ago
Closed 4 years ago
#9 closed defect (fixed)
Corrupt written data on FAT32 partitions while downloading files
| Reported by: | M. Seidel | Owned by: | somebody |
|---|---|---|---|
| Priority: | Feedback Pending | Milestone: | Future |
| Component: | IFS | Version: | |
| Severity: | highest | Keywords: | |
| Cc: | KO Myung-Hun |
Description
Hi, I use ECS1.2R and XP on the same machine. Only ECS has access to the www, so files I need for XP are only downloaded with ECS and stored with the installed fat32.ifs directly at the fat32 drives accessible under both systems.
Since longer time and trying different versions of fat32 (including 0.9.13 latest) I wondered that the downloaded files under XP are corrupt. Selfexpanding exe's are stopping with a CRC error or zip files cannot be expand.
Under ECS I use different versions of Seamonkey (latest 1.1.13)
On my formerly PC I had sometimes the same problems.
I have done some experiments: Downloading the file: http://startdownload.filefront.com/4286905//6fe471f91cdc109738f8eac44a258057618d7941c6ca8a291bbc33ef2fe926a0a3cdf95c098932f4// (should be project_51.zip ~130MB) direct to a fat32 partition
and a 2nd download to a JFS patition
results in the following:
Download to FAT32 drive: J:\>md5suml project_51.zip 9935917E3288FF884E9B595280BBBCCD project_51.zip filesize 131127388 Bytes
Archive: J:/project_51.zip
inflating: pro51.PK3 error: invalid compressed data to inflate inflating: readme.txt
ERROR
Download to JFS drive: F:\>md5suml project_51.zip 6D6A29E52F1055886A4271E53C3FB968 project_51.zip filesize 131127388 Bytes
Archive: F:/project_51.zip
inflating: pro51.PK3 inflating: readme.txt
OK
So I have written a little file comparison tool. It compares the good and the wrong file and logs all issues. See the attached files. Each line shows the adress, databyte good, databyte bad, total error count and the nummer of continued errors in follow. Please watch the follows carefully!
If I copy such larger files from HPFS or JFS to a FAT32 drive with a filecommander, like FC220, no problem occurs. With my 2Mbit www connection the files are written only slow while downloading them. May be a small buffer allocation problem in the IFS or cachef32.exe?
Attachments (4)
Change History (14)
Changed 15 years ago by
| Attachment: | datvergl.txt added |
|---|
Changed 15 years ago by
| Attachment: | DATVERGL.PAS added |
|---|
comment:1 Changed 15 years ago by
| Cc: | KO Myung-Hun added |
|---|
Hi/2.
What versions of FAT32 were you using ?
And are you using lazy-write cache ? If so, how about without lazy-write cache ? Still corrupted ?
KO Myung-Hun
Replying to mst:
Hi, I use ECS1.2R and XP on the same machine. Only ECS has access to the www, so files I need for XP are only downloaded with ECS and stored with the installed fat32.ifs directly at the fat32 drives accessible under both systems.
Since longer time and trying different versions of fat32 (including 0.9.13 latest) I wondered that the downloaded files under XP are corrupt. Selfexpanding exe's are stopping with a CRC error or zip files cannot be expand.
Under ECS I use different versions of Seamonkey (latest 1.1.13)
On my formerly PC I had sometimes the same problems.
I have done some experiments: Downloading the file: http://startdownload.filefront.com/4286905//6fe471f91cdc109738f8eac44a258057618d7941c6ca8a291bbc33ef2fe926a0a3cdf95c098932f4// (should be project_51.zip ~130MB) direct to a fat32 partition
and a 2nd download to a JFS patition
results in the following:
Download to FAT32 drive: J:\>md5suml project_51.zip 9935917E3288FF884E9B595280BBBCCD project_51.zip filesize 131127388 Bytes
Archive: J:/project_51.zip
inflating: pro51.PK3 error: invalid compressed data to inflate inflating: readme.txt
ERROR
Download to JFS drive: F:\>md5suml project_51.zip 6D6A29E52F1055886A4271E53C3FB968 project_51.zip filesize 131127388 Bytes
Archive: F:/project_51.zip
inflating: pro51.PK3 inflating: readme.txt
OK
So I have written a little file comparison tool. It compares the good and the wrong file and logs all issues. See the attached files. Each line shows the adress, databyte good, databyte bad, total error count and the nummer of continued errors in follow. Please watch the follows carefully!
If I copy such larger files from HPFS or JFS to a FAT32 drive with a filecommander, like FC220, no problem occurs. With my 2Mbit www connection the files are written only slow while downloading them. May be a small buffer allocation problem in the IFS or cachef32.exe?
Changed 15 years ago by
| Attachment: | config.sys added |
|---|
comment:2 follow-up: 3 Changed 15 years ago by
Hi²
I'm actually using this release: ftp://ftp.netlabs.org/pub/fat32/fat32_0913.wpi
Installed over a older version without changing any parameters 'by hand' in config.sys
I have attached my config.sys
But the defect occurs since ~2 years, also with older versions.
Let me know if you need more info.
Thanks in advance!
comment:3 Changed 15 years ago by
Hi/2.
You can turn off lazy-write cache by specifying '/L' option to 'cachef32.exe' as the following.
cachef32 /L:off
To turn it on again,
cachef32 /L:on
I want to know the result when turning off lazy-write cache.
KO Myung-Hun
Replying to mst:
Hi²
I'm actually using this release: ftp://ftp.netlabs.org/pub/fat32/fat32_0913.wpi
Installed over a older version without changing any parameters 'by hand' in config.sys
I have attached my config.sys
But the defect occurs since ~2 years, also with older versions.
Let me know if you need more info.
Thanks in advance!
comment:4 follow-up: 5 Changed 15 years ago by
Hi²
I have downloaded the file again. The server was very busy and I needed 1,5h until it was complete. Test with /L:off. File was corrupt, found over 13000 Errors. See attached datvergl2.txt
new download 311208 with cachef32.exe /S /F /L:off J:\>md5suml project_51.zip DA0116023F85A48FBA70E9295FD97246 project_51.zip CRC fail
Archive: J:/project_51.zip
inflating: pro51.PK3 bad CRC a3e8c100 (should be 1c6e2954) inflating: readme.txt
Today I've tested again with /L:on. File was OK, but I had a fast download with 2Mbit/s
new download 020109 with cachef32.exe /S /F /L:on J:\>md5suml project_51.zip 6D6A29E52F1055886A4271E53C3FB968 project_51.zip
Archive: J:/project_51.zip
inflating: pro51.PK3 inflating: readme.txt
OK
I think the problem occurs if only small blocks are written with pauses to a file.
Greetings, Mario
comment:5 Changed 15 years ago by
Hi/2.
It seems that my explanation is not enough.
You should specify '/L:off' on command line not in config.sys to disable lazy-write cache.
And you should confirm the following message when launching 'cachef32' without any parameters.
LAZY WRITING is NOT active!
Anyway, as you said, small size writing slowly seems to cause the problem.
However, to determine whether this problem occur in file-writing routine or in lazy-write cache routine, is very important.
Thanks.
KO Myung-Hun
Replying to mst:
Hi²
I have downloaded the file again. The server was very busy and I needed 1,5h until it was complete. Test with /L:off. File was corrupt, found over 13000 Errors. See attached datvergl2.txt
new download 311208 with cachef32.exe /S /F /L:off J:\>md5suml project_51.zip DA0116023F85A48FBA70E9295FD97246 project_51.zip CRC fail
Archive: J:/project_51.zip
inflating: pro51.PK3 bad CRC a3e8c100 (should be 1c6e2954) inflating: readme.txt
Today I've tested again with /L:on. File was OK, but I had a fast download with 2Mbit/s
new download 020109 with cachef32.exe /S /F /L:on J:\>md5suml project_51.zip 6D6A29E52F1055886A4271E53C3FB968 project_51.zip
Archive: J:/project_51.zip
inflating: pro51.PK3 inflating: readme.txt
OK
I think the problem occurs if only small blocks are written with pauses to a file.
Greetings, Mario
comment:6 follow-up: 7 Changed 15 years ago by
Hi², I think I catched it!
F:\fat32>cachef32 /L:OFF FAT32 cache helper version 0.9.13. FAT32: WARNING: Lazy writing is not active! CACHEF32 is already running. Current priority is Idle time. LAZY WRITING is NOT active!
Read-Ahead sector count for drive I: is 64. Read-Ahead sector count for drive J: is 64.
CACHE has all space in high memory. CACHE has space for 4096 sectors CACHE contains 4096 sectors There are 0 dirty sectors in cache. The cache hits ratio is 96%. FAT32.IFS has currently 7 GDT segments allocated.
approx 1,5h each download, slow
download 070109 try 1 with commandline: cachef32.exe /L:OFF J:\>md5suml project_51.zip 6D6A29E52F1055886A4271E53C3FB968 project_51.zip CRC = CRC file on JFS OK
download 070109 try 2 with commandline: cachef32.exe /L:OFF J:\>md5suml project_51.zip 6D6A29E52F1055886A4271E53C3FB968 project_51.zip OK
download 070109 try 3 with commandline: cachef32.exe /L:OFF J:\>md5suml project_51.zip 6D6A29E52F1055886A4271E53C3FB968 project_51.zip OK
Date 080109: C:\OS2WARE\FC2_220>cachef32.exe /L:ON FAT32 cache helper version 0.9.13. FAT32: WARNING: Lazy writing is already active or cachesize is 0! CACHEF32 is already running. Current priority is Idle time.
DISKIDLE : 1000 milliseconds. BUFFERIDLE: 500 milliseconds. MAXAGE : 5000 milliseconds.
Read-Ahead sector count for drive I: is 64. Read-Ahead sector count for drive J: is 64.
CACHE has all space in high memory. CACHE has space for 4096 sectors CACHE contains 132 sectors There are 0 dirty sectors in cache. The cache hits ratio is 73%. FAT32.IFS has currently 7 GDT segments allocated.
approx 1,5h each download, slow
J:\>md5suml project_51.zip 0F1C481DBCEB4417B788C477D89F6B1C project_51.zip FAIL!!!!!!!!!!!!!!!!!
After the failure dl. I will switch off the lazy writing. But I miss the line: FAT32: WARNING: Lazy writing is not active''' ???
F:\fat32>cachef32.exe /L:OFF FAT32 cache helper version 0.9.13. CACHEF32 is already running. Current priority is Idle time.
DISKIDLE : 1000 milliseconds. BUFFERIDLE: 500 milliseconds. MAXAGE : 5000 milliseconds.
Read-Ahead sector count for drive I: is 64. Read-Ahead sector count for drive J: is 64.
CACHE has all space in high memory. CACHE has space for 4096 sectors CACHE contains 4096 sectors There are 0 dirty sectors in cache. The cache hits ratio is 96%. FAT32.IFS has currently 7 GDT segments allocated.
I try it again and see the line now. What happens here?
F:\fat32>cachef32.exe /L:OFF FAT32 cache helper version 0.9.13. FAT32: WARNING: Lazy writing is not active! CACHEF32 is already running. Current priority is Idle time. LAZY WRITING is NOT active!
Read-Ahead sector count for drive I: is 64. Read-Ahead sector count for drive J: is 64.
CACHE has all space in high memory. CACHE has space for 4096 sectors CACHE contains 4096 sectors There are 0 dirty sectors in cache. The cache hits ratio is 96%. FAT32.IFS has currently 7 GDT segments allocated.
download time ~2h
J:\>md5suml project_51.zip 6D6A29E52F1055886A4271E53C3FB968 project_51.zip CRC OK
Summary: with L:OFF 4 of 4 downloads are OK with L:ON 1 of 1 download is corrupt cachef32 is still loaded in config.sys: CALL=C:\OS2\CACHEF32.EXE /S /F /L:on
lazy writing is switching on or off by running cachef32 from command line.
I hope this helps. Greetings, Mario.
comment:7 Changed 15 years ago by
Hi/2.
I greatly appreciate your efforts.
Cache seems to have a problem, but I cannot figure out the problem currently.
Maybe it'll takes very long time. Or other people such as Lars, can solve this problem.
Until then, do not download to FAT32 partition in low speed with cache.
KO Myung-Hun
Replying to mst:
Hi², I think I catched it!
F:\fat32>cachef32 /L:OFF FAT32 cache helper version 0.9.13. FAT32: WARNING: Lazy writing is not active! CACHEF32 is already running. Current priority is Idle time. LAZY WRITING is NOT active!
Read-Ahead sector count for drive I: is 64. Read-Ahead sector count for drive J: is 64.
CACHE has all space in high memory. CACHE has space for 4096 sectors CACHE contains 4096 sectors There are 0 dirty sectors in cache. The cache hits ratio is 96%. FAT32.IFS has currently 7 GDT segments allocated.
approx 1,5h each download, slow
download 070109 try 1 with commandline: cachef32.exe /L:OFF J:\>md5suml project_51.zip 6D6A29E52F1055886A4271E53C3FB968 project_51.zip CRC = CRC file on JFS OK
download 070109 try 2 with commandline: cachef32.exe /L:OFF J:\>md5suml project_51.zip 6D6A29E52F1055886A4271E53C3FB968 project_51.zip OK
download 070109 try 3 with commandline: cachef32.exe /L:OFF J:\>md5suml project_51.zip 6D6A29E52F1055886A4271E53C3FB968 project_51.zip OK
Date 080109: C:\OS2WARE\FC2_220>cachef32.exe /L:ON FAT32 cache helper version 0.9.13. FAT32: WARNING: Lazy writing is already active or cachesize is 0! CACHEF32 is already running. Current priority is Idle time.
DISKIDLE : 1000 milliseconds. BUFFERIDLE: 500 milliseconds. MAXAGE : 5000 milliseconds.
Read-Ahead sector count for drive I: is 64. Read-Ahead sector count for drive J: is 64.
CACHE has all space in high memory. CACHE has space for 4096 sectors CACHE contains 132 sectors There are 0 dirty sectors in cache. The cache hits ratio is 73%. FAT32.IFS has currently 7 GDT segments allocated.
approx 1,5h each download, slow
J:\>md5suml project_51.zip 0F1C481DBCEB4417B788C477D89F6B1C project_51.zip FAIL!!!!!!!!!!!!!!!!!
After the failure dl. I will switch off the lazy writing. But I miss the line: FAT32: WARNING: Lazy writing is not active''' ???
F:\fat32>cachef32.exe /L:OFF FAT32 cache helper version 0.9.13. CACHEF32 is already running. Current priority is Idle time.
DISKIDLE : 1000 milliseconds. BUFFERIDLE: 500 milliseconds. MAXAGE : 5000 milliseconds.
Read-Ahead sector count for drive I: is 64. Read-Ahead sector count for drive J: is 64.
CACHE has all space in high memory. CACHE has space for 4096 sectors CACHE contains 4096 sectors There are 0 dirty sectors in cache. The cache hits ratio is 96%. FAT32.IFS has currently 7 GDT segments allocated.
I try it again and see the line now. What happens here?
F:\fat32>cachef32.exe /L:OFF FAT32 cache helper version 0.9.13. FAT32: WARNING: Lazy writing is not active! CACHEF32 is already running. Current priority is Idle time. LAZY WRITING is NOT active!
Read-Ahead sector count for drive I: is 64. Read-Ahead sector count for drive J: is 64.
CACHE has all space in high memory. CACHE has space for 4096 sectors CACHE contains 4096 sectors There are 0 dirty sectors in cache. The cache hits ratio is 96%. FAT32.IFS has currently 7 GDT segments allocated.
download time ~2h
J:\>md5suml project_51.zip 6D6A29E52F1055886A4271E53C3FB968 project_51.zip CRC OK
Summary: with L:OFF 4 of 4 downloads are OK with L:ON 1 of 1 download is corrupt cachef32 is still loaded in config.sys: CALL=C:\OS2\CACHEF32.EXE /S /F /L:on
lazy writing is switching on or off by running cachef32 from command line.
I hope this helps. Greetings, Mario.
comment:8 Changed 8 years ago by
| Milestone: | → Future |
|---|---|
| Severity: | → highest |
did anyone test with latest alpha?
comment:9 Changed 7 years ago by
| Priority: | critical → Feedback Pending |
|---|
comment:10 Changed 4 years ago by
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
Hi
Since this issue was opened 11 years ago and last modified 3 years ago, I'm closing it. In case the issue exist on the latest test version (0.10 r383) please reopen the ticket.
Regards
source