Opened 2 years ago

Closed 21 months ago

#194 closed defect (no change needed)

Unusable zip arhive of huge volume

Reported by: Shmuel (Seymour J.) Metz Owned by:
Priority: Feedback pending Milestone:
Component: *none Version:
Severity: medium Keywords: zip huge incomplete
Cc:

Description

When I zip a volume containg more than 4 MiB compressed, Arctool fails with the message

Z0001: Unable to open file
K:\M.zip
The data is invalid

An unzip test shows

Thu 10-18-18 4:01:16{9}[h:\] unzip -t K:\M
Archive: K:/M.zip

End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.

Attachments (6)

unzipver (1.3 KB) - added by Shmuel (Seymour J.) Metz 23 months ago.
unzip -v output
zipver (1.3 KB) - added by Shmuel (Seymour J.) Metz 23 months ago.
zip -v output
BackupUSB.cmd (4.5 KB) - added by Shmuel (Seymour J.) Metz 22 months ago.
Script used to generate zip commands
unsplit.cmd (1.5 KB) - added by Shmuel (Seymour J.) Metz 22 months ago.
Script used to concatenate z?? file writte by zip -s
zipM.zip (133.7 KB) - added by Shmuel (Seymour J.) Metz 22 months ago.
Logs of zip -s 1G and unzip -t of zip (last) segment
unzipM.zip (98.1 KB) - added by Shmuel (Seymour J.) Metz 22 months ago.
Logs of unzip -l and unzip of concatenated split files

Download all attachments as: .zip

Change History (35)

comment:1 Changed 2 years ago by Silvan Scherrer

Priority: majorFeedback pending

4 MiB ??? or 4 GiB???

and to what filesystem do you zip it?

comment:2 Changed 2 years ago by Shmuel (Seymour J.) Metz

Priority: Feedback pendingmajor

Sorry, 4 GiB. The archive volume was JFS.

comment:3 Changed 2 years ago by Silvan Scherrer

could you test if the zip file is valid? means if it can be opened on another operating system? as it might be a unzip issue and not zip. please try to find the real cause first.

comment:4 Changed 2 years ago by Shmuel (Seymour J.) Metz

The unzip fails with the same error on x86 Linux, both 32-bit and 64-bit.

comment:5 Changed 2 years ago by Silvan Scherrer

ok so it seems like a zip issue. I might have a look when time permits. But most likely not very soon.

comment:6 Changed 2 years ago by Shmuel (Seymour J.) Metz

Is there any way to generate diagnostic output from zip that would help you track it down?

comment:7 Changed 2 years ago by Silvan Scherrer

you could try the split option. eventually this works.

How to diagnostic? I would need to look at the code first. But as said. I do when time permits, or someone sponsors this fix immediately.

comment:8 Changed 23 months ago by Shmuel (Seymour J.) Metz

Doesn't split generate the same archive name on multiple volumes rather than multiple archive names on a single volume?

Is the a command line 7z utility for ArcaOS and, if so, what is the package name?

comment:9 Changed 23 months ago by Silvan Scherrer

split just splits the zip file after a certain amount of bytes. It's well documented in the help screen iirc.
7z??? is something other and has nothing to do with this ticket. And I have no idea at all.

And as said, if you need a fix either wait until we have time do look at it or sponsor the fix. We can't work for free sorry.

comment:10 Changed 23 months ago by ak120

Please provide the output of

zip -v

and

unzip -v

which are involved in your operations. Please read the file ziplimit.txt for further information.

comment:11 Changed 23 months ago by Shmuel (Seymour J.) Metz

I'm away from my computer, but I'll check the versions when I get home. Meanwhile, the "-s 3G" option didn't help; should I try it with a smaller size?

BTW, in my previous reply I was confusing split with spanning in my previous reply.

comment:12 Changed 23 months ago by Silvan Scherrer

start with 1GB and then go up until it blows

comment:13 Changed 23 months ago by Shmuel (Seymour J.) Metz

  1. Some error messages only appear with the -v (verbose) option
  1. with -s 1G, Arctool will still not handle the zip file or the concatenated file.
  1. unzip -t recognizes the zip fil as split
  1. If I concatenate Z0a Z02 , ..., ZIP, unzip -t gives warnings but no fatal errors

Changed 23 months ago by Shmuel (Seymour J.) Metz

Attachment: unzipver added

unzip -v output

Changed 23 months ago by Shmuel (Seymour J.) Metz

Attachment: zipver added

zip -v output

comment:14 Changed 23 months ago by Silvan Scherrer

what Arctool does is out of this scope, as this is not maintained by us. So with split all runs?
and try to see which split value is the last working.

Last edited 23 months ago by Silvan Scherrer (previous) (diff)

comment:15 Changed 23 months ago by Shmuel (Seymour J.) Metz

If I use -s (split) and concatenate the segments in sequence then unzip gives warnings but successfully extracts a file.

JFS supports large files, and Info-zip has support for 64-bit file addressing on 32-bit systems, so it may be easier to support large archives than to fully support split.

I ran into a couple of small problems and wonder whether I should open separate tickets on them:

  1. Some error messages are only written with -v (verbose)
  1. Some error messages, e.g., permission, are written without any context, so -v is needed to determine the relevant file
  1. The Disk full message does not distinguish between the volume for the new archive and the -b (work) volume

comment:16 Changed 22 months ago by Silvan Scherrer

please do what I asked for and try to find out which split value is the highest working one.
We have 64-bit addressing enabled, thatswhy I ask about the highest working split value, to see why it might be broken.
For the other issues you need to open individual tickets.

But as I told already: don't expect fast fixes, as zip is right now not top priority.
If you want a fast fix, we are more than glad to send you a price quote.

Changed 22 months ago by Shmuel (Seymour J.) Metz

Attachment: BackupUSB.cmd added

Script used to generate zip commands

Changed 22 months ago by Shmuel (Seymour J.) Metz

Attachment: unsplit.cmd added

Script used to concatenate z?? file writte by zip -s

comment:17 Changed 22 months ago by Shmuel (Seymour J.) Metz

The zip command treats -s 8G as -s 4G; I get the same warnings as with 1G, the same error messages:

testing: PDF/honeywell/Multics/AG90-03_PgmgIntro_Dec81.pdf

error: invalid compressed data to inflate

and I am able to extract files other than the one with the above error message. I've attached the script used to build the zip commands and the script used to concatenate the pieces from zip -s; would it help to attach logs from the zip, the unzip -t or the unzip to extract?

Meanwhile I'll take a look at gzip. Thanks.

comment:18 Changed 22 months ago by Shmuel (Seymour J.) Metz

I spoke too soon. It is true that I can unzip *some* files, but it turns out that I can't unzip all of them. The attached files are the logs from generating and attempt to use a large zip archive with -s 1G.

comment:19 Changed 22 months ago by Shmuel (Seymour J.) Metz

Sorry: the attachments are too large to upload. I can try to trim them, or can provide them through alternate channels.

I zipped them in pairs and that shrank them enough to upload.

Last edited 22 months ago by Shmuel (Seymour J.) Metz (previous) (diff)

Changed 22 months ago by Shmuel (Seymour J.) Metz

Attachment: zipM.zip added

Logs of zip -s 1G and unzip -t of zip (last) segment

Changed 22 months ago by Shmuel (Seymour J.) Metz

Attachment: unzipM.zip added

Logs of unzip -l and unzip of concatenated split files

comment:20 Changed 22 months ago by Silvan Scherrer

thanks for the info. When time permits or someone sponsors the fix we will look at it.

comment:21 Changed 22 months ago by Shmuel (Seymour J.) Metz

comment:22 Changed 22 months ago by Silvan Scherrer

this discussion is completely useless, at least for the issues you have. Because all of that I decided to put aside zip for now and might look at it next year again. Why should I invest time, which no one pays and then some individuals (not referring to the reporter of this ticket) write such bullshit? I really have a lot better things to do with my time.

So I repeat again, if one sponsors it, I will look at it, but if no one steps up, I will not look at those issues for longer.

Last edited 22 months ago by Silvan Scherrer (previous) (diff)

comment:23 Changed 21 months ago by Shmuel (Seymour J.) Metz

In case someone else runs into this, /pub/os2/util/archiver/p7zip-9.20.1-os2.zip in http://hobbes.nmsu.edu/h-browse.php?dir=/pub/os2/util/archiver provides a viable workaround; the newer http://hobbes.nmsu.edu/download/pub/os2/util/archiver/p7zip-16.02-os2-1.zip didn't work.

comment:24 Changed 21 months ago by Shmuel (Seymour J.) Metz

It turns out that the problem only occurs with the work directory (-b) option. Removing it is a viable bypass, although it does more than double the elapsed time when the archive goes to a USB thumb drive.

comment:25 Changed 21 months ago by Silvan Scherrer

USB is slow. But that's not a zip issue.

comment:26 Changed 21 months ago by Shmuel (Seymour J.) Metz

@diver Of course not, but it is an issue with the acceptability of the bypass, which is why I mentioned it.

comment:27 Changed 21 months ago by Silvan Scherrer

Priority: majorFeedback pending

it it true that all works as it should when you don't use the -b option? If so please open a new ticket with a valid description about the -b option and close this ticket.

comment:28 Changed 21 months ago by Shmuel (Seymour J.) Metz

I created ticket 201 explicitly referencing -b. Should I also open a ticket for the split (-s) issue, or is that not supported?

comment:29 Changed 21 months ago by Silvan Scherrer

Resolution: no change needed
Status: newclosed

Every issue needs a own ticket. Please verify first in other platforms as well. As it might be a crossbug.
Marking this one as solved in favour of the new tickets

Note: See TracTickets for help on using tickets.