Opened 6 years ago

Closed 6 years ago

#194 closed defect (no change needed)

Unusable zip arhive of huge volume

Reported by: Shmuel (Seymour J.) Metz Owned by:
Priority: Feedback pending Milestone:
Component: *none Version:
Severity: medium Keywords: zip huge incomplete
Cc:

Description

When I zip a volume containg more than 4 MiB compressed, Arctool fails with the message

Z0001: Unable to open file
K:\M.zip
The data is invalid

An unzip test shows

Thu 10-18-18 4:01:16{9}[h:\] unzip -t K:\M
Archive: K:/M.zip

End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.

Attachments (6)

unzipver (1.3 KB ) - added by Shmuel (Seymour J.) Metz 6 years ago.
unzip -v output
zipver (1.3 KB ) - added by Shmuel (Seymour J.) Metz 6 years ago.
zip -v output
BackupUSB.cmd (4.5 KB ) - added by Shmuel (Seymour J.) Metz 6 years ago.
Script used to generate zip commands
unsplit.cmd (1.5 KB ) - added by Shmuel (Seymour J.) Metz 6 years ago.
Script used to concatenate z?? file writte by zip -s
zipM.zip (133.7 KB ) - added by Shmuel (Seymour J.) Metz 6 years ago.
Logs of zip -s 1G and unzip -t of zip (last) segment
unzipM.zip (98.1 KB ) - added by Shmuel (Seymour J.) Metz 6 years ago.
Logs of unzip -l and unzip of concatenated split files

Download all attachments as: .zip

Change History (35)

comment:1 by Silvan Scherrer, 6 years ago

Priority: majorFeedback pending

4 MiB ??? or 4 GiB???

and to what filesystem do you zip it?

comment:2 by Shmuel (Seymour J.) Metz, 6 years ago

Priority: Feedback pendingmajor

Sorry, 4 GiB. The archive volume was JFS.

comment:3 by Silvan Scherrer, 6 years ago

could you test if the zip file is valid? means if it can be opened on another operating system? as it might be a unzip issue and not zip. please try to find the real cause first.

comment:4 by Shmuel (Seymour J.) Metz, 6 years ago

The unzip fails with the same error on x86 Linux, both 32-bit and 64-bit.

comment:5 by Silvan Scherrer, 6 years ago

ok so it seems like a zip issue. I might have a look when time permits. But most likely not very soon.

comment:6 by Shmuel (Seymour J.) Metz, 6 years ago

Is there any way to generate diagnostic output from zip that would help you track it down?

comment:7 by Silvan Scherrer, 6 years ago

you could try the split option. eventually this works.

How to diagnostic? I would need to look at the code first. But as said. I do when time permits, or someone sponsors this fix immediately.

comment:8 by Shmuel (Seymour J.) Metz, 6 years ago

Doesn't split generate the same archive name on multiple volumes rather than multiple archive names on a single volume?

Is the a command line 7z utility for ArcaOS and, if so, what is the package name?

comment:9 by Silvan Scherrer, 6 years ago

split just splits the zip file after a certain amount of bytes. It's well documented in the help screen iirc.
7z??? is something other and has nothing to do with this ticket. And I have no idea at all.

And as said, if you need a fix either wait until we have time do look at it or sponsor the fix. We can't work for free sorry.

comment:10 by ak120, 6 years ago

Please provide the output of

zip -v

and

unzip -v

which are involved in your operations. Please read the file ziplimit.txt for further information.

comment:11 by Shmuel (Seymour J.) Metz, 6 years ago

I'm away from my computer, but I'll check the versions when I get home. Meanwhile, the "-s 3G" option didn't help; should I try it with a smaller size?

BTW, in my previous reply I was confusing split with spanning in my previous reply.

comment:12 by Silvan Scherrer, 6 years ago

start with 1GB and then go up until it blows

comment:13 by Shmuel (Seymour J.) Metz, 6 years ago

  1. Some error messages only appear with the -v (verbose) option
  1. with -s 1G, Arctool will still not handle the zip file or the concatenated file.
  1. unzip -t recognizes the zip fil as split
  1. If I concatenate Z0a Z02 , ..., ZIP, unzip -t gives warnings but no fatal errors

by Shmuel (Seymour J.) Metz, 6 years ago

Attachment: unzipver added

unzip -v output

by Shmuel (Seymour J.) Metz, 6 years ago

Attachment: zipver added

zip -v output

comment:14 by Silvan Scherrer, 6 years ago

what Arctool does is out of this scope, as this is not maintained by us. So with split all runs?
and try to see which split value is the last working.

Last edited 6 years ago by Silvan Scherrer (previous) (diff)

comment:15 by Shmuel (Seymour J.) Metz, 6 years ago

If I use -s (split) and concatenate the segments in sequence then unzip gives warnings but successfully extracts a file.

JFS supports large files, and Info-zip has support for 64-bit file addressing on 32-bit systems, so it may be easier to support large archives than to fully support split.

I ran into a couple of small problems and wonder whether I should open separate tickets on them:

  1. Some error messages are only written with -v (verbose)
  1. Some error messages, e.g., permission, are written without any context, so -v is needed to determine the relevant file
  1. The Disk full message does not distinguish between the volume for the new archive and the -b (work) volume

comment:16 by Silvan Scherrer, 6 years ago

please do what I asked for and try to find out which split value is the highest working one.
We have 64-bit addressing enabled, thatswhy I ask about the highest working split value, to see why it might be broken.
For the other issues you need to open individual tickets.

But as I told already: don't expect fast fixes, as zip is right now not top priority.
If you want a fast fix, we are more than glad to send you a price quote.

by Shmuel (Seymour J.) Metz, 6 years ago

Attachment: BackupUSB.cmd added

Script used to generate zip commands

by Shmuel (Seymour J.) Metz, 6 years ago

Attachment: unsplit.cmd added

Script used to concatenate z?? file writte by zip -s

comment:17 by Shmuel (Seymour J.) Metz, 6 years ago

The zip command treats -s 8G as -s 4G; I get the same warnings as with 1G, the same error messages:

testing: PDF/honeywell/Multics/AG90-03_PgmgIntro_Dec81.pdf

error: invalid compressed data to inflate

and I am able to extract files other than the one with the above error message. I've attached the script used to build the zip commands and the script used to concatenate the pieces from zip -s; would it help to attach logs from the zip, the unzip -t or the unzip to extract?

Meanwhile I'll take a look at gzip. Thanks.

comment:18 by Shmuel (Seymour J.) Metz, 6 years ago

I spoke too soon. It is true that I can unzip *some* files, but it turns out that I can't unzip all of them. The attached files are the logs from generating and attempt to use a large zip archive with -s 1G.

comment:19 by Shmuel (Seymour J.) Metz, 6 years ago

Sorry: the attachments are too large to upload. I can try to trim them, or can provide them through alternate channels.

I zipped them in pairs and that shrank them enough to upload.

Last edited 6 years ago by Shmuel (Seymour J.) Metz (previous) (diff)

by Shmuel (Seymour J.) Metz, 6 years ago

Attachment: zipM.zip added

Logs of zip -s 1G and unzip -t of zip (last) segment

by Shmuel (Seymour J.) Metz, 6 years ago

Attachment: unzipM.zip added

Logs of unzip -l and unzip of concatenated split files

comment:20 by Silvan Scherrer, 6 years ago

thanks for the info. When time permits or someone sponsors the fix we will look at it.

comment:21 by Shmuel (Seymour J.) Metz, 6 years ago

comment:22 by Silvan Scherrer, 6 years ago

this discussion is completely useless, at least for the issues you have. Because all of that I decided to put aside zip for now and might look at it next year again. Why should I invest time, which no one pays and then some individuals (not referring to the reporter of this ticket) write such bullshit? I really have a lot better things to do with my time.

So I repeat again, if one sponsors it, I will look at it, but if no one steps up, I will not look at those issues for longer.

Last edited 6 years ago by Silvan Scherrer (previous) (diff)

comment:23 by Shmuel (Seymour J.) Metz, 6 years ago

In case someone else runs into this, /pub/os2/util/archiver/p7zip-9.20.1-os2.zip in http://hobbes.nmsu.edu/h-browse.php?dir=/pub/os2/util/archiver provides a viable workaround; the newer http://hobbes.nmsu.edu/download/pub/os2/util/archiver/p7zip-16.02-os2-1.zip didn't work.

comment:24 by Shmuel (Seymour J.) Metz, 6 years ago

It turns out that the problem only occurs with the work directory (-b) option. Removing it is a viable bypass, although it does more than double the elapsed time when the archive goes to a USB thumb drive.

comment:25 by Silvan Scherrer, 6 years ago

USB is slow. But that's not a zip issue.

comment:26 by Shmuel (Seymour J.) Metz, 6 years ago

@diver Of course not, but it is an issue with the acceptability of the bypass, which is why I mentioned it.

comment:27 by Silvan Scherrer, 6 years ago

Priority: majorFeedback pending

it it true that all works as it should when you don't use the -b option? If so please open a new ticket with a valid description about the -b option and close this ticket.

comment:28 by Shmuel (Seymour J.) Metz, 6 years ago

I created ticket 201 explicitly referencing -b. Should I also open a ticket for the split (-s) issue, or is that not supported?

comment:29 by Silvan Scherrer, 6 years ago

Resolution: no change needed
Status: newclosed

Every issue needs a own ticket. Please verify first in other platforms as well. As it might be a crossbug.
Marking this one as solved in favour of the new tickets

Note: See TracTickets for help on using tickets.