Cannot run yum when logged onto non-writable volume or volume not supporting long filenames

[Lewis Rosenthal]

Paths (and path rewriters) all set correctly.

%UNIXROOT% is set to J: (no trailing backslash)

Running just yum from the following volumes produces different output:

A: [FAT16]            fails - see message, below
C: [JFS]              succeeds
D: [HPFS]             succeeds
E: [DVD]              fails - see message, below
F: [NetWare]          succeeds
I: [NetDrive - ISOFS] succeeds
J: [JFS]              succeeds
N: [HPFS]             succeeds
O: [FAT16]            fails - see message, below
Q: [FAT32]            succeeds

Failure text (no yum.log):

[e:\]yum
Traceback (most recent call last):
  File "/@unixroot/usr/lib/python2.7/site.py", line 62, in <module>
    import os
  File "/@unixroot/usr/lib/python2.7/os.py", line 400, in <module>
    import UserDict
ImportError: No module named UserDict

Obviously, UserDict.py and friends look just fine, or we would not have success anywhere.

Some package verisons:

yum-plugin-verify-1.1.31-3.oc00.noarch
yum-utils-1.1.31-3.oc00.noarch
yum-3.4.3-10.oc00.pentium4
yum-plugin-versionlock-1.1.31-3.oc00.noarch
yum-metadata-parser-1.1.4-7.oc00.pentium4
python-devel-2.7.6-14.oc00.pentium4
python-2.7.6-14.oc00.pentium4
python-libs-2.7.6-14.oc00.pentium4
libxml2-python-2.7.7-6.oc00.i386
python-pycurl-7.19.5.1-2.oc00.pentium4
rpm-python-4.13.0-9.oc00.pentium4

The mystery to me is how the NetDrive-mapped read-only media is not a problem.

Marking as critical because we need to be able to run yum from the Blue Lion installation media (DVD, usually). This was not the case with the previous bootstrap, but my latest one seems to have the latest files (the test above was from my production machine, which is regularly updated, and does not involve the bootstrap; the issue is easily reproduced on multiple systems with both fresh and maintained installations).

Possibly related to #71?

[Silvan Scherrer]

is this still the case? As ArcaOS is out since long it seems to me it works as it should.

[Lewis Rosenthal]

This is still the case. We just work around it in ArcaOS by ensuring that we are logged onto a writable drive before running yum.

[a:\] yum
Traceback (most recent call last):
  File "/@unixroot/usr/lib/python2.7/site.py", line 62, in <module>
    import os
  File "/@unixroot/usr/lib/python2.7/os.py", line 400, in <module>
    import UserDict
ImportError: No module named UserDict

(The shell does not matter. Identical results are obtained under dash.)

Note change in summary.

This is a PATH problem. The above is with PYTHONPATH unset. Setting PYTHONPATH to:

PYTHONPATH=j:/usr/lib/python2.7;j:/usr/lib/python2.7/lib-dynload;j:
/usr/lib/python2.7/site-packages/yum;j:/usr/lib/python2.7/site-packages/rpmUtils

gets us a bit farther, though we still fail ultimately with:

There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:

   No module named rpmUtils.transaction

Please install a package which provides this module, or
verify that the module is installed correctly.

It seems that we can't properly locate rpmUtils/transaction.py.

This is also a kludge, as different plugins may have different module imports, some of which may be outside the "normal" (default, unset) PYTHONPATH, so PYTHONPATH would have to be continually edited (non optimal).

[Silvan Scherrer]

it will never work on FAT16, as yum and rpm needs long filenames. and it might even need symlinks.
so for me the only issue here is the one with running from non writable drives. I wonder if this could be due to the fact yum writes cache files and log files. Could you retest that?
Could you eventualy test the same with some unix?

[Silvan Scherrer]

hmm no feedback since 7 month? Will be closed within 2 weeks.

[Lewis Rosenthal]

Sorry, somehow I missed your last request for retesting.

I'll re-test under ArcaOS with a DVD. As for testing with Unix, almost all of my Linux installs are SuSE (zypper, not yum). I'll probably have to do a separate Fedora install. I assume you'll want a version before DNF replaced yum. If you have a specific version in mind (or a different distro and version), please advise.

[Silvan Scherrer]

no I have nothing specifig in mind. Just to see how other yum based systems work with DVD (readonly). That FAT16 is not working is a fact. This will never work.

[Lewis Rosenthal]

That FAT16 is not working is a fact. This will never work.

Why not? If %TEMP% is pointing to a filesystem supporting long filenames and EAs, and %UNIXROOT% is pointing to a volume supporting long filenames and EAs, then why if I am just currently logged onto a FAT16 volume should this matter? I am still running YUM from the %UNIXROOT% (JFS or HPFS volume). In fact, there may not even be a local repository on the FAT16 volume; I'm just at a prompt on, say, Z: (FAT16 RAM disk) and type "yum install <blah>". yum is going to run from %PATH% (%UNIXROOT%\usr\bin\yum.exe), read/write to cache on %UNIXROOT%\var\cache\yum, and write temporary files to %TEMP% (on the JFS or HPFS volume). It should not matter what my current logged-on drive is, just as it should not matter whether I am logged onto a read-only filesystem (e.g., ISO 9660 or UDF). In short, yum should not be trying to write to the current volume.

I'll run some more tests over the weekend with a DVD and set up YUM on some Linux system to see if this is an upstream issue or not.

[Lewis Rosenthal]

Testing this was easier than I thought.

I created a second RAM 1GB disk with the ArcaOS loader, and formatted it FAT16. I also just inserted any CD I had on hand into the DVD drive. Both attempts to run yum (just yum, with no parameters, or yum update) fail as originally described:

[s:\]yum
Traceback (most recent call last):
  File "/@unixroot/usr/lib/python2.7/site.py", line 62, in <module>
    import os
  File "/@unixroot/usr/lib/python2.7/os.py", line 400, in <module>
    import UserDict
ImportError: No module named UserDict

man yum works, as does rpm -qa. In addition, the python shell works.

Linux tests:

Centos 4.8 / yum 2.3.2: No problem running yum as root while pointing to the mounted directory of a 720K floppy (/media/floppy/), with or without the write-protect tab set. The CD in the server was uncooperative (unable to mount).

openSUSE Leap 43.2 / yum 3.4.3: (I was surprised that yum 3.4.3 was available for openSUSE, but it installed via zypper.) No problem running yum while in the mounted directory of a CD-ROM (/var/run/media/lewis/D101GGC/). No floppy drive in that box, and no USB floppy handy.

In short, a much older version of yum on Linux does not seem to have a problem, and neither does a version concurrent with ours. This appears to be unique to our environment.

Whether this is worth investigating further I do not know. It's an interesting puzzle, but we have a workaround (don't run yum while logged onto a FAT16 volume or a read-only filesystem). Future versions of ArcaOS should get past this when we have HPFS support for the RAM disk during install.

[Silvan Scherrer]

@fat16 this was s missunderstanding. I meant having temp or whatever in fat16 will not work. Just issue yum on such a volume should work in theory. If this is ging to be fixed is another q. For sure not within 2-3 month. As i have absolutely no time loking further into it until after august.

[Silvan Scherrer]

add PYTHONCASEOK=1 to the env and it should work. This is just a workaround until the real cause is found.

[Lewis Rosenthal]

Confirmed that this workaround allows yum to run when logged onto on a non-writable volume (CD) or a FAT16 (writable) one.

What is odd here is that AIUI, PYTHONCASEOK allows Python to import modules case-insensitively. Normally, when importing modules, Python follows sys.path, so this behavior is interesting. On my test system, sys.path seems to be:

C:/USR/lib/python27.zip
C:/USR/lib/python2.7
C:/USR/lib/python2.7/plat-os2knix
C:/USR/lib/python2.7/lib-tk
C:/USR/lib/python2.7/lib-old
C:/USR/lib/python2.7/lib-dynload
C:/USR/lib/python2.7/site-packages

So, I'm baffled as to how PYTHONCASEOK works (or how you found it). ;-)

Further reference for PYTHONCASEOK is ​here. FWIW, Python itself is perfectly content to run when logged onto read-only media or FAT16 volumes.

Nice work, Silvan. This has me thinking in some other directions, now.

[Silvan Scherrer]

reading code that is. you are just lucky that another project got stalled and I had some spare. We found even the root cause. very very ugly bug in libc :( we are thinking about a fix now

[Silvan Scherrer]

​https://github.com/bitwiseworks/libc/issues/39 for more reference of the bug.

next libcn will have the fix. please close it here when you are confident with it.

[Silvan Scherrer]

feel free to test the fix as well ​http://rpm.netlabs.org/test/ln2.zip if not wait until the new libc is available. should be very soon

[Lewis Rosenthal]

Confirmed that the new libcn allows yum to start and function on FAT16 and optical (read-only) media. Kudos!