Opened 10 years ago
Closed 5 years ago
#90 closed defect (duplicate)
python: (2.7.6) problem determining filesystem encoding
Reported by: | Lewis Rosenthal | Owned by: | Yuri Dario |
---|---|---|---|
Priority: | Feedback Pending | Milestone: | |
Component: | python | Version: | |
Severity: | highest | Keywords: | |
Cc: |
Description
This is tangentially related to getting SSL support for yum repos, so I'll open this here. Frankly, python is so complex that we should probably break it out into its own project, here, but I'll leave that to you, Yuri. ;-)
I needed to install M2Crypto as I am having difficulty accessing my repo mirror via SSL (different ticket - to be created). M2Crypto seems to want to be installed via setuptools. Installing setuptools should be fairly straightforward:
wget --no-check-certificate https://bootstrap.pypa.io/ez_setup.py -O - | python
however, once the installation starts, the fun begins:
Extracting in c:/var/temp/tmpcqr6rs Now working in c:/var/temp/tmpcqr6rs/setuptools-5.6 Installing Setuptools running install running bdist_egg running egg_info writing requirements to setuptools.egg-info/requires.txt writing setuptools.egg-info/PKG-INFO writing top-level names to setuptools.egg-info/top_level.txt writing dependency_links to setuptools.egg-info/dependency_links.txt writing entry points to setuptools.egg-info/entry_points.txt Traceback (most recent call last): File "setup.py", line 219, in <module> dist = setuptools.setup(**setup_params) File "J:/usr/lib/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "J:/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "J:/usr/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "C:/var/temp/tmpcqr6rs/setuptools-5.6/setuptools/command/install.py", line 67, in run self.do_egg_install() File "C:/var/temp/tmpcqr6rs/setuptools-5.6/setuptools/command/install.py", line 109, in do_egg_install self.run_command('bdist_egg') File "J:/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "J:/usr/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "C:/var/temp/tmpcqr6rs/setuptools-5.6/setuptools/command/bdist_egg.py", line 152, in run self.run_command("egg_info") File "J:/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "J:/usr/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "C:/var/temp/tmpcqr6rs/setuptools-5.6/setuptools/command/egg_info.py", line 168, in run self.find_sources() File "C:/var/temp/tmpcqr6rs/setuptools-5.6/setuptools/command/egg_info.py", line 193, in find_sources mm.run() File "C:/var/temp/tmpcqr6rs/setuptools-5.6/setuptools/command/egg_info.py", line 277, in run self.add_defaults() File "C:/var/temp/tmpcqr6rs/setuptools-5.6/setuptools/command/egg_info.py", line 306, in add_defaults sdist.add_defaults(self) File "C:/var/temp/tmpcqr6rs/setuptools-5.6/setuptools/command/sdist.py", line 151, in add_defaults self.filelist.append(fn) File "C:/var/temp/tmpcqr6rs/setuptools-5.6/setuptools/command/egg_info.py", line 220, in append if self._safe_path(path): File "C:/var/temp/tmpcqr6rs/setuptools-5.6/setuptools/command/egg_info.py", line 240, in _safe_path u_path = unicode_utils.filesys_decode(path) File "C:/var/temp/tmpcqr6rs/setuptools-5.6/setuptools/unicode_utils.py", line 31, in filesys_decode return path.decode(enc) TypeError: decode() argument 1 must be string, not None Something went wrong during the installation. See the error message above.
Looking at the section in question in unicode_utils.py, he see:
def filesys_decode(path): """ Ensure that the given path is decoded, NONE when no expected encoding works """ fs_enc = sys.getfilesystemencoding() if isinstance(path, decoded_string): return path for enc in (fs_enc, "utf-8"): try: return path.decode(enc) except UnicodeDecodeError: continue
Result: cannot install setuptools.
I've tried this under a variety of shells, too, but to no avail. Filesystem is JFS.
Change History (12)
comment:1 by , 10 years ago
Owner: | set to |
---|---|
Status: | new → accepted |
comment:2 by , 10 years ago
Confirming that your edit works for me, Yuri; thanks. Unfortunately, I don't know enough python to be of much use, so I truly appreciate the pointer. However, as you say, we should keep this open as this is merely a workaround and not a fix to the underlying problem.
comment:3 by , 6 years ago
Priority: | major → Feedback Pending |
---|---|
Severity: | → highest |
I'm not sure if this is still the case, as setuptools changed a lot since this ticket was opened
comment:4 by , 6 years ago
Well, we still fail during the install, but in a different place:
Traceback (most recent call last): File "setup.py", line 181, in <module> dist = setuptools.setup(**setup_params) File "N:/USR/lib/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "N:/USR/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "N:/USR/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "N:/var/temp/tmpfmn52e/setuptools-33.1.1/setuptools/command/install.py", line 67, in run self.do_egg_install() File "N:/var/temp/tmpfmn52e/setuptools-33.1.1/setuptools/command/install.py", line 109, in do_egg_install self.run_command('bdist_egg') File "N:/USR/lib/python2.7/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "N:/USR/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "N:/var/temp/tmpfmn52e/setuptools-33.1.1/setuptools/command/bdist_egg.py", line 161, in run cmd = self.call_command('install_lib', warn_dir=0) File "N:/var/temp/tmpfmn52e/setuptools-33.1.1/setuptools/command/bdist_egg.py", line 147, in call_command self.run_command(cmdname) File "N:/USR/lib/python2.7/distutils/cmd.py", line 326, in run_command self.distribution.run_command(command) File "N:/USR/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "N:/var/temp/tmpfmn52e/setuptools-33.1.1/setuptools/command/install_lib.py", line 12, in run outfiles = self.install() File "N:/USR/lib/python2.7/distutils/command/install_lib.py", line 115, in install outfiles = self.copy_tree(self.build_dir, self.install_dir, preserve_symlinks=1) File "N:/var/temp/tmpfmn52e/setuptools-33.1.1/setuptools/command/install_lib.py", line 90, in copy_tree assert preserve_mode and preserve_times and not preserve_symlinks AssertionError Something went wrong during the installation. See the error message above.
install_lib.py beginning at line 86 looks like this:
def copy_tree( self, infile, outfile, preserve_mode=1, preserve_times=1, preserve_symlinks=0, level=1 ): assert preserve_mode and preserve_times and not preserve_symlinks exclude = self.get_exclusions()
On the surface, I don't see anything wrong with the assertion which would cause it to be false.
comment:5 by , 6 years ago
I wonder what setuptool you use. As have version 34.4.1 installed. And I have exactly this fixed in this version.
comment:8 by , 6 years ago
Resolution: | → fixed |
---|---|
Status: | accepted → closed |
The original issue (problem determining filesystem encoding) was resolved for the testcase by using Yuri's workaround. I can confirm that installing setuptools from RPM works; that was never the issue. I was installing setuptools from ez_setup.py. Perhaps the build of setuptools downloaded using that method is incompatible with our environment.
This issue started as an inability to install pip (and wheel) necessary to install M2Crypto. Now that setuptools works, I should open specific tickets for what is failing during the installation of pip.
comment:9 by , 5 years ago
Resolution: | fixed |
---|---|
Status: | closed → reopened |
I've just faced a very similar issue here: https://github.com/bitwiseworks/qtwebengine-chromium-os2/issues/3#issuecomment-628684752. There, Python own SAX code also assumes that sys.getfilesystemencoding()
never returns None while the docs clearly say otherwise.
Since the original issue was just worked around instead of fixing, I'm reopening this.
Given that this is the second failing case, I'm going to make sys.getfilesystemencoding()
return what sys.getdefaultencoding()
returns instead of None. This will fix both cases w/o any workarounds needed. Note that `sys.getdefaultencoding()` is not supposed to return None so we should be on the safe side here.
comment:10 by , 5 years ago
Doing what I suggested above fixes both issues but I'm now getting problems rebuilding Python. Seems that our sqlite3 package has changed since we built the Python RPM last time and we no more provide sqlite3.lib or such and the Python EMX compiler setup script can't find sqlite3_dll.a
. Without the sqlite3 module in Python, yum can't function (at all).
I need to fix more bits in Python it seems.
comment:11 by , 5 years ago
Closing this in favor of https://github.com/bitwiseworks/python-os2/issues/1.
comment:12 by , 5 years ago
Resolution: | → duplicate |
---|---|
Status: | reopened → closed |
Since encoding is None, .decode fails... This workaround works for me:
and instead of easy setup, I followed the standard way of installing python packages
I'm leaving this ticket open since we need a fix for path.decode() function.