tahoe-lafs/trac-2024-07-25

UnicodeEncodeError in test_client.Basic.test_absolute_storage_dir on linux when LANG= is not set #2912

New issue

Closed

opened 2018-03-28 18:07:57 +00:00 by warner · 1 comment

warner commented

2018-03-28 18:07:57 +00:00

Owner

In 1f92879 we got a new test, test_client.Basic.test_absolute_storage_dir, which exercises the new storage storage_dir= control in tahoe.cfg . Apparently this fails on linux systems where LANG= is not set (usually it's set to something like en_US.UTF-8), which makes one of the system encodings to be "ascii" instead of "utf-8", which is causing a UnicodeEncodeError inside an os.makedirs() call in the test:

[ERROR]
Traceback (most recent call last):
  File "/home/warner/stuff/tahoe/tahoe/src/allmydata/test/test_client.py", line 330, in test_absolute_storage_dir
    expected_path,
  File "/home/warner/stuff/tahoe/tahoe/src/allmydata/test/test_client.py", line 270, in _storage_dir_test
    c = client.create_client(basedir)
  File "/home/warner/stuff/tahoe/tahoe/src/allmydata/client.py", line 163, in create_client
    basedir=basedir
  File "/home/warner/stuff/tahoe/tahoe/src/allmydata/client.py", line 202, in __init__
    self.init_storage()
  File "/home/warner/stuff/tahoe/tahoe/src/allmydata/client.py", line 401, in init_storage
    expiration_sharetypes=expiration_sharetypes)
  File "/home/warner/stuff/tahoe/tahoe/src/allmydata/storage/server.py", line 55, in __init__
    fileutil.make_dirs(sharedir)
  File "/home/warner/stuff/tahoe/tahoe/src/allmydata/util/fileutil.py", line 183, in make_dirs
    os.makedirs(dirname, mode)
  File "/home/warner/stuff/tahoe/tahoe/.tox/py27/lib/python2.7/os.py", line 148, in makedirs
    if head and tail and not path.exists(head):
  File "/home/warner/stuff/tahoe/tahoe/.tox/py27/lib/python2.7/genericpath.py", line 26, in exists
    os.stat(path)
exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\u2603' in position 91: ordinal not in range(128)

allmydata.test.test_client.Basic.test_absolute_storage_dir

This is affecting a couple of the buildbot builders (Centos 7, Fedora 24, and Slackware).

We might want to add an additional Travis test, with LANG=, to try and catch this on pull-requests (since the buildbot currently only runs after-the-fact on trunk).

In 1f92879 we got a new test, `test_client.Basic.test_absolute_storage_dir`, which exercises the new `storage storage_dir=` control in tahoe.cfg . Apparently this fails on linux systems where LANG= is not set (usually it's set to something like `en_US.UTF-8`), which makes one of the system encodings to be "ascii" instead of "utf-8", which is causing a UnicodeEncodeError inside an `os.makedirs()` call in the test: ``` [ERROR] Traceback (most recent call last): File "/home/warner/stuff/tahoe/tahoe/src/allmydata/test/test_client.py", line 330, in test_absolute_storage_dir expected_path, File "/home/warner/stuff/tahoe/tahoe/src/allmydata/test/test_client.py", line 270, in _storage_dir_test c = client.create_client(basedir) File "/home/warner/stuff/tahoe/tahoe/src/allmydata/client.py", line 163, in create_client basedir=basedir File "/home/warner/stuff/tahoe/tahoe/src/allmydata/client.py", line 202, in __init__ self.init_storage() File "/home/warner/stuff/tahoe/tahoe/src/allmydata/client.py", line 401, in init_storage expiration_sharetypes=expiration_sharetypes) File "/home/warner/stuff/tahoe/tahoe/src/allmydata/storage/server.py", line 55, in __init__ fileutil.make_dirs(sharedir) File "/home/warner/stuff/tahoe/tahoe/src/allmydata/util/fileutil.py", line 183, in make_dirs os.makedirs(dirname, mode) File "/home/warner/stuff/tahoe/tahoe/.tox/py27/lib/python2.7/os.py", line 148, in makedirs if head and tail and not path.exists(head): File "/home/warner/stuff/tahoe/tahoe/.tox/py27/lib/python2.7/genericpath.py", line 26, in exists os.stat(path) exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\u2603' in position 91: ordinal not in range(128) allmydata.test.test_client.Basic.test_absolute_storage_dir ``` This is affecting a couple of the buildbot builders (Centos 7, Fedora 24, and Slackware). We might want to add an additional Travis test, with `LANG=`, to try and catch this on pull-requests (since the buildbot currently only runs after-the-fact on trunk).

tahoe-lafs added the

labels 2018-03-28 18:07:57 +00:00

tahoe-lafs added this to the 1.13.0 milestone 2018-03-28 18:07:57 +00:00

Brian Warner <warner@lothar.com> commented

2018-03-29 00:40:59 +00:00

Author

Owner

In de41ece/trunk:

Merge branch '2912-unicode-error'

closes ticket:2912

Also adds a new travis builder (with LANG=C) to exercise the same unicode
problems that several of our buildbot workers see.

In [de41ece/trunk](/tahoe-lafs/trac-2024-07-25/commit/de41ecee46e9695007f86596080433dba6bd6f74): ``` Merge branch '2912-unicode-error' closes ticket:2912 Also adds a new travis builder (with LANG=C) to exercise the same unicode problems that several of our buildbot workers see. ```

tahoe-lafs added the

fixed

label 2018-03-29 00:40:59 +00:00

Brian Warner <warner@lothar.com> closed this issue

2018-03-29 00:40:59 +00:00