"tahoe --version" shouldn't show scary diagnostic warnings #2436

Closed
opened 2015-05-27 21:14:16 +00:00 by warner · 13 comments
warner commented 2015-05-27 21:14:16 +00:00
Owner

It is very common for tahoe --version to report diagnostics warnings about the difference between what pkg_resources claims to provide, and what the actual imported modules (and their *version*) reports. These warnings have convinced several users that their tahoe installation has failed, when in fact everything will would work normally apart from those warnings.

Zooko and I agreed on IRC to change tahoe --version to report the versions of Tahoe itself and its dependencies (foolscap, pycryptopp, etc), but to remove the import-vs-pkg_resources warnings. We'll add a separate tahoe debug versions command that emits complete diagnostics.

For reference, this is what the problematic output looks like:

% bin/tahoe -V
allmydata-tahoe: 1.10.0
foolscap: 0.8.0
pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958
zfec: 1.4.24
Twisted: 13.0.0
Nevow: 0.11.1
zope.interface: unknown
python: 2.7.9
platform: Linux-fedora_22-x86_64-64bit_ELF
pyOpenSSL: 0.14
simplejson: 3.7.2
pycrypto: 2.6.1
pyasn1: 0.1.7
mock: 1.0.1
setuptools: 0.6c16dev4

Warning: dependency 'pycparser' (version '2.10') found by pkg_resources not found by import.
Warning: dependency 'six' (version '1.9.0') found by pkg_resources not found by import.
Warning: dependency 'cryptography' (version '0.8.2') found by pkg_resources not found by import.
Warning: dependency 'cffi' (version '0.8.6') found by pkg_resources not found by import.
Warning: dependency 'pyasn1-modules' (version '0.0.5') found by pkg_resources not found by import.
Warning: dependency 'enum34' (version '1.0') found by pkg_resources not found by import.
Warning: dependency 'characteristic' (version '14.3.0') found by pkg_resources not found by import.
Warning: dependency 'service-identity' (version '14.0.0') found by pkg_resources not found by import.

For debugging purposes, the PYTHONPATH was
  '/home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages'
install_requires was
  ['setuptools >= 0.6c6', 'zfec >= 1.1.0', 'simplejson >= 1.4', 'zope.interface == 3.6.0, == 3.6.1, == 3.6.2, >= 3.6.5', 'Twisted >= 11.0.0', 'foolscap >= 0.6.3', 'pyOpenSSL', 'Nevow >= 0.6.0', 'pycrypto == 2.1.0, == 2.3, >= 2.4.1', 'pyasn1 >= 0.0.8a', 'mock >= 0.8.0', 'pycryptopp >= 0.6.0']
sys.path after importing pkg_resources was
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/bin:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/setuptools-0.6c16dev4.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/src:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-x86_64.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/mock-1.0.1-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/Nevow-0.11.1-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/foolscap-0.8.0-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/zfec-1.4.24-py2.7-linux-x86_64.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/service_identity-14.0.0-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pyutil-2.0.0-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pyasn1_modules-0.0.5-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/characteristic-14.3.0-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/zbase32-1.1.5-py2.7.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/Twisted-13.0.0-py2.7-linux-x86_64.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/zope.interface-4.1.2-py2.7-linux-x86_64.egg:
  /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages:
  /usr/lib64/python27.zip:
  /usr/lib64/python2.7:
  /usr/lib64/python2.7/plat-linux2:
  /usr/lib64/python2.7/lib-tk:
  /usr/lib64/python2.7/lib-old:
  /usr/lib64/python2.7/lib-dynload:
  /usr/lib64/python2.7/site-packages:
  /usr/lib64/python2.7/site-packages/gst-0.10:
  /usr/lib64/python2.7/site-packages/gtk-2.0:
  /usr/lib/python2.7/site-packages

The output is large enough that, depending upon the size of your screen, you may wind up with the scary "Warning:... not found by import" lines at the top of the screen, which sounds an awful lot like an ImportError.

It is very common for `tahoe --version` to report diagnostics warnings about the difference between what `pkg_resources` claims to provide, and what the actual imported modules (and their `*version*`) reports. These warnings have convinced several users that their tahoe installation has failed, when in fact everything will would work normally apart from those warnings. Zooko and I agreed on IRC to change `tahoe --version` to report the versions of Tahoe itself and its dependencies (foolscap, pycryptopp, etc), but to remove the import-vs-pkg_resources warnings. We'll add a separate `tahoe debug versions` command that emits complete diagnostics. For reference, this is what the problematic output looks like: ``` % bin/tahoe -V allmydata-tahoe: 1.10.0 foolscap: 0.8.0 pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958 zfec: 1.4.24 Twisted: 13.0.0 Nevow: 0.11.1 zope.interface: unknown python: 2.7.9 platform: Linux-fedora_22-x86_64-64bit_ELF pyOpenSSL: 0.14 simplejson: 3.7.2 pycrypto: 2.6.1 pyasn1: 0.1.7 mock: 1.0.1 setuptools: 0.6c16dev4 Warning: dependency 'pycparser' (version '2.10') found by pkg_resources not found by import. Warning: dependency 'six' (version '1.9.0') found by pkg_resources not found by import. Warning: dependency 'cryptography' (version '0.8.2') found by pkg_resources not found by import. Warning: dependency 'cffi' (version '0.8.6') found by pkg_resources not found by import. Warning: dependency 'pyasn1-modules' (version '0.0.5') found by pkg_resources not found by import. Warning: dependency 'enum34' (version '1.0') found by pkg_resources not found by import. Warning: dependency 'characteristic' (version '14.3.0') found by pkg_resources not found by import. Warning: dependency 'service-identity' (version '14.0.0') found by pkg_resources not found by import. For debugging purposes, the PYTHONPATH was '/home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages' install_requires was ['setuptools >= 0.6c6', 'zfec >= 1.1.0', 'simplejson >= 1.4', 'zope.interface == 3.6.0, == 3.6.1, == 3.6.2, >= 3.6.5', 'Twisted >= 11.0.0', 'foolscap >= 0.6.3', 'pyOpenSSL', 'Nevow >= 0.6.0', 'pycrypto == 2.1.0, == 2.3, >= 2.4.1', 'pyasn1 >= 0.0.8a', 'mock >= 0.8.0', 'pycryptopp >= 0.6.0'] sys.path after importing pkg_resources was /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/bin: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/setuptools-0.6c16dev4.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/src: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-x86_64.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/mock-1.0.1-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/Nevow-0.11.1-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/foolscap-0.8.0-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/zfec-1.4.24-py2.7-linux-x86_64.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/service_identity-14.0.0-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pyutil-2.0.0-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/pyasn1_modules-0.0.5-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/characteristic-14.3.0-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/zbase32-1.1.5-py2.7.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/Twisted-13.0.0-py2.7-linux-x86_64.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages/zope.interface-4.1.2-py2.7-linux-x86_64.egg: /home/olabini/workspace/SPEAR/tahoe/allmydata-tahoe-1.10.0/support/lib/python2.7/site-packages: /usr/lib64/python27.zip: /usr/lib64/python2.7: /usr/lib64/python2.7/plat-linux2: /usr/lib64/python2.7/lib-tk: /usr/lib64/python2.7/lib-old: /usr/lib64/python2.7/lib-dynload: /usr/lib64/python2.7/site-packages: /usr/lib64/python2.7/site-packages/gst-0.10: /usr/lib64/python2.7/site-packages/gtk-2.0: /usr/lib/python2.7/site-packages ``` The output is large enough that, depending upon the size of your screen, you may wind up with the scary "Warning:... not found by import" lines at the top of the screen, which sounds an awful lot like an ImportError.
tahoe-lafs added the
code
normal
defect
1.10.0
labels 2015-05-27 21:14:16 +00:00
tahoe-lafs added this to the 1.10.1 milestone 2015-05-27 21:14:16 +00:00
zooko commented 2015-05-29 14:53:59 +00:00
Author
Owner

On IRC Daira said that she thinks all of the warnings shown in the user-supplied output above have been fixed in trunk. I'll ask the user to try the 10.1a1post9 tarball.

On IRC Daira said that she thinks all of the warnings shown in the user-supplied output above have been fixed in trunk. I'll ask the user to try the 10.1a1post9 tarball.
zooko commented 2015-05-29 15:05:50 +00:00
Author
Owner

(https://twitter.com/olabini/status/604300869832597504)

"the printing of sys.path REALLY looks like a stack trace btw."

I think a lesson here is never to show output to a user that looks like a stack trace, i.e. has a list of lines where the beginning of each line is a filename/pathname. I've seen this before: if a user sees something that looks like a stack trace, they immediately avert their gaze and assume that the software is malfunctioning.

(https://twitter.com/olabini/status/604300869832597504) "the printing of sys.path REALLY looks like a stack trace btw." I think a lesson here is never to show output to a user that looks like a stack trace, i.e. has a list of lines where the beginning of each line is a filename/pathname. I've seen this before: if a user sees something that looks like a stack trace, they immediately avert their gaze and assume that the software is malfunctioning.
warner commented 2015-05-29 19:11:15 +00:00
Author
Owner

What if tahoe --version emitted just the versions and a single line that said "warning: not all dependency versions matched expectations. Run 'tahoe debug versions' for more info." ? And then we move all that other (scary) stuff to the other command?

What if `tahoe --version` emitted just the versions and a single line that said "warning: not all dependency versions matched expectations. Run 'tahoe debug versions' for more info." ? And then we move all that other (scary) stuff to the other command?
warner commented 2015-06-01 05:34:22 +00:00
Author
Owner

Moving this out of 1.10.1, insufficient consensus to make a last-minute change.

Moving this out of 1.10.1, insufficient consensus to make a last-minute change.
tahoe-lafs modified the milestone from 1.10.1 to 1.11.0 2015-06-01 05:34:22 +00:00
tahoe-lafs modified the milestone from 1.11.0 to 1.10.2 2015-07-17 21:41:02 +00:00
daira commented 2015-07-17 21:43:06 +00:00
Author
Owner

The issue I have with not displaying a warning is that the printed versions may not be correct. However, I'm okay with printing a short warning on the line that displays each offending version (in the "comment" field), and suppressing the long debugging information in cases where that is the only problem.

The issue I have with not displaying a warning is that the printed versions *may not be correct*. However, I'm okay with printing a short warning on the line that displays each offending version (in the "comment" field), and suppressing the long debugging information in cases where that is the only problem.
daira commented 2015-07-17 21:49:10 +00:00
Author
Owner

So in the example in the description, for instance, the output would be:

allmydata-tahoe: 1.10.0
foolscap: 0.8.0
pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958
zfec: 1.4.24
Twisted: 13.0.0
Nevow: 0.11.1
zope.interface: unknown
python: 2.7.9
platform: Linux-fedora_22-x86_64-64bit_ELF
pyOpenSSL: 0.14
simplejson: 3.7.2
pycrypto: 2.6.1
pyasn1: 0.1.7
mock: 1.0.1
setuptools: 0.6c16dev4
pycparser: 2.10 [according to pkg_resources]
six: 1.9.0 [according to pkg_resources]
cryptography: 0.8.2 [according to pkg_resources]
cffi: 0.8.6 [according to pkg_resources]
pyasn1-modules: 0.0.5 [according to pkg_resources]
enum34: 1.0 [according to pkg_resources]
characteristic: 14.3.0 [according to pkg_resources]
service-identity: 14.0.0 [according to pkg_resources]
So in the example in the description, for instance, the output would be: ``` allmydata-tahoe: 1.10.0 foolscap: 0.8.0 pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958 zfec: 1.4.24 Twisted: 13.0.0 Nevow: 0.11.1 zope.interface: unknown python: 2.7.9 platform: Linux-fedora_22-x86_64-64bit_ELF pyOpenSSL: 0.14 simplejson: 3.7.2 pycrypto: 2.6.1 pyasn1: 0.1.7 mock: 1.0.1 setuptools: 0.6c16dev4 pycparser: 2.10 [according to pkg_resources] six: 1.9.0 [according to pkg_resources] cryptography: 0.8.2 [according to pkg_resources] cffi: 0.8.6 [according to pkg_resources] pyasn1-modules: 0.0.5 [according to pkg_resources] enum34: 1.0 [according to pkg_resources] characteristic: 14.3.0 [according to pkg_resources] service-identity: 14.0.0 [according to pkg_resources] ```
tahoe-lafs added
packaging
and removed
code
labels 2015-07-17 21:51:23 +00:00
warner commented 2015-07-17 23:32:32 +00:00
Author
Owner

That sounds like a good compromise to me. I think I'll use () instead of [].

That sounds like a good compromise to me. I think I'll use `()` instead of `[]`.
daira commented 2015-07-18 01:23:32 +00:00
Author
Owner

[] is already used for comments; () is used for paths when --version-and-path is given. I want the output to remain consistently parsable.

`[]` is already used for comments; `()` is used for paths when `--version-and-path` is given. I want the output to remain consistently parsable.
daira commented 2015-07-21 17:57:00 +00:00
Author
Owner
Review needed for <https://github.com/tahoe-lafs/tahoe-lafs/pull/176>.
Daira Hopwood <daira@jacaranda.org> commented 2015-07-21 17:59:34 +00:00
Author
Owner

In /tahoe-lafs/trac-2024-07-25/commit/431728f8f854e02d9ab2f731675f12ce18cda122:

Don't show scary diagnostic warnings from --version[-and-path]
if the only problem is that extra packages are found by pkg_resources.
fixes ticket:2436

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
In [/tahoe-lafs/trac-2024-07-25/commit/431728f8f854e02d9ab2f731675f12ce18cda122](/tahoe-lafs/trac-2024-07-25/commit/431728f8f854e02d9ab2f731675f12ce18cda122): ``` Don't show scary diagnostic warnings from --version[-and-path] if the only problem is that extra packages are found by pkg_resources. fixes ticket:2436 Signed-off-by: Daira Hopwood <daira@jacaranda.org> ```
tahoe-lafs added the
fixed
label 2015-07-21 17:59:34 +00:00
Daira Hopwood <daira@jacaranda.org> closed this issue 2015-07-21 17:59:34 +00:00
daira commented 2015-07-21 19:46:56 +00:00
Author
Owner

Oops, I just spotted a bug. get_package_versions_string extends _vers_and_locs_list each time it is called (assuming not hasattr(sys, 'frozen')), but it may be called more than once.

I will fix it.

Oops, I just spotted a bug. `get_package_versions_string` extends `_vers_and_locs_list` each time it is called (assuming `not hasattr(sys, 'frozen')`), but it may be called more than once. I will fix it.
tahoe-lafs removed the
fixed
label 2015-07-21 19:46:56 +00:00
daira reopened this issue 2015-07-21 19:46:56 +00:00
Daira Hopwood <daira@jacaranda.org> commented 2015-07-29 00:40:35 +00:00
Author
Owner

In /tahoe-lafs/trac-2024-07-25/commit/2c99294a4faa0e0f94fdb53ed60954e1e0724955:

Don't show scary diagnostic warnings from --version[-and-path] (corrected). refs ticket:2436

The previous version would incorrectly add to the output of
get_package_versions_string each time it was called.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
In [/tahoe-lafs/trac-2024-07-25/commit/2c99294a4faa0e0f94fdb53ed60954e1e0724955](/tahoe-lafs/trac-2024-07-25/commit/2c99294a4faa0e0f94fdb53ed60954e1e0724955): ``` Don't show scary diagnostic warnings from --version[-and-path] (corrected). refs ticket:2436 The previous version would incorrectly add to the output of get_package_versions_string each time it was called. Signed-off-by: Daira Hopwood <daira@jacaranda.org> ```
warner commented 2015-07-29 00:41:33 +00:00
Author
Owner

Landed in 28399b6.

Landed in 28399b6.
tahoe-lafs added the
fixed
label 2015-07-29 00:41:33 +00:00
warner closed this issue 2015-07-29 00:41:33 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: tahoe-lafs/trac-2024-07-25#2436
No description provided.