Running unit tests throws DeprecationWarnings #859

New issue

Closed

opened 2009-12-14 20:24:34 +00:00 by imhavoc · 19 comments

imhavoc commented

2009-12-14 20:24:34 +00:00

Owner

bigpig@~ $ tahoe restart
STOPPING /home/bigpig/.tahoe
process 11011 is dead
STARTING /home/bigpig/.tahoe
/usr/lib/pymodules/python2.6/foolscap/banana.py:2: [DeprecationWarning](wiki/DeprecationWarning): the sets module is deprecated
> import struct, sets, time
/usr/lib/python2.6/dist-packages/formless/annotate.py:730: [DeprecationWarning](wiki/DeprecationWarning): object.*new*() takes no parameters
> rv = cls = [InterfaceClass](wiki/InterfaceClass).*new*(cls, name, bases, dct)
/usr/lib/python2.6/dist-packages/nevow/testutil.py:7: [DeprecationWarning](wiki/DeprecationWarning): The popen2 module is deprecated.  Use the subprocess module.
> from popen2 import Popen3
/usr/lib/python2.6/dist-packages/nevow/guard.py:15: [DeprecationWarning](wiki/DeprecationWarning): the md5 module is deprecated; use hashlib instead
> import md5
/usr/lib/pymodules/python2.6/allmydata/storage_client.py:31: [DeprecationWarning](wiki/DeprecationWarning): the sha module is deprecated; use the hashlib module instead
> import sha, time
client node probably started

System is AMD64, dual-core. Tahoe-lafs runs fine.

This tahoe-lafs was installed form apt-get. Let me know if I can supply more informatoin.

``` bigpig@~ $ tahoe restart STOPPING /home/bigpig/.tahoe process 11011 is dead STARTING /home/bigpig/.tahoe /usr/lib/pymodules/python2.6/foolscap/banana.py:2: [DeprecationWarning](wiki/DeprecationWarning): the sets module is deprecated > import struct, sets, time /usr/lib/python2.6/dist-packages/formless/annotate.py:730: [DeprecationWarning](wiki/DeprecationWarning): object.*new*() takes no parameters > rv = cls = [InterfaceClass](wiki/InterfaceClass).*new*(cls, name, bases, dct) /usr/lib/python2.6/dist-packages/nevow/testutil.py:7: [DeprecationWarning](wiki/DeprecationWarning): The popen2 module is deprecated. Use the subprocess module. > from popen2 import Popen3 /usr/lib/python2.6/dist-packages/nevow/guard.py:15: [DeprecationWarning](wiki/DeprecationWarning): the md5 module is deprecated; use hashlib instead > import md5 /usr/lib/pymodules/python2.6/allmydata/storage_client.py:31: [DeprecationWarning](wiki/DeprecationWarning): the sha module is deprecated; use the hashlib module instead > import sha, time client node probably started ``` System is AMD64, dual-core. Tahoe-lafs runs fine. This tahoe-lafs was installed form apt-get. Let me know if I can supply more informatoin.

tahoe-lafs added the

labels 2009-12-14 20:24:34 +00:00

tahoe-lafs added this to the undecided milestone 2009-12-14 20:24:34 +00:00

zooko commented

2009-12-14 22:00:07 +00:00

Thank you for the bug report!

The deprecation warnings from foolscap have been fixed in foolscap trunk but a new release of foolscap hasn't made containing the fix: http://foolscap.lothar.com/trac/ticket/124

The deprecation warnings from nevow (which includes the one from the package named formless in your bug report) I don't know about. I just searched the Nevow trac and didn't find evidence that those have been fixed. Could you run tahoe --version and tell me what version of Nevow is in use? I can see from the Nevow trac that the latest release of Nevow is v0.10.0.

If these deprecation warnings are still in Nevow v0.10.0 then we should open a ticket with them.

The last one is our fault:

/usr/lib/pymodules/python2.6/allmydata/storage_client.py:31: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha, time

That one is fixed by this patch I just committed: changeset:b69e1c600dc7ff7f.

Okay so please see what version of Nevow you have, if it isn't v0.10.0 then upgrade it to v0.10.0 and see if the DeprecationWarnings are still there, and then we'll open a bug report with Nevow.

Thank you for the bug report! The deprecation warnings from foolscap have been fixed in foolscap trunk but a new release of foolscap hasn't made containing the fix: <http://foolscap.lothar.com/trac/ticket/124> The deprecation warnings from nevow (which includes the one from the package named `formless` in your bug report) I don't know about. I just searched [the Nevow trac](http://divmod.org/trac/wiki/DivmodNevow) and didn't find evidence that those have been fixed. Could you run `tahoe --version` and tell me what version of Nevow is in use? I can see from [the Nevow trac](http://divmod.org/trac/wiki/DivmodNevow) that the latest release of Nevow is v0.10.0. If these deprecation warnings are still in Nevow v0.10.0 then we should open a ticket with them. The last one is our fault: ``` /usr/lib/pymodules/python2.6/allmydata/storage_client.py:31: DeprecationWarning: the sha module is deprecated; use the hashlib module instead import sha, time ``` That one is fixed by this patch I just committed: changeset:b69e1c600dc7ff7f. Okay so please see what version of Nevow you have, if it isn't v0.10.0 then upgrade it to v0.10.0 and see if the DeprecationWarnings are still there, and then we'll open a bug report with Nevow.

imhavoc commented

2009-12-14 22:11:22 +00:00

Author

Owner

tahoe --version tells me:

allmydata-tahoe: 1.5.0, foolscap: 0.4.2, pycryptopp: 0.5.17, zfec: 1.4.5, Twisted: 8.2.0, Nevow: 0.9.31, zope.interface: unknown, python: 2.6.4, platform: Linux-Ubuntu_9.10-x86_64-64bit_ELF, sqlite: 3.6.16, simplejson: 2.0.9, pyOpenSSL: 0.9, setuptools: 0.6c9, pysqlite: 2.4.1

Is there a graceful way to update all my Python modules?

`tahoe --version` tells me: ``` allmydata-tahoe: 1.5.0, foolscap: 0.4.2, pycryptopp: 0.5.17, zfec: 1.4.5, Twisted: 8.2.0, Nevow: 0.9.31, zope.interface: unknown, python: 2.6.4, platform: Linux-Ubuntu_9.10-x86_64-64bit_ELF, sqlite: 3.6.16, simplejson: 2.0.9, pyOpenSSL: 0.9, setuptools: 0.6c9, pysqlite: 2.4.1 ``` Is there a graceful way to update all my Python modules?

zooko commented

2009-12-14 22:35:19 +00:00

Opinions differ. I would suggest:

sudo apt-get install python-setuptools
sudo easy_install -U Nevow

Other people might say that this is insanity and they might plead with you to think twice before you throw away all the beauty and perfection of your Ubuntu packaging system.

But don't listen to them.

Opinions differ. I would suggest: ``` sudo apt-get install python-setuptools sudo easy_install -U Nevow ``` Other people might say that this is insanity and they might plead with you to think twice before you throw away all the beauty and perfection of your Ubuntu packaging system. But don't listen to them.

imhavoc commented

2009-12-14 22:53:02 +00:00

Author

Owner

Well, it worked fine. Thank you. apt-get is all fine and dandy until you want to update something outside of their timeline.

Well, it worked fine. Thank you. `apt-get` is all fine and dandy until you want to update something outside of their timeline.

tahoe-lafs changed title from ~~Ubuntu 9.10, Starting Tahoe-lafs through multiple DeprecationWarnings~~ to Ubuntu 9.10, Starting Tahoe-lafs throws multiple DeprecationWarnings

2009-12-16 20:00:23 +00:00

zooko commented

2009-12-16 20:23:59 +00:00

So did you sudo easy_install -U Nevow, and when you did, was it v0.10.0 of Nevow that was installed, and if it was, did the Nevow- and formless- related DeprecationWarnings stop happening?

So did you `sudo easy_install -U Nevow`, and when you did, was it v0.10.0 of Nevow that was installed, and if it was, did the Nevow- and formless- related DeprecationWarnings stop happening?

imhavoc commented

2009-12-16 21:00:58 +00:00

Author

Owner

Yes, I ran the sudo easy_install -U Nevow command, and it updated Nevow to v0.10.0 flawlessly. That did not stop the DeprecationWarnings, though:

This is what a tahoe restart looks like on the system in question after the update of Nevow:

$ tahoe restart
STOPPING /home/bigpig/.tahoe
process 14341 is dead
STARTING /home/bigpig/.tahoe
/usr/lib/pymodules/python2.6/foolscap/banana.py:2: DeprecationWarning: the sets module is deprecated
  import struct, sets, time
/usr/local/lib/python2.6/dist-packages/Nevow-0.10.0-py2.6.egg/formless/annotate.py:730: DeprecationWarning: object.__new__() takes no parameters
  rv = cls = InterfaceClass.__new__(cls, name, bases, dct)
/usr/local/lib/python2.6/dist-packages/Nevow-0.10.0-py2.6.egg/nevow/testutil.py:7: DeprecationWarning: The popen2 module is deprecated.  Use the subprocess module.
  from popen2 import Popen3
/usr/local/lib/python2.6/dist-packages/Nevow-0.10.0-py2.6.egg/nevow/guard.py:15: DeprecationWarning: the md5 module is deprecated; use hashlib instead
  import md5
/usr/lib/pymodules/python2.6/allmydata/storage_client.py:31: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha, time
client node probably started

Yes, I ran the `sudo easy_install -U Nevow` command, and it updated Nevow to v0.10.0 flawlessly. That did not stop the `DeprecationWarning`s, though: This is what a `tahoe restart` looks like on the system in question after the update of Nevow: ``` $ tahoe restart STOPPING /home/bigpig/.tahoe process 14341 is dead STARTING /home/bigpig/.tahoe /usr/lib/pymodules/python2.6/foolscap/banana.py:2: DeprecationWarning: the sets module is deprecated import struct, sets, time /usr/local/lib/python2.6/dist-packages/Nevow-0.10.0-py2.6.egg/formless/annotate.py:730: DeprecationWarning: object.__new__() takes no parameters rv = cls = InterfaceClass.__new__(cls, name, bases, dct) /usr/local/lib/python2.6/dist-packages/Nevow-0.10.0-py2.6.egg/nevow/testutil.py:7: DeprecationWarning: The popen2 module is deprecated. Use the subprocess module. from popen2 import Popen3 /usr/local/lib/python2.6/dist-packages/Nevow-0.10.0-py2.6.egg/nevow/guard.py:15: DeprecationWarning: the md5 module is deprecated; use hashlib instead import md5 /usr/lib/pymodules/python2.6/allmydata/storage_client.py:31: DeprecationWarning: the sha module is deprecated; use the hashlib module instead import sha, time client node probably started ```

warner commented

2009-12-30 01:03:38 +00:00

FYI, the next release of Foolscap will fix the first of those warnings. The change is already in the hg repo, and I'm hoping to cut a release before tahoe-1.6.0 is out.

zooko commented

2010-01-09 23:47:59 +00:00

I updated (@@http://divmod.org/trac/ticket/2994#comment:374810@@) explaining that we hope to get this fixed for Lucid. If the Nevow folks (with our help) don't put out a new Nevow release in time and get it into Lucid then I propose that we suppress these warnings in Tahoe-LAFS using the Python warnings module from the stdlib.

I updated (@@http://divmod.org/trac/ticket/2994#[comment:374810](/tahoe-lafs/trac/issues/859#issuecomment-374810)@@) explaining that we hope to get this fixed for Lucid. If the Nevow folks (with our help) don't put out a new Nevow release in time and get it into Lucid then I propose that we suppress these warnings in Tahoe-LAFS using the Python `warnings` module from the stdlib.

zooko commented

2010-01-09 23:53:58 +00:00

I updated (@@http://foolscap.lothar.com/trac/ticket/124#comment:374813@@) mentioning that we need a new foolscap release or else suppress this warning in Tahoe-LAFS.

I updated (@@http://foolscap.lothar.com/trac/ticket/124#[comment:374813](/tahoe-lafs/trac/issues/859#issuecomment-374813)@@) mentioning that we need a new foolscap release or else suppress this warning in Tahoe-LAFS.

daira commented

2010-01-16 00:54:45 +00:00

Not specific to Ubuntu; I get the same warnings on cygwin.

daira added

c/code

and removed

c/unknown

labels 2010-01-16 00:54:45 +00:00

daira changed title from ~~Ubuntu 9.10, Starting Tahoe-lafs throws multiple DeprecationWarnings~~ to Starting Tahoe-lafs throws multiple DeprecationWarnings

2010-01-16 00:54:45 +00:00

zooko commented

2010-01-20 04:18:36 +00:00

Foolscap v0.5 is out and it allegedly contains the fix for http://foolscap.lothar.com/trac/ticket/124 . If someone could confirm that Tahoe-LAFS emits no DeprecationWarnings about "sets" when run with Foolscap v0.5 then we can consider that issue fixed.

Foolscap v0.5 is out and it allegedly contains the fix for <http://foolscap.lothar.com/trac/ticket/124> . If someone could confirm that Tahoe-LAFS emits no DeprecationWarnings about "sets" when run with Foolscap v0.5 then we can consider that issue fixed.

zooko commented

2010-01-29 22:39:31 +00:00

So if Foolscap v0.5.0 gets into Ubuntu 10.04 then that DeprecationWarning will be fixed in 10.04. Here is the issue ticket for Debian requesting that Debian upgrade foolscap: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=567145 .

For Nevow it doesn't look like it is going to get fixed in Nevow trunk, a new version of Nevow released, and that new version accepted into Ubuntu in time for 10.04, so I think we should suppress the DeprecationWarning in Tahoe-LAFS. There is sample code to do that in the Nevow ticket: (@@http://divmod.org/trac/ticket/2994#comment:-1@@)

To apply this to Tahoe-LAFS you find the line that emits the warning, which I assume is import nevow, and wrap it in:

import warnings
warnings.filterwarnings("ignore", category=DeprecationWarning, 
    message="object.__new__\(\) takes no parameters", 
    append=True) 
try:
    import nevow
finally:
    warnings.filters.pop()

Also we should update source:src/allmydata/test/test_runner.py to make the code-under-test fail if it emits anything to stderr when the tahoe command is executed. (test_runner.py used to do that, but I made it stop doing that and allow the code-under-test to pass while emitting DeprecationWarnings. Oh, here is the patch: changeset:3a1196701f088a5f.)

So if Foolscap v0.5.0 gets into Ubuntu 10.04 then that DeprecationWarning will be fixed in 10.04. Here is the issue ticket for Debian requesting that Debian upgrade foolscap: <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=567145> . For Nevow it doesn't look like it is going to get fixed in Nevow trunk, a new version of Nevow released, and that new version accepted into Ubuntu in time for 10.04, so I think we should suppress the DeprecationWarning in Tahoe-LAFS. There is sample code to do that in the Nevow ticket: (@@http://divmod.org/trac/ticket/2994#[comment:-1](/tahoe-lafs/trac/issues/859#issuecomment--1)@@) To apply this to Tahoe-LAFS you find the line that emits the warning, which I assume is `import nevow`, and wrap it in: ``` import warnings warnings.filterwarnings("ignore", category=DeprecationWarning, message="object.__new__ takes no parameters", append=True) try: import nevow finally: warnings.filters.pop() ``` Also we should update source:src/allmydata/test/test_runner.py to make the code-under-test fail if it emits anything to stderr when the `tahoe` command is executed. (test_runner.py used to do that, but I made it stop doing that and allow the code-under-test to pass while emitting DeprecationWarnings. Oh, here is the patch: changeset:3a1196701f088a5f.)

zooko modified the milestone from undecided to 1.6.0

2010-01-29 22:39:31 +00:00

zooko commented

2010-01-31 15:52:08 +00:00

Attachment suppress-DepWarn.darcspatch.txt (50732 bytes) added

This patch suppresses the warnings that result from import nevow and adds back in a unit test that makes sure that the code under test doesn't emit anything to stdout when run in --quiet mode.

**Attachment** suppress-DepWarn.darcspatch.txt (50732 bytes) added This patch suppresses the warnings that result from `import nevow` and adds back in a unit test that makes sure that the code under test doesn't emit anything to stdout when run in --quiet mode.

suppress-DepWarn.darcspatch.txt

50 KiB

zooko commented

2010-01-31 15:53:21 +00:00

Folks: review-needed! This one is very easy to review. The only thing that you should think about is whether importing nevow in source:src/allmydata/init.py will cause some weird packaging/loading problem. I can't think of anything. Anyway, as soon as someone says "This is okay" then we'll see how the buildbots like it.

Folks: review-needed! This one is very easy to review. The only thing that you should think about is whether importing nevow in source:src/allmydata/*init*.py will cause some weird packaging/loading problem. I can't think of anything. Anyway, as soon as someone says "This is okay" then we'll see how the buildbots like it.

daira commented

2010-02-01 00:36:03 +00:00

This is okay. It does introduce a pyflakes warning, which can be suppressed by

    hush_pyflakes = nevow
    del hush_pyflakes

after import nevow. Also it doesn't suppress the warnings that occur before running unit tests:

c:\python26\lib\site-packages\twisted-8.2.0-py2.6-win32.egg\twisted\persisted\sob.py:12: DeprecationWarning: the md5 module is deprecated; use hashlib instead
  import os, md5, sys
c:\python26\lib\site-packages\twisted-8.2.0-py2.6-win32.egg\twisted\python\filepath.py:12: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha

This is okay. It does introduce a pyflakes warning, which can be suppressed by ``` hush_pyflakes = nevow del hush_pyflakes ``` after `import nevow`. Also it doesn't suppress the warnings that occur before running unit tests: ``` c:\python26\lib\site-packages\twisted-8.2.0-py2.6-win32.egg\twisted\persisted\sob.py:12: DeprecationWarning: the md5 module is deprecated; use hashlib instead import os, md5, sys c:\python26\lib\site-packages\twisted-8.2.0-py2.6-win32.egg\twisted\python\filepath.py:12: DeprecationWarning: the sha module is deprecated; use the hashlib module instead import sha ```

daira commented

2010-02-01 00:52:39 +00:00

Attachment suppress-DepWarn-improved.darcspatch.txt (51311 bytes) added

cli: suppress DeprecationWarnings emitted from importing nevow and twisted. Fixes #859

**Attachment** suppress-DepWarn-improved.darcspatch.txt (51311 bytes) added cli: suppress [DeprecationWarnings](wiki/DeprecationWarnings) emitted from importing nevow and twisted. Fixes #859

suppress-DepWarn-improved.darcspatch.txt

50 KiB

zooko commented

2010-02-01 04:04:47 +00:00

applied David-Sarah's version: changeset:6215ebd95e292cca. Thanks!

zooko added the

r/fixed

label 2010-02-01 04:04:47 +00:00

zooko closed this issue

2010-02-01 04:04:47 +00:00

daira commented

2010-02-02 00:40:40 +00:00

The warnings on starting a node are fixed, but after the 1.6 release we should also fix the ones that occur on running setup.py trial (see comment:374826 above). I don't know where to put the warning suppression code to do that.

The warnings on starting a node are fixed, but after the 1.6 release we should also fix the ones that occur on running `setup.py trial` (see [comment:374826](/tahoe-lafs/trac/issues/859#issuecomment-374826) above). I don't know where to put the warning suppression code to do that.

daira removed the

r/fixed

label 2010-02-02 00:40:40 +00:00

daira modified the milestone from 1.6.0 to 1.7.0

2010-02-02 00:40:40 +00:00

daira reopened this issue

2010-02-02 00:40:40 +00:00

daira changed title from ~~Starting Tahoe-lafs throws multiple DeprecationWarnings~~ to Running unit tests throws DeprecationWarnings

2010-02-02 00:40:40 +00:00

zooko commented

2010-06-17 04:21:46 +00:00

I think these are all fixed. Let's not worry about suppressing the warnings from Twisted-8.2.0 since newer versions of Twisted have already been released that don't emit those warnings.