provide binary .egg's for pycryptopp for all supported platforms and versions of Python #804
Labels
No labels
0.2.0
0.3.0
0.4.0
0.5.0
0.5.1
0.6.0
0.6.1
0.7.0
0.8.0
0.9.0
1.0.0
1.1.0
1.10.0
1.10.1
1.10.2
1.10a2
1.11.0
1.12.0
1.12.1
1.13.0
1.14.0
1.15.0
1.15.1
1.2.0
1.3.0
1.4.1
1.5.0
1.6.0
1.6.1
1.7.0
1.7.1
1.7β
1.8.0
1.8.1
1.8.2
1.8.3
1.8β
1.9.0
1.9.0-s3branch
1.9.0a1
1.9.0a2
1.9.0b1
1.9.1
1.9.2
1.9.2a1
LeastAuthority.com automation
blocker
cannot reproduce
cloud-branch
code
code-dirnodes
code-encoding
code-frontend
code-frontend-cli
code-frontend-ftp-sftp
code-frontend-magic-folder
code-frontend-web
code-mutable
code-network
code-nodeadmin
code-peerselection
code-storage
contrib
critical
defect
dev-infrastructure
documentation
duplicate
enhancement
fixed
invalid
major
minor
n/a
normal
operational
packaging
somebody else's problem
supercritical
task
trivial
unknown
was already fixed
website
wontfix
worksforme
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: tahoe-lafs/trac-2024-07-25#804
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Some System Configuration Information
Downloads
Per these instructions I ran:
I've deviated from the prescribed protocol since the default install docs do not recommend acquiring the source from the darcs repo.
Perhaps differences between the tarball and the repo versions generate this bug.
To check this I also attempt to build from the tarball in a separate location.
I.E. I ran:
Build From Repo
Per these instructions I ran:
in the "from-repo" directory.
Upon inspection of the output (attached) I see several error-types.
I guess these two to be not-fatal:
Following those two error-types the next error is:
Difference Between First and Second Build Attempts in From-Tarball Version
In the "from-tar" directory I ran:
(Actually I changed the log names to be more descriptive, after the fact.)
I do not notice obvious differences between repo and tar.
I do notice differences between initial and subsequent build-attempts.
(Perhaps I should be writing "setup-" vice "build-" attempts.)
(Log names changed after the fact for clarity.)
Intriguingly "zope" appears to be missing "Python.h" on the first build attempt, this is not reported on subsequent build attempts.BR The missing Python.h in pycryptopp is reported in initial and subsequent build attempts:
First Versus Second From Repo Setup
Is the difference between initial and subsequent setup bugs also
seen in the from repo version?
To find out I tested a fresh from-repo setup attempt.
So the difference between zope and pycryptopp initial vs.
second installs generalizes to installs from tar and from repo.
Bug Interpretation
It looks like Python.h of pycryptopp is not found so I guess that I lack some necessary dependency.
Calling All Dependencies
(Since my bug does not seem to vary between from-repo and from-tar, I stick to from repo from here on.)
Where're these elusive Python.h's? Sound like C headers to me....
Naive:
OK, not surprising since docs say all repo is python, and what's missing is C. So, my belief that I have a dependency problem grows stronger.
Following [http://allmydata.org/source/tahoe/trunk/docs/install.html More Details], I learn that I can view "the canonical list of dependencies" in _auto_deps.py.
Of course, I cannot run "tahoe --version" yet so I skip that advice...
Aha! A "complete list of all dependencies..."
Hmmm.... next step... submit and wait... if no help then dissect above expression...
OK, I did try to dissect a tiny bit:
In ipython interpreter:
(And just to be sure I tried above with explicit path name from root to relevant site-packages.)
Which is:
Attachment from_repo_initial_setup.txt (35493 bytes) added
Attachment from_repo_second_setup.txt (8740 bytes) added
Attachment from_tar_build_log_initial.txt (35894 bytes) added
Attachment from_tar_build_log_second.txt (9267 bytes) added
OK, solved.
For my system:
Unfortunately when I ran setup.py install without sudo, I received a complaint about test_suite needing to be a list...
Anywho, it looks like basic install's OK now. Maybe I should write up an amateur install doc, for ubuntu people?
Cannot Install, No Python.h (zope, pycryptopp), Cannot Obtain Dependency Listto Cannot Install,Thank you for the bug report. This isn't fixed yet though, until either the process in the current install.html works, or the install.html is changed to describe a process that works.
I'm not sure exactly how to accomplish that, yet...
sudo? You had to use sudo? I'm sorry, I haven't read your bug report thoroughly enough yet. You definitely don't need to use sudo.
As to your issue "i.", the goal is to make it so that the list of dependencies in http://allmydata.org/source/tahoe/trunk/docs/install.html is really sufficient:
(Indeed, the goal would be to automate the installation of pywin32 too, so that the list of dependencies would be just Python. See http://twistedmatrix.com/trac/ticket/3238 (patch: declare that twisted requires pywin32 if it is to offer process management or iocp reactor on Windows), #756 (if pywin32 has been manually installed, setuptools still doesn't detect it), #142 (pywin32 can't be installed with setuptools), http://bugs.python.org/setuptools/issue18 (can't install pywin32), http://sourceforge.net/tracker/?func=detail&atid=551954&aid=1799934&group_id=78018 (easy_install silently fails).)
Anyway, my point is that the problem you encountered is that pycryptopp couldn't be automatically installed, and there are two ways to improve the situation: either (a) make pycryptopp be automatically installed on your platform and/or (b) document pycryptopp as something that has to be manually installed on your platform.
The former would be accomplished by building a binary egg of pycryptopp for your platform. I intend to configure the pycryptopp buildslaves to build eggs and upload them to http://allmydata.org in the same way that they currently build .deb's and upload those. One problem with this is http://bugs.python.org/setuptools/issue78 (egg platform names don't reflect unicode variant (UCS2, UCS4)).
The latter would be accomplished by editing the InstallDetails wiki page to tell Ubuntu users to do what you did. (sudo apt-get install python-dev)
Assigning to arc.
arc: what shall we do? I think we should narrow this ticket to be about the lack of a binary egg for pycryptopp on Linux. What do you think? If you agree, please change the title of this ticket and set it to be assigned to "nobody" instead of yourself. Perhaps we could also open another ticket saying that the InstallDetails page should include information about binary eggs of pycryptopp. Or instead of spending time opening that ticket, just spend the same time editing the wiki.
This bug has surfaced again.
System info:
tahoe@cat:~/allmydata-tahoe-1.5.0$ uname -a
Linux cat 2.6.24-24-generic #1 SMP Fri Sep 18 16:49:39 UTC 2009 i686 GNU/Linux
tahoe@cat:~/allmydata-tahoe-1.5.0$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 8.04.3 LTS
Release: 8.04
Codename: hardy
tahoe@cat:~/allmydata-tahoe-1.5.0$ cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 79
model name : AMD Athlon(tm) 64 Processor 3500+
stepping : 2
cpu MHz : 2210.208
cache size : 512 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow up pni cx16 lahf_lm svm extapic cr8_legacy ts fid vid ttp tm stc
bogomips : 4420.41
clflush size : 64
I tried using the solutions listed in this ticket. No joy.
This bug has surfaced again.
System info:
I tried using the solutions listed in this ticket. Build still failed.
Okay I'm changing this ticket to say that we should take strategy (a) from comment:114744 -- make binary .egg's of pycryptopp be installed for supported systems. arc's and imhavoc's systems are both amd64 with UCS4, and we have a pycryptopp buildbot that fits that description: http://allmydata.org/buildbot-pycryptopp/builders/linux-amd64-ubuntu-karmic-yukyuk . So to close this ticket, configure that buildbot to generate binary eggs and upload them to the tahoe-deps repository. (Then check if the builds start working for arc and imhavoc.) Oh, but arc was using Python 2.5 and imhavoc was using Python 2.6. Actually I think arc has subsequently upgraded his Ubuntu to Karmic so he has 2.6 as well. At least for now I'm defining this ticket as "provide binary .egg's for pycryptopp for linux-amd64-py2.6-ucs4".
Cannot Install,to provide binary .egg's for pycryptopp for linux-amd64-py2.6-ucs4Replying to zooko:
Actually, on this system, Python is at version 2.5.2. Sorry.
Okay, to close this ticket we need to generate binary .egg's for both Python 2.5 and Python 2.6 on Linux-amd64-ucs4.
provide binary .egg's for pycryptopp for linux-amd64-py2.6-ucs4to provide binary .egg's for pycryptopp for linux-amd64-py2.[56]-ucs4It turns out that my /home is a separate partition.
That partition was being mounted as 'noexec.'
When I edited fstab to make the partition mount as 'exec' and rebooted, tahoe compiled fine on the system in question.
Okay, so basically it turn out that you did have a C compiler, once you allowed it to be executable. Wait, was the cc1plus executable on your home partition?
In any case this ticket is about making precompiled pycryptopp binaries available for people who don't have a compiler.
I may have needed to install build-essential also.... I think I did. So, yes, I did have a C compiler available. So my problem may have been miss-files in this ticket.
So, the problem wasn't that your home directory was mounted no-exec, it was that you hadn't installed
build-essential? That would explain the error message saying thatcc1plusdidn't work. Hey waitasecond, where is the error message? I thought I saw a paste from you showingcc1plusfailing, but scanning back up through this ticket I don't see it.What was the exact error message that you got?
I emailed it to you. I think that I never pasted it here.
Oh, okay here was the original error message that imhavoc mailed to me:
Yep, that looks like
build-essentialwasn't installed. Now, once this ticket is fixed, then it will just get the binary .egg of pycryptopp and it won't even try to executegcc.One of my grid participants has run into this bug again.
Ubuntu 8.10
AMD (32-bit CPU)
Looks like pycrypto is not available. There is also no python-pycryptopp available in apt.
We're trying to get a grid online, but installing tahoe-lafs seems to be, by far, the hardest part of getting a grid online.
The workaround for Ubuntu is to install
build-essentialso that a pycryptopp will be built automatically from source.python -V
gcc -v
cat /proc/cpuinfo
build fail output:
Replying to zooko:
The informative line in that voluminous error output is:
So the work-around is to install
build-essentialandpython-dev. :-)Replying to zooko:
It would be immensely helpful if this information was included in the download/install documentation. Python is not throwing useful errors for people who are not familiar with python builds.
So, let me start a list of system requirements that should be in place if you expect to see a successful build:
build-essential,python-dev...I see that I also installed
python-opensslon my 8.04 system before getting a successful build.Am I missing anything?
There are some notes about such things on http://allmydata.org/trac/tahoe/wiki/InstallDetails . Those notes aren't on http://allmydata.org/source/tahoe/trunk/docs/install.html because those steps aren't supposed to be necessary. The fact that they are necessary (on some platforms) is a bug, and is the subject of this ticket. Improving the documentation for those platforms where binaries of pycryptopp and the other dependencies are not available would be the subject of... let's see... #282 (more detailed and targeted docs about installing from source). Hm, see also #701 (The instructions in "Installing Tahoe" do not mention C compiler requirement.) which is pretty much the same issue as you are having but opened by my brother Nathan.
I stumbled across this page from the InstallDetails page: http://allmydata.org/trac/tahoe/wiki/DownloadDebianPackages
That's useful information. It should be linked from the install.html page.
It is!
Err, sorry, I meant that the InstallDetails page is linked from the install.html page. The DownloadDebianPackages page is too platform-specific to be linked from the install.html page.
Since imhavoc's friend is using 32-bit, I hereby enlarge this ticket to also provide 32-bit binaries of pycryptopp.
provide binary .egg's for pycryptopp for linux-amd64-py2.[56]-ucs4to provide binary .egg's for pycryptopp for linux-{i386,amd64}-py2.{5,6}-ucs4Okay, we've now fixed #1094 which is to automatically build bdist_eggs of pycryptopp on all supported platforms. Now the question is whether the supported platforms (see the pycryptopp buildbot and the tahoe-dep-eggs directory on tahoe-lafs.org) include the platforms that the users on this ticket needed...
provide binary .egg's for pycryptopp for linux-{i386,amd64}-py2.{5,6}-ucs4to provide binary .egg's for pycryptopp for all supported platforms and versions of PythonHere is a nice table which shows which binary packages we are currently hosting for which platforms.
Any platform in http://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-dep-eggs/README.html which has a row that goes all the way across with every cell filled in is a platform where Tahoe-LAFS can be easily installed by someone who doesn't have a compiler or development libraries.
Attachment pycryptopp-0.5.25_r757-py2.6-netbsd-5.0.2-i386.egg (1919317 bytes) added
here's a binary egg for netbsd-5.0.2 i386, from the non-syslib buildslave.
I'm going to call this good enough. We have binary pycryptopp eggs for every platform on our Supported Builder list (go to http://tahoe-lafs.org/buildbot/ and click on "Supported Builders") except for Solaris. I guess I'll open a new ticket asking for binary eggs for all of the needed dependencies for Solaris (those are: Twisted, pyOpenSSL, pycrypto, pycryptopp, pywin32, zfec, zope.interface as shown on http://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-dep-eggs/README.html .) There, the Solaris row of the matrix is now #1294.
The bug still occurred for me on my raspberry pi running Linux 3.6.11+ armv6l and Python 2.7.3 where there is no pycryptoapp binary package. installing the suggested dependencies (python-dev python-openssl) solved this for me.
Thanks for the bug report, tilllt. Could you please paste in the output from
python misc/build_helpers/show-tool-versions.py? That will tell us what kind of binary someone would need to provide to make it install on your platform without needing the dev packages.well, i was too quick in assuming the build had worked. actually the python setup.py build command finishes without error, but then tahoe won't run, complaining it cannot find it's dependency pycryptopp >=0.6. which is kind of weird. as even the first half hour of python setup.py trial did not have complains until i stopped it.
this is the output
http://pastebin.com/eASzrxzk you asked for.
so apparently somehow an outdated pycryptopp was installed along the way.
now i did easy_install for pycryptopp from this page:
https://tahoe-lafs.org/trac/pycryptopp
and now tahoe fails with
so i guess it is a different bug than the others in here had.
ok, final update, after running:
sudo easy_install Twisted>=11.0.0
the build helpers look like this (and tahoe runs):
Replying to tilllt:
Hey wait a minute. The part of the output that I wanted is what comes before
locale.localeandbuildbot. The first line should beplatformand the secondmachine. Here's what it looks like on my system:ah sorry, i am running the raspberry pi headless and had "screen" running in the terminal which cut off the beginning of the output:
Cool, thanks! Okay, I think you will be able to install Tahoe-LAFS on Raspberry Pi without compiling anything from source as soon as there is a new row on https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/README.html in the "Python 2.7" table which row is labeled "linux-armv6l" and which has an egg in each of its six columns.
Okay, so which Raspberry Pi owner wants to build these six eggs for us?
Perhaps we should have a ticket for each platform that needs eggs building for it?
Note that some eggs need newer versions because of these dependency increases for 1.10:
Replying to daira:
Good idea. I created #1983.
Replying to daira:
Filed as #1978 to #1986.
Now we need to recruit some egg-building volunteers to assign the tickets to.