webdav frontend #451

Open
opened 2008-06-03 05:24:57 +00:00 by warner · 9 comments
warner commented 2008-06-03 05:24:57 +00:00
Owner

I suspect that the most convenient front-end in the long run is going to be a WEBDAV server. Most operating systems know how to mount DAV shares, we wouldn't need any special per-platform code, and our webapi server is pretty close to being a DAV server anyways.

OTOH there is some funky locking that must be done, and there might be some impedance mismatches that need to be resolved.

I suspect that the most convenient front-end in the long run is going to be a WEBDAV server. Most operating systems know how to mount DAV shares, we wouldn't need any special per-platform code, and our webapi server is pretty close to being a DAV server anyways. OTOH there is some funky locking that must be done, and there might be some impedance mismatches that need to be resolved.
tahoe-lafs added the
code-frontend-web
major
enhancement
1.0.0
labels 2008-06-03 05:24:57 +00:00
tahoe-lafs added this to the undecided milestone 2008-06-03 05:24:57 +00:00
warner commented 2008-09-09 20:12:45 +00:00
Author
Owner

I looked a little bit at this, and I think it's feasible. Our existing RESTful interface is pretty close to what a webdav client appears to use. The main difference is that directory listing uses a special PROPsomething method (instead of a special GET), and servers can implement locking. The PROPFIND/PROPSET methods use a bizarre XML syntax (as befits a microsoft-inspired protocol :-).

The best way to pursue this will start by building the apple calendar server (http://trac.calendarserver.org/), which includes a webdav server written in twisted (using web2). If we could get this server running inside a tahoe node, I think the rest would be pretty easy. Getting twisted.web2 into a tahoe process when it isn't in the regular twisted distribution is a hassle, but should be possible. (web2 was recently removed from twisted SVN, so it could spend some time maturing. The challenge is to manage the namespaces properly). web2 doesn't play well with Nevow, but I think we might want a separate port/server for the webdav stuff anyways, so that wouldn't be a problem.

Once that's running, the question is how people should use it. One possibility that might make regular users a bit more comfortable would be to give the tahoe node a table of username/password/rootcap, and have the webdav server implement HTTP Digest authentication, and teach the twisted.vfs layer that when a request shows up for /foo/bar.txt via username Bob, that means it should turn that into a GET for /uri/BOB's-rootcap/foo/bar.txt . Another possibility is that users could tell their clients to mount <http://localhost:8123/dav/ROOTCAP>, which would be more capabilitiesish.

After we build this, we might want to look into caching dirnodes for some limited period of time (5 seconds?), since at least Mac OS-X likes to look for a lot of hidden OS-X -specific files in each directory (to decide how to arrange the icons, where to open the window, etc).

The unix tool 'cadaver' is a DAV client that acts a lot like the old FTP client, and is useful for running tests. The easiest way to get a server running is to enable mod_dav in apache.

I looked a little bit at this, and I think it's feasible. Our existing RESTful interface is pretty close to what a webdav client appears to use. The main difference is that directory listing uses a special PROPsomething method (instead of a special GET), and servers can implement locking. The PROPFIND/PROPSET methods use a bizarre XML syntax (as befits a microsoft-inspired protocol :-). The best way to pursue this will start by building the apple calendar server (<http://trac.calendarserver.org/>), which includes a webdav server written in twisted (using web2). If we could get this server running inside a tahoe node, I think the rest would be pretty easy. Getting twisted.web2 into a tahoe process when it isn't in the regular twisted distribution is a hassle, but should be possible. (web2 was recently removed from twisted SVN, so it could spend some time maturing. The challenge is to manage the namespaces properly). web2 doesn't play well with Nevow, but I think we might want a separate port/server for the webdav stuff anyways, so that wouldn't be a problem. Once that's running, the question is how people should use it. One possibility that might make regular users a bit more comfortable would be to give the tahoe node a table of username/password/rootcap, and have the webdav server implement HTTP Digest authentication, and teach the twisted.vfs layer that when a request shows up for /foo/bar.txt via username Bob, that means it should turn that into a GET for /uri/BOB's-rootcap/foo/bar.txt . Another possibility is that users could tell their clients to mount `<http://localhost:8123/dav/ROOTCAP>`, which would be more capabilitiesish. After we build this, we might want to look into caching dirnodes for some limited period of time (5 seconds?), since at least Mac OS-X likes to look for a lot of hidden OS-X -specific files in each directory (to decide how to arrange the icons, where to open the window, etc). The unix tool 'cadaver' is a DAV client that acts a lot like the old FTP client, and is useful for running tests. The easiest way to get a server running is to enable mod_dav in apache.
tahoe-lafs modified the milestone from undecided to eventually 2010-02-11 03:36:36 +00:00
davidsarah commented 2010-04-04 22:35:33 +00:00
Author
Owner
See [wiki/GSoCIdeas#WebDAVSupport](wiki/GSoCIdeas#WebDAVSupport)
zooko commented 2010-10-19 03:09:32 +00:00
Author
Owner

There exists a very old project to add webdav to twisted: http://akadav.sourceforge.net/

And a branch within twisted: (@@http://www.mail-archive.com/twisted-web@twistedmatrix.com/msg02169.html@@)

There exists a very old project to add webdav to twisted: <http://akadav.sourceforge.net/> And a branch within twisted: (@@http://www.mail-archive.com/twisted-web@twistedmatrix.com/msg02169.html@@)
davidsarah commented 2010-10-20 02:18:30 +00:00
Author
Owner

Replying to zooko:

There exists a very old project to add webdav to twisted: http://akadav.sourceforge.net/

This has a dependency on PyXML, which is unmaintained.

And a branch within twisted: (@@http://www.mail-archive.com/twisted-web@twistedmatrix.com/msg02169.html@@)

This code depended on Twisted web2, which is dead. Progress on reanimating this branch seems slow.

I think that wsgidav is probably a better choice, because it seems to be actively maintained.

Replying to [zooko](/tahoe-lafs/trac-2024-07-25/issues/451#issuecomment-108689): > There exists a very old project to add webdav to twisted: <http://akadav.sourceforge.net/> This has a dependency on [PyXML](http://sourceforge.net/projects/pyxml/files/), which is unmaintained. > And a branch within twisted: (@@http://www.mail-archive.com/twisted-web@twistedmatrix.com/msg02169.html@@) This code depended on Twisted web2, which is dead. [Progress on reanimating this branch](@@http://twistedmatrix.com/trac/ticket/3081#[comment:-1](/tahoe-lafs/trac-2024-07-25/issues/451#issuecomment--1)@@) seems slow. I think that [wsgidav](http://code.google.com/p/wsgidav/) is probably a better choice, because it [seems to be actively maintained](http://code.google.com/p/wsgidav/source/list).
warner commented 2010-10-20 03:39:29 +00:00
Author
Owner

Yay! A new option for WebDAV!

But, doesn't WSGI usually means synchronous+blocking? twisted.web can call out to WSGI applications, but usually it's hard for a WSGI server to call out to Deferred-returning APIs. But, maybe there's a way to hack it in. It'd be awesome to get DAV working.

Yay! A new option for WebDAV! But, doesn't WSGI usually means synchronous+blocking? twisted.web can call out to WSGI applications, but usually it's hard for a WSGI server to call out to Deferred-returning APIs. But, maybe there's a way to hack it in. It'd be awesome to get DAV working.
davidsarah commented 2010-10-20 04:27:40 +00:00
Author
Owner

Replying to warner:

Yay! A new option for WebDAV!

But, doesn't WSGI usually means synchronous+blocking?

Yes. However this appears to say that someone got wsgidav working with Twisted's wsgi-container (see code in comment 10).

As you say, there might be problems calling back into Tahoe's APIs, since wsgi-container works by using deferToThread. But maybe you can have a WSGI thread block until the main Tahoe thread has responded to a message from it.

Replying to [warner](/tahoe-lafs/trac-2024-07-25/issues/451#issuecomment-108691): > Yay! A new option for WebDAV! > > But, doesn't WSGI usually means synchronous+blocking? Yes. However [this](http://code.google.com/p/wsgidav/issues/detail?id=41) appears to say that someone got wsgidav working with Twisted's wsgi-container (see code in comment 10). As you say, there might be problems calling back into Tahoe's APIs, since wsgi-container works by using `deferToThread`. But maybe you can have a WSGI thread block until the main Tahoe thread has responded to a message from it.
davidsarah commented 2011-01-03 20:34:00 +00:00
Author
Owner

Zope2 apparently has a WebDAV server, as does Zope3 (source of Zope3's implementation).

It appears to be fairly tightly dependent on the rest of Zope (example for COPY/MOVE), but is still worth looking at for inspiration, and for how it has solved client-compatibility issues.

[Zope2 apparently has a WebDAV server](http://wiki.zope.org/zope2/ZopeAndWebDAV), [as does Zope3](https://launchpad.net/z3c.dav) ([source of Zope3's implementation](http://svn.zope.org/z3c.dav/)). It appears to be fairly tightly dependent on the rest of Zope ([example for COPY/MOVE](http://svn.zope.org/z3c.dav/trunk/src/z3c/dav/copymove.py?view=markup)), but is still worth looking at for inspiration, and for how it has solved client-compatibility issues.
zooko commented 2011-03-10 23:45:04 +00:00
Author
Owner

I just discovered pyWebDAV. I heard about it because it had a security vulnerability announced on http://lwn.net .

I just discovered [pyWebDAV](http://code.google.com/p/pywebdav/). I heard about it because it had a security vulnerability announced on <http://lwn.net> .
tahoe-lafs added
normal
and removed
major
labels 2012-06-30 22:24:19 +00:00
tahoe-lafs modified the milestone from eventually to undecided 2012-06-30 22:24:19 +00:00
amontero commented 2013-12-28 17:18:43 +00:00
Author
Owner

Call to Twisted/WSGI/webserver wizards: Could not #2144 help achieve WebDAV, just using an out-of-the-box web server?

Call to Twisted/WSGI/webserver wizards: Could not #2144 help achieve WebDAV, just using an out-of-the-box web server?
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: tahoe-lafs/trac-2024-07-25#451
No description provided.