tahoe-lafs/trac-2024-07-25

Factor functionality related to running a storage service into a separate class from `_Client` #3931

New issue

Open

opened 2022-10-03 13:52:48 +00:00 by exarkun · 0 comments

exarkun commented

2022-10-03 13:52:48 +00:00

Owner

allmydata.client._Client combines many concerns. At least:

metrics collection
secrets management
webui frontend initialization
storage services
storage client services
blacklist management
file "node" creation
helper management
sftp frontend initialization

This is just the list that's obvious from reading the names of its methods.

Most of the logic for most of these things is already implemented by something other than _Client. However, _Client ties them all together in an obligatory "super object". Much of the functionality can be disabled - but only through the very awkward interface of the bytes in a tahoe.cfg file.

Instead of making tahoe.cfg the only control structure for what _Client does and does not do, we should arrange to have a convenient Python API that allows for composition of functionality. For starters, this could exactly mirror the capabilities of tahoe.cfg (and should, in fact, be the API that the contents of tahoe.cfg drives). The point would be to make it easier to make these choices from Python code - without a trip through an ini-style configuration file.

We could start by pulling any one of these pieces out but the storage service piece is particularly interesting as it is undergoing a non-trivial amount of maintenance these days so it makes sense as a starting place for me.

This refactoring would make it easier to test existing and new features of the storage system. It should also make the implementation easier to maintain and experiment with in the future.

Presently a _Client is almost always created by create_client_from_config which mediates between tahoe.cfg and the current _Client Python API. A reasonable goal for this particular ticket could be a refactoring with allows create_client_from_config to perform a single, simple composition between _Client and an API responsible for the storage service (for example, create the storage service object and pass it to _Client, or vice-versa, or pass both to a third thing that does the necessary Twisted Service setup, etc).

This would replace the current implementation which does some some storage service initialization in create_client_from_config and creates a _Client and passes some of the storage service values to a method on the partially-initialized _Client where there is further tahoe.cfg inspection and the rest of the storage service setup process.

`allmydata.client._Client` combines many concerns. At least: * metrics collection * secrets management * webui frontend initialization * storage services * storage client services * blacklist management * file "node" creation * helper management * sftp frontend initialization This is just the list that's obvious from reading the names of its methods. Most of the logic for most of these things is already implemented by something other than `_Client`. However, `_Client` ties them all together in an obligatory "super object". Much of the functionality can be disabled - but only through the very awkward interface of the bytes in a `tahoe.cfg` file. Instead of making `tahoe.cfg` the only control structure for what `_Client` does and does not do, we should arrange to have a convenient Python API that allows for composition of functionality. For starters, this could exactly mirror the capabilities of `tahoe.cfg` (and should, in fact, be the API that the contents of `tahoe.cfg` drives). The point would be to make it easier to make these choices from Python code - without a trip through an ini-style configuration file. We could start by pulling any one of these pieces out but the storage service piece is particularly interesting as it is undergoing a non-trivial amount of maintenance these days so it makes sense as a starting place for me. This refactoring would make it easier to test existing and new features of the storage system. It should also make the implementation easier to maintain and experiment with in the future. Presently a `_Client` is almost always created by `create_client_from_config` which mediates between `tahoe.cfg` and the current `_Client` Python API. A reasonable goal for this particular ticket could be a refactoring with allows `create_client_from_config` to perform a single, simple composition between `_Client` and an API responsible for the storage service (for example, create the storage service object and pass it to `_Client`, or vice-versa, or pass both to a third thing that does the necessary Twisted `Service` setup, etc). This would replace the current implementation which does some some storage service initialization in `create_client_from_config` _and_ creates a `_Client` _and_ passes some of the storage service values to a method on the partially-initialized `_Client` where there is further `tahoe.cfg` inspection and the rest of the storage service setup process.

Factor functionality related to running a storage service into a separate class from _Client #3931

Factor functionality related to running a storage service into a separate class from `_Client` #3931