tahoe-lafs/trac-2024-07-25

document known scaling issues #1199

New issue

Open

opened 2010-09-18 01:00:49 +00:00 by zooko · 0 comments

zooko commented

2010-09-18 01:00:49 +00:00

Owner

There are some known issues with scaling Tahoe-LAFS up in terms of servers, users, files, aggregate data. Most of these issues were learned from the allmydata.com experience. Most of them are "soft limits" -- things start taking an inconveniently long time -- rather than hard limits where something that used to work fails.

Examples:

it took a long time to do deep-add-lease and deep-repair
user data was spread over all the servers and some files and directories were reachable only through other directories, so if a few servers were (even temporarily) off-line then some users accounts couldn't be deep-add-leased, deep-repaired, or recursively downloaded. There was no good support for a deep- operation to stop cleanly when it encounters an unrecoverable file or directory and show its partial results to the user or preserve its state for later retry.

There are some known issues with scaling Tahoe-LAFS up in terms of servers, users, files, aggregate data. Most of these issues were learned from the allmydata.com experience. Most of them are "soft limits" -- things start taking an inconveniently long time -- rather than hard limits where something that used to work fails. Examples: * it took a long time to do deep-add-lease and deep-repair * user data was spread over all the servers and some files and directories were reachable only through other directories, so if a few servers were (even temporarily) off-line then some users accounts couldn't be deep-add-leased, deep-repaired, or recursively downloaded. There was no good support for a deep- operation to stop cleanly when it encounters an unrecoverable file or directory and show its partial results to the user or preserve its state for later retry.

tahoe-lafs added the

labels 2010-09-18 01:00:49 +00:00

tahoe-lafs added this to the eventually milestone 2010-09-18 01:00:49 +00:00

Sign in to join this conversation.

No labels

LeastAuthority.com automation

cannot reproduce

code-frontend-cli

code-frontend-ftp-sftp

code-frontend-magic-folder

code-frontend-web

code-peerselection

dev-infrastructure

somebody else's problem

was already fixed

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: tahoe-lafs/trac-2024-07-25#1199

No description provided.