tahoe-lafs/trac-2024-07-25
2
0
Issues 1.1k Wiki Activity

Move off Trac - Initiation #4095

New issue
Open
opened 2024-03-06 16:14:54 +00:00 by btlogy · 6 comments
btlogy commented 2024-03-06 16:14:54 +00:00
Owner
Copy link

As discussed during the nuts-and-bolts meeting this Tue 5th of Mars 2024, I'm creating this issue to initiate the possible migration of Trac.

The main deliverable of this task would be a document (Wiki page?) describing, at least:

  • the goals and initial requirements
  • the current situation
  • the possible foreseeable options
As discussed during the nuts-and-bolts meeting this Tue 5th of Mars 2024, I'm creating this issue to initiate the possible migration of Trac. The main deliverable of this task would be a document (Wiki page?) describing, at least: * the goals and initial requirements * the current situation * the possible foreseeable options
tahoe-lafs added the
dev-infrastructure
normal
task
n/a
 labels 2024-03-06 16:14:54 +00:00
tahoe-lafs added this to the undecided milestone 2024-03-06 16:14:54 +00:00
btlogy commented 2024-03-06 16:47:00 +00:00
Author
Owner
Copy link

I've already compiled some notes from the last Tuesday meeting and assessed the current situation to some extend.

But I'm unsure if I should/could share what I have so far in a Wiki page or an other live document (cryptpad.fr) before we have something sound to start the next phase (planning) - if we do.

I've already compiled some notes from the last Tuesday meeting and assessed the current situation to some extend. But I'm unsure if I should/could share what I have so far in a Wiki page or an other live document (cryptpad.fr) before we have something sound to start the next phase (planning) - if we do.
btlogy commented 2024-03-08 14:36:34 +00:00
Author
Owner
Copy link

I've a few questions I'd like to find the answer in order to update my notes about this task:

  1. #3742 is about a repository on gitlab.com for infra/Terraform code : does it exist? is it used to deploy this Trac instance? how is it deployed otherwise?
  2. there are many other services running on the same system, including 7 other Trac projects, a Darks server and at least 3 Builbot masters: should those be in the scope too?
  3. the version of this Trac instance (v1.0.13) and the system hosting it (Ubuntu 19) have a few new releases (Trac 1.6.0 / Ubuntu 22): is there something blocking some upgrade?
  4. as mentioned in my previous comment: what would be the best way to share my notes so we can collaborate to improve them (cryptpad.fr)?
I've a few questions I'd like to find the answer in order to update my notes about this task: 1. #3742 is about a repository on gitlab.com for infra/Terraform code : does it exist? is it used to deploy this Trac instance? how is it deployed otherwise? 2. there are many other services running on the same system, including 7 other Trac projects, a Darks server and at least 3 Builbot masters: should those be in the scope too? 3. the version of this Trac instance (v1.0.13) and the system hosting it (Ubuntu 19) have a few new releases (Trac 1.6.0 / Ubuntu 22): is there something blocking some upgrade? 4. as mentioned in my previous comment: what would be the best way to share my notes so we can collaborate to improve them (cryptpad.fr)?
meejah commented 2024-03-08 18:49:15 +00:00
Author
Owner
Copy link

I'm not aware of any infratructure repository, and don't immediately see anything on gitlab.com (at least under tahoe-lafs org).

Sharing notes could be done on cryptpad.fr for sure. Another option could be to find a suitable place in docs/* and do that via a pull-request.

I'm not aware of any infratructure repository, and don't immediately see anything on gitlab.com (at least under tahoe-lafs org). Sharing notes could be done on cryptpad.fr for sure. Another option could be to find a suitable place in docs/* and do that via a pull-request.
btlogy commented 2024-03-19 17:09:09 +00:00
Author
Owner
Copy link

As discussed last week in the Nuts and Bolt meeting, I've published a summary of my assessment of the current state of the infrastructure around Trac in a new Wiki page: DevInfra

As discussed last week in the Nuts and Bolt meeting, I've published a summary of my assessment of the current state of the infrastructure around Trac in a new Wiki page: [DevInfra](wiki/DevInfra)
btlogy commented 2024-05-30 09:00:03 +00:00
Author
Owner
Copy link

Wherever the Trac data will end up being moved, here are the notes I've collected while trying to reproduce the current Trac setup from a hotcopy to a docker environment:

  1. Initiate trac-admin <project> then hotcopy <path> on the source
  2. Remove log directory from the hotcopy path/log
  3. Purge session from the hotcopy with trac-admin path and session purge ...
  4. Compact the DB with sqlit3 path/db/trac.db 'VACUUM;'
  5. Prepare to transfer the curated hotcopy directory with public_html folder and trac.htdigest file
  6. Checkout https://code.lafs.eval.latfa.net/tahoe-lafs/MoveOffTrac on the target
  7. Transfer and restore Trac hotcopy in trac/tahoe-lafs on the target
  8. Transfer and restore extra files (living outside of Trac so far) on the target:
    • trac/public_html
    • trac/trac.htdigest
  9. Clone the Git repo: git clone <https://github.com/tahoe-lafs/tahoe-lafs.git> trac/repos/tahoe-lafs.git
  10. Create missing folders (e.g.: trac/thaoe-lafs/log)
  11. Patch trac/tahoe-lafs/conf/trac.ini
  12. Gain admin access (if needed):
    1. docker-compose run --rm --entrypoint bash trac
    2. trac-admin tahoe-lafs
    3. permission add <username_here> admin
    4. exit and exit
  13. Start the local Trac: docker-compose run --rm trac
  14. Login on http://localhost:8000/tahoe-lafs/login
  15. Edit the http://localhost:8000/tahoe-lafs/wiki/WikiStart?action=edit page to replace link: /~trac/ -> ../chrome/site/ or transform them as attachment

These steps will likely be improved with more iteration, but they should help reproducing a workable environment to actually test/start the migration.

Wherever the Trac data will end up being moved, here are the notes I've collected while trying to reproduce the current Trac setup from a hotcopy to a docker environment: 1. Initiate `trac-admin <project>` then `hotcopy <path>` on the source 2. Remove log directory from the hotcopy `path/log` 3. Purge session from the hotcopy with `trac-admin path` and `session purge ...` 4. Compact the DB with `sqlit3 path/db/trac.db 'VACUUM;'` 5. Prepare to transfer the curated hotcopy directory with `public_html` folder and `trac.htdigest` file 6. Checkout <https://code.lafs.eval.latfa.net/tahoe-lafs/MoveOffTrac> on the target 7. Transfer and restore Trac hotcopy in `trac/tahoe-lafs` on the target 8. Transfer and restore extra files (living outside of Trac so far) on the target: - `trac/public_html` - `trac/trac.htdigest` 9. Clone the Git repo: `git clone <https://github.com/tahoe-lafs/tahoe-lafs.git> trac/repos/tahoe-lafs.git` 10. Create missing folders (e.g.: `trac/thaoe-lafs/log`) 11. Patch `trac/tahoe-lafs/conf/trac.ini` 12. Gain admin access (if needed): 1. `docker-compose run --rm --entrypoint bash trac` 2. `trac-admin tahoe-lafs` 3. `permission add <username_here> admin` 4. `exit` and `exit` 13. Start the local Trac: `docker-compose run --rm trac` 14. Login on <http://localhost:8000/tahoe-lafs/login> 15. Edit the <http://localhost:8000/tahoe-lafs/wiki/WikiStart?action=edit> page to replace link: `/~trac/` -> `../chrome/site/` or transform them as attachment These steps will likely be improved with more iteration, but they should help reproducing a workable environment to actually test/start the migration.
btlogy commented 2024-06-11 10:18:29 +00:00
Author
Owner
Copy link

While the execution phase has already been started (test of migration to Gitea), we'd better agree on the final definition of the scope and the deliverables so we can plan the actual migration:

  • Scope of the migration:

    • INCLUDES:
      • Trac issues of the Tahoe-LAFS project
      • Trac wiki pages of the Tahoe-LAFS project
      • Trac HTML home page of the Tahoe-LAFS project
      • Hall of Fame HTML page of the Tahoe-LAFS project
      • The related DNS records (mostly: tahoe-lafs.org)
    • EXCLUDES:
      • Binary repository for Tahoe-LAFS releases (https://tahoe-lafs.org/downloads)
      • Other Trac projects (see DevInfra)
      • Buildbot master instances for some other Trac projects
      • DARCS SCM for some of the other Trac projects
      • Any other services provided by the current server and not yet documented in DevInfra

  • Deliverables:

    1. A VPS (hosted by Hetzner) providing 4 features powered by NixOS and Gitea:
      • a tracking system provisioned with the issues migrated from Trac (same numbers)
      • a Wiki system provisioned with the relevant pages migrated from Trac (same names)
      • a static website replacing the landing page from Trac with the code required for CI and CD
      • a repository defining the VPS it-self and its configuration as code (including the secrets)
    2. A detailed migration plan to handle the transition (DNS changes and/or redirections)
    3. An high-level migration plan for the 3 first features above to be later hosted on Codeberg SaaS (assuming it is possible)

I'll discuss this proposal in the next Nuts&Bolts meeting (Tue 11th of June).

While the execution phase has already been started (test of migration to Gitea), we'd better agree on the final definition of the scope and the deliverables so we can plan the actual migration: * Scope of the migration: * INCLUDES: * Trac issues of the Tahoe-LAFS project * Trac wiki pages of the Tahoe-LAFS project * Trac HTML home page of the Tahoe-LAFS project * Hall of Fame HTML page of the Tahoe-LAFS project * The related DNS records (mostly: tahoe-lafs.org) * EXCLUDES: * Binary repository for Tahoe-LAFS releases (<https://tahoe-lafs.org/downloads>) * Other Trac projects (see [DevInfra](wiki/DevInfra)) * Buildbot master instances for some other Trac projects * DARCS SCM for some of the other Trac projects * Any other services provided by the current server and not yet documented in [DevInfra](wiki/DevInfra) * Deliverables: 1. A VPS (hosted by Hetzner) providing 4 features powered by NixOS and Gitea: * a tracking system provisioned with the issues migrated from Trac (same numbers) * a Wiki system provisioned with the relevant pages migrated from Trac (same names) * a static website replacing the landing page from Trac with the code required for CI and CD * a repository defining the VPS it-self and its configuration as code (including the secrets) 2. A detailed migration plan to handle the transition (DNS changes and/or redirections) 3. An high-level migration plan for the 3 first features above to be later hosted on Codeberg SaaS (assuming it is possible) I'll discuss this proposal in the next Nuts&Bolts meeting (Tue 11th of June).
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
No results found.
No results found.
Labels
Clear labels   
0.2.0

   
0.3.0

   
0.4.0

   
0.5.0

   
0.5.1

   
0.6.0

   
0.6.1

   
0.7.0

   
0.8.0

   
0.9.0

   
1.0.0

   
1.1.0

   
1.10.0

   
1.10.1

   
1.10.2

   
1.10a2

   
1.11.0

   
1.12.0

   
1.12.1

   
1.13.0

   
1.14.0

   
1.15.0

   
1.15.1

   
1.2.0

   
1.3.0

   
1.4.1

   
1.5.0

   
1.6.0

   
1.6.1

   
1.7.0

   
1.7.1

   
1.7β

   
1.8.0

   
1.8.1

   
1.8.2

   
1.8.3

   
1.8β

   
1.9.0

   
1.9.0-s3branch

   
1.9.0a1

   
1.9.0a2

   
1.9.0b1

   
1.9.1

   
1.9.2

   
1.9.2a1

   
LeastAuthority.com automation

   
blocker

   
cannot reproduce

   
cloud-branch

   
code

catch-all for anything that can be fixed by writing code but isn't classified better

  
code-dirnodes

anything involving tahoe directories

  
code-encoding

immutable-file encoding format/layout, cap format, encoding/decoding algorithms

  
code-frontend

general frontend things: new user-facing interfaces, FTP/SFTP servers, FUSE-like interfaces

  
code-frontend-cli

CLI tools

  
code-frontend-ftp-sftp

   
code-frontend-magic-folder

   
code-frontend-web

webapi interfaces, user-facing "WUI" interfaces

  
code-mutable

anything involving mutable files: encoding/decoding, share format

  
code-network

Introducer, locating storage servers, establishing/maintaining connections to them

  
code-nodeadmin

node startup/shutdown, tools to create nodes, logging tools, management/monitoring tools, manhole, disk-space-limiting tools

  
code-peerselection

server selection/sorting algorithm, rebalancing code, allocation policy, swarming-download

  
code-storage

storage-server implementation, share-storage directory layout, client-to-server share transfer protocol

  
contrib

problems with non-core things that live in the source:contrib/ directory

  
critical

   
defect

   
dev-infrastructure

things that help us build Tahoe: buildbot, automated speed/memory/code-coverage tests and results-displayers, IRC channels, mailing lists, this Trac instance

  
documentation

docs items. Also use 'docs' keyword on tickets in more specific components.

  
duplicate

   
enhancement

   
fixed

   
invalid

   
major

   
minor

   
n/a

   
normal

   
operational

things involved in running a tahoe grid, external tools to monitor/maintain a grid (like "nodeadmin" but larger-scale). Was also used for issues with the allmydata.com grid.

  
packaging

setup.py script, library dependencies, binary packages, licensing, installation instructions

  
somebody else's problem

   
supercritical

   
task

   
trivial

   
unknown

Uncategorized. All tickets default to this component until someone updates it.

  
was already fixed

   
website

things about the tahoe-lafs.org website (and sometimes this Trac instance, shared with 'dev-infrastructure')

  
wontfix

   
worksforme
No labels
0.2.0
0.3.0
0.4.0
0.5.0
0.5.1
0.6.0
0.6.1
0.7.0
0.8.0
0.9.0
1.0.0
1.1.0
1.10.0
1.10.1
1.10.2
1.10a2
1.11.0
1.12.0
1.12.1
1.13.0
1.14.0
1.15.0
1.15.1
1.2.0
1.3.0
1.4.1
1.5.0
1.6.0
1.6.1
1.7.0
1.7.1
1.7β
1.8.0
1.8.1
1.8.2
1.8.3
1.8β
1.9.0
1.9.0-s3branch
1.9.0a1
1.9.0a2
1.9.0b1
1.9.1
1.9.2
1.9.2a1
LeastAuthority.com automation
blocker
cannot reproduce
cloud-branch
code
code-dirnodes
code-encoding
code-frontend
code-frontend-cli
code-frontend-ftp-sftp
code-frontend-magic-folder
code-frontend-web
code-mutable
code-network
code-nodeadmin
code-peerselection
code-storage
contrib
critical
defect
dev-infrastructure
documentation
duplicate
enhancement
fixed
invalid
major
minor
n/a
normal
operational
packaging
somebody else's problem
supercritical
task
trivial
unknown
was already fixed
website
wontfix
worksforme
Milestone
Clear milestone
No items
No milestone
undecided
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: tahoe-lafs/trac-2024-07-25#4095
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?