Upgrade to Trac 1.6 when released

[stoecker]

Trac 1.4 was released. Upgrade should be done after waiting some time first the bugs to settle after a major version upgrade.

• Change site html to new format ​https://trac.edgewall.org/wiki/TracInterfaceCustomization#SiteAppearance
• switch plugins to 1.4 branches
• iniadminpanel plugin wont work, as the Genshi support seems broken in a major way: ​https://trac.edgewall.org/ticket/13237 - update plugin to get it work, still needs a proper Jinja update for real 1.4 support
• Add a modified CSS to get an acceptable layout back.
• Check if JOSMs simple minded wiki parser survives the layout changes

Plugins:

• AdvancedTicketWorkflowPlugin ok
• BackLinksMacro not used, deleted
• HudsonTrac ok
• NavAdd no longer needed
• TracAccountManager probably ok
• TracAdvParseArgsPlugin not used, deleted
• TracIniAdminPanel ok
• TracSpamFilter in progress
• TracStats
• Tracticketstats
• TracVote
• TracXMLRPC
• TranslatedPages
• WantedPages not used, deleted

[stoecker]

Ooops, wrong component :-)

[Klumbumbus]

I hope the translation is included properly this time :)

[Don-vip]

1.4 looks very promising:

• Switch to Jinja2 template engine for faster and more memory lenient server-side content generation
• 5 times speed-up when rendering query results, thanks to the migration from Genshi to Jinja2.

Maybe we should nevertheless try this version on another machine, to make sure all bugs impacting us are properly identified and fixed in the next maintenance update. Otherwise I fear some bugs could remain undetected until we switch.

[stoecker]

That "Genshi to Jinja2" was actually the reason why I went away from Trac mailinglists. They wanted to drop Genshi without a time when both are supported and I voiced the opinion that this is the worst sign you can sent to plugin authors. And instead of reacting on what I said they started to blame me for the way I said it (which was relatively harmless, I do worse when really upset :-).

Regarding the speedup: I don't think you will notice it. E.g. everything which is slow in our wiki (like the translation tables) neither uses Genshi nor Jinja and the main speed issue is database access and client side rendering, not the content generating.

[stoecker]

TODO:

• change site.html to new format: ​https://trac.edgewall.org/wiki/TracInterfaceCustomization#SiteAppearance

[stoecker]

Lots of work and little benefit.

Let's hope situation get's better.

[Don-vip]

Why so late? Are you waiting for a new Trac release?

[stoecker]

I either hope that the workload gets reduced or that there comes up a reason why we should upgrade (i.e. a wonderful new feature :-)

[simon04]

May I move this to the longterm goals?

[stoecker]

Ticket #20382 has been marked as a duplicate of this ticket.

[anonymous]

1. Any updates? Is this really that much work? ​https://trac.edgewall.org/wiki/TracUpgrade

2. Well if going Trac 1.4 is so much work can we at lest get Trac 1.2.6 (latest stable)?

[stoecker]

Replying to anonym:

1. Any updates? Is this really that much work? ​https://trac.edgewall.org/wiki/TracUpgrade

A lot. I did this for 2 other (less intensively used) instances and it took some time.

2. Well if going Trac 1.4 is so much work can we at lest get Trac 1.2.6 (latest stable)?

I think so. Have to check.

[stoecker]

There was trouble with trac 1.2.x upgrade as well: See #16651

[stoecker]

Seems trac 1.6 will finally have python 3 support, so that's the next logical step for update. I'd think we skip 1.4 completely.

​https://trac.edgewall.org/wiki/TracDev/ReleaseNotes/1.6

[anonymous]

Quite some time has passed and trac 1.6 is nowhere to be seen. Can we reapproach the question of updating to 1.4.3 or at least 1.2.6?

[gaben]

Python 3.5+ compatibility added with ​Trac 1.5.3.

[stoecker]

Replying to gaben:

Python 3.5+ compatibility added with ​Trac 1.5.3.

I know. There are already people officially using the devel version instead of the release. Probably that's really the way forward.

[anonymous]

So trac 1.6 seems to be ready. We can start the upgrade process or if we don’t want to maintain trac consider moving to GitHub

[stoecker]

Ticket #23224 has been marked as a duplicate of this ticket.

[stoecker]

Replying to anonym:

So trac 1.6 seems to be ready. We can start the upgrade process or if we don’t want to maintain trac consider moving to GitHub

Who is "we" and why should "we" consider moving to GitHub?

If "we" will start the update process, then "we" could start by adapting AdvancedTicketWorkflowPlugin, BackLinksMacro, HudsonTrac, NavAdd, TracAccountManager, TracAdvParseArgsPlugin, TracIniAdminPanel, TracSpamFilter, TracStats, Tracticketstats, TracVote, TracXMLRPC and WantedPages plugins to Trac 1.6. I doubt most of them will work from scratch.

If the plugins work it still will take me several days of work to get a working Trac 1.6 instance including updating the internal plugins, the configuration and other stuff. I'm surely not doing this two weeks after a release which is a bit dubious, as not even the trac wiki reflects that release yet.

[anonymous]

Moving to GitHub would considerably lower the workload needed to maintain Trac. Imagine a scenario where Trac has a vulnerability it is fixed but you haven’t updated trac here. As a result someone can insert unauthorized code into JOSM. That would be bad, and since we all know that trac updating has been neglected well it may not be as far fetched as one may think

[anonymous]

Keeping one’s workspace updated is not something that should be questioned. However we all can see that this has been grossly neglected. As such there are two ways out of this debt: keep trec up to date or outsource this task by moving to GitHub

[stoecker]

Discussing with an "anonym" one makes not much sense, so this will be my last answer to you.

It seems you actually have little knowledge about the software you're talking about because compromising Trac will not give you a chance to insert any code in JOSM. You have to compromise the server behind it to do so.

Also you have simply NO understanding of the costs involved to maintain an infrastructre under maintainer control compared to an infrastructure under control of a company which is known to exploit their users. You have no understanding which services this server provides and what the costs of moving them would be.

JOSM has never neglected to update the infrastructure, especially Trac, but we have REJECTED these updates, because the quality of the updated versions was worse than the quality of the running version. Reason for this is probably the toxic environment caused by the political corrections groups which drove away the majority of real Trac contributors.

And last it seems like these groups you're only talk and do nothing, as I don't see much progress with the plugins mentioned above.

[taylor.smock]

Moving to GitHub would considerably lower the workload needed to maintain Trac

See also #16857. We'd also have to move to git from subversion, which would be a PITA. I'd want to do it right, which would mean:

• Contacting old contributors to see what name and email combination they want to be used for attribution
• Writing a script to parse out patch by <foo> so that we have proper patch authorship information
• Rewriting the version code (git rev-list --count HEAD is off with our git mirror -- 18540 versus 18874, a difference of 334)
• Converting svn:externals to git submodules (git submodule add -b <branch> <url> -- if <branch> is ., it matches the current branch of the current repository; the -b <branch> is important, since otherwise we would have to update the submodule constantly)

All of that takes work. If you (@anonymous) want to help out, you can help update the plugins mentioned in comment:27, update ​tracboat to work with current gitlab versions, or look into other self-hosted alternatives.

Anyway, endpoints that are consumed by JOSM and other applications:

• latest
• tested
• plugin
• styles
• presets
• rules
• maps

There are probably a few more endpoints that I don't know about/don't remember.

Then there are some plugins which parse wiki pages. Some of our integration tests parse wiki pages for example.

[stoecker]

AdvancedTicketWorkflowPlugin

[taylor.smock]

• ​BackLinksMacro -- provides [[BackLinks]] and [[BackLinks(pagename)]] macros. I don't know if we use either. It isn't currently maintained. And seems to be partially broken as is.
  • ​TracBacksPlugin
  • ​TrackBackLinkPlugin
• ​HudsonTrac
  • Only needed if we use the build timeline. Otherwise Trac 1.4 allows for custom navigation items.
• TracAccountManager -- Probably ​AccountManagerPlugin. Probably fine (last change 2023-06-08).
• TracAdvParseArgsPlugin -- Probably ​AdvParseArgsPlugin
• TracSpamFilter -- Probably ​SpamFilter. Probably works (has a 1.5.x version).
• TracStats -- Probably ​TracStatsPlugin. But we don't seem to be using it.
• TracTicketStats -- Probably ​TracTicketStatsPlugin. It isn't currently maintained. I think this is a "nice-to-have", but is definitely not critical.
• TracVote -- Probably ​VotePlugin, appears to be used by trac.edgewall.org, so probably works with 1.6 with no changes.
• TracXMLRPC -- Probably ​XmlRpcPlugin. Probably fine (last change on 2023-03-13).
• WantedPages -- Probably ​WantedPagesMacro, but could be ​WantedPagesPlugin (which redirects to WantedPagesMaco]).

[stoecker]

TracSpamFilter -- Probably ​SpamFilter. Probably works (has a 1.5.x version).

Not fine. Major work. Ask the author ;-)

• Spambayes doesn't work properly for python3 - I'm thinking about integrating the relevant parts into the core
  • If somebody could help me with that it would be appreciated. The plugin only uses a small part of spambayes, so stripping the code to minimum and include it seems better than to wait for a proper spambayes for python3. When keeping the interface using the external lib remains an option.
• Only a single page has been converted to Jinja2. For IniAdmin one page took me about 6 hours, so that's quite a task. Gets probably faster now that I know most details.

TracStats -- Probably ​TracStatsPlugin. But we don't seem to be using it.
TracTicketStats -- Probably ​TracTicketStatsPlugin. It isn't currently maintained. I think this is a "nice-to-have", but is definitely not critical.

Some of the stats plugins results are probably only visible for admins. Have to check.

​BackLinksMacro -- provides BackLinks and BackLinks(pagename) macros. I don't know if we use either.
WantedPages -- Probably ​WantedPagesMacro, but could be ​WantedPagesPlugin (which redirects to WantedPagesMaco]).

Have to check the wiki, when not used I'll drop them.

TracVote -- Probably ​VotePlugin, appears to be used by trac.edgewall.org, so probably works with 1.6 with no changes.

trac.edgewall.org runs with 1.4.3. No 1.6 yet (e.g. because of spamfilter plugin).

[stoecker]

BackLinksMacro

[stoecker]

WantedPages

[stoecker]

TracAdvParseArgsPlugin

[stoecker]

Forgot TranslatedPages

[skyper]

Unfortunately, I do not have time for (J)OSM, atm.

Replying to stoecker:

Replying to taylor.smock:

​BackLinksMacro -- provides BackLinks and BackLinks(pagename) macros. I don't know if we use either.

Have to check the wiki, when not used I'll drop them.

Currently, back links are added manually.
If macros are not currently used on any wiki page does not mean that they are not used at all. I used BackLinks quite a lot to find wiki pages which link to the wiki page I was editing and to make sure that I did not break to many links. If possible, please, keep this option.

[taylor.smock]

@skyper: Would either one of the maintained options work for you? I linked to them in comment:34. Both of them are "automatic" backlinks. ​TracBacksPlugin is more for the ticket section, while ​TracBackLinkPlugin adds them at the bottom of a wiki page (as a collapsible Back Links section).

[stoecker]

One problem is, that this only work in some cases. There are so many linking methods, that the backlinks are more a guess than exact results. Also all the TranslatedPages stuff is a black box to all the macros ;-)

[stoecker]

A small progress report: I'm nearly at the stage to replace one of my own instances of Trac with 1.6. Lots of work to upgrade all the stuff, but it seems to be running now. Tomorrow I'll test a lot and if no major errors appear, I'll start the next one (needing Spamfilter plugin).

P.S. I'm now also maintainer of HudsonPlugin ;-)