advice needed - streams forte

Streams

Thu, 19 Sep 2024 06:54:31 -0400 from Y

Einer von Vielen wrote the following post Thu, 19 Sep 2024 06:54:31 -0400

advice needed - streams forte
In short:
Wich way to go in the long run (month, years): streams or forte?

Context
One of my two servers broke down a while ago and needs replacement. On the second server I want a fresh installation.

Usage

cloud storage for my family
clones of family accounts on a second server as backup in case the hardware of one server fails
stay connected with contacts from hubzilla/streams/forte, optionally mastodon. Temporal bugs don't bother me much.
maintenance / improvement of addons: I would like to avoid to implement and test for both systems streams and forte.
running instance(s) inside virtual machine(s) to debug an addon before making merge requests and rolling the changes out to live servers
maintenance and live test of debian install script

@Einer von Vielen

Jupiter Rowland's (streams) outlet

Thu, 19 Sep 2024 07:22:43 -0400 from Streams at els Mussols

Normally, I'd say Hubzilla.

But since you absolutely need to be able to clone existing (streams) channels...

When did your family create their accounts? Before July 20th? In that case, the most reliable option is the "Streams classic" branch of @Bill Statler's fork. However, it's unmaintained because Bill is not a dev, so you'd have to stick with it as-is until (streams) itself gets its bugs ironed out.

Any account registered since July 20th? (streams) and hope for the best.

Forte is maybe a month old. It's super-experimental and far from being fit for daily-driving. The last thing Mike needs now is having to maintain two server applications that people depend on as daily drivers because someone sets up a Forte family instance.

Unless Mike steps up and declares Forte daily-driver-grade stable, leave it to him to tinker with.

Einer von Vielen

Thu, 19 Sep 2024 08:21:56 -0400 from streams

Thanks, @Jupiter Rowland's (streams) outlet

@Mike Macgirvin 🖥️
Regarding compatibility of /uexport and /import_items How long will this be compatible between streams and forte, possibly?

Einer von Vielen

Thu, 19 Sep 2024 08:38:40 -0400 from streams

The more I think about the more obsolete my original question becomes

Wich way to go in the long run (month, years): streams or forte?

I am tempted to say, sorry for asking

Bill Statler

Thu, 19 Sep 2024 11:32:40 -0400 from Bunny of Doom

the "Streams classic" branch of @Bill Statler's fork

It's here if anyone wants it:
https://codeberg.org/billstatler/streams-classic

At the moment, it is identical to commit 22f54d23c from the 'dev' branch of Mike's streams repository. So it should work identically to that version. (I still have not installed and tested it myself. Maybe next week.)

There are probably bugs. In the future, I would like to sort through all of Mike's commits after this point, and find the bugfixes that apply to the "classic" code. That is what is commonly called a "learning experience"

meaning that I will probably break things.

Mike Macgirvin 🖥️

Thu, 19 Sep 2024 17:38:14 -0400 from Y

Here's the situation. Yes, we had a number of issues with the streams FEP-ef61 update; some of these were critical issues which didn't show up in testing and took quite a while to solve. Most of them are solved now. Apologies that some of you were bitten by these issues. I'm seeing pretty good federation now between "classic" streams, streams/FEP-ef61, forte (FEP-ef61), and ActivityPub. I'm running multiple streams and forte sites and if I find something wrong and/or annoying, I'll often do what I can to fix it.

I can't solve Giac's federation issues with ActivityPub. No idea why this is affecting his site and nobody else - or what to do about it.

I'm no longer interested in developing decentralised communication software. After 45+ years of doing so and bleeding out on hilltop after hilltop trying to interest others in the concepts of portable identity, permissions, and online safety, my priorities have changed.

Nomadic identity in the streams repository is mature and stable (both before and after the FEP-ef61 update).

Forte should make for a decent ActivityPub server with "nearly" all of the features of the streams repository - except there's some unfinished development work related to nomadic identity and import/export. There is also still a bit of regression testing to do - as there were thousands of code changes. There is no schedule or roadmap for doing this.

Sun, 06 Oct 2024 05:01:16 -0400 from hubzilla

@Mike Macgirvin 🖥️

I'm no longer interested in developing decentralised communication software. After 45+ years of doing so and bleeding out on hilltop after hilltop trying to interest others in the concepts of portable identity, permissions, and online safety,

A damning indictment of the software industry, both commercial and volunteer; however, a quick re-call of the extant programmer demographic (despite all the various "initiatives"!), not a surprise.

Enjoy retirement; history is littered with people who have made contributions that are not appreciated many decades after the innovations.

Marcus Rohrmoser 🌻

Sun, 06 Oct 2024 16:15:02 -0400 from mastodon

Hi @Mike Macgirvin 🖥️,
that's sad to hear, but I can understand too well and wish you all the best. You are perhaps the one adult voice in this ball pool I'm grateful to have found. Your understanding of agency and responsibility always appeared crystal clear, rare and will be missed direly. Never say never, right?

Jupiter Rowland's (streams) outlet

Thu, 19 Sep 2024 18:00:22 -0400 from Streams at els Mussols

The release branch of the streams repository was last updated three days ago. Maybe the existing (streams) instances could upgrade and see how things are now.

El veciet emergency

Thu, 19 Sep 2024 18:38:27 -0400 from Get Ready To Rumbly

I already did the update, my problem is still there, but I pretend to use some site channel without the activitypub plugin
How do I set the default for a new channel to start without activitypub already ON?
And, plz, Amin turn off your AP plugin because a like or comment in this group sends the site in a coma for x minutes
I have a channel that I use just for read AP post from some contacts, just this , all the others channels will turn AP plugin off,
I will try to keep this site to interact with the zot/nomad universe only, because without activitypub it works fine.

Bill Statler

Thu, 19 Sep 2024 20:30:01 -0400 from Bunny of Doom

@Mike Macgirvin 🖥️ - Thanks for that update.

I hope I haven't stepped on any toes with "streams classic", and I just wanted to explain my thinking. You've always said that the streams repository had "a community-driven model" with "no dedicated developers". Until you announced your retirement 3 weeks ago, that was not how things were working in practice. Now, it is. So if the community wants to keep using streams software, the individuals in the community had better do something about it.

I assumed you were never going to touch the code again. I see you've made some fixes over the past 3 weeks, and I appreciate that a lot. I'm not assuming you'll keep doing this forever. I don't want to be dependent on your efforts.

My reason for forking "classic" is for a backup instance that I, personally, might be able to keep in operation for the next few years. There is no way I could debug FEP-ef61 issues. But I might be able to handle small bugs in "classic" (especially if I discover that you already fixed them in 'dev'!).

Anyway, the situation is a bit different than it was 3 weeks ago, since you have squashed more bugs. I'm still running 'dev' and I'm not seeing obvious problems with it. So this makes 'classic' less important to me. I'll probably try it anyway, just for the experience.

So that leaves Giac's freezeup problems. I've been looking at some of his logs (and I know you've done that too). What I see is, after he comments on a post (via ActivityPub), within a few seconds he gets dozens of HTTP "GET" requests for that comment (via 'apgateway'). They all seem to come from the instances of the followers of the original poster. For some reason, Giac's server responds slowly to these GETs, and eventually they pile up until his server replies with "504 Gateway Timeout" to every request.

I don't know if 'apgateway' is super slow, or if something else is interfering. Anyway, that's the best sleuthing I've been able to do.

Mike Macgirvin 🖥️

Fri, 20 Sep 2024 01:07:13 -0400 from Y

There's nothing magic about the apgateway. It takes a nomadic id and a nomadic path and looks up the id and turns it into a local (non-nomadic) path. Then it pretends you actually called the local (non-nomadic) path and delivers the goods you asked for. It hardly uses any resources to do this. So it's unlikely this is at fault. I can't stop a thousand servers from requesting the same information at the same time. The only thing I can do is space out deliveries so maybe these things don't all happen at once.

El veciet emergency

Fri, 20 Sep 2024 09:21:43 -0400 last edited: Fri, 20 Sep 2024 09:23:37 -0400 from Get Ready To Rumbly

one thing I can tell you for sure,

the longest comas ever are the ones that happen after I interact from diversispiritus in this group/forum, and, one explanation could be that there are a lot of contacts and activitypub instances involved here.

silverpill

Fri, 20 Sep 2024 10:09:19 -0400 from mitra

@elvecietemergency @Streams @Einer von Vielen @Mike Macgirvin 🖥️ This is probably caused by Mastodon: https://github.com/mastodon/mastodon/issues/31921

Bill Statler

Fri, 20 Sep 2024 14:15:55 -0400 from Bunny of Doom

@silverpill - I took another look at one of Giac's logs, and I think you're right that this is contributing. Looks like it took at least 40 seconds for diversispiritus to respond to a single "GET /replies/..." request (with "404 Not Found"). I can't tell whether this was the only cause of the slowdown, or just one contributor.

Jupiter Rowland's (streams) outlet

Fri, 20 Sep 2024 14:44:01 -0400 from Streams at els Mussols

Is that Mastodon looking for a place that doesn't exist by itself or (streams) guiding to Mastodon to a place that doesn't exist?

Bill Statler

Fri, 20 Sep 2024 15:27:37 -0400 from Bunny of Doom

I think it's Mastodon going to a place that does exist but is empty. As soon as they receive a post or comment, they ask for replies to it. Which is stupid.

Mike Macgirvin 🖥️

Fri, 20 Sep 2024 17:13:29 -0400 from Y

Get yourself on the dev branch (for now), and add the following to your .htconfig.php file:

App::$config['system']['show_replies_collection'] = '0'.

Mastodon sites won't know you've got a replies collection because we won't tell them. Mastodon has destroyed that endpont anyway with Mastodon specific quirks (*), so it has little value. The streams repository uses the 'context' property for holding the real replies collection.

(*) Mastodon's replies collection doesn't really provide a collection of all replies and sub-replies anyway. They provide a replies collection with different modes that you can select by visiting different pages of the results. The replies on 'page=1' of the results could show 0 results, but if you visit 'page=2' of the search results, you might find 75 replies. These I think are sub-replies, but there's no documentation of this horrible hack - so it's hard to know for sure what it is going on.

Mike Macgirvin 🖥️

Fri, 20 Sep 2024 19:02:00 -0400 from Y

Looks like good timing for this...

jonny (good kind) wrote the following post Thu, 19 Sep 2024 00:24:50 -0400

alright, after like a year of halfheartedly trying on and off, #FetchAllReplies is pretty much finished - the problem of not being able to see all replies to a post is one of the largest complaints that people have with mastodon in particular but also the fedi in general. It is an especially potent problem for smaller servers, making them feel lonely, and making the whole fedi seem quiet. It is also a large contributor to the 'reply guy' problem where a moderately popular post will get the same replies over and over again and people won't even know they're doing it.

This patch recursively fetches replies using activitypub collections. it does it respectfully, only when someone is explicitly looking at a post (rather than fetching all replies for everything all the time) with some debounce, and spaces out the recursive calls to the other servers in deep threads.

the only thing left is to make the posts get inserted into the web client as they are received, currently you need to refresh to see them.

trying it locally now and it is a game changer.

i'm not "good at ruby" so if you ever wanna see this upstream, kindly spare a code review?

https://github.com/NeuromatchAcademy/mastodon/pull/44

#FediDev #MastoDev #UnFuckTheFedi #PubSubIsCoolButPresentsPrettySeriousUsabilityProblems #JustSmallInstanceThings

Bill Statler

Fri, 20 Sep 2024 19:42:07 -0400 from Bunny of Doom

That sounds like a good thing. I think.

Diversi Spiritus supervisor

Mon, 23 Sep 2024 13:14:55 -0400 from Diversi Spiritus

Just another test (here in this channel AP is off now)