Talk:Code Stewardship

In reality, bugs with priority "normal" are not fixed within 30 days in most cases. Some teams are responsible for multiple code bases with hundreds of such bugs (for example, there are 656 bugs tagged "Collaboration-Team-Triage" with "normal" priority). For these SLAs to be realistic, one of three things needs to happen:

1. The standard SLA's timeline for "normal" priority bugs is extended, or eliminated entirely
2. On a case by case basis, individual code bases have laxer SLAs
3. Thousands of "normal" priority bugs are changed to have "low" priority
4. Bugs filed prior to this policy taking effect are grandfathered in

Also, the huge backlog that many code bases (even ones with WMF teams as stewards) currently have shows that, if these expectations are going to be agreed to (even just the ones about fixing high priority bugs), they will likely have to come with more resourcing or fewer expectations of non-maintenance work.

Thanks for the feedback Roan. I've received a fair bit of feedback on this topic and I agree with you're assessment of where we're at today and the challenges we'd face with the current set of SLAs. I'll be updating the stewardship model and SLAs over the next day or so to reflect that feedback.

For the records, when it comes to component maintenance responsibilities of WMF teams, I am aware of the following documentation apart from Developers/Maintainers:

• Reading/Component responsibility
• Wikimedia Language engineering/Code review statement of intent
• https://phabricator.wikimedia.org/project/view/20/ (RelEng)
• https://phabricator.wikimedia.org/project/profile/1212/ (Perf)
• https://phabricator.wikimedia.org/project/view/1025/ (Ops)
• https://phabricator.wikimedia.org/project/profile/48/ (Multimedia)
• https://phabricator.wikimedia.org/project/profile/45/ (Research)
• https://phabricator.wikimedia.org/project/profile/1174/ (Discovery)
• https://phabricator.wikimedia.org/project/profile/65/ (Collaboration)

Maybe helpful, for reference.

Addition: https://meta.wikimedia.org/wiki/Community_Tech/Maintenance (Community Tech). Just got aware of it in T178645.

Thanks Andre, it is helpful. I was aware of a couple of these areas but hadn't seen several of them.

Cheers,

JR

Regarding "bug resolution", I'm glad to see "responsiveness" mentioned, but what does "insuring timely resolutions" imply? Is "resolution" meant in a broader sense, or does it refer to closing an open task?

As our projects are FOSS and (only in theory) anyone could provide a patch to fix any open task, in most projects there is only a weak culture to close tasks as "declined" (e.g. not enough (wo)manpower in our team to fix this bug / too hard to fix when it comes to required architecture changes / etc) as that can require spending more time to explain such a decision than to simply ignore open but unlikely-to-be-ever-fixed tasks. Hence I would not see "timely resolutions" of tasks as a goal per se.

Also, to which extent can this also be applied to feature requests?

I'm not sure what is meant by "broader sense", but what I was going for is making sure that as a steward, those bugs that are identified as highly impactful get fixed. Although I think that having less noise in the bug tracking system has it's benefits, I don't think there's anything inherently wrong with allowing things to remain open even if only as a low priority item.

So in short, I'm not thinking about it in the context of bug report management, but actual code fixes.

As for feature requests, I think that's different at least in how one defines importance/priority. I'm not yet too familiar with how we prioritize new features today, but if I understand correctly, as WMF we solicit feedback from the users en mass to help us prioritize the work, which also seems to be part of the quarterly/yearly planning process.

In addition, there have been times when I've declined a task only for the filer to insist on reopening it, at which point I sometimes triage it as "lowest" and plan to ignore it forever instead of trying to argue over it.

Does that new priority stick or does the filer assign a higher priority to it?

Rarely you can see edit wars, whether that's on some Wikipedia or in a task tracker. :)

I don't have Phab notifications turned on for priority changes, and I don't often pay any attention to the priority, so I don't know.

Regarding SLAs for code review, there has been some previous discussion in https://phabricator.wikimedia.org/T113707

Thanks, I'd not seen that one before.

Based on some conversations that I've started having, I think it's quite likely that SLAs will be an area of concern. The primary driver as I understand it is that we don't always have a clear path to getting things addressed in our code base. Lack of clear ownership (stewardship) and expectations of that commitment have become concerns for both external and internal contributors/stakeholders.

Some thoughts that I have on this are:

- Teams could define their own SLAs initially as we figure out if it makes sense to have standard SLAs.

- SLAs could be two way contracts. SLAs only come into play if prerequisites are met (i.e., we won't prioritize code reviews if you've not run linters, ran tests, etc...).

- We could use SLO vs SLA (Objective vs Agreement). Less of a commitment/contract and more of an internal measure.

- Initially I think the idea is to set some targets, see how often we meet them, and adjust as necessary. These learnings will help us do yearly resource planning.

Reading group already does this to some extent: Reading/Component responsibility.

Other thoughts on the topic if SLAs?

...and Wikimedia Language engineering/Code review statement of intent also something I'd consider an SLA on Code Review / maintenance.

@JBranaa (WMF) It is unclear to me in your proposed SLAs section if the times listed are a) time to first action, b) time to complete resolution, or c) max time between each subsequent interaction. Could you clarify either here or in the document?

@BDavis (WMF) My initial thoughts were time to first action. I think it would be hard to commit to any one timeline for all bugs. I think that if we could get a first pass review and ETA to fix within a committed amount of time, it would go a long way to satisfy the submitter.

Under "Base Service Level Agreements", how is "non-production" defined? Two examples coming to my mind: Beta features to be promoted to stable at some point? Support in MW Core for database backends which are not used on Wikimedia servers?

Also, the Priority values (?) imply that someone would have to both regularly and quickly (for UBN/High: less than a week) set the "Priority" field in every task, while I'd say that some teams pretty much ignore the Priority field and prefer for example workboard columns to express planning/urgency of tasks.

Originally the idea of "production" vs "non-production" was focused on whether the code in question was being used in WMF production. We could use "Beta" as a flag for this provided Beta code is isolated and does not impact non-beta code. However, as the application of the stewardship model gets applied beyond WMF, it probably doesn't make sense to include that in a base SLA definition. In fact, I don't know at this point if a base service level agreement event makes sense. As WMF, we may get to a point where we share a common base SLA, but I don't think we'll be there for a while.

As for priority values, this goes back to the discussion about how to differentiate between different tasks, especially those that are bugs. Not all bugs/tasks are created equally :-) Provided that we are able to effectively differentiate between tasks/bugs, it'll probably make sense to commit to/or at least target some level of responsiveness.

All that being said, it may not even make sense to define SLAs for bugs/tasks. It just initially seemed like a natural extension to SLAs for Code Reviews.

I see a lot of description about what must be done by a Code Steward, but I'm having a hard time seeing what benefit the CS themselves gets from the process.

I think that's a fair question. Although I don't necessarily see them as sticks, it wouldn't hurt to list out some benefits. That being said, the original context for defining CSes was to ensure that those components that WMF was dependent on in production were properly resourced. Hence the "Responsible Wikimedia Team" column on the developers/maintainers. Once I dove into it a bit more, it seemed like "stewardship" was a better fit than "ownership/responsible", and that there was no reason to limit the stewardship model to WMF teams or to code that was used in WMF production.

One of the interesting discussions that I had with Kunal was the need to have a two-sided contract when it came to SLAs. Requiring certain activities to happen prior to commits (i.e, linting, testing, etc...) before the stewards would perform code reviews. So in a way, being a steward provides a mechanism to encourage behavior by offering up support provided the contributor meets a set of steward-defined prerequisites. These prerequisites can be existing global coding standards or steward specific.

That being said, there'll likely be some prerequisites to being a steward, like current voting status (+2), etc...

I also don't think all code has to have stewards. If the code in question is important enough to a person or group and they depend on it in anyway, it'll be in their best interested to be a steward and establish more rigor around the development and support of the code.

I'm not sure if that additional context helps at all and/or give you any ideas about explicit benefits, but I'll think more about some benefits as well.

Only vaguely related: User:AKlapper (WMF)/Code Review lists benefits of code review as "knowledge transfer, increased team awareness, and finding alternative solutions. Good debates help to get to a higher standard of coding and drives quality." (and also lists all the related problems to face).

But that's of course just the benefit of having such a process, not the benefit for the stewards themselves.