Repository navigation
Pattern Idea : Using Extensions to manage Contributions at scale #439
Description
Might be related to https://github.com/InnerSourceCommons/InnerSourcePatterns/blob/main/patterns/1-initial/reluctance-to-accept-contributions.md
Problem
As the number of new feature contributions to a mature inner-source repository rapidly increase, it adds more burden on code reviews and maintenance. How do we allow for faster release of new features, encouraging innovation and experimentation; while also keeping repositories well maintained?
Context:
-
More organizations or company employees are contributing to strategic inner-source codebases, some of which are scaling rapidly with new feature contributions. This is adding a huge code review backlog on a smaller group of maintainers, resulting in maintainer burnout as well as slowing down release of new capabilities.
-
Adding an excessive number of capabilities and code to repositories without due diligence is making them difficult to maintain.
-
New features are getting released without thorough documentation, hardening or testing, creating a poor user experience.
-
You want to allow for expansion and encourage innovation and experimentation without restricting contributions, while also keeping good code and quality standards for user experience.
-
A large amount of time goes into hardening and thorough testing of features to meet product standards, but you want to allow for faster release of new innovations for adopting products to explore before investing time in productionization of capabilities.
-
You want to encourage the community to share innovations, combining product capabilities with other use-cases without adding more dependencies to primary repository.
Solutions
Allowing extensions/plugins to high-scale inner-source codebases or libraries can relieve the maintenance burden on repository maintainers and allow faster release of new features for adopting products to explore. This shifts maintenance of capabilities to extension owners and allows the primary repository to support capabilities that have been adopted more widely and are more strategic.
-
Create library or repository extension templates to allow addition of new features in new repositories, retaining the same modular structure and APIs as the primary repository.
-
Loosen the requirements for contributors to create extensions by bypassing reviews to allow for faster release or experimentation.
-
Allow registration of extensions with primary libraries for users to leverage extensions alongside the original project.
-
Make extensions easily discoverable with a publishing page showing the best extensions that users have created, which are ready for product usage.
-
Establish the lifecycle for extensions from creation to porting into the primary library. For instance:
- Extension creators continue maintaining the extension, providing any support and fixing defects. Any extension left unmaintained will be unlisted from the publishing page.
- Create criteria for when an extension can be ported to the primary repository, such as adoption of the extension by internal products and demand for the feature.
- Porting process of the extension to the primary repository will follow more stringent code review guidelines as set by library maintainers.
Following the principles outlined above ensures that:
-
Contributing an extension feels identical to adding a new feature to the primary repository and is a similar amount of work.
-
Extensions are discoverable in a repeatable manner to all users of the primary project; just because code doesn't live in the main repository yet does not mean it is not valuable.
-
The maintainer burden is reduced until an extension has demonstrated that it fills an important gap in the primary project.
Authors:
Sukriti Sharma
Alexander Brooks
Activity
Ssukriti commentedon Aug 16, 2022
I am unable to attach labels, but this is an initial idea that I will contribute if you all think is worthwhile to have as a pattern.
We have adopted this in our organization and have seen success at managing inner-source projects at scale
NewMexicoKid commentedon Aug 16, 2022
Ssukriti commentedon Aug 16, 2022
Thank you for the feedback @NewMexicoKid ! Yes you are right, I will move the last three points to forces as tradeoffs while drafting the pull request.
Some thoughts on the architecture:
Extension templates will require the base software to be installed a dependency and they will build on top of that. Since the extension will be in a separate Github repository, we can also include the same performance testing framework in the extension repositories, where the tests can run in an automatic fashion with every version upgrade of base software. This way the ownership and maintenance of the tests with extensions would be on the extension owners. If any changes in the base software cause the extension to break, the owner can either limit the compatibility to older versions or will have to update the extension accordingly to adopt latest versions of base software. (we think this scenario should be rare unless there is a major release upgrade of base software)
We would avoid adding more tests to the base repository in order to keep it lightweight and have lesser maintenance on primary maintainer group.
This is an interesting point and maybe a way to include this would be that the extensions repositories include all the performance tests for all the capabilities offered by the base software as well . Any drop in performance should be documented in the extension so users are made aware of the downsides.
About future-proofing and long term maintenance, I think we could use more brainstorming, but in general changes required on extensions side to keep up with base repository could be minimized with a modular structure in base software, such that unless there are major API changes (major release), the functionality remains independent. We also allow existing feature enhancements directly into the primary repository and recommend extensions only for a new use case or capability.
Having a well defined cycle for accepting the extension into primary library could also help move the strategic capabilities into the primary repository sooner. (such as if the extension received adoption by X products. ) If an extension was not able to build a community around it, such that maintenance had to dropped after X amount of time, it would be unlisted, which is also fine I think as it would be similar to deprecating features in software. We need to do our best to offer the same discoverability to extension modules to give them a fair chance to be adopted and build a community/ leverage the community of the base repository.
spier commentedon Aug 16, 2022
@Ssukriti thank you for sharing this approach with us!
I especially like that it addresses challenges related to "software architecture for InnerSource".
For those we don't have many patterns yet.
For inspiration you could look at these patterns.
They address quite the same thing as your Extensions idea though.
You could theoretically submit your pattern as Structured immediately (Level 2), as you have one Known Instance already (your own org).
However I would recommend to start by submitting it as Initial (Level 1), and then maturing it over time to Level 2.
That has these benefits:
If this approach works for you, you would start this by:
Also take a look at our Contributor Handbook. The requirement descriptions or Level 1 patterns should be helpful.
I hope that you will find the contribution experience to be a smooth one.
We are here to help and also open to any feedback for improving our process.
NewMexicoKid commentedon Aug 16, 2022
Ssukriti commentedon Aug 29, 2022
I have opened a pull request with the submission as an initial idea #444
Thanks
spier commentedon Sep 5, 2022
@Ssukriti as promised the pattern is now merged and available here:
https://github.com/InnerSourceCommons/InnerSourcePatterns/blob/main/patterns/1-initial/extensions-to-manage-contributions-at-scale.md
I am closing this issue here as well.
Let's open new issues and PRs as we see fit to improve this pattern further in the future.
And thanks again for sharing your great pattern!