Feature rollout process

Pre-release features

We always aim to communicate clearly about pre-release features so that we do not cause confusion around customers thinking features are generally available, that they aren’t going to change, or even that they couldn’t be taken away if we find they aren’t working.

1. Prior to the official launch of the feature, we should focus on the problems we are solving, rather than that feature x will definitely include scope y and/or be delivered on z date. For example, we can talk about our ultimate goal of making it easy to search and discover open source code and the reason why we are focused on it, or how we’re working on helping teams problems around a specific batch changes use case, and that we’re looking for feedback. This can (and should) include some specifics, but that doesn’t need to be the emphasis because people care more about the problems we’re solving anyway.
2. Whenever communicating publicly about pre-release features, we should use the beta and experimental labels so we don’t create confusion, and be sure we note that final capabilities of the feature might change. This includes social media, the Sourcegraph website, documentation, release post, changelog, handbook, in the app, or anywhere else. Documentation, as it is long-lived and not tied to a moment in time, should indicate when a feature became generally available.

Public beta testing by its nature involves sharing more detail, showing and letting people use things that are still unreleased, and which can change or even be abandoned before they go live. That’s fine to do, as long as we follow the above principles.

L1 launches and milestones

L1 launches (full definition in marketing handbook) are special: we don’t do them frequently, but when we do we engage an additional layer of collaboration around them to avoid misunderstandings around how these important features will move from development to general availability.

1. Two months prior to an L1 feature’s planned release date, the PMM for the product should open a collaboration document (using this planning template) to be shared with marketing, product, PR, and any other teams who have a role in the launch, depending on the feature.
2. At the same time, a regular sync should be created for the team. This could be a sync meeting, async Slack channel, a mix or other approach, as long as everyone agrees. Regardless of what communication mode you choose, it should allow for teammates to follow async.

Within the document and as part of keeping in sync, you should track the following (linking to other sources of truth as necessary, rather than creating an additional source of truth here):

• What the current estimate of launch date is
• High level how the feature will work and what capabilities it will have
• How it will be priced
• Marketing plan
• Any beta/experimental phases that are planned, including which components of the beta will remain free or become paid
• What (if any) parts of the feature will have public communications minimized so that they can be used by PR/marketing around the launch

It’s expected that these points will change over the period heading towards launch; the purpose is not to get everything right from the beginning and lock it in, but to stay on the same page on the important points as they evolve. The same procedure would also be followed for milestone-type features, such as hitting a certain repository indexing milestone, but the document would be relatively simpler in that case.

1. Product, product marketing, and comms will work together on the launch and comms plan for every stage through GA to ensure a cohesive message for prospects, customers, and the market.

2. All customer-facing/public comms should be reviewed by marketing for L1 launches.

Before merge

• Run hallway tests with internal users
• Complete a final design review
• Review documentation
• Review analytics and ensure desired metrics have been added to the feature
• Confirm feature flag functionality

After merge, before launch

1. Gather internal feedback: Enable the feature flag in the Sourcegraph organization settings to enable your feature for all Sourcegraph team members. Be sure to leave enough time for folks to experience the feature in their workflows and provide feedback. The #feedback-dogfood channel is a great place to ask for internal feedback.
2. Analytics: Validate logging is working for critical flows
3. Approvals: Receive approval from key stakeholders.
4. Bug Tracking: Keep track of all feedback.
   • Track high priority bugs in GitHub issues. Ensure they are closed before launch.
   • Track all lower priority bugs that have to be fixed soon.

Launch

1. Enable for all Sourcegraph Cloud users
   • Follow the steps in this document to enable your feature in global settings and to push it to all users on Sourcegraph.com.
   • In the PR that pushes changes live, add everyone who gave approval for launch as reviewers.
2. Metrics
   • Share analytics for monitoring the feature shipped. Track metrics for regressions.

Post-launch

1. Marketing: Share update with marketing.
2. Metrics: continue to track metrics to ensure expected outcomes are achieved.
3. Enable by default: Once a feature has been validated on Sourcegraph Cloud it can be enabled by default to be included in the next release. The feature flag should be left in place for at least one release cycle so that customers can disable the feature if problems arise.

Features unique to Sourcegraph Server

Features that are specific to Sourcegraph Server and can’t be tested on Sourcegraph Cloud still go through the same process before merge. Then:

1. Test on Sourcegraph dogfood instance: follow the before launch steps above on k8s.sgdev.org.
2. Release feature with flag off: for the next release, the feature flag is disabled by default.
3. Run external user tests: Reach out to select customers to turn on the feature flag for a specific time period, usually those with expressed interest in testing the feature.
4. Announce to customers they can enable the feature flag: Let customers know that they can turn on the feature.
5. Enable the feature flag: We turn on the flag by default.
6. Remove the feature flag: Follow the post-launch process.

Before merge

• Run hallway tests with internal users, using the #feedback-dogfood channel.
• Complete a final design review
• Review documentation
• Review analytics and ensure desired metrics have been added to the feature
• If there are privacy or security related changes, get approval from our security partners

After merge, before launch

1. Analytics: Validate logging is working for critical flows
2. Bug Tracking: Keep track of all feedback.
   • Track and ensure high priority bugs in GitHub issues. Ensure they are closed before launch.
   • Track all lower priority bugs that have to be fixed soon.

Launch

1. Release Process
   • Release process is different for each product. Please refer to the Browser Extensions & IDE Extensions pages for more details.
2. Metrics
   • Share analytics for monitoring the feature shipped. Track metrics for regressions.

Post-launch

1. Notify partners: Share the release status with partners on the #integrations channel.
2. Metrics: continue to track metrics to ensure expected outcomes are achieved.