Adding support for a new code host

Sourcegraph currently supports a number of different code hosts.

Not all code hosts are supported “natively”. Native support means a code host type is listed as an option in our UI and it is seamless to add. In some cases it is possible to work around this if the code host is based on git, in which case it can be added via our other git repository hosts mechanism.

The following are the list of changes required to natively support another code host type.

Steps required for all new code host types

We’ll use Pagure as an example. Here’s the commit that introduced the majority of the changes required.

  1. Add a feature flag to guard it until the implementation is complete
  2. Implement the Source interface. To do this you’ll most likely need to create a client for the code host
  3. Add code host configuration schema
  4. Add a new kind and type
  5. It also needs to be added to our GraphQL schema
  6. Update to ensure all the cases where we check for types and kinds are covered
  7. The same needs to be done for places where we need unmarshal config
  8. Update our UI code so that the new code host type is displayed

A rough estimate for the above changes is two weeks of development work for 1 onboarded developer.

Additional steps required if the code host it not git based

If the code host does not use git then we need to create a custom VCSSyncer implementation that can import the repository from its original format into a git repository.

We’ll use Perforce as an example.

  1. Implement the VCSSyncer interface
  2. You may need to update the gitserver and server Dockerfile so that any required CLI tools are available at runtime

A rough estimate for the above effort is two weeks of development work for 1 onboarded developer, assuming third party tooling already supports incremental imports from the source repo type.

Extra steps if permissions syncing is required

In order to support permissions syncing, we need a way of mapping permissions from the code host with users in the Sourcegraph instance. The decision on how to perform the mapping varies, but the most common option is to map users based on verified e-mail addresses.

  1. Implement the AuthProvider interface
  2. Provide the authorization section in config

Bear in mind that some code hosts support permissions that are more granular than a repo. For example, Perforce support per path permissions. In this case, sub-repo permissions need to be imported.

We currently only support per path filtering in sub-repo permissions.

The implementation can vary a lot depending on the specifics of each code host, but again, a rough estimate is about two weeks of work.

Considerations

  • Converting to git from another source can take a long time
  • Incremental importing is necessary so that changes can be pulled more frequently
  • Concepts like branches sometimes don’t map directly to the way they are supported in git