Integrate GitHub with Backstage

Backstage is capable of integrating with a variety of Source Code Management Systems, such as GitHub or GitLab. This integration is very powerful, as it allows Backstage to automatically discover resources in your repository, create new repositories using Software Templates and much more. Finally, you can also use the SCM integration to authenticate users against your SCM system, so they can use their existing credentials to log in to Backstage, which we'll also set up in this section.

GitHub Preparations

For the Backstage GitHub integration, we'll need a GitHub Personal Access Token for Backstage to access your repositories. Furthermore, we'll need to set up an OAuth App in GitHub to allow users to log in to Backstage using their GitHub credentials. Let's start with the Personal Access Token.

You can create a new Personal Access Token in your GitHub account settings under "Developer settings" -> "Personal access tokens" -> "Tokens (classic)". Click on "Generate new token (classic)", give it a name (e.g., "Backstage Token") and select the following scopes (for this lab, we will keep it simple; in a production environment you should follow the principle of least privilege and only assign the necessary scopes):

repo
read:org
read:user
user:email
workflow

Please also ensure that the token expires at a reasonable time, for this lab you can set it to 7 days. Store the generated token somewhere safe (e.g. in your password manager), as you won't be able to see it again.

As this Personal Access Token only allows Backstage to access your repositories but doesn't allow users to log in, we also need to set up an OAuth App in GitHub. Navigate to "Developer settings" in your GitHub Account or Organization -> "OAuth Apps" and click on "New OAuth App". Fill in the following details:

Application name: Backstage
Homepage URL: http://localhost:3000
Authorization callback URL: http://localhost:3000/api/auth/github/handler/frame

We will need to adjust these URLs later, when we deploy Backstage to a real environment. For now, this is sufficient for our local setup.

After creating the OAuth App, you should see a Client ID and get the option to generate a Client Secret. Please generate a new Client Secret and store it somewhere safe, as you won't be able to see it again.

Furthermore, please create a team in your GitHub Organization (e.g. "Backstage Users") and add the users who should have access to Backstage to this team. We will use this team later to restrict access to Backstage.

Now we have everything we need to integrate GitHub with Backstage, let's move on to the Backstage configuration.

Configure Catalog Integration

Firstly, a word about security: Never store sensitive information, such as Personal Access Tokens or Client Secrets, directly in your configuration files. Instead, use environment variables to inject these values at runtime. In this lab, we will use environment variables for this purpose. If you are using a password manager, store these values there and retrieve them when needed (e.g., using direnv).

It is possible to use environment variables in the Backstage configuration files by using the ${ENV_VAR} syntax. As we can assume that the GitHub integration will be used in every Backstage installation in our company (lab), we will add the configuration globally in the app-config.yaml file. Open the app-config.yaml file in your favorite editor and make the following changes (if they are not already present from the initial setup):

integrations:
  github:
    - host: github.com
      token: ${GITHUB_TOKEN} # Personal Access Token

This configuration adds the GitHub integration to Backstage, using the Personal Access Token we created earlier. Now, let's also configure the Catalog to automatically discover resources in our GitHub repositories. Add the following section to the app-config.yaml file (remove the existing catalog section if present):

catalog:
  providers:
    github:
      backstageLab:
        organization: your-github-organization
        catalogPath: /catalog-info.yaml
        filters:
          branch: main
          repository: '.*'
        schedule: 
          frequency: { minutes: 5 } # every 5 minutes, turn to a lower interval in production
          timeout: { minutes: 3 }   # 3 minutes

This configuration tells Backstage to scan all repositories in your GitHub organization for files named catalog-info.yaml in the main branch every 5 minutes. These files contain the resource definitions that will be added to the Catalog.

Additionally, you might want to let Backstage know your organization data, such as Teams and Users, to enhance the user experience. Add the following section to the app-config.yaml file:

catalog:
  providers:
    githubOrg:
      id: backstageLabOrg
      githubUrl: https://github.com
      orgs: ['your-github-organization']
      schedule:
        initialDelay: { seconds: 30 }
        frequency: { hours: 1 }
        timeout: { minutes: 50 }

This configuration tells Backstage to sync the organization data from your GitHub organization every hour.

After making these changes, save the app-config.yaml file and restart your Backstage application. Stop the running application (Ctrl+C) and run the following command again:

yarn start

You will find that the catalog is still empty and does not contain any resources or users yet. This is because we didn't add the GitHub integration to Backstage yet.

Install GitHub Catalog Plugin

As stated at the beginning of the lab, Backstage can be seen as a framework to build your own Developer Portal; therefore, it is highly extensible using plugins. There are two main types of plugins in Backstage: Frontend Plugins and Backend Plugins:

Frontend Plugins: These plugins extend the user interface of Backstage, adding new pages, components or functionality to the frontend application.
Backend Plugins: These plugins extend the backend functionality of Backstage, adding new APIs, data sources, and integrations.

In our case, we need to install backend plugins to integrate with GitHub, one for the Catalog and one for the organization data. Run the following commands in your Backstage project directory:

yarn --cwd packages/backend add @backstage/plugin-catalog-backend-module-github
yarn --cwd packages/backend add @backstage/plugin-catalog-backend-module-github-org

Afterwards, we need to register these plugins in the backend application. Open the packages/backend/src/index.ts file and add the following lines in the catalog-plugin section (if not already present):

backend.add(import('@backstage/plugin-catalog-backend-module-github'));
backend.add(import('@backstage/plugin-catalog-backend-module-github-org'));

📖Example backend/src/index.ts structure

If you're unsure where to add these lines, here's what a typical index.ts file looks like:

import { createBackend } from '@backstage/backend-defaults';

const backend = createBackend();

// Core plugins
backend.add(import('@backstage/plugin-app-backend'));
backend.add(import('@backstage/plugin-catalog-backend'));

// GitHub integration plugins
backend.add(import('@backstage/plugin-catalog-backend-module-github'));
backend.add(import('@backstage/plugin-catalog-backend-module-github-org'));

backend.start();

The order typically doesn't matter, but it's good practice to group related plugins together.

After making these changes, ensure that you have your Personal Access Token available as an environment variable named GITHUB_TOKEN, then restart your Backstage application again:

export GITHUB_TOKEN=your-personal-access-token
yarn start

After some time (depending on your catalog size and the general organization setup), you should see at least your users and teams appearing in the Catalog. If you have repositories with a catalog-info.yaml file, these should appear as well.

If you see nothing appearing, please check the logs of your Backstage application for any errors. Common issues are misconfigured Personal Access Tokens or missing catalog-info.yaml files in your repositories.

As we have successfully integrated GitHub with Backstage and filled our Catalog with resources, let's move on to the next step and set up authentication for our Backstage application.

Configure GitHub Authentication

To allow users to log in to Backstage using their GitHub credentials, we need to configure the GitHub authentication provider in Backstage. Now, let's establish a routine for adding new plugins: we first install the necessary packages, then register them in the backend application, and finally configure them in the app-config.yaml file.

Install the GitHub Auth Plugin: Run the following command in your Backstage project directory:

yarn --cwd packages/backend add @backstage/plugin-auth-backend-module-github-provider

Register the Plugin: Open the packages/backend/src/index.ts file and add the following line in the auth-plugin section:

backend.add(import('@backstage/plugin-auth-backend-module-github-provider'));

Configure the Plugin: Open the app-config.yaml file and add the following section:

auth:
  environment: development
  providers:
    github:
      development:
        clientId: ${AUTH_GITHUB_CLIENT_ID}
        clientSecret: ${AUTH_GITHUB_CLIENT_SECRET}
        ## uncomment if using GitHub Enterprise
        # enterpriseInstanceUrl: ${AUTH_GITHUB_ENTERPRISE_INSTANCE_URL}
        ## uncomment to set lifespan of user session
        # sessionDuration: { hours: 24 } # supports `ms` library format (e.g. '24h', '2 days'), ISO duration, "human duration" as used in code
        signIn:
          resolvers:
            # See https://backstage.io/docs/auth/github/provider#resolvers for more resolvers
            - resolver: usernameMatchingUserEntityName

This configuration uses environment variables to inject the Client ID and Client Secret we created earlier. Please ensure that you have these values available as environment variables named AUTH_GITHUB_CLIENT_ID and AUTH_GITHUB_CLIENT_SECRET. Furthermore, it will match the GitHub username with the Backstage user entity name to control access. This means that only users who are present in the Backstage Catalog (which we populated earlier) will be able to log in.

When restarting your Backstage application, you might notice that you still only see the "Guest" user and it doesn't work. This is because we removed the configuration for the Guest user in the app-config.yaml file and didn't add the authentication to the frontend application yet. Let's add the GitHub authentication provider to the frontend application as well.

Open the file packages/app/src/App.tsx and add the following import at the top of the file:

import { githubAuthApiRef } from '@backstage/core-plugin-api';
import { SignInPage } from '@backstage/core-components';

Then, locate the SignInPage component in the same file and modify it as follows:

components: {
    SignInPage: props => (
      <SignInPage
        {...props}
        auto
        provider={{
          id: 'github-auth-provider',
          title: 'GitHub',
          message: 'Sign in using GitHub',
          apiRef: githubAuthApiRef,
        }}
      />
    ),
  },

It's okay to remove the Guest user configuration, as we won't use it anymore. After making these changes, ensure that you have the Client ID and Client Secret available as environment variables, then restart your Backstage application again:

export GITHUB_TOKEN=your-personal-access-token
export AUTH_GITHUB_CLIENT_ID=your-client-id
export AUTH_GITHUB_CLIENT_SECRET=your-client-secret
yarn start

When opening your Backstage application now, you should see the option to log in using GitHub. After logging in, you should get a consent screen from GitHub, asking you to authorize Backstage to access your account. After authorizing, you should be logged in to Backstage as your GitHub user. It might take some time until your user data is initially synced from GitHub, so if you can't log in right away, please be patient and try again after some minutes.

From now on, users in your GitHub organization can log in to Backstage using their GitHub credentials, as long as they are present in the Backstage Catalog.

Now that we have successfully set up authentication for our Backstage application, let's move on and set up our first repository for Backstage.