Skip to main content
Performance Improvement with Parallel Benchmarks
Running benchmarks can be quite long and slow down your CI process. With CodSpeed, you can run multiple benchmark commands in the same CI workflow. When running heavy benchmark suites, this can divide the total runtime by the number of jobs, dramatically speeding up your CI pipeline.
We recommend limiting each shard to fewer than 1,000 benchmarks.
Multiple Benchmark Overview
You can speed up your CI even further by combining sharded benchmarks with partial runs. This way, you can run only the benchmark suites that are relevant to the code changes in your pull request.

​
How to split the execution of benchmark suites?

Sharding allows you to run your benchmarks in several commands, with each command running a subset of your benchmarks. There are two main ways of sharding your benchmarks:
  • splitting them in several files, and running each file or folder independently
  • using a sharding tool provided by your benchmarking framework
Check the sharding docs corresponding to your benchmarking framework:

​
Configure your CI workflow to use sharded benchmarks

Once you’ve sharded your benchmarks, you can run using several CI jobs within the same workflow, depending on your CI provider:

​
Multiple languages benchmarks

With benchmarks written in several languages, it can be difficult to get a unified performance overview of your project. With CodSpeed, you can run benchmarks written in multiple languages. When run in the same CI workflow, CodSpeed will aggregate the results of these benchmarks into a single report. For example, using pytest and vitest:
.github/workflows/codspeed.yml
permissions: # optional for public repositories
  contents: read # required for actions/checkout
  id-token: write # required for OIDC authentication with CodSpeed

jobs:
  python-benchmarks:
    name: "Run Python benchmarks"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - name: Install required-version defined in uv.toml
        uses: astral-sh/setup-uv@v7
      - uses: actions/setup-python@v6
        with:
          python-version: 3.12.8
      - name: Run benchmarks
        uses: CodSpeedHQ/action@v4
        with:
          mode: simulation
          run: uv run pytest tests/benchmarks/ --codspeed

  nodejs-benchmarks:
    name: "Run NodeJS benchmarks"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: "actions/setup-node@v6"
      - name: Install dependencies
        run: npm install
      - name: Run benchmarks
        uses: CodSpeedHQ/action@v4
        with:
          mode: simulation
          run: npm exec vitest bench