Benchmarking Vitest Against Jest • Kwicherbelliaken Studio

TL;DR

I used Hyperfine to benchmark Jest and Vitest and found no significant performance difference.

I converted 33 test files covering 216 tests from [[Jest]] to [[Vitest]] and benchmarked their performance. I used hyperfine as my benchmarking tool after finding an example of it being used to compare differences between git branches in Adam Johnsons blog.

This is the command I ran with hyperfine:

1
 hyperfine \
2
    --warmup 1 \
3
    --command-name testing-with-jest \
4
    --prepare 'git switch master' \
5
    'npm run jest <LIST_OF_33_TEST_FILES> --silent' \
6
    --command-name testing-with-vitest \
7
    --prepare 'git switch jack/refactor/add-vitest' \
8
    'npm run vitest <LIST_OF_33_TEST_FILES> --silent'

If you are interested in understanding the structure of the command passed to hyperfine, Adam does a good job in his blog Shell: benchmark the difference between two Git branches with hyperfine. The rub is that on master we run our tests with Jest, while on the feature branch we use Vitest. Hyperfine runs enough samples—about 10 per test runner on the 33 test files—to reliably spot performance differences. It also does a warmup run to reduce cold start effects.

Here are the results:

1
Benchmark 1: testing-with-jest
2
  Time (mean ± σ):     15.554 s ±  1.700 s    [User: 71.014 s, System: 9.626 s]
3
  Range (min … max):   13.871 s … 20.071 s    10 runs
4

5
Benchmark 2: testing-with-vitest
6
  Time (mean ± σ):     16.403 s ±  0.749 s    [User: 73.026 s, System: 11.010 s]
7
  Range (min … max):   15.626 s … 18.251 s    10 runs
8

9
Summary
10
  testing-with-jest ran
11
    1.05 ± 0.12 times faster than testing-with-vitest

For this test suite, there’s no major difference between Jest and Vitest—though Jest is slightly faster.

It is worth noting that, before I discovered Hyperfine, I used Claude to write a shell script that ran the same comparison, comparing their average execution times.

1
#!/bin/bash
2

3
# Number of runs for each test runner
4
RUNS=10
5

6
echo "Running benchmarks. Each test sweep runs $RUNS times"
7
echo "----------------------------------------"
8

9
# Function to convert time to seconds
10
to_seconds() {
11
    local time=$1
12
    local minutes=$(echo $time | cut -d'm' -f1)
13
    local seconds=$(echo $time | cut -d'm' -f2 | cut -d's' -f1)
14
    echo "$minutes * 60 + $seconds" | bc
15
}
16

17
# Function to calculate average time
18
calculate_average() {
19
    local total=0
20
    local count=0
21
    while read -r time; do
22
        seconds=$(to_seconds "$time")
23
        total=$(echo "$total + $seconds" | bc)
24
        count=$((count + 1))
25
    done
26
    if [ $count -gt 0 ]; then
27
        echo "scale=3; $total / $count" | bc
28
    else
29
        echo "0"
30
    fi
31
}
32

33
# Create temporary directories for results
34
mkdir -p .benchmark-results
35

36
echo "Running Vitest benchmarks..."
37
for i in $(seq 1 $RUNS); do
38
    echo "Vitest run $i/$RUNS"
39
    { time npm run vitest run <LIST_OF_33_JEST_TEST_FILES> --run --silent; } 2> .benchmark-results/vitest_run_$i.txt
40
done
41

42
echo "Running Jest benchmarks..."
43
for i in $(seq 1 $RUNS); do
44
    echo "Jest run $i/$RUNS"
45
    { time npm run test <LIST_OF_33_VITEST_TEST_FILES> --run --silent; } 2> .benchmark-results/jest_run_$i.txt
46
done
47

48
# Calculate averages
49
echo "----------------------------------------"
50
echo "Results:"
51
echo "Individual run times:"
52
echo "Vitest runs:"
53
for i in $(seq 1 $RUNS); do
54
    time=$(grep "real" .benchmark-results/vitest_run_$i.txt | awk '{print $2}')
55
    seconds=$(to_seconds "$time")
56
    echo "Run $i: ${seconds}s"
57
done
58

59
echo "Jest runs:"
60
for i in $(seq 1 $RUNS); do
61
    time=$(grep "real" .benchmark-results/jest_run_$i.txt | awk '{print $2}')
62
    seconds=$(to_seconds "$time")
63
    echo "Run $i: ${seconds}s"
64
done
65

66
echo "----------------------------------------"
67
echo "Averages:"
68
echo "Vitest average time: $(grep "real" .benchmark-results/vitest_run_* | awk '{print $2}' | calculate_average)s"
69
echo "Jest average time: $(grep "real" .benchmark-results/jest_run_* | awk '{print $2}' | calculate_average)s"
70

71
# Save test output for analysis
72
echo "----------------------------------------"
73
echo "Saving detailed test output for analysis..."
74
echo "Vitest output:" > .benchmark-results/vitest_analysis.txt
75
cat .benchmark-results/vitest_run_1.txt >> .benchmark-results/vitest_analysis.txt
76
echo "Jest output:" > .benchmark-results/jest_analysis.txt
77
cat .benchmark-results/jest_run_1.txt >> .benchmark-results/jest_analysis.txt
78

79
# Don't cleanup so we can analyze the results
80
# rm -rf .benchmark-results

Below are the results. (It’s not from running all 33 test files, but from 10 test runs of a single file, and gives an idea of what I produced.)

1
Results:
2
Individual run times:
3

4
Vitest runs:
5
Run 1: 7.369s
6
Run 2: 7.049s
7
Run 3: 6.771s
8
Run 4: 6.553s
9
Run 5: 6.389s
10
Run 6: 6.179s
11
Run 7: 6.136s
12
Run 8: 6.363s
13
Run 9: 6.379s
14
Run 10: 6.665s
15

16
Jest runs:
17
Run 1: 8.542s
18
Run 2: 7.688s
19
Run 3: 7.543s
20
Run 4: 7.359s
21
Run 5: 7.257s
22
Run 6: 7.231s
23
Run 7: 7.201s
24
Run 8: 7.082s
25
Run 9: 7.226s
26
Run 10: 7.117s
27

28
Averages:
29
Vitest average time: 6.585s
30
Jest average time: 7.424s