how to write a custom Semgrep rule

Not specified by the original author.

TL;DR

A Semgrep rule is a small YAML file with an id, the languages it applies to, a severity, and a pattern written in the target language with metavariables like [ARG] for the parts that vary. Write the pattern to match the dangerous code shape, add a clear message explaining the safe alternative, test it with semgrep --test against good and bad examples, then drop it into your CI config. One focused rule per bad pattern beats one clever rule that tries to catch everything.

how to write a custom Semgrep rule

Use this when

• The same bug class keeps recurring and you want CI to catch it forever
• You need to ban a dangerous API (eval-like calls, raw SQL concatenation) in your codebase
• A security review produced a 'never do this again' pattern
• You are evaluating whether Semgrep fits your team's workflow

Not for

• Installing or configuring Semgrep in CI (setup docs cover that)
• Writing rules for other SAST engines
• Browsing the public Semgrep rule registry

Steps

1. Write the smallest YAML skeleton. Every rule needs rules, an id, languages, severity (INFO, WARNING, or ERROR), a message, and a pattern. Keep ids namespaced like team-name.no-eval so they sort and read well. Expected output: a YAML file that semgrep --validate accepts without errors.

2. Express the bad pattern in the target language. Use metavariables in brackets for the varying parts, e.g. a pattern like os.system([CMD]) matches any call to os.system regardless of argument. Semgrep understands the languages grammar, so it matches code shape, not text, and ignores formatting differences. Expected output: the pattern matches the vulnerable snippet from your last incident.

3. Add pattern-not for the safe exceptions. If there is a legitimate use (a wrapper that sanitizes first), exclude it with pattern-not so the rule doesnt cry wolf on reviewed code. Expected output: the sanitized wrapper call no longer triggers the rule.

4. Test with semgrep --test. Put vulnerable examples in a test file marked with ruleid comments for expected hits, and safe examples that must not match. Run the test command; it fails if any expectation is wrong. Expected output: semgrep --test reports all expectations met, with zero unexpected matches.

5. Write a message that teaches. The message shows up in CI output, so make it say what is wrong and what to do instead in one or two sentences, with a link to your internal guidance if you have it. Expected output: a developer who has never seen the rule understands the fix from the message alone.

6. Ship it as WARNING first, promote to ERROR later. Run the rule in CI for a week or two, fix or waive the backlog it finds, then flip it to blocking once the codebase is clean. Expected output: the first ERROR-mode run breaks no innocent builds because the backlog was already handled.

Variant phrasings

• "Semgrep custom rule tutorial"
• "Semgrep pattern vs pattern-either"
• "how to test Semgrep rules locally"
• "ban function with Semgrep rule"

Edge cases and pitfalls

• Overly broad patterns (matching every function call with one argument) drown the team in noise; anchor on the specific dangerous callee.
• Different language versions parse differently; pin the languages list to what you actually ship.
• Rules rot: review custom rules yearly and delete the ones for APIs you no longer use.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_8y7ibS39tqUeUDTNqo1Fgw

Resolved from the public thread: https://vectle.com/posts/pst_8y7ibS39tqUeUDTNqo1Fgw

Thread query: how to write a custom Semgrep rule

TL;DR: A Semgrep rule is a small YAML file with an id, the languages it applies to, a severity, and a pattern written in the target language with metavariables like [ARG] for the parts that vary. Write the pattern to match the dangerous code shape, add a clear message explaining the safe alternative, test it with semgrep --test against good and bad examples, then drop it into your CI config. One focused rule per bad pattern beats one clever rule that tries to catch everything.

Not specified by the original author.

TL;DR

The real cause is a rule producing too many matches, not a slow scan. Raise the per-file match ceiling via the engine env var: export SEMGREP\CORE\EXTRA="-max\match\per\_file 200000" and set the scan timeout to 0 (unlimited) if your pipeline allows it.

Steps

1. Semgrep times out scanning a small (~6K lines) JS file in Azure Pipelines PR analysis, with a confusing "Timeout" error. The real cause is a rule producing too many matches, not a slow scan. Increasing the timeout alone does not help.
2. Raise the per-file match ceiling via the engine env var: export SEMGREP_CORE_EXTRA="-max_match_per_file 200000" and set the scan timeout to 0 (unlimited) if your pipeline allows it. The reporter confirmed this combination got their scan through. If a single rule is matching pathologically, also consider scoping that rule down, the timeout error is semgrep's way of saying the match budget was exhausted.

When to use

You are seeing this: Semgrep times out scanning a small (~6K lines) JS file in Azure Pipelines PR analysis, with a confusing "Timeout" error. Use this skill when you run into "Semgrep confusing Timeout error on rules with too many matches maxmatchper_file".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Question asked in the thread: Semgrep times out scanning a small (~6K lines) JS file in Azure Pipelines PR analysis, with a confusing "Timeout" error. The real cause is a rule producing too many matches, not a slow scan. Increasing the timeout alone does not help.

Verified answer: Raise the per-file match ceiling via the engine env var: export SEMGREP_CORE_EXTRA="-max_match_per_file 200000" and set the scan timeout to 0 (unlimited) if your pipeline allows it. The reporter confirmed this combination got their scan through. If a single rule is matching pathologically, also consider scoping that rule down, the timeout error is semgrep's way of saying the match budget was exhausted.

Source: https://github.com/semgrep/semgrep/issues/2411

Thread: https://vectle.com/threads/thr_Pv4VtKbDNgFQVtbNEvfBmg

Not specified by the original author.

TL;DR

[bstpierre (top answer)] Using pattern-inside works: ` rules: - id: chat patterns: - pattern-inside: | var $X = GrpcChannel.ForAddress(...);... - pattern: var $Y = new ChatService.ChatServiceClient($X); languages: - csharp message: [pass] severity: INFO with this test case: using Grpc.Net.Client; using GrpcChat.ProtoBuf; var channel = GrpcChannel.ForAddress("YOUR\_HOST (port 8888)"); // ruleid: chat var client = new ChatService.ChatServiceClient(channel); when I run a test: % semgrep --test rules/ ✓ All tests passed! Context: Stack Overflow #71459777 (top answer, 4 votes, 1 answers)

Steps

1. [bstpierre (top answer)] Using pattern-inside works:

``` using Grpc.Net.Client; using GrpcChat.ProtoBuf; var channel = GrpcChannel.ForAddress("YOUR_HOST (port 8888)"); // ruleid: chat var client = new ChatService.ChatServiceClient(channel); ```

``` % semgrep --test rules/ ✓ All tests passed! ```

``` rules: - id: create-chat-client patterns: - pattern: var $X = GrpcChannel.ForAddress(...); - pattern: var $Y = new ChatService.ChatServiceClient($X); languages: - csharp message: [pass] severity: INFO ```

2. And I am trying to match this code:

``` using Grpc.Net.Client; using GrpcChat.ProtoBuf; var channel = GrpcChannel.ForAddress("YOUR_HOST (port 8888)"); var client = new ChatService.ChatServiceClient(channel); ```

3. These match separately, but the 'patterns' should be a "AND" match and it fails. I must be missing something o

## When to use

You are seeing this: I must be missing something o Use this skill when you run into "Semgrep not finding two lines of code with a 'patterns' section".

## When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

## Versions

No specific versions are mentioned in the source material, so treat the fix as generally applicable and check the examples against whatever you have installed.

## Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Source: Stack Overflow #71459777 (top answer, 4 votes, 1 answers): I have a Semgrep rule: `` rules: - id: create-chat-client patterns: - pattern: var $X = GrpcChannel.ForAddress(...); - pattern: var $Y = new ChatService.ChatServiceClient($X); languages: - csharp message: [pass] severity: INFO And I am trying to match this code: using Grpc.Net.Client; using GrpcChat.ProtoBuf; var channel = GrpcChannel.ForAddress("YOUR_HOST (port 8888)"); var client = new ChatService.ChatServiceClient(channel); `` These match separately, but the 'patterns' should be a "AND" match and it fails. I must be missing something o

Verified resolution: [bstpierre (top answer)] Using pattern-inside works: `` rules: - id: chat patterns: - pattern-inside: | var $X = GrpcChannel.ForAddress(...); ... - pattern: var $Y = new ChatService.ChatServiceClient($X); languages: - csharp message: [pass] severity: INFO with this test case: using Grpc.Net.Client; using GrpcChat.ProtoBuf; var channel = GrpcChannel.ForAddress("YOUR_HOST (port 8888)"); // ruleid: chat var client = new ChatService.ChatServiceClient(channel); when I run a test: % semgrep --test rules/ ✓ All tests passed!

Source URL: https://stackoverflow.com/questions/71459777