Skip to content

C++: Add ECMAScript std::regex parser - #22300

Open
jketema wants to merge 40 commits into
github:mainfrom
jketema:jketema/ecma-regex
Open

C++: Add ECMAScript std::regex parser#22300
jketema wants to merge 40 commits into
github:mainfrom
jketema:jketema/ecma-regex

Conversation

@jketema

@jketema jketema commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Commit-by-commit review recommended. This is not hooked into anything yet, as this is getting quite big already, so I didn't want to add the flow config here, or other regex alternatives that can be used with std::regex.

Approach:

  1. Import the Ruby parser, make QL and tests compile, fix position issue with raw strings
  2. Remove unsupported features
  3. Add features that are not supported by the Ruby parser
  4. Add missing test cases
  5. Some after-the-fact fixes.

Copilot AI and others added 30 commits August 7, 2026 14:26
Note that we are currently still implementing what Ruby thinks a regex is.
This is not correct as C++ by default uses a variant of ECMAScript regexes.
We will address this in the follow-up commits.
Strings can have a prefix in C++, which affects the location.
Clean up test source while here
Comment thread cpp/ql/lib/semmle/code/cpp/regex/RegexTreeView.qll Fixed
Comment thread cpp/ql/lib/semmle/code/cpp/regex/RegexTreeView.qll Fixed
Comment thread cpp/ql/lib/semmle/code/cpp/regex/RegexTreeView.qll Fixed
@jketema
jketema marked this pull request as ready for review August 10, 2026 09:25
@jketema
jketema requested a review from a team as a code owner August 10, 2026 09:25
Copilot AI balanced review requested due to automatic review settings August 10, 2026 09:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an ECMAScript-compatible std::regex parser and parse-tree model for future C++ data-flow integration.

Changes:

  • Adds parser and regex tree-view libraries.
  • Supports raw-string source locations and shared regex utilities.
  • Adds comprehensive parsing, value, and location tests.
Show a summary per file
File Description
cpp/ql/lib/semmle/code/cpp/regex/internal/ParseRegExp.qll Implements regex parsing.
cpp/ql/lib/semmle/code/cpp/regex/RegexTreeView.qll Exposes parsed regex trees.
cpp/ql/lib/semmle/code/cpp/exprs/Literal.qll Models raw string literals.
cpp/ql/lib/qlpack.yml Adds the regex dependency.
cpp/ql/lib/change-notes/2026-07-23-std-regex-ecmascript-parser.md Documents the feature.
cpp/ql/test/library-tests/regex/regexp.cpp Provides regex test inputs.
cpp/ql/test/library-tests/regex/regexp.ql Tests parsed terms and values.
cpp/ql/test/library-tests/regex/regexp.expected Records expected query results.
cpp/ql/test/library-tests/regex/parse.ql Produces parse-tree graphs.
cpp/ql/test/library-tests/regex/parse.expected Records expected parse trees.
cpp/ql/test/library-tests/regex/locations.ql Tests source locations.
cpp/ql/test/library-tests/regex/locations.expected Records expected locations.

Review details

Tip

Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Suppressed comments (3)

cpp/ql/lib/semmle/code/cpp/regex/RegexTreeView.qll:834

  • The single-character claim contradicts this implementation and the new abc test result, where one RegExpConstant represents the complete three-character constant. Remove that claim to keep the public class documentation accurate.
   * A constant regular expression term, that is, a regular expression
   * term matching a single string. Currently, this will always be a single character.

cpp/ql/lib/semmle/code/cpp/regex/internal/ParseRegExp.qll:65

  • Remove the accidentally duplicated explanation.
            // check if the character that comes before the previous closing bracket
            // is an opening bracket (taking `^` into account)
            // check if the character that comes before the previous closing bracket
            // is an opening bracket (taking `^` into account)

cpp/ql/lib/semmle/code/cpp/regex/RegexTreeView.qll:604

  • Add the missing possessive apostrophe.
    /** Holds if this terms name is given by the part following the escape character. */
  • Files reviewed: 12/12 changed files
  • Comments generated: 5
  • Review effort level: Balanced

result.getEnd() = part_end
}

/** Hodls if this term may match an unlimited number of times. */
Comment on lines +806 to +807
* A normal character in a regular expression, that is, a character
* without special meaning. This includes escaped characters.
}

/**
* Holds if `root` has the `s` flag for multi-line matching.
*/
abstract class RegExp extends StringLiteral {
/**
* Holds if this `RegExp` has the `s` flag for multi-line matching.
Comment thread cpp/ql/lib/semmle/code/cpp/regex/internal/ParseRegExp.qll Outdated
jketema and others added 2 commits August 10, 2026 11:30
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants