Why isn't a pattern which is terminated with \\$ working correctly on Windows?

[andrey-starodubtsev]

Please tick this box to confirm you have reviewed the above.

• I have a different issue.

What version of ripgrep are you using?

C:\> rg --version
ripgrep 15.1.0 (rev af60c2de9d)

features:+pcre2
simd(compile):+SSE2,-SSSE3,-AVX2
simd(runtime):+SSE2,+SSSE3,+AVX2

PCRE2 10.45 is available (JIT is available)

How did you install ripgrep?

winget install BurntSushi.ripgrep.MSVC

What operating system are you using ripgrep on?

Windows 11 Home

Describe your bug.

ripgrep can't find patterns which are terminated with \\$

What are the steps to reproduce the behavior?

C:\>cat 1.h
#pragma once

#define FOO(x) \
    static int a = x; \
    ++a;

C:\>grep "static.*\\\\$" 1.h
    static int a = x; \

C:\>grep --version
grep (GNU grep) 3.0
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.

C:\>rg "static.*\\\\$" 1.h

WSL version with the latest release of ripgrep works fine:

➜  /tmp cat 1.h
#pragma once

#define FOO(x)        \
    static int a = x; \
    ++a;
➜  /tmp grep "static.*\\\\$" 1.h
    static int a = x; \
➜  /tmp rg "static.*\\\\$" 1.h
4:    static int a = x; \
➜  grep --version
grep (GNU grep) 3.7
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and others; see
<https://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.
➜  /tmp rg --version
ripgrep 15.1.0 (rev af60c2de9d)

features:+pcre2
simd(compile):+SSE2,-SSSE3,-AVX2
simd(runtime):+SSE2,+SSSE3,+AVX2

PCRE2 10.45 is available (JIT is available)

What is the actual behavior?

empty match

What is the expected behavior?

non-empty match

[BurntSushi]

Since you're on Windows, I can't tell whether your input files use CRLF or LF line endings. Have you tried the --crlf flag? This is relevant because it changes $ from matching only \n to also matching \r\n.

Maybe your grep program is using CRLF automatically? It's hard to tell with the information you've provided. But it's easy to demonstrate the difference between LF and CRLF. First, LF:

$ xxd /tmp/1.h
00000000: 2370 7261 676d 6120 6f6e 6365 0a0a 2364  #pragma once..#d
00000010: 6566 696e 6520 464f 4f28 7829 205c 0a20  efine FOO(x) \.
00000020: 2020 2073 7461 7469 6320 696e 7420 6120     static int a
00000030: 3d20 783b 205c 0a20 2020 202b 2b61 3b0a  = x; \.    ++a;.
$ grep "static.*\\\\$" /tmp/1.h
    static int a = x; \
$ rg "static.*\\\\$" /tmp/1.h
4:    static int a = x; \

And now with CRLF:

$ xxd /tmp/1.h
00000000: 2370 7261 676d 6120 6f6e 6365 0d0a 0d0a  #pragma once....
00000010: 2364 6566 696e 6520 464f 4f28 7829 205c  #define FOO(x) \
00000020: 0d0a 2020 2020 7374 6174 6963 2069 6e74  ..    static int
00000030: 2061 203d 2078 3b20 5c0d 0a20 2020 202b   a = x; \..    +
00000040: 2b61 3b0d 0a                             +a;..
$ grep "static.*\\\\$" /tmp/1.h
$ rg "static.*\\\\$" /tmp/1.h
$ rg --crlf "static.*\\\\$" /tmp/1.h
4:    static int a = x; \

My version of grep doesn't work with CRLF line terminators. But maybe yours does.

Also, on Unix at least, using single quotes makes the escaping easier:

$ rg --crlf 'static.*\\$' /tmp/1.h
4:    static int a = x; \

[andrey-starodubtsev]

Okay, it turns out that:

1. grep and rg on Windows interpret pattern differently. To match backslash at the end of the line for grep it's "\\\\$" and for rg - "\\$". For WSL matching pattern is the same though - "\\\\$".
2. --crlf helps, thank you; it turns out that header file had Unix endlines, and because of that $ didn't match. But nonetheless, grep matches lines with any terminator without any additional configuration (grep.exe I use is distributed with Git); and rg matches only one type of line terminator depending on whether --crlf present or not. That means that if, for example, project contains files with different terminators to find all occurrences of matching pattern one need to run rg twice.

PS: Sadly ' doesn't work for cmd.exe, so I got used to use " to be able to copy commands between Windows and Linux shells; doesn't work in 100% cases, but handy most of the times.

[BurntSushi]

That means that if, for example, project contains files with different terminators to find all occurrences of matching pattern one need to run rg twice.

That's incorrect. If you use --crlf, then $ will match both CRLF and LF line endings. It's very easy to demonstrate:

$ printf 'a\r\n' | rg 'a$'
$ printf 'a\r\n' | rg 'a$' --crlf
a
$ printf 'a\n' | rg 'a$'
a
$ printf 'a\n' | rg 'a$' --crlf
a

In other words, $ will always match Unix line endings (unless you use --null-data).

[andrey-starodubtsev]

Yes, you are right; I must have tried pattern from grep, which doesn't match but not because of line endings.

So, putting --crlf in RIPGREP_CONFIG_PATH solves the issue completely on both platforms.

Thanks again.