More from Steven Levithan:
“Regular Expressions Cookbook manages to be simultaneously accessible and almost ridiculously comprehensive.”
—Jeff Atwood
New flags
About flags
XRegExp provides two new flags, which can be combined with native flags and arranged in any order. Unlike native flags, non-native flags do not show up as properties on regular expression objects.
- New flags
s— Dot matches all (aka dotall or singleline mode)x— Free-spacing and line comments (aka extended mode)
- Native flags
g— All matches (global)i— Case insensitive (ignoreCase)m—^and$match at newlines (multiline)y— Matches anchored tolastIndex(sticky) — New in JavaScript 1.8 (Firefox 3)
Dot matches all (s)
The now abandoned ECMAScript 4 proposals called for recognizing the C1/Unicode NEL "next line" control character (\u0085) as an additional newline character in that standard.
Usually, dot does not match newlines. However, a mode in which dot matches newlines can be as useful as one where dot doesn't. The s flag allows the mode to be selected on a per-regex basis. Escaped dots and dots within character classes (e.g. [.a-z]) are always equivalent to literal dots. The newline characters are listed below:
\u000a— Line feed —\n\u000d— Carriage return —\r\u2028— Line separator\u2029— Paragraph separator
Annotations
- Rationale: All common Perl-like regular expression flavors except JavaScript include a flag that allows dot to match newlines. Without this mode, matching any single character requires, e.g.,
[\s\S],[\0-\uFFFF], or god forbid(.|\s). - Compatibility: No known problems; the
sflag is illegal in native ECMAScript 3 regular expressions. - Prior art: The flag comes from Perl.
Free-spacing and line comments (x)
It might be better to think of whitespace and comments as do-nothing (rather than ignore-me) metacharacters. This distinction is important with something like \12 3, which with the x flag is taken as \12 followed by 3, and not \123. However, quantifiers following whitespace or comments apply to the preceeding token, so x + is equivalent to x+.
This flag has two complementary effects. First, it causes most whitespace to be ignored, so you can free-format the regex pattern for readability. Second, it allows comments with a leading #. Specifically, it turns most whitespace into an "ignore me" metacharacter, and # into an "ignore me, and everything else up to the next newline" metacharacter. They aren't taken as metacharacters within character classes (which means that classes are not free-format, even with x), and as with other metacharacters, you can escape whitespace and # that you want to be taken literally. Of course, you can always use \s to match whitespace.
ECMA-262 Edition 3 uses an interpretation of whitespace based on Unicode's Basic Multilingual Plane, from version 2.1 or later of the Unicode standard. Following are the characters that should be matched by \s according to ECMA-262 Edition 3 and Unicode 5.1:
JavaScript's \s is similar but not equivalent to \p{Z} from regex libraries that support Unicode properties, including XRegExp's own Unicode plugin. The difference is that \s includes characters \u0009–\u000d, which are not assigned the Separator property in the Unicode character database.
Note that not all shorthand character classes and other JavaScript regex syntax is Unicode-aware. According to ECMA-262 Edition 3, \s, \S, ., ^, and $ use Unicode-based interpretations of whitespace and newline, while \d, \D, \w, \W, \b, and \B use ASCII-only interpretations of digit, word character, and word boundary (e.g. /a\b/.test("naïve") returns true). Actual browser implementations differ on these points. For example, Firefox 2 and lower considers \d and \D to be Unicode-aware, whereas Firefox 3 fixes this bug—making \d equivalent to [0-9] as with most other browsers.
To test which characters or positions are matched by the tokens just mentioned in your browser, see the JavaScript regex Unicode compatibility test.
\u0009— Tab —\t\u000a— Line feed —\n— newline\u000b— Vertical tab —\v\u000c— Form feed —\f\u000d— Carriage return —\r— newline\u0020— Space\u00a0— No-break space\u1680— Ogham space mark\u180e— Mongolian vowel separator\u2000— En quad\u2001— Em quad\u2002— En space\u2003— Em space\u2004— Three-per-em space\u2005— Four-per-em space\u2006— Six-per-em space\u2007— Figure space\u2008— Punctuation space\u2009— Thin space\u200a— Hair space\u2028— Line separator — newline\u2029— Paragraph separator — newline\u202f— Narrow no-break space\u205f— Medium mathematical space\u3000— Ideographic space
Annotations
- Rationale: Regular expressions are notoriously hard to read; adding whitespace and comments makes regular expressions easier to read.
- Compatibility: No known problems; the
xflag is illegal in native ECMAScript 3 regular expressions. Note that line comments cannot contain invalid patterns if the user expects to apply this flag post-compilation usingaddFlags. - Prior art: The flag comes from Perl, and was originally inspired by Jeffrey Friedl's pretty-printing of complex regexes.
References and sources
- ECMA International (1999-12). Standard ECMA-262, ECMAScript Language Specification, 3rd Edition. ISO/IEC 16262.
- ES4 Wiki (2007-09-03; retrieved 2009-06-24). Proposals: Extended regular expressions.
- Friedl, Jeffrey (2006-08). Mastering Regular Expressions, 3rd Edition. O'Reilly Media. ISBN 978-0-596-52812-6.
- The Unicode Standard 5.1 (2008; retrieved 2009-06-24). Final Unicode 5.1 names list.