Regular Expressions

Introduction

Regular expressions, or "regexps", provide a way to find patterns within text. This is useful in many contexts, for example a regexp can be used to check whether a piece of text meets some criteria. In Switch a regular expression can be used to sort files based on some naming scheme: only files with a name that matches the regular expression are allowed to pass through a certain connection.

While this is almost surely of no concern for the use of regexps in Switch, it is good to know that not all regexp implementations are created equal. Most implementations provide identical results for basic expressions, but the results for more advanced expressions and matches may differ in subtle ways.

The Switch regexp language is modeled on Perl's regexp language, A good reference on regexps is Mastering Regular Expressions: Powerful Techniques for Perl and Other Tools by Jeffrey E. Friedl, ISBN 1565922573. Several web sites offer tools for working with regexps or provide libraries with examples for matching all sorts of patterns. One simple tool to start playing with regexps can be found at http://www.roblocher.com/technotes/regexp.aspx.

Brief tutorial

Regexps are built up from expressions, quantifiers, and assertions. The simplest form of expression is simply a character, example, x or 5. An expression can also be a set of characters. For example, [ABCD] will match an A or a B or a C or a D. As a shorthand we could write this as [A-D]. If we want to match any of the capital letters in the English alphabet we can write [A-Z]. A quantifier tells the regexp engine how many occurrences of the expression we want, e.g. x{1,1} means match an x which occurs at least once and at most once. We will look at assertions and more complex expressions later.

We will start by writing a regexp to match integers in the range 0 to 99. We will require at least one digit so we will start with [0-9]{1,1} which means match a digit exactly once. This regexp alone will match integers in the range 0 to 9. To match one or two digits we can increase the maximum number of occurrences so the regexp becomes [0-9]{1,2} meaning match a digit at least once and at most twice. However, this regexp as it stands will not match correctly. This regexp will match one or two digits within a string. To ensure that we match against the whole string we must use the anchor assertions. We need ^ (caret) which when it is the first character in the regexp means that the regexp must match from the beginning of the string. And we also need $ (dollar) which when it is the last character in the regexp means that the regexp must match until the end of the string. So now our regexp is ^[0-9]{1,2}$. Note that assertions, such as ^ and $, do not match any characters.

If you have seen regexps elsewhere, they may have looked different from the ones above.

This is because some sets of characters and some quantifiers are so common that they have special symbols to represent them. [0-9] can be replaced with the symbol \d.

The quantifier to match exactly one occurrence, {1,1}, can be replaced with the expression itself. This means that x{1,1} is exactly the same as x alone. So our 0 to 99 matcher could be written ^\d{1,2}$. Another way of writing it would be ^\d\d{0,1}$, i.e. from the start of the string match a digit followed by zero or one digits. In practice most people would write it ^\d\d?$. The ? is a shorthand for the quantifier {0,1}, i.e. a minimum of no occurrences and a maximum of one occurrence. This is used to make an expression optional. The regexp ^\d\d?$ means "from the beginning of the string match one digit followed by zero or one digits and then the end of the string".

Our second example is matching the words 'mail', 'letter' or 'correspondence' but without matching 'email', 'mailman', 'mailer', 'letterbox' etc. We will start by just matching 'mail'. In full the regexp is m{1,1}a{1,1}i{1,1}l{1,1}, but since each expression itself is automatically quantified by {1,1} we can simply write this as mail; an m followed by an a followed by an i followed by an l. The symbol | (bar) is used for alternation, so our regexp now becomes mail|letter|correspondence which means match 'mail' or 'letter' or 'correspondence'. Whilst this regexp will find the words we want it will also find words we do not want such as 'email'.

We will start by putting our regexp in parentheses, (mail|letter|correspondence).

Parentheses have two effects, firstly they group expressions together and secondly they identify parts of the regexp that we wish to capture for reuse later on. Our regexp still matches any of the three words but now they are grouped together as a unit. This is useful for building up more complex regexps. It is also useful because it allows us to examine which of the words actually matched.

We need to use another assertion, this time \b "word boundary": \b(mail|letter|correspondence)\b.

This regexp means "match a word boundary followed by the expression in parentheses followed by another word boundary". The \b assertion matches at a position in the regexp not a character in the regexp. A word boundary is any non-word character such as a space, a newline or the beginning or end of the string.

Characters and abbreviations for sets of characters

Sets of characters

Square brackets are used to match any character in the set of characters contained within the square brackets. All the character set abbreviations described above can be used within square brackets. Apart from the character set abbreviations and the following two exceptions no characters have special meanings in square brackets.

Using the predefined character set abbreviations is more portable than using character ranges across platforms and languages. For example, [0-9] matches a digit in Western alphabets but \d matches a digit in any alphabet.

Note that in most regexp literature sets of characters are called "character classes".

Quantifiers

By default an expression is automatically quantified by {1,1}, i.e. it should occur exactly once. In the following list E stands for any expression. An expression is a character or an abbreviation for a set of characters or a set of characters in square brackets or any parenthesized expression.

If we wish to apply a quantifier to more than just the preceding character we can use parentheses to group characters together in an expression. For example, tag+ matches a 't' followed by an 'a' followed by at least one 'g', whereas (tag)+ matches at least one occurrence of 'tag'.

Note that quantifiers are "greedy". They will match as much text as they can. For example, 0+ will match as many zeros as it can from the first zero it finds, e.g. '2.0005'.

Capturing text

Parentheses allow us to group elements together so that we can quantify and capture them. For example if we have the expression mail|letter|correspondence that matches a string we know that one of the words matched but not which one. Using parentheses allows us to "capture" whatever is matched within their bounds, so if we used (mail|letter|correspondence) and matched this regexp against the string "I sent you some email" we would capture 'mail'.

We can use captured text within the regexp itself. To refer to the captured text we use backreferences which are indexed from 1. For example we could search for duplicate words in a string using \b(\w+)\W+\1\b which means match a word boundary followed by one or more word characters followed by one or more non-word characters followed by the same text as the first parenthesized expression followed by a word boundary.

If we want to use parentheses purely for grouping and not for capturing we can use the non-capturing syntax, e.g. (?:green|blue). Non-capturing parentheses begin '(?:' and end ')'. In this example we match either 'green' or 'blue' but we do not capture the match so we only know whether or not we matched but not which color we actually found. Using non-capturing parentheses is more efficient than using capturing parentheses since the regexp engine has to do less book-keeping.

Both capturing and non-capturing parentheses may be nested.

Anchors

Anchors match the beginning or end of the string but they do not match any characters.

Regular expressions entered in the Switch user interface always match the complete string. In other words the regexp is automatically anchored at the beginning and at the end of the string, and you shouldn't specify explicit anchors. (Regular expressions specified in a Switch JavaScript program should include explicit anchors as needed).

Assertions

Assertions make some statement about the text at the point where they occur in the regexp but they do not match any characters. In the following list E stands for any expression.

Case insensitive matching

You can make the complete regexp case insensitive by surrounding it by the /.../i constuct. More precisely, if the regexp starts with a forward slash and ends with a forward slash followed by the letter i, then matching for the complete regexp is case insensitive.

For example: