Regular expressions are a means to describe a text pattern,^[1] so that you can look for that pattern in a block of data. The best way to read any regular expression is one character at a time, so you need to know what each character represents.

These are the basic building blocks that you will use when writing regular expressions. If you don’t already know regex syntax, you’ll want to stick a bookmark on this page, since you’ll be referring to it until you become familiar with these characters. The Regular Expression Vocabulary table is your key to translating a line of seemingly random characters into a meaningful pattern. The table will be followed by further explanations and examples for each of the items in the table.

That’s not all there is to regular expressions, but it’s a really good starting point. Each regular expression presented in this book will have an explanation of what it’s doing, which will help you see in practical examples what each of the above characters actually ends up meaning in the wild. And, in my experience, regular expressions are understood much more quickly via examples rather than via lectures.

What follows is a more detailed explanation of each of the items in the table above, with examples.

The . character in a regular expression matches any character. For example, consider the following pattern:

That pattern matches a string containing a, followed by any character, followed by c. So, that pattern matches the strings "abc", "ancient", and "warcraft", each of which contain that pattern. It does not match "tragic", on the other hand, because there are two characters between the a and the c. That is, the . by itself, matches a single character only.

The . character is very frequently used in connection with to mean "match everything". You’ll see the (.) pattern appearing often throughout this book, and throughout examples that you see online. And while it’s often what you want, it’s just as often used incorrectly. Remember that while (.*) matches any string, so will the simpler and faster pattern ^ because every string has a start (even an empty string) and so ^ matches it.

It’s faster, too, because while (.) has to match all the way out to the end of the string, ^ only has to note that the string has a beginning, and then it is done. Note also that the pattern (.) has parenthesis and therefore captures the matched string into the variable $1. If you’re not planning to use $1 in a later substitution, then this, in addition to being a waste of computation cycles, is a waste of memory.

While considerations of this kind probably won’t save you a noticeable amount of time, getting into the habit of writing efficient regular expressions will, in the long run, not only save you these small amounts, but will result in rules that are easier to understand and easier to maintain, because they match only what you’re interested in, and nothing more.

The backslash, or escape character, either adds special meaning to a character, or removes it, depending on the context. For example, you’ve already been told that the . character has special meaning. But if you want to match the literal . character, then you need to escape it with the backslash. So, while . means "any character," \. means a literal "." character.

Conversely, some characters gain special meaning when prefixed by a \ character. For example, while s means a literal "s" character, \s means a "whitespace" character. That is, a space or a tab.

The Metacharacter table below lists useful escape characters that you’ll see throughout the book and can be used as shorthand for more verbose patterns.

The term "metacharacter" is often also applied to characters such as . and $ which have special meanings within regular expressions.

Referred to as anchor characters, these ensure that a string starts with, or ends with, a particular character, or sequence of characters. Since this is a very common need, these are included in this basic vocabulary. Consider the examples in the `anchor examples table`_

Remember, as you craft your regular expressions, that they are, by default, a substring match. Which is to say, a pattern of cow matches cow, scow, coward, and pericowperitis, because they all contain "cow" somewhere in them. Using the anchor characters allow you to be more specific as to what you wanted to match. The \b metacharacter, introduced above, can also be useful in some contexts, but perhaps less so when you’re dealing with URLs.

The + character allows a pattern or character to match more than once. For example, the following pattern will allow for common misspellings of the word "giraffe".

This pattern will allow one or more f’s, as well as one or more e’s. So it matches "girafe", "giraffe", and "giraffee". It will also match "girafffffeeeeee".

Be sure to use + rather than * when you want to ensure non-empty matches.

The * character allows the previous character to match zero or more times. That is to say, it’s exactly the same as +, except that it also allows for the pattern to not match at all. This is often used when + was meant, which can result in some confusion when it matches an empty string. As an example, we’ll use a slight modification of the pattern used in the above section:

This pattern matches the same strings listed above ("giraffe", "girafe" and "giraffee") but will also match the string "giraeeeee", which contains zero "f" characters, as well as the string "gira", which contains zero "f" characters and zero "e" characters.

Most commonly, you’ll see it used in conjunction with the . character, meaning "match anything." Frequently, in that case, the person using it has forgotten that regular expressions are substring matches. For example, consider this pattern:

The intent of that pattern is to match any string ending in .gif. The $ insists that it is at the end of the string, and the \ before the . makes that a literal . character, rather than the wildcard . character. In this particular case, the .* was there to mean "starts with anything," but is completely unnecessary, and will only serve to consume time in the matching process.

A more useful example of the * character is one which checks for a comment line in an Apache configuration file. The first non-space character needs to be a #, but the spaces are optional:

This pattern, then, matches a string that might (but doesn’t have to) begin with whitespace, followed by a . This ensures that the first non-space character of the line is a .

If you want to match a particular number of times, you can use the {n,m} quantifier to specify the range of times you wish to match. The possibilities of how you can specify this are shown in the table below.

These repetition quantifiers may be applied to a single character, or to a grouping. For example:

will match 1, 2, or 3 digits.

Will match anywhere from 2 to 5 instances of a, b, or c.

In the case of all of the repetition characters above, matching is greedy. That is, the regular expression matches as much as it possibly can. That is, if you apply the regular expression a+ to the string aaaa, matches the entire string, and not be satisfied by just the first a. This is particularly important when you are using the .* syntax, which can occasionally match more than you thought it would. I’ll give some examples of this after we’ve discussed a few more metacharacters.

On the other hand, if you wish for matches to not be greedy, you can offset the greedy nature of the repetition character by putting a ? after it.

Consider, for example, a scenario where I want to match everything between two slashes in a URL. I’ll be applying the regular expression to the URI /one/two/three/, and I’ll try a greedy, and not-greedy, regular expression. The `table of greedy examples`_ shows the results of these patterns.

The first regex is greedy, and matches as much as it possibly can, going out to the last slash. The second is non-greedy, and so stops as early as it can, when it encounters the second slash.

The ? character makes a single character match optional. This is extremely useful for common misspellings, or elements that may, or may not, appear in a string. For example, you might use it in a word when you’re not sure whether it’s supposed to be hyphenated:

The above pattern matches both "email" and "e-mail", so that either spelling will be accepted. Likewise, you could use:

to match the word color both as it is spelled in the USA, and the way that it is spelled in the rest of the world.

Additionally, the ? character turns off the "greedy" nature of the + and characters. Thus, putting a ? after a + or a will make it match as little as it possibly can. See Greedy Matching.

Further examples of the greedy vs. non-greed behavior will follow once we have learned about backreferences.

Parentheses allow you to group several characters as a unit, and also to capture the results of a match for later use. The ability to treat several characters as a unit is extremely useful in pattern matching. Think of it as combining several atoms into a single molecule. For example, consider this example:

This will look for the sequence "abc" appearing one or more times, and so would match the string "abc" and the string "abcabc".

Even more useful is the "capturing" functionality of the parentheses. Once a pattern has matched, you often want to know what matched, so that you can use it later. This is usually referred to as "backreferences."

For example, you may be looking for a .gif file, as in the example above, and you really want to know what .gif file you matched. By capturing the filename with parentheses, you can use it later on:

In the event that this pattern matches, we will capture the matching value in a special variable, $1. (In some contexts, the variable may be called %1 instead.) If you have more than one set of parentheses, the second one will be captured to the variable $2, the third to $3, and so on. Only values up through $9 are available, however. The reason for this is that $10 would be ambiguous. It might mean $1, followed by a literal zero (0), or it might mean $10. Rather than providing additional syntax to disambiguate this term, the designer of mod_rewrite instead chose to only provide backreferences through $9.

The exact way in which you can exploit this feature will be more obvious later, once we start looking at the RewriteRule directive in :ref:`RewriteRule`

Consider these two patterns, applied to the string "canadian".

The first pattern will return with a value of "anadia" in $1, since it will match as much as it possibly can between the first c and the last n it sees. The second, on the other hand, will return with $1 set to "a", since it is non-greedy, and so stops at the first n it sees.

TODO Recommend the correct regex tool

It is instructive to acquire a tool such as Regex Coach, or Rebug, mentioned in the Regex tools section below, and feed them these patterns and strings, to watch them match the different parts of the string. Mastering Regular Expressions also has a very complete treatment of backreferences, greedy matching, and what actually happens during the matching phase.

A character class allows you to define a set of characters, and match any one of them. There are several built-in character classes, like the \s metacharacter that you saw above. Using the [ ] notation lets you define your own custom character classes. As a very simple example, consider the following:

This character class matches the letter a, or the letter b, or the letter c. For example, if we wanted to match the subset of users whose usernames started with one of those letters, we might look for the pattern:

This combines several of the characters that have been described above. It ends up matching a directory path for that subset of users, and the username ends up in the $1 variable. Well, actually, not quite, as we’ll see in a minute, but almost.

The character class syntax also allows you to specify a range of characters fairly easily. For example, if you wanted to match a number between 1 and 5, you can use the character class [1-5].

Within a character class, the ^ character has special meaning, if it is the first character in the class. The character class [^abc] is the opposite of the character class [abc]. That is, it matches any character which is not a, b, or c.

Which brings us back to the example above, where we are attempting to match a username starting with a, b, or c. The problem with the example is that the * character is greedy, meaning that it attempts to match as much as it possibly can. If we want to force it to stop matching when it reaches a slash, we need to match only "not slash" characters:

I’ve replaced the . with [^/]+ which has the effect that, rather than matching any character, it matches only not-slash characters. In other words, it will only match up to a slash, or the end of the string, whichever comes first. Also, I’ve used + instead of , since one-character usernames are typically not permitted. Now, $1 will contain the username, whereas, before, it could possibly have contained other directory path components after the username.

Finally, if you wish to negate an entire regular expression match, prefix it with !. This is not consistent across all regular expression implementations, but can be used in a number of them. A very common use of this in the context of rewrite rules will be to indicate that you want a pattern to apply to all directories except for one. So, for example, if we wanted to exclude the /images directory from consideration, we would match the /images directory, but then negate the match, thus:

This matches any path not starting with /images. We’ll see more of this kind of pattern match especially in the chapter :ref:`Proxying with mod_rewrite`.

A few examples may be instructive in your understanding of how regular expressions work. We’ll start with a few of the cases that you may frequently encounter, and suggest a few alternate solutions to each.

TODO Ensure that these tools all still exist.

If you’re going to spend more than just a little time messing with regexes, you’re eventually going to want a tool that helps you visualize what’s going on. There are a number of them available, each of which has different strengths and weaknesses. You’ll find that most of the really good tools for regular expression development come out of the Perl community, where regular expressions are particularly popular, and tend to get used in almost every program.

You may find various websites that purport to be RewriteRule generators. I strongly encourage you to avoid these, and instead to learn how to craft your own rules. I’ve evaluated several of these sites, and every one has resulted in RewriteRule directives that were either enormously inefficient, or completely wrong.

Having a good grasp of Regular Expressions is a necessary prerequisite to working with mod_rewrite. All too often, people try to build regular expressions by the brute-force method, trying various different combinations at random until something seems to mostly work. This results in expressions that are inefficient and fragile, as well as a great waste of time, and much frustration.

Keep a bookmark in this chapter, and refer back to it when you’re trying to figure out what a particular regex is doing.

Other recommended reference sources include the Perl regular expression documentation, which you can find online at http://www.perldoc.com/perl5.8.0/pod/perlre.html or by typing perldoc perlre at your command line, and the PCRE documentation, which you can find online at http://pcre.org/pcre.txt. This is useful even if you’re using regex in other implementations (like mod_rewrite, for example), since the syntax is largely the same across implementations.