Basics of writing a Logparser SQL Query

A basic SQL query must have, at a minimum two basic building blocks: the SELECT clause, and the FROM clause. For starters: start Log Parser Lizard, click on the “New Query” button on the toolbar, from a drop down list select “Windows Event Log” and in the Query text box in the bottom of the window write the following command:


SELECT * FROM System

 

The SELECT clause is used to specify which input record fields we want to appear in the output. The FROM clause is used to specify which specific data source we want the Input Format to process. Different Input Formats interpret the value of the FROM clause in different ways; for instance, the EVT Input Format requires the value of the FROM clause to be the name of a Windows Event Log, which in our example is the "System" Event Log.

 

The special "*" wildcard after a SELECT keyword means "all the fields" (like in standard SQL). Most of the times, an output of all of the fields of the log records might not be desired. You might only want to see only the fields that are of your interest. To accomplish this, instead of the "*" wildcard in the SELECT clause, you will have to write a comma-separated list of the names of the fields you wish to be displayed.

 

SELECT TimeGenerated, EventTypeName, SourceName FROM System.

 

The Log Parser SQL-Like language also supports a wide variety of functions, including arithmetical functions (e.g. ADD, SUB, MUL, DIV, MOD, QUANTIZE, etc.), string manipulation functions (e.g. SUBSTR, STRCAT, STRLEN, EXTRACT_TOKEN, etc.), and timestamp manipulation functions (e.g. TO_DATE, TO_TIME, TO_UTCTIME, etc.). Functions can also appear as arguments of other functions.

 

SELECT TO_DATE(TimeGenerated), TO_UPPERCASE( EXTRACT_TOKEN(EventTypeName, 0, ' ') ), SourceName FROM System

 

То change the name of a field-expression in the SELECT clause by using an alias you can use the AS keyword followed by the new name of the field.

 

SELECT TO_DATE(TimeGenerated) AS DateGenerated, TO_UPPERCASE( EXTRACT_TOKEN(EventTypeName, 0, ' ') ) AS TypeName, SourceName FROM System

 

When retrieving data from an Input Format, it is often needed to filter out unneeded records and only keep those that match specific criteria. To accomplish this task, you can use another basic building block of the Log Parser SQL language: the WHERE clause which is used to specify a Boolean expression that must be satisfied by an input record for that record to be listed in the output. Input records that do not satisfy the condition will be discarded. Conditions specified in the WHERE clause can be more complex, making use of comparison operators (such as ">", "<=", "<>", "LIKE", "BETWEEN", etc.) and boolean operators (such as "AND", "OR", "NOT"). The WHERE clause must immediately follow the FROM clause.

 

SELECT TimeGenerated, EventTypeName, SourceName FROM System WHERE ( SourceName = 'Service Control Manager' AND EventID >= 7024)

 

The ORDER BY clause can be used to specify that the output records should be sorted according to the values of selected fields. By default, output records are sorted according to ascending values. We can change the sort direction by appending the DESC (for descending) or ASC (for ascending) keywords to the ORDER BY clause.

 

SELECT SourceName, EventID, TimeGenerated FROM System ORDER BY TimeGenerated

 

Sometimes we might need to aggregate multiple input records together and perform some operation on groups of input records. To accomplish this, the Log Parser SQL like language has a set of aggregate functions (also referred to as "SQL functions") that can be used to perform basic calculations on multiple records. These functions include SUM, COUNT, MAX, MIN, and AVG. The GROUP BY clause is used to specify which fields we want the group subdivision to be based on. After the input records have been divided into these groups, all the aggregate functions in the SELECT clause will be calculated separately on each of these groups, and the query will return an output record for each group created.

 

SELECT EventTypeName, Count(*) FROM system GROUP BY EventTypeName

 

For filtering results from groups you can use the HAVING clause. The HAVING clause works just like the WHERE clause, with the only difference being that the HAVING clause is evaluated after groups have been created, which makes it possible for the HAVING clause to specify aggregate functions.

 

SELECT EventTypeName, Count(*) from system group by EventTypeName HAVING EventTypeName =’Error event'

 

The DISTINCT keyword is used to indicate that the output of a query should consist of unique records. Duplicate output records are discarded. It is also possible to use the DISTINCT keyword inside the COUNT aggregate function, in order to retrieve the total number of different values appearing in the data.

 

SELECT DISTINCT SourceName from System

 

SELECT COUNT( DISTINCT SourceName) from System

 

Use the TOP keyword in the SELECT clause to return only a few records at the top of the ordered output.

 

SELECT TOP 10 SourceName, Count(*) as Total FROM System GROUP BY SourceName ORDER BY Total DESC

 

These are simple queries, but they are good example that this log tool is more powerful for analyzing syslog events than any other event log viewer. For more samples, you can always look in examples provided with the program. They don’t all work out-of-a-box but can be very helpful.

 

There are many additional resources for learning about and using Log Parser on the Internet. Please check the following links.

  1. Logparser Download
  2. Logparser Forums
  3. Using Log Parser Lizard with SharePoint
  4. Examples (SQL) queries for IIS Analysis
  5. Under the hood of Logparser
  6. Microsoft Script Center page - The Microsoft logparser overview page
  7. Forensic IIS log exploration with LogParser
  8. Using the Logparser Utility to Analyze Exchange/IIS Logs
  9. LogParser 2.2 and ASP.NET
  10. Auditing the Event Logs
  11. Log Parser Plus
  12. Aggressive Virus Defense

Feedback and Knowledge Base