org.apache.jmeter.protocol.http.util.accesslog

## Class TCLogParser

- java.lang.Object
- - org.apache.jmeter.protocol.http.util.accesslog.TCLogParser

- All Implemented Interfaces:
    LogParser
  Direct Known Subclasses:
    SharedTCLogParser

      public class TCLogParser
      extends java.lang.Object
      implements LogParser

  Description:
  Currently the parser only handles GET/POST requests. It's easy enough to add support for other request methods by changing checkMethod. The is a complete rewrite of a tool I wrote for myself earlier. The older algorithm was basic and did not provide the same level of flexibility I want, so I wrote a new one using a totally new algorithm. This implementation reads one line at a time using BufferedReader. When it gets to the end of the file and the sampler needs to get more requests, the parser will re-initialize the BufferedReader. The implementation uses StringTokenizer to create tokens.

  The parse algorithm is the following:

  1. cleans the entry by looking for backslash "\"
  2. looks to see if GET or POST is in the line
  3. tokenizes using quotes "
  4. finds the token with the request method
  5. gets the string of the token and tokenizes it using space
  6. finds the first token beginning with slash character
  7. tokenizes the string using question mark "?"
  8. get the path from the first token
  9. returns the second token and checks it for parameters
  10. tokenizes the string using ampersand "&"
  11. parses each token to name/value pairs

  Extending this class is fairly simple. Most access logs use the same format starting from the request method. Therefore, changing the implementation of cleanURL(string) method should be sufficient to support new log formats. Tomcat uses common log format, so any webserver that uses the format should work with this parser. Servers that are known to use non standard formats are IIS and Netscape.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected boolean` | `decode` by default, we probably should decode the parameter values
    `protected java.lang.String` | `FILENAME`
    `protected Filter` | `FILTER` Handles to supporting classes
    `static java.lang.String` | `GET`
    `static java.lang.String` | `HEAD`
    `protected static org.slf4j.Logger` | `log`
    `static java.lang.String` | `POST`
    `protected java.io.BufferedReader` | `READER`
    `protected java.lang.String` | `RMETHOD` protected members *
    `protected java.io.File` | `SOURCE`
    `protected java.lang.String` | `URL_PATH` The path to the access log file
    `protected boolean` | `useFILE`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `TCLogParser()`
    `TCLogParser(java.lang.String source)`

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `boolean` | `checkMethod(java.lang.String text)` The method checks for `POST`, `GET` and `HEAD` methods currently.
    `boolean` | `checkParamFormat(java.lang.String text)` Checks the string to see if it contains "&" and "=".
    `boolean` | `checkURL(java.lang.String url)` Checks the string to make sure it has `/path/file?`
    `java.lang.String` | `cleanURL(java.lang.String entry)` The method cleans the URL using the following algorithm.
    `void` | `close()` close the any streams or readers.
    `void` | `convertStringToJMRequest(java.lang.String text, TestElement el)` Convert a single line into XML
    `NVPair[]` | `convertStringtoNVPair(java.lang.String stringparams)` Parse the string parameters into NVPair[] array.
    `boolean` | `decodeParameterValue()` decode the parameter values is to true by default
    `protected int` | `parse(java.io.BufferedReader breader, TestElement el, int parseCount)` The method is responsible for reading each line, and breaking out of the while loop if a set number of lines is given.
    `int` | `parse(TestElement el, int parseCount)` parse the entire file.
    `int` | `parseAndConfigure(int count, TestElement el)` parse a set number of lines from the access log.
    `protected int` | `parseLine(java.lang.String line, TestElement el)` parseLine calls the other parse methods to parse the given text.
    `protected NVPair` | `parseOneParameter(java.lang.String parameter)` Method expects name and value to be separated by an equal sign "=".
    `protected java.util.List<java.lang.String>` | `parseParameters(java.lang.String parameters)` Method uses StringTokenizer to convert the string into single pairs.
    `void` | `setDecodeParameterValues(boolean decodeparams)` by default decode is set to true. if the parameters shouldn't be decoded, call the method with false
    `void` | `setFilter(Filter filter)` Use the filter to include/exclude files in the access logs.
    `void` | `setSourceFile(java.lang.String source)` Sets the source file.
    `void` | `setUseParsedFile(boolean file)` Calls this method to set whether or not to use the path in the log.
    `java.lang.String` | `stripFile(java.lang.String url, TestElement el)` Tokenize the URL into two tokens.
    `java.util.StringTokenizer` | `tokenize(java.lang.String line, java.lang.String delim)` Parses the line using java.util.StringTokenizer.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### log

          protected static final org.slf4j.Logger log

    - #### GET

          public static final java.lang.String GET

      See Also:
        Constant Field Values

    - #### POST

          public static final java.lang.String POST

      See Also:
        Constant Field Values

    - #### HEAD

          public static final java.lang.String HEAD

      See Also:
        Constant Field Values

    - #### RMETHOD

          protected java.lang.String RMETHOD

      protected members *

    - #### URL_PATH

          protected java.lang.String URL_PATH

      The path to the access log file

    - #### useFILE

          protected boolean useFILE

    - #### SOURCE

          protected java.io.File SOURCE

    - #### FILENAME

          protected java.lang.String FILENAME

    - #### READER

          protected java.io.BufferedReader READER

    - #### FILTER

          protected Filter FILTER

      Handles to supporting classes

    - #### decode

          protected boolean decode

      by default, we probably should decode the parameter values

  - ### Constructor Detail

    - #### TCLogParser

          public TCLogParser()

    - #### TCLogParser

          public TCLogParser(java.lang.String source)

      Parameters:
        `source` - name of the source file

  - ### Method Detail

    - #### setDecodeParameterValues

          public void setDecodeParameterValues(boolean decodeparams)

      by default decode is set to true. if the parameters shouldn't be decoded, call the method with false
      Parameters:
        `decodeparams` - flag whether parameters should be decoded

    - #### decodeParameterValue

          public boolean decodeParameterValue()

      decode the parameter values is to true by default
      Returns:
        `true` if parameter values should be decoded, `false` otherwise

    - #### setUseParsedFile

          public void setUseParsedFile(boolean file)

      Calls this method to set whether or not to use the path in the log. We may want to provide the ability to filter the log file later on. By default, the parser uses the file in the log.
      Parameters:
        `file` - flag whether to use the path from the log

    - #### setFilter

          public void setFilter(Filter filter)

      Use the filter to include/exclude files in the access logs. This is provided as a convenience and reduce the need to spend hours cleaning up log files.
      Specified by:
        `setFilter` in interface `LogParser`
      Parameters:
        `filter` - `Filter` to be used while reading the log lines

    - #### setSourceFile

          public void setSourceFile(java.lang.String source)

      Sets the source file.
      Specified by:
        `setSourceFile` in interface `LogParser`
      Parameters:
        `source` - name of the source file

    - #### parse

          public int parse(TestElement el,
                           int parseCount)

      parse the entire file.
      Parameters:
        `el` - TestElement to read the lines into
        `parseCount` - number of max lines to read
      Returns:
        number of read lines, or `-1` if an error occurred while reading

    - #### parseAndConfigure

          public int parseAndConfigure(int count,
                                       TestElement el)

      parse a set number of lines from the access log. Keep in mind the number of lines parsed will depend on the filter and number of lines in the log. The method returns the actual number of lines parsed.
      Specified by:
        `parseAndConfigure` in interface `LogParser`
      Parameters:
        `count` - number of lines to read
        `el` - `TestElement` to read lines into
      Returns:
        lines parsed

    - #### parse

          protected int parse(java.io.BufferedReader breader,
                              TestElement el,
                              int parseCount)

      The method is responsible for reading each line, and breaking out of the while loop if a set number of lines is given.
      Note: empty lines will not be counted
      Parameters:
        `breader` - `BufferedReader` to read lines from
        `el` - `TestElement` to read lines into
        `parseCount` - number of lines to read
      Returns:
        number of lines parsed

    - #### parseLine

          protected int parseLine(java.lang.String line,
                                  TestElement el)

      parseLine calls the other parse methods to parse the given text.
      Parameters:
        `line` - single line to be parsed
        `el` - `TestElement` in which the line will be added
      Returns:
        number of lines parsed (zero or one, actually)

    - #### cleanURL

          public java.lang.String cleanURL(java.lang.String entry)

      The method cleans the URL using the following algorithm.
      1. check for double quotes
      2. check the request method
      3. tokenize using double quotes
      4. find first token containing request method
      5. tokenize string using space
      6. find first token that begins with "/"
      Example Tomcat log entry:

      127.0.0.1 - - [08/Jan/2003:07:03:54 -0500] "GET /addrbook/ HTTP/1.1" 200 1981

      would result in the extracted url `/addrbook/`

      Parameters:
        `entry` - line from which the url is to be extracted
      Returns:
        cleaned url

    - #### checkMethod

          public boolean checkMethod(java.lang.String text)

      The method checks for `POST`, `GET` and `HEAD` methods currently. The other methods aren't supported yet.
      Parameters:
        `text` - text to be checked for HTTP method
      Returns:
        `true` if method is supported, `false` otherwise

    - #### stripFile

          public java.lang.String stripFile(java.lang.String url,
                                            TestElement el)

      Tokenize the URL into two tokens. If the URL has more than one "?", the parse may fail. Only the first two tokens are used. The first token is automatically parsed and set at `URL_PATH`.
      Parameters:
        `url` - url which should be stripped from parameters
        `el` - `TestElement` to parse url into
      Returns:
        String presenting the parameters, or `null` when none where found

    - #### checkURL

          public boolean checkURL(java.lang.String url)

      Checks the string to make sure it has `/path/file?name=value` format. If the string doesn't contains a "?", it will return `false`.
      Parameters:
        `url` - url to check for parameters
      Returns:
        `true` if url contains a `?`, `false` otherwise

    - #### checkParamFormat

          public boolean checkParamFormat(java.lang.String text)

      Checks the string to see if it contains "&" and "=". If it does, return `true`, so that it can be parsed.
      Parameters:
        `text` - text to be checked for `&` and `=`
      Returns:
        `true` if `text` contains both `&` and `=`, `false` otherwise

    - #### convertStringToJMRequest

          public void convertStringToJMRequest(java.lang.String text,
                                               TestElement el)

      Convert a single line into XML
      Parameters:
        `text` - to be converted
        `el` - `HTTPSamplerBase` which consumes the `text`

    - #### convertStringtoNVPair

          public NVPair[] convertStringtoNVPair(java.lang.String stringparams)

      Parse the string parameters into NVPair[] array. Once they are parsed, it is returned. The method uses parseOneParameter(string) to convert each pair.
      Parameters:
        `stringparams` - String with parameters to be parsed
      Returns:
        array of `NVPair`s

    - #### parseOneParameter

          protected NVPair parseOneParameter(java.lang.String parameter)

      Method expects name and value to be separated by an equal sign "=". The method uses StringTokenizer to make a NVPair object. If there happens to be more than one "=" sign, the others are ignored. The chance of a string containing more than one is unlikely and would not conform to HTTP spec. I should double check the protocol spec to make sure this is accurate.
      Parameters:
        `parameter` - to be parsed
      Returns:
        `NVPair` with the parsed name and value of the parameter

    - #### parseParameters

          protected java.util.List<java.lang.String> parseParameters(java.lang.String parameters)

      Method uses StringTokenizer to convert the string into single pairs. The string should conform to HTTP protocol spec, which means the name/value pairs are separated by the ampersand symbol "&". Someone could write the querystrings by hand, but that would be round about and go against the purpose of this utility.
      Parameters:
        `parameters` - string to be parsed
      Returns:
        List of name/value pairs

    - #### tokenize

          public java.util.StringTokenizer tokenize(java.lang.String line,
                                                    java.lang.String delim)

      Parses the line using java.util.StringTokenizer.
      Parameters:
        `line` - line to be parsed
        `delim` - delimiter
      Returns:
        StringTokenizer constructed with `line` and `delim`

    - #### close

          public void close()

      Description copied from interface: `LogParser`
      close the any streams or readers.
      Specified by:
        `close` in interface `LogParser`

Copyright © 1998-2018 Apache Software Foundation. All Rights Reserved.