Function rustc_lexer::tokenize [−][src]
pub fn tokenize(input: &str) -> impl Iterator<Item = Token> + '_Expand description
Creates an iterator that produces tokens from the input string.
pub fn tokenize(input: &str) -> impl Iterator<Item = Token> + '_Creates an iterator that produces tokens from the input string.