JSON for Modern C++  3.0
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Groups
nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer Class Reference

lexical analysis More...

Collaboration diagram for nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer:

Public Types

enum  token_type {
  token_type::uninitialized, token_type::literal_true, token_type::literal_false, token_type::literal_null,
  token_type::value_string, token_type::value_number, token_type::begin_array, token_type::begin_object,
  token_type::end_array, token_type::end_object, token_type::name_separator, token_type::value_separator,
  token_type::parse_error, token_type::end_of_input
}
 token types for the parser More...
 
using lexer_char_t = unsigned char
 the char type to use in the lexer More...
 

Public Member Functions

 lexer (const string_t &s) noexcept
 constructor with a given buffer More...
 
 lexer (std::istream *s) noexcept
 
 lexer ()=default
 default constructor More...
 
token_type scan () noexcept
 
void yyfill () noexcept
 append data from the stream to the internal buffer More...
 
string_t get_token () const noexcept
 return string representation of last read token More...
 
string_t get_string () const
 return string value for string tokens More...
 
number_float_t get_number () const
 return number value for number tokens More...
 

Static Public Member Functions

static string_t to_unicode (const size_t codepoint1, const size_t codepoint2=0)
 create a string from a Unicode code point More...
 
static std::string token_type_name (token_type t) noexcept
 return name of values of type token_type More...
 

Private Attributes

std::istream * m_stream
 optional input stream More...
 
string_t m_buffer
 the buffer More...
 
const lexer_char_tm_content = nullptr
 the buffer pointer More...
 
const lexer_char_tm_start = nullptr
 pointer to the beginning of the current symbol More...
 
const lexer_char_tm_marker = nullptr
 pointer for backtracking information More...
 
const lexer_char_tm_cursor = nullptr
 pointer to the current symbol More...
 
const lexer_char_tm_limit = nullptr
 pointer to the end of the buffer More...
 

Detailed Description

template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
class nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer

This class organizes the lexical analysis during JSON deserialization. The core of it is a scanner generated by re2c http://re2c.org that processes a buffer and recognizes tokens according to RFC 7159 and ECMA-404.

Definition at line 3336 of file json.hpp.

Member Typedef Documentation

template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
using nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::lexer_char_t = unsigned char

Definition at line 3359 of file json.hpp.

Member Enumeration Documentation

template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
enum nlohmann::basic_json::lexer::token_type
strong
Enumerator
uninitialized 

indicating the scanner is uninitialized

literal_true 

the "true" literal

literal_false 

the "false" literal

literal_null 

the "null" literal

value_string 

a string - use get_string() for actual value

value_number 

a number - use get_number() for actual value

begin_array 

the character for array begin "["

begin_object 

the character for object begin "{"

end_array 

the character for array end "]"

end_object 

the character for object end "}"

name_separator 

the name separator ":"

value_separator 

the value separator ","

parse_error 

indicating a parse error

end_of_input 

indicating the end of the input buffer

Definition at line 3340 of file json.hpp.

3341  {
3342  uninitialized,
3343  literal_true,
3344  literal_false,
3345  literal_null,
3346  value_string,
3347  value_number,
3348  begin_array,
3349  begin_object,
3350  end_array,
3351  end_object,
3352  name_separator,
3353  value_separator,
3354  parse_error,
3355  end_of_input
3356  };

Constructor & Destructor Documentation

template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::lexer ( const string_t s)
inlinenoexcept

Definition at line 3362 of file json.hpp.

3363  : m_stream(nullptr), m_buffer(s)
3364  {
3365  m_content = reinterpret_cast<const lexer_char_t*>(s.c_str());
3367  m_limit = m_content + s.size();
3368  }
std::istream * m_stream
optional input stream
Definition: json.hpp:4450
const lexer_char_t * m_content
the buffer pointer
Definition: json.hpp:4454
const lexer_char_t * m_start
pointer to the beginning of the current symbol
Definition: json.hpp:4456
string_t m_buffer
the buffer
Definition: json.hpp:4452
unsigned char lexer_char_t
the char type to use in the lexer
Definition: json.hpp:3359
const lexer_char_t * m_cursor
pointer to the current symbol
Definition: json.hpp:4460
const lexer_char_t * m_limit
pointer to the end of the buffer
Definition: json.hpp:4462
template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::lexer ( std::istream *  s)
inlinenoexcept

Definition at line 3369 of file json.hpp.

3370  : m_stream(s)
3371  {
3372  getline(*m_stream, m_buffer);
3373  m_content = reinterpret_cast<const lexer_char_t*>(m_buffer.c_str());
3375  m_limit = m_content + m_buffer.size();
3376  }
std::istream * m_stream
optional input stream
Definition: json.hpp:4450
const lexer_char_t * m_content
the buffer pointer
Definition: json.hpp:4454
const lexer_char_t * m_start
pointer to the beginning of the current symbol
Definition: json.hpp:4456
string_t m_buffer
the buffer
Definition: json.hpp:4452
unsigned char lexer_char_t
the char type to use in the lexer
Definition: json.hpp:3359
const lexer_char_t * m_cursor
pointer to the current symbol
Definition: json.hpp:4460
const lexer_char_t * m_limit
pointer to the end of the buffer
Definition: json.hpp:4462
template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::lexer ( )
inlinedefault

Member Function Documentation

template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
number_float_t nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::get_number ( ) const
inline

This function translates the last token into a floating point number. The pointer m_begin points to the beginning of the parsed number. We pass this pointer to std::strtod which sets endptr to the first character past the converted number. If this pointer is not the same as m_cursor, then either more or less characters have been used during the comparison. This can happen for inputs like "01" which will be treated like number 0 followed by number 1.

Returns
the result of the number conversion or NAN if the conversion read past the current token. The latter case needs to be treated by the caller function.
Exceptions
std::range_errorif passed value is out of range

Definition at line 4436 of file json.hpp.

4437  {
4438  // conversion
4439  typename string_t::value_type* endptr;
4440  const auto float_val = std::strtod(reinterpret_cast<typename string_t::const_pointer>(m_start),
4441  &endptr);
4442 
4443  // return float_val if the whole number was translated and NAN
4444  // otherwise
4445  return (reinterpret_cast<lexer_char_t*>(endptr) == m_cursor) ? float_val : NAN;
4446  }
const lexer_char_t * m_start
pointer to the beginning of the current symbol
Definition: json.hpp:4456
const lexer_char_t * m_cursor
pointer to the current symbol
Definition: json.hpp:4460
template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
string_t nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::get_string ( ) const
inline

The function iterates the characters between the opening and closing quotes of the string value. The complete string is the range [m_start,m_cursor). Consequently, we iterate from m_start+1 to m_cursor-1.

We differentiate two cases:

  1. Escaped characters. In this case, a new character is constructed according to the nature of the escape. Some escapes create new characters (e.g., "\\n" is replaced by "\n"), some are copied as is (e.g., "\\\\"). Furthermore, Unicode escapes of the shape "\\uxxxx" need special care. In this case, to_unicode takes care of the construction of the values.
  2. Unescaped characters are copied as is.
Returns
string value of current token without opening and closing quotes
Exceptions
std::out_of_rangeif to_unicode fails

Definition at line 4315 of file json.hpp.

4316  {
4317  string_t result;
4318  result.reserve(static_cast<size_t>(m_cursor - m_start - 2));
4319 
4320  // iterate the result between the quotes
4321  for (const lexer_char_t* i = m_start + 1; i < m_cursor - 1; ++i)
4322  {
4323  // process escaped characters
4324  if (*i == '\\')
4325  {
4326  // read next character
4327  ++i;
4328 
4329  switch (*i)
4330  {
4331  // the default escapes
4332  case 't':
4333  {
4334  result += "\t";
4335  break;
4336  }
4337  case 'b':
4338  {
4339  result += "\b";
4340  break;
4341  }
4342  case 'f':
4343  {
4344  result += "\f";
4345  break;
4346  }
4347  case 'n':
4348  {
4349  result += "\n";
4350  break;
4351  }
4352  case 'r':
4353  {
4354  result += "\r";
4355  break;
4356  }
4357 
4358  // characters that are not "un"escsaped
4359  case '\\':
4360  {
4361  result += "\\\\";
4362  break;
4363  }
4364  case '/':
4365  {
4366  result += "\\/";
4367  break;
4368  }
4369  case '"':
4370  {
4371  result += "\\\"";
4372  break;
4373  }
4374 
4375  // unicode
4376  case 'u':
4377  {
4378  // get code xxxx from uxxxx
4379  auto codepoint = std::strtoul(std::string(reinterpret_cast<typename string_t::const_pointer>(i + 1),
4380  4).c_str(), nullptr, 16);
4381 
4382  if (codepoint >= 0xD800 and codepoint <= 0xDBFF)
4383  {
4384  // make sure there is a subsequent unicode
4385  if ((i + 6 >= m_limit) or * (i + 5) != '\\' or * (i + 6) != 'u')
4386  {
4387  throw std::invalid_argument("missing low surrogate");
4388  }
4389 
4390  // get code yyyy from uxxxx\uyyyy
4391  auto codepoint2 = std::strtoul(std::string(reinterpret_cast<typename string_t::const_pointer>
4392  (i + 7), 4).c_str(), nullptr, 16);
4393  result += to_unicode(codepoint, codepoint2);
4394  // skip the next 11 characters (xxxx\uyyyy)
4395  i += 11;
4396  }
4397  else
4398  {
4399  // add unicode character(s)
4400  result += to_unicode(codepoint);
4401  // skip the next four characters (xxxx)
4402  i += 4;
4403  }
4404  break;
4405  }
4406  }
4407  }
4408  else
4409  {
4410  // all other characters are just copied to the end of the
4411  // string
4412  result.append(1, static_cast<typename string_t::value_type>(*i));
4413  }
4414  }
4415 
4416  return result;
4417  }
StringType string_t
a type for a string
Definition: json.hpp:194
const lexer_char_t * m_start
pointer to the beginning of the current symbol
Definition: json.hpp:4456
unsigned char lexer_char_t
the char type to use in the lexer
Definition: json.hpp:3359
const lexer_char_t * m_cursor
pointer to the current symbol
Definition: json.hpp:4460
static string_t to_unicode(const size_t codepoint1, const size_t codepoint2=0)
create a string from a Unicode code point
Definition: json.hpp:3392
const lexer_char_t * m_limit
pointer to the end of the buffer
Definition: json.hpp:4462
template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
string_t nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::get_token ( ) const
inlinenoexcept

Definition at line 4288 of file json.hpp.

4289  {
4290  return string_t(reinterpret_cast<typename string_t::const_pointer>(m_start),
4291  static_cast<size_t>(m_cursor - m_start));
4292  }
StringType string_t
a type for a string
Definition: json.hpp:194
const lexer_char_t * m_start
pointer to the beginning of the current symbol
Definition: json.hpp:4456
const lexer_char_t * m_cursor
pointer to the current symbol
Definition: json.hpp:4460
template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
token_type nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::scan ( )
inlinenoexcept

This function implements a scanner for JSON. It is specified using regular expressions that try to follow RFC 7159 and ECMA-404 as close as possible. These regular expressions are then translated into a deterministic finite automaton (DFA) by the tool re2c http://re2c.org. As a result, the translated code for this function consists of a large block of code with goto jumps.

Returns
the class of the next token read from the buffer

Definition at line 3499 of file json.hpp.

3500  {
3501  // pointer for backtracking information
3502  m_marker = nullptr;
3503 
3504  // remember the begin of the token
3505  m_start = m_cursor;
3506 
3507 
3508  {
3509  lexer_char_t yych;
3510  unsigned int yyaccept = 0;
3511  static const unsigned char yybm[] =
3512  {
3513  0, 64, 64, 64, 64, 64, 64, 64,
3514  64, 96, 96, 64, 64, 96, 64, 64,
3515  64, 64, 64, 64, 64, 64, 64, 64,
3516  64, 64, 64, 64, 64, 64, 64, 64,
3517  96, 64, 0, 64, 64, 64, 64, 64,
3518  64, 64, 64, 64, 64, 64, 64, 64,
3519  192, 192, 192, 192, 192, 192, 192, 192,
3520  192, 192, 64, 64, 64, 64, 64, 64,
3521  64, 64, 64, 64, 64, 64, 64, 64,
3522  64, 64, 64, 64, 64, 64, 64, 64,
3523  64, 64, 64, 64, 64, 64, 64, 64,
3524  64, 64, 64, 64, 0, 64, 64, 64,
3525  64, 64, 64, 64, 64, 64, 64, 64,
3526  64, 64, 64, 64, 64, 64, 64, 64,
3527  64, 64, 64, 64, 64, 64, 64, 64,
3528  64, 64, 64, 64, 64, 64, 64, 64,
3529  64, 64, 64, 64, 64, 64, 64, 64,
3530  64, 64, 64, 64, 64, 64, 64, 64,
3531  64, 64, 64, 64, 64, 64, 64, 64,
3532  64, 64, 64, 64, 64, 64, 64, 64,
3533  64, 64, 64, 64, 64, 64, 64, 64,
3534  64, 64, 64, 64, 64, 64, 64, 64,
3535  64, 64, 64, 64, 64, 64, 64, 64,
3536  64, 64, 64, 64, 64, 64, 64, 64,
3537  64, 64, 64, 64, 64, 64, 64, 64,
3538  64, 64, 64, 64, 64, 64, 64, 64,
3539  64, 64, 64, 64, 64, 64, 64, 64,
3540  64, 64, 64, 64, 64, 64, 64, 64,
3541  64, 64, 64, 64, 64, 64, 64, 64,
3542  64, 64, 64, 64, 64, 64, 64, 64,
3543  64, 64, 64, 64, 64, 64, 64, 64,
3544  64, 64, 64, 64, 64, 64, 64, 64,
3545  };
3546 
3547  if ((m_limit - m_cursor) < 5)
3548  {
3549  yyfill();
3550  };
3551  yych = *m_cursor;
3552  if (yych <= '9')
3553  {
3554  if (yych <= ' ')
3555  {
3556  if (yych <= '\n')
3557  {
3558  if (yych <= 0x00)
3559  {
3560  goto basic_json_parser_27;
3561  }
3562  if (yych <= 0x08)
3563  {
3564  goto basic_json_parser_29;
3565  }
3566  if (yych >= '\n')
3567  {
3568  goto basic_json_parser_4;
3569  }
3570  }
3571  else
3572  {
3573  if (yych == '\r')
3574  {
3575  goto basic_json_parser_2;
3576  }
3577  if (yych <= 0x1F)
3578  {
3579  goto basic_json_parser_29;
3580  }
3581  }
3582  }
3583  else
3584  {
3585  if (yych <= ',')
3586  {
3587  if (yych == '"')
3588  {
3589  goto basic_json_parser_26;
3590  }
3591  if (yych <= '+')
3592  {
3593  goto basic_json_parser_29;
3594  }
3595  goto basic_json_parser_14;
3596  }
3597  else
3598  {
3599  if (yych <= '-')
3600  {
3601  goto basic_json_parser_22;
3602  }
3603  if (yych <= '/')
3604  {
3605  goto basic_json_parser_29;
3606  }
3607  if (yych <= '0')
3608  {
3609  goto basic_json_parser_23;
3610  }
3611  goto basic_json_parser_25;
3612  }
3613  }
3614  }
3615  else
3616  {
3617  if (yych <= 'm')
3618  {
3619  if (yych <= '\\')
3620  {
3621  if (yych <= ':')
3622  {
3623  goto basic_json_parser_16;
3624  }
3625  if (yych == '[')
3626  {
3627  goto basic_json_parser_6;
3628  }
3629  goto basic_json_parser_29;
3630  }
3631  else
3632  {
3633  if (yych <= ']')
3634  {
3635  goto basic_json_parser_8;
3636  }
3637  if (yych == 'f')
3638  {
3639  goto basic_json_parser_21;
3640  }
3641  goto basic_json_parser_29;
3642  }
3643  }
3644  else
3645  {
3646  if (yych <= 'z')
3647  {
3648  if (yych <= 'n')
3649  {
3650  goto basic_json_parser_18;
3651  }
3652  if (yych == 't')
3653  {
3654  goto basic_json_parser_20;
3655  }
3656  goto basic_json_parser_29;
3657  }
3658  else
3659  {
3660  if (yych <= '{')
3661  {
3662  goto basic_json_parser_10;
3663  }
3664  if (yych == '}')
3665  {
3666  goto basic_json_parser_12;
3667  }
3668  goto basic_json_parser_29;
3669  }
3670  }
3671  }
3672 basic_json_parser_2:
3673  ++m_cursor;
3674  yych = *m_cursor;
3675  goto basic_json_parser_5;
3676 basic_json_parser_3:
3677  {
3678  return scan();
3679  }
3680 basic_json_parser_4:
3681  ++m_cursor;
3682  if (m_limit <= m_cursor)
3683  {
3684  yyfill();
3685  };
3686  yych = *m_cursor;
3687 basic_json_parser_5:
3688  if (yybm[0 + yych] & 32)
3689  {
3690  goto basic_json_parser_4;
3691  }
3692  goto basic_json_parser_3;
3693 basic_json_parser_6:
3694  ++m_cursor;
3695  {
3696  return token_type::begin_array;
3697  }
3698 basic_json_parser_8:
3699  ++m_cursor;
3700  {
3701  return token_type::end_array;
3702  }
3703 basic_json_parser_10:
3704  ++m_cursor;
3705  {
3706  return token_type::begin_object;
3707  }
3708 basic_json_parser_12:
3709  ++m_cursor;
3710  {
3711  return token_type::end_object;
3712  }
3713 basic_json_parser_14:
3714  ++m_cursor;
3715  {
3717  }
3718 basic_json_parser_16:
3719  ++m_cursor;
3720  {
3722  }
3723 basic_json_parser_18:
3724  yyaccept = 0;
3725  yych = *(m_marker = ++m_cursor);
3726  if (yych == 'u')
3727  {
3728  goto basic_json_parser_59;
3729  }
3730 basic_json_parser_19:
3731  {
3732  return token_type::parse_error;
3733  }
3734 basic_json_parser_20:
3735  yyaccept = 0;
3736  yych = *(m_marker = ++m_cursor);
3737  if (yych == 'r')
3738  {
3739  goto basic_json_parser_55;
3740  }
3741  goto basic_json_parser_19;
3742 basic_json_parser_21:
3743  yyaccept = 0;
3744  yych = *(m_marker = ++m_cursor);
3745  if (yych == 'a')
3746  {
3747  goto basic_json_parser_50;
3748  }
3749  goto basic_json_parser_19;
3750 basic_json_parser_22:
3751  yych = *++m_cursor;
3752  if (yych <= '/')
3753  {
3754  goto basic_json_parser_19;
3755  }
3756  if (yych <= '0')
3757  {
3758  goto basic_json_parser_49;
3759  }
3760  if (yych <= '9')
3761  {
3762  goto basic_json_parser_40;
3763  }
3764  goto basic_json_parser_19;
3765 basic_json_parser_23:
3766  yyaccept = 1;
3767  yych = *(m_marker = ++m_cursor);
3768  if (yych <= 'D')
3769  {
3770  if (yych == '.')
3771  {
3772  goto basic_json_parser_42;
3773  }
3774  }
3775  else
3776  {
3777  if (yych <= 'E')
3778  {
3779  goto basic_json_parser_43;
3780  }
3781  if (yych == 'e')
3782  {
3783  goto basic_json_parser_43;
3784  }
3785  }
3786 basic_json_parser_24:
3787  {
3788  return token_type::value_number;
3789  }
3790 basic_json_parser_25:
3791  yyaccept = 1;
3792  yych = *(m_marker = ++m_cursor);
3793  goto basic_json_parser_41;
3794 basic_json_parser_26:
3795  yyaccept = 0;
3796  yych = *(m_marker = ++m_cursor);
3797  if (yych <= 0x00)
3798  {
3799  goto basic_json_parser_19;
3800  }
3801  goto basic_json_parser_31;
3802 basic_json_parser_27:
3803  ++m_cursor;
3804  {
3805  return token_type::end_of_input;
3806  }
3807 basic_json_parser_29:
3808  yych = *++m_cursor;
3809  goto basic_json_parser_19;
3810 basic_json_parser_30:
3811  ++m_cursor;
3812  if (m_limit <= m_cursor)
3813  {
3814  yyfill();
3815  };
3816  yych = *m_cursor;
3817 basic_json_parser_31:
3818  if (yybm[0 + yych] & 64)
3819  {
3820  goto basic_json_parser_30;
3821  }
3822  if (yych <= 0x00)
3823  {
3824  goto basic_json_parser_32;
3825  }
3826  if (yych <= '"')
3827  {
3828  goto basic_json_parser_34;
3829  }
3830  goto basic_json_parser_33;
3831 basic_json_parser_32:
3832  m_cursor = m_marker;
3833  if (yyaccept == 0)
3834  {
3835  goto basic_json_parser_19;
3836  }
3837  else
3838  {
3839  goto basic_json_parser_24;
3840  }
3841 basic_json_parser_33:
3842  ++m_cursor;
3843  if (m_limit <= m_cursor)
3844  {
3845  yyfill();
3846  };
3847  yych = *m_cursor;
3848  if (yych <= 'e')
3849  {
3850  if (yych <= '/')
3851  {
3852  if (yych == '"')
3853  {
3854  goto basic_json_parser_30;
3855  }
3856  if (yych <= '.')
3857  {
3858  goto basic_json_parser_32;
3859  }
3860  goto basic_json_parser_30;
3861  }
3862  else
3863  {
3864  if (yych <= '\\')
3865  {
3866  if (yych <= '[')
3867  {
3868  goto basic_json_parser_32;
3869  }
3870  goto basic_json_parser_30;
3871  }
3872  else
3873  {
3874  if (yych == 'b')
3875  {
3876  goto basic_json_parser_30;
3877  }
3878  goto basic_json_parser_32;
3879  }
3880  }
3881  }
3882  else
3883  {
3884  if (yych <= 'q')
3885  {
3886  if (yych <= 'f')
3887  {
3888  goto basic_json_parser_30;
3889  }
3890  if (yych == 'n')
3891  {
3892  goto basic_json_parser_30;
3893  }
3894  goto basic_json_parser_32;
3895  }
3896  else
3897  {
3898  if (yych <= 's')
3899  {
3900  if (yych <= 'r')
3901  {
3902  goto basic_json_parser_30;
3903  }
3904  goto basic_json_parser_32;
3905  }
3906  else
3907  {
3908  if (yych <= 't')
3909  {
3910  goto basic_json_parser_30;
3911  }
3912  if (yych <= 'u')
3913  {
3914  goto basic_json_parser_36;
3915  }
3916  goto basic_json_parser_32;
3917  }
3918  }
3919  }
3920 basic_json_parser_34:
3921  ++m_cursor;
3922  {
3923  return token_type::value_string;
3924  }
3925 basic_json_parser_36:
3926  ++m_cursor;
3927  if (m_limit <= m_cursor)
3928  {
3929  yyfill();
3930  };
3931  yych = *m_cursor;
3932  if (yych <= '@')
3933  {
3934  if (yych <= '/')
3935  {
3936  goto basic_json_parser_32;
3937  }
3938  if (yych >= ':')
3939  {
3940  goto basic_json_parser_32;
3941  }
3942  }
3943  else
3944  {
3945  if (yych <= 'F')
3946  {
3947  goto basic_json_parser_37;
3948  }
3949  if (yych <= '`')
3950  {
3951  goto basic_json_parser_32;
3952  }
3953  if (yych >= 'g')
3954  {
3955  goto basic_json_parser_32;
3956  }
3957  }
3958 basic_json_parser_37:
3959  ++m_cursor;
3960  if (m_limit <= m_cursor)
3961  {
3962  yyfill();
3963  };
3964  yych = *m_cursor;
3965  if (yych <= '@')
3966  {
3967  if (yych <= '/')
3968  {
3969  goto basic_json_parser_32;
3970  }
3971  if (yych >= ':')
3972  {
3973  goto basic_json_parser_32;
3974  }
3975  }
3976  else
3977  {
3978  if (yych <= 'F')
3979  {
3980  goto basic_json_parser_38;
3981  }
3982  if (yych <= '`')
3983  {
3984  goto basic_json_parser_32;
3985  }
3986  if (yych >= 'g')
3987  {
3988  goto basic_json_parser_32;
3989  }
3990  }
3991 basic_json_parser_38:
3992  ++m_cursor;
3993  if (m_limit <= m_cursor)
3994  {
3995  yyfill();
3996  };
3997  yych = *m_cursor;
3998  if (yych <= '@')
3999  {
4000  if (yych <= '/')
4001  {
4002  goto basic_json_parser_32;
4003  }
4004  if (yych >= ':')
4005  {
4006  goto basic_json_parser_32;
4007  }
4008  }
4009  else
4010  {
4011  if (yych <= 'F')
4012  {
4013  goto basic_json_parser_39;
4014  }
4015  if (yych <= '`')
4016  {
4017  goto basic_json_parser_32;
4018  }
4019  if (yych >= 'g')
4020  {
4021  goto basic_json_parser_32;
4022  }
4023  }
4024 basic_json_parser_39:
4025  ++m_cursor;
4026  if (m_limit <= m_cursor)
4027  {
4028  yyfill();
4029  };
4030  yych = *m_cursor;
4031  if (yych <= '@')
4032  {
4033  if (yych <= '/')
4034  {
4035  goto basic_json_parser_32;
4036  }
4037  if (yych <= '9')
4038  {
4039  goto basic_json_parser_30;
4040  }
4041  goto basic_json_parser_32;
4042  }
4043  else
4044  {
4045  if (yych <= 'F')
4046  {
4047  goto basic_json_parser_30;
4048  }
4049  if (yych <= '`')
4050  {
4051  goto basic_json_parser_32;
4052  }
4053  if (yych <= 'f')
4054  {
4055  goto basic_json_parser_30;
4056  }
4057  goto basic_json_parser_32;
4058  }
4059 basic_json_parser_40:
4060  yyaccept = 1;
4061  m_marker = ++m_cursor;
4062  if ((m_limit - m_cursor) < 3)
4063  {
4064  yyfill();
4065  };
4066  yych = *m_cursor;
4067 basic_json_parser_41:
4068  if (yybm[0 + yych] & 128)
4069  {
4070  goto basic_json_parser_40;
4071  }
4072  if (yych <= 'D')
4073  {
4074  if (yych != '.')
4075  {
4076  goto basic_json_parser_24;
4077  }
4078  }
4079  else
4080  {
4081  if (yych <= 'E')
4082  {
4083  goto basic_json_parser_43;
4084  }
4085  if (yych == 'e')
4086  {
4087  goto basic_json_parser_43;
4088  }
4089  goto basic_json_parser_24;
4090  }
4091 basic_json_parser_42:
4092  yych = *++m_cursor;
4093  if (yych <= '/')
4094  {
4095  goto basic_json_parser_32;
4096  }
4097  if (yych <= '9')
4098  {
4099  goto basic_json_parser_47;
4100  }
4101  goto basic_json_parser_32;
4102 basic_json_parser_43:
4103  yych = *++m_cursor;
4104  if (yych <= ',')
4105  {
4106  if (yych != '+')
4107  {
4108  goto basic_json_parser_32;
4109  }
4110  }
4111  else
4112  {
4113  if (yych <= '-')
4114  {
4115  goto basic_json_parser_44;
4116  }
4117  if (yych <= '/')
4118  {
4119  goto basic_json_parser_32;
4120  }
4121  if (yych <= '9')
4122  {
4123  goto basic_json_parser_45;
4124  }
4125  goto basic_json_parser_32;
4126  }
4127 basic_json_parser_44:
4128  yych = *++m_cursor;
4129  if (yych <= '/')
4130  {
4131  goto basic_json_parser_32;
4132  }
4133  if (yych >= ':')
4134  {
4135  goto basic_json_parser_32;
4136  }
4137 basic_json_parser_45:
4138  ++m_cursor;
4139  if (m_limit <= m_cursor)
4140  {
4141  yyfill();
4142  };
4143  yych = *m_cursor;
4144  if (yych <= '/')
4145  {
4146  goto basic_json_parser_24;
4147  }
4148  if (yych <= '9')
4149  {
4150  goto basic_json_parser_45;
4151  }
4152  goto basic_json_parser_24;
4153 basic_json_parser_47:
4154  yyaccept = 1;
4155  m_marker = ++m_cursor;
4156  if ((m_limit - m_cursor) < 3)
4157  {
4158  yyfill();
4159  };
4160  yych = *m_cursor;
4161  if (yych <= 'D')
4162  {
4163  if (yych <= '/')
4164  {
4165  goto basic_json_parser_24;
4166  }
4167  if (yych <= '9')
4168  {
4169  goto basic_json_parser_47;
4170  }
4171  goto basic_json_parser_24;
4172  }
4173  else
4174  {
4175  if (yych <= 'E')
4176  {
4177  goto basic_json_parser_43;
4178  }
4179  if (yych == 'e')
4180  {
4181  goto basic_json_parser_43;
4182  }
4183  goto basic_json_parser_24;
4184  }
4185 basic_json_parser_49:
4186  yyaccept = 1;
4187  yych = *(m_marker = ++m_cursor);
4188  if (yych <= 'D')
4189  {
4190  if (yych == '.')
4191  {
4192  goto basic_json_parser_42;
4193  }
4194  goto basic_json_parser_24;
4195  }
4196  else
4197  {
4198  if (yych <= 'E')
4199  {
4200  goto basic_json_parser_43;
4201  }
4202  if (yych == 'e')
4203  {
4204  goto basic_json_parser_43;
4205  }
4206  goto basic_json_parser_24;
4207  }
4208 basic_json_parser_50:
4209  yych = *++m_cursor;
4210  if (yych != 'l')
4211  {
4212  goto basic_json_parser_32;
4213  }
4214  yych = *++m_cursor;
4215  if (yych != 's')
4216  {
4217  goto basic_json_parser_32;
4218  }
4219  yych = *++m_cursor;
4220  if (yych != 'e')
4221  {
4222  goto basic_json_parser_32;
4223  }
4224  ++m_cursor;
4225  {
4227  }
4228 basic_json_parser_55:
4229  yych = *++m_cursor;
4230  if (yych != 'u')
4231  {
4232  goto basic_json_parser_32;
4233  }
4234  yych = *++m_cursor;
4235  if (yych != 'e')
4236  {
4237  goto basic_json_parser_32;
4238  }
4239  ++m_cursor;
4240  {
4241  return token_type::literal_true;
4242  }
4243 basic_json_parser_59:
4244  yych = *++m_cursor;
4245  if (yych != 'l')
4246  {
4247  goto basic_json_parser_32;
4248  }
4249  yych = *++m_cursor;
4250  if (yych != 'l')
4251  {
4252  goto basic_json_parser_32;
4253  }
4254  ++m_cursor;
4255  {
4256  return token_type::literal_null;
4257  }
4258  }
4259 
4260 
4261  }
the character for array begin &quot;[&quot;
const lexer_char_t * m_marker
pointer for backtracking information
Definition: json.hpp:4458
the character for object end &quot;}&quot;
indicating the end of the input buffer
a string - use get_string() for actual value
const lexer_char_t * m_start
pointer to the beginning of the current symbol
Definition: json.hpp:4456
void yyfill() noexcept
append data from the stream to the internal buffer
Definition: json.hpp:4264
unsigned char lexer_char_t
the char type to use in the lexer
Definition: json.hpp:3359
const lexer_char_t * m_cursor
pointer to the current symbol
Definition: json.hpp:4460
token_type scan() noexcept
Definition: json.hpp:3499
const lexer_char_t * m_limit
pointer to the end of the buffer
Definition: json.hpp:4462
the character for array end &quot;]&quot;
a number - use get_number() for actual value
the character for object begin &quot;{&quot;
template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
static string_t nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::to_unicode ( const size_t  codepoint1,
const size_t  codepoint2 = 0 
)
inlinestatic
Parameters
codepoint1the code point (can be high surrogate)
codepoint2the code point (can be low surrogate or 0)
Returns
string representation of the code point
Exceptions
std::out_of_rangeif code point is >0x10ffff
std::invalid_argumentif the low surrogate is invalid
See Also
http://en.wikipedia.org/wiki/UTF-8#Sample_code

Definition at line 3392 of file json.hpp.

3394  {
3395  string_t result;
3396 
3397  // calculate the codepoint from the given code points
3398  size_t codepoint = codepoint1;
3399  if (codepoint1 >= 0xD800 and codepoint1 <= 0xDBFF)
3400  {
3401  if (codepoint2 >= 0xDC00 and codepoint2 <= 0xDFFF)
3402  {
3403  codepoint =
3404  // high surrogate occupies the most significant 22 bits
3405  (codepoint1 << 10)
3406  // low surrogate occupies the least significant 15 bits
3407  + codepoint2
3408  // there is still the 0xD800, 0xDC00 and 0x10000 noise
3409  // in the result so we have to substract with:
3410  // (0xD800 << 10) + DC00 - 0x10000 = 0x35FDC00
3411  - 0x35FDC00;
3412  }
3413  else
3414  {
3415  throw std::invalid_argument("missing or wrong low surrogate");
3416  }
3417  }
3418 
3419  if (codepoint <= 0x7f)
3420  {
3421  // 1-byte characters: 0xxxxxxx (ASCII)
3422  result.append(1, static_cast<typename string_t::value_type>(codepoint));
3423  }
3424  else if (codepoint <= 0x7ff)
3425  {
3426  // 2-byte characters: 110xxxxx 10xxxxxx
3427  result.append(1, static_cast<typename string_t::value_type>(0xC0 | ((codepoint >> 6) & 0x1F)));
3428  result.append(1, static_cast<typename string_t::value_type>(0x80 | (codepoint & 0x3F)));
3429  }
3430  else if (codepoint <= 0xffff)
3431  {
3432  // 3-byte characters: 1110xxxx 10xxxxxx 10xxxxxx
3433  result.append(1, static_cast<typename string_t::value_type>(0xE0 | ((codepoint >> 12) & 0x0F)));
3434  result.append(1, static_cast<typename string_t::value_type>(0x80 | ((codepoint >> 6) & 0x3F)));
3435  result.append(1, static_cast<typename string_t::value_type>(0x80 | (codepoint & 0x3F)));
3436  }
3437  else if (codepoint <= 0x10ffff)
3438  {
3439  // 4-byte characters: 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
3440  result.append(1, static_cast<typename string_t::value_type>(0xF0 | ((codepoint >> 18) & 0x07)));
3441  result.append(1, static_cast<typename string_t::value_type>(0x80 | ((codepoint >> 12) & 0x3F)));
3442  result.append(1, static_cast<typename string_t::value_type>(0x80 | ((codepoint >> 6) & 0x3F)));
3443  result.append(1, static_cast<typename string_t::value_type>(0x80 | (codepoint & 0x3F)));
3444  }
3445  else
3446  {
3447  throw std::out_of_range("code points above 0x10FFFF are invalid");
3448  }
3449 
3450  return result;
3451  }
StringType string_t
a type for a string
Definition: json.hpp:194
template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
static std::string nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::token_type_name ( token_type  t)
inlinestaticnoexcept

Definition at line 3454 of file json.hpp.

3455  {
3456  switch (t)
3457  {
3459  return "<uninitialized>";
3460  case (token_type::literal_true):
3461  return "true literal";
3463  return "false literal";
3464  case (token_type::literal_null):
3465  return "null literal";
3466  case (token_type::value_string):
3467  return "string literal";
3468  case (token_type::value_number):
3469  return "number literal";
3470  case (token_type::begin_array):
3471  return "[";
3472  case (token_type::begin_object):
3473  return "{";
3474  case (token_type::end_array):
3475  return "]";
3476  case (token_type::end_object):
3477  return "}";
3479  return ":";
3481  return ",";
3482  case (token_type::end_of_input):
3483  return "<end of input>";
3484  default:
3485  return "<parse error>";
3486  }
3487  }
the character for array begin &quot;[&quot;
the character for object end &quot;}&quot;
indicating the end of the input buffer
a string - use get_string() for actual value
indicating the scanner is uninitialized
the character for array end &quot;]&quot;
a number - use get_number() for actual value
the character for object begin &quot;{&quot;
template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
void nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::yyfill ( )
inlinenoexcept

Definition at line 4264 of file json.hpp.

4265  {
4266  if (not m_stream or not * m_stream)
4267  {
4268  return;
4269  }
4270 
4271  const ssize_t offset_start = m_start - m_content;
4272  const ssize_t offset_marker = m_marker - m_start;
4273  const ssize_t offset_cursor = m_cursor - m_start;
4274 
4275  m_buffer.erase(0, static_cast<size_t>(offset_start));
4276  std::string line;
4277  std::getline(*m_stream, line);
4278  m_buffer += line;
4279 
4280  m_content = reinterpret_cast<const lexer_char_t*>(m_buffer.c_str());
4281  m_start = m_content;
4282  m_marker = m_start + offset_marker;
4283  m_cursor = m_start + offset_cursor;
4284  m_limit = m_start + m_buffer.size() - 1;
4285  }
const lexer_char_t * m_marker
pointer for backtracking information
Definition: json.hpp:4458
std::istream * m_stream
optional input stream
Definition: json.hpp:4450
const lexer_char_t * m_content
the buffer pointer
Definition: json.hpp:4454
const lexer_char_t * m_start
pointer to the beginning of the current symbol
Definition: json.hpp:4456
string_t m_buffer
the buffer
Definition: json.hpp:4452
unsigned char lexer_char_t
the char type to use in the lexer
Definition: json.hpp:3359
const lexer_char_t * m_cursor
pointer to the current symbol
Definition: json.hpp:4460
const lexer_char_t * m_limit
pointer to the end of the buffer
Definition: json.hpp:4462

Member Data Documentation

template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
string_t nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::m_buffer
private

Definition at line 4452 of file json.hpp.

template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
const lexer_char_t* nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::m_content = nullptr
private

Definition at line 4454 of file json.hpp.

template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
const lexer_char_t* nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::m_cursor = nullptr
private

Definition at line 4460 of file json.hpp.

template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
const lexer_char_t* nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::m_limit = nullptr
private

Definition at line 4462 of file json.hpp.

template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
const lexer_char_t* nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::m_marker = nullptr
private

Definition at line 4458 of file json.hpp.

template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
const lexer_char_t* nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::m_start = nullptr
private

Definition at line 4456 of file json.hpp.

template<template< typename U, typename V, typename...Args > class ObjectType = std::map, template< typename U, typename...Args > class ArrayType = std::vector, class StringType = std::string, class BooleanType = bool, class NumberIntegerType = int64_t, class NumberFloatType = double, template< typename U > class Allocator = std::allocator>
std::istream* nlohmann::basic_json< ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberFloatType, Allocator >::lexer::m_stream
private

Definition at line 4450 of file json.hpp.


The documentation for this class was generated from the following file: