[soci-users] Exchanging data types

Discussion:

Nicolas Deroche

2013-03-05 14:21:12 UTC

Hello,

I just started using SOCI, and I have one quick question about casting

Consider this structure:
struct A
{ unsigned long one; unsigned int two; unsigned short three; }

In MySQL, the fields are defined like:
one : INT unsigned
two: SMALLINT unsigned
three: TINYINT unsigned

How am I supposed to cast using the static templated function from_base
the SQL values into my structure?
static void from_base(values const & v, indicator /* ind */, A& s)
{
s.one = v.get<unsigned long>("id"); /// Throws std::bad_cast()
}

Thanks,

nico

Pawel Aleksander Fedorynski

2013-03-05 19:32:27 UTC

Permalink

Hello Nico,

Right now SOCI (or at least the MySQL backend) is very strict about the
types when using the row interface. This is often inconvenient and we
should probably change it. But for now the rules are:

- All integer types smaller than INT are promoted to int, regardless
whether the original type was signed or unsigned. So, e.g., if you select
a value of type TINYINT UNSIGNED, you should retrieve it from the rowset
using v.get<int>("id").

- INT is retrieved as int: v.get<int>("id")

- INT UNSIGNED is retrieved as (signed) long long: v.get<long long>("id"),
although this will work too: v.get<unsigned>("id").

- BIGINT is retrieved as long long.

- BIGINT UNSIGNED is retrieved as unsigned long long.

If you don't know the type that was selected form the database, you can use
v.get_properties("id").get_data_type() to find out. It will be
dt_unsigned_long_long for BIGINT UNSIGNED, dt_long_long for BIGINT and INT
UNSIGNED, and dt_integer for all the smaller integral types.

Unless I'm mistaken this problem only pertains to the row interface
(dynamic binding), when you use static binding, i.e., into(n) this should
be more forgiving with respect to types.

Thanks,

Aleksander

Post by Nicolas Deroche
Hello,
I just started using SOCI, and I have one quick question about casting
struct A
{ unsigned long one; unsigned int two; unsigned short three; }
one : INT unsigned
two: SMALLINT unsigned
three: TINYINT unsigned
How am I supposed to cast using the static templated function from_base
the SQL values into my structure?
static void from_base(values const & v, indicator /* ind */, A& s)
{
s.one = v.get<unsigned long>("id"); /// Throws std::bad_cast()
}
Thanks,
nico
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

Nicolas Deroche

2013-03-05 20:18:56 UTC

Permalink

Thanks for the reply,
I figured out casting from v.get<int> to C++'s unsigned long doesn't make
any data loss, as you said in your mail

I'd like to point out the lack of this information on the documentation on
the website !

In the future, I hope this could change, and be able to use directly C++
types in the get<T> template function, or at least something less strict
like
- short (0 to 255); int (0 to 65K); long (0 to 4M); and long 64 for greate
values
- bool's, and something else for other specific types.

IMO, this could give some noob-friendly startup for the library !

Thanks again,

nico

Post by Pawel Aleksander Fedorynski
Hello Nico,
Right now SOCI (or at least the MySQL backend) is very strict about the
types when using the row interface. This is often inconvenient and we
- All integer types smaller than INT are promoted to int, regardless
whether the original type was signed or unsigned. So, e.g., if you select
a value of type TINYINT UNSIGNED, you should retrieve it from the rowset
using v.get<int>("id").
- INT is retrieved as int: v.get<int>("id")
- INT UNSIGNED is retrieved as (signed) long long: v.get<long long>("id"),
although this will work too: v.get<unsigned>("id").
- BIGINT is retrieved as long long.
- BIGINT UNSIGNED is retrieved as unsigned long long.
If you don't know the type that was selected form the database, you can
use v.get_properties("id").get_data_type() to find out. It will be
dt_unsigned_long_long for BIGINT UNSIGNED, dt_long_long for BIGINT and INT
UNSIGNED, and dt_integer for all the smaller integral types.
Unless I'm mistaken this problem only pertains to the row interface
(dynamic binding), when you use static binding, i.e., into(n) this should
be more forgiving with respect to types.
Thanks,
Aleksander

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

Pawel Aleksander Fedorynski

2013-03-05 20:32:51 UTC

Permalink

Post by Nicolas Deroche
Thanks for the reply,
I figured out casting from v.get<int> to C++'s unsigned long doesn't make
any data loss, as you said in your mail
I'd like to point out the lack of this information on the documentation on
the website !

You are right, sorry about that!

I completely agree, there is no reason why v.get<short>(...) shouldn't work
as long as the actual value is in the short's range, no matter what type it
had in the database.

Thanks,

Aleksander

Post by Nicolas Deroche
- bool's, and something else for other specific types.
IMO, this could give some noob-friendly startup for the library !
Thanks again,
nico

Post by Pawel Aleksander Fedorynski
Hello Nico,
Right now SOCI (or at least the MySQL backend) is very strict about the
types when using the row interface. This is often inconvenient and we
- All integer types smaller than INT are promoted to int, regardless
whether the original type was signed or unsigned. So, e.g., if you select
a value of type TINYINT UNSIGNED, you should retrieve it from the rowset
using v.get<int>("id").
- INT is retrieved as int: v.get<int>("id")
- INT UNSIGNED is retrieved as (signed) long long: v.get<long
long>("id"), although this will work too: v.get<unsigned>("id").
- BIGINT is retrieved as long long.
- BIGINT UNSIGNED is retrieved as unsigned long long.
If you don't know the type that was selected form the database, you can
use v.get_properties("id").get_data_type() to find out. It will be
dt_unsigned_long_long for BIGINT UNSIGNED, dt_long_long for BIGINT and INT
UNSIGNED, and dt_integer for all the smaller integral types.
Unless I'm mistaken this problem only pertains to the row interface
(dynamic binding), when you use static binding, i.e., into(n) this should
be more forgiving with respect to types.
Thanks,
Aleksander

Nicolas Deroche

2013-03-05 21:27:32 UTC

Permalink

I use this very bad code to partiallly hide the lack of the feature

namespace soci
{
template<typename T>
T get_integer_casted(const values& v, const std::string& field)
{
return static_cast<T>(v.get<int>(field));
}
}

Post by Pawel Aleksander Fedorynski

You are right, sorry about that!

I completely agree, there is no reason why v.get<short>(...) shouldn't
work as long as the actual value is in the short's range, no matter what
type it had in the database.
Thanks,
Aleksander

Post by Nicolas Deroche
- bool's, and something else for other specific types.
IMO, this could give some noob-friendly startup for the library !
Thanks again,
nico

Post by Pawel Aleksander Fedorynski
Hello Nico,
Right now SOCI (or at least the MySQL backend) is very strict about the
types when using the row interface. This is often inconvenient and we
- All integer types smaller than INT are promoted to int, regardless
whether the original type was signed or unsigned. So, e.g., if you select
a value of type TINYINT UNSIGNED, you should retrieve it from the rowset
using v.get<int>("id").
- INT is retrieved as int: v.get<int>("id")
- INT UNSIGNED is retrieved as (signed) long long: v.get<long
long>("id"), although this will work too: v.get<unsigned>("id").
- BIGINT is retrieved as long long.
- BIGINT UNSIGNED is retrieved as unsigned long long.
If you don't know the type that was selected form the database, you can
use v.get_properties("id").get_data_type() to find out. It will be
dt_unsigned_long_long for BIGINT UNSIGNED, dt_long_long for BIGINT and INT
UNSIGNED, and dt_integer for all the smaller integral types.
Unless I'm mistaken this problem only pertains to the row interface
(dynamic binding), when you use static binding, i.e., into(n) this should
be more forgiving with respect to types.
Thanks,
Aleksander

Mateusz Loskot

2013-03-06 01:41:21 UTC

Permalink

There is however documentation here
http://soci.sourceforge.net/doc/exchange.html#dynamic
and link to backend-specific pages below the small table.
I agree, this could be documented better, certainly.

If you see anything missing or you'd like to suggest any improvement,
feel more than welcome to open issue ticket at
https://github.com/SOCI/soci

Post by Nicolas Deroche
In the future, I hope this could change, and be able to use directly C++
types in the get<T> template function, or at least something less strict
like
- short (0 to 255); int (0 to 65K); long (0 to 4M); and long 64 for greate
values
- bool's, and something else for other specific types.
IMO, this could give some noob-friendly startup for the library !

You're right. We are aware it causes lots of confusions [1], including
to myself [2].
As Aleksander mentioned, we most likely will change that in future.

Personally, I think we should gear towards "C++ types first" approach
and try to cover them in a way most natural from C++ perspective.
I'm slowly sketching document [3] where I'm going to describe proposal on how
we want to handle C++ integers, in core and backends.
Any input will be appreciated.

[1] https://github.com/SOCI/soci/issues/90
[2] http://sourceforge.net/mailarchive/message.php?msg_id=30443672
[3] https://github.com/SOCI/soci/wiki/RFC1

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

Pawel Aleksander Fedorynski

2013-03-06 05:02:01 UTC

Permalink

Post by Mateusz Loskot

Post by Nicolas Deroche
the website !

I updated the documentation for the MySQL backend listing all the types and
what they currently convert to.

Post by Mateusz Loskot
If you see anything missing or you'd like to suggest any improvement,
feel more than welcome to open issue ticket at
https://github.com/SOCI/soci

greate

Post by Nicolas Deroche
values
- bool's, and something else for other specific types.
IMO, this could give some noob-friendly startup for the library !

There are two questions here.

1. What is the mapping between SQL types and SOCI types. The answer will
probably be backend specific, since various databases may have different
integral types. The current answer for MySQL is (as of my most recent
commit) documented in backends/mysql.html. I think what we have there is
pretty natural from the C++ perspective:

- All integral types (signed or unsigned) with less than 4 bytes should be
mapped to dt_integer, which corresponds to C++ type int, which in practice
is always 4 bytes in C++.

- Four byte signed integer should also be mapped to dt_integer.

- Four byte unsigned integer should be mapped to dt_long_long, which
corresponds to C++ type long long.

- Eight byte signed integer should be mapped to dt_long_long.

- Eight byte unsigned integer should be mapped to dt_unsigned_long_long.

The backends probably shouldn't change their mapping too willy-nilly since
existing code may depend on it.

2. How flexible SOCI should be when the user tries to read a field value
into a variable of a different type.

This is the actual problem. We're rather unforgiving. We should become
more lenient: it is just sad that when you select a value from a SMALLINT
column and try to read it using r.get<short>(...), which is exactly the
matching type, you get std::bad_cast.

It is not clear how much more lenient exactly.

(a) One option is to look at the actual value and only throw when the value
is out of range for the requested type. I.e., if the column is of type
BIGINT but the actual value read is 1234 then let the user interpret it as
short. If the column is INT but the actual value is positive then let the
user interpret it as unsigned.

(b) Another option is to look at the ranges of types, e.g., if the column
is of type SMALLINT (two byte signed integer), the user should be able to
interpret the values as short, or int, or long long, but not as char.
Also, interpreting values from column of signed types as unsigned C++
types would never be allowed.

As a user I'd slightly prefer (a). I don't know which will be easier to
implement.

Thanks,

Aleksander

I'm slowly sketching document [3] where I'm going to describe proposal on

Post by Mateusz Loskot
how
we want to handle C++ integers, in core and backends.
Any input will be appreciated.
[1] https://github.com/SOCI/soci/issues/90
[2] http://sourceforge.net/mailarchive/message.php?msg_id=30443672
[3] https://github.com/SOCI/soci/wiki/RFC1
Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
------------------------------------------------------------------------------
Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester
Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the
endpoint security space. For insight on selecting the right partner to
tackle endpoint security challenges, access the full report.
http://p.sf.net/sfu/symantec-dev2dev
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users