[soci-users] Using a unit testing framework?

[Vadim Zeitlin]

Hello,

I've done my first modifications to the SOCI unit tests recently and it
wasn't a very pleasant experience. The 2 main reasons for this are the
rather unusual tests organization and the use of assert() instead of some
more advanced mechanism. I'm especially riled by the latter because
assert() is just not good enough: it doesn't show you which test failed, it
doesn't show you the values of the variables with which it failed and it
doesn't allow you to continue running past the first failure.

So I wonder what do you think about using some testing framework instead.
Considering SOCI existing dependencies on Boost it could make sense to use
Boost.Test (http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/index.html)
but OTOH all the current dependencies are optional, while this one would be
semi-required as it's really nice to be able to run tests even if you don't
use Boost.

Hence an alternative solution: CATCH. This means "C++ AutomatedTest Cases
in Headers" and, as you can see at https://github.com/philsquared/Catch [1]
it is just a single header and all you need to do to start using it is to
just include it. It is also really nice in that it's enough to write

CHECK( ul == 4000000000ul );

to see the value of "ul" if the test fails, i.e. you have just a single
CHECK() which does everything, whereas with Boost.Test you need to use a
specific macro:

BOOST_CHECK_EQUAL( ul, 4000000000ul );

Personally I think this is a great idea but I admit that I haven't used
CATCH in really big test suites so I don't know what effect all the
meta-programming machinery necessary to make the above work has on the
compile times. Still, I believe CATCH would be a good fit for SOCI, so what
do you think about starting to use it?

And if not, using Boost.Test would still be much better than relying on
bad old assert()...

Finally, this is definitely not 3.2-critical but it would be nice to have
an agreement about this before the tests need to be modified the next time.

Regards,
VZ

[1] If you want to see the project activity, look at "integration" branch,
not the default "master".

[Mateusz Loskot]

Post by Vadim Zeitlin
[...]
Finally, this is definitely not 3.2-critical but it would be nice to have
an agreement about this before the tests need to be modified the next time.

Vadim,

I have no time today to respond in details, but this is important topic.
Briefly, I am planning to create new tests, please see related bullets here
https://github.com/SOCI/soci/wiki/Roadmap
There also is link to Alex's article with my short discussion with Alex.

I'll get back to this discussion.

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

[Denis Arnaud]

Hi Vadim,

Boost.Test, in combination with CMake, works pretty well. Moreover, it
integrates smoothly with many CI (continuous integration) frameworks such
as Jenkins/Hudson and Travis-CI.
You can browse through a full example with the following project:
http://github.com/airsim/stdair/. By default, (Boost.Test-based) test
checking is activated; it may be de-activated thanks to the
'-DENABLE_TEST:BOOL=OFF'
option passed to the CMake command-line. (In order for you to see how the
dependency on Boost.Test is handled at the installation/deployment level)
an example of packaging such a component in Fedora/CentOS/RedHat is given
by the RPM specification file:
http://pkgs.fedoraproject.org/cgit/stdair.git/tree/stdair.spec.

Hope that helps.

Kind regards

Denis

Post by Vadim Zeitlin
Hello,
I've done my first modifications to the SOCI unit tests recently and it
wasn't a very pleasant experience. The 2 main reasons for this are the
rather unusual tests organization and the use of assert() instead of some
more advanced mechanism. I'm especially riled by the latter because
assert() is just not good enough: it doesn't show you which test failed, it
doesn't show you the values of the variables with which it failed and it
doesn't allow you to continue running past the first failure.
So I wonder what do you think about using some testing framework instead.
Considering SOCI existing dependencies on Boost it could make sense to use
Boost.Test (
http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/index.html)
but OTOH all the current dependencies are optional, while this one would be
semi-required as it's really nice to be able to run tests even if you don't
use Boost.
Hence an alternative solution: CATCH. This means "C++ AutomatedTest Cases
in Headers" and, as you can see at https://github.com/philsquared/Catch[1]
it is just a single header and all you need to do to start using it is to
just include it. It is also really nice in that it's enough to write
CHECK( ul == 4000000000ul );
to see the value of "ul" if the test fails, i.e. you have just a single
CHECK() which does everything, whereas with Boost.Test you need to use a
BOOST_CHECK_EQUAL( ul, 4000000000ul );
Personally I think this is a great idea but I admit that I haven't used
CATCH in really big test suites so I don't know what effect all the
meta-programming machinery necessary to make the above work has on the
compile times. Still, I believe CATCH would be a good fit for SOCI, so what
do you think about starting to use it?
And if not, using Boost.Test would still be much better than relying on
bad old assert()...
Finally, this is definitely not 3.2-critical but it would be nice to have
an agreement about this before the tests need to be modified the next time.
Regards,
VZ
[1] If you want to see the project activity, look at "integration" branch,
not the default "master".
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

[Alex Ott]

Hi

I also use the Boost.Test + CMake + Jenkins - it's work pretty well...

Regarding packaging - have you tried CPack? For us it works without
much problems

Post by Denis Arnaud
Hi Vadim,
Boost.Test, in combination with CMake, works pretty well. Moreover, it
integrates smoothly with many CI (continuous integration) frameworks such as
Jenkins/Hudson and Travis-CI.
http://github.com/airsim/stdair/. By default, (Boost.Test-based) test
checking is activated; it may be de-activated thanks to the
'-DENABLE_TEST:BOOL=OFF' option passed to the CMake command-line. (In order
for you to see how the dependency on Boost.Test is handled at the
installation/deployment level) an example of packaging such a component in
http://pkgs.fedoraproject.org/cgit/stdair.git/tree/stdair.spec.
Hope that helps.
Kind regards
Denis

Post by Vadim Zeitlin
Hello,
I've done my first modifications to the SOCI unit tests recently and it
wasn't a very pleasant experience. The 2 main reasons for this are the
rather unusual tests organization and the use of assert() instead of some
more advanced mechanism. I'm especially riled by the latter because
assert() is just not good enough: it doesn't show you which test failed, it
doesn't show you the values of the variables with which it failed and it
doesn't allow you to continue running past the first failure.
So I wonder what do you think about using some testing framework instead.
Considering SOCI existing dependencies on Boost it could make sense to use
Boost.Test
(http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/index.html)
but OTOH all the current dependencies are optional, while this one would be
semi-required as it's really nice to be able to run tests even if you don't
use Boost.
Hence an alternative solution: CATCH. This means "C++ AutomatedTest Cases
in Headers" and, as you can see at https://github.com/philsquared/Catch [1]
it is just a single header and all you need to do to start using it is to
just include it. It is also really nice in that it's enough to write
CHECK( ul == 4000000000ul );
to see the value of "ul" if the test fails, i.e. you have just a single
CHECK() which does everything, whereas with Boost.Test you need to use a
BOOST_CHECK_EQUAL( ul, 4000000000ul );
Personally I think this is a great idea but I admit that I haven't used
CATCH in really big test suites so I don't know what effect all the
meta-programming machinery necessary to make the above work has on the
compile times. Still, I believe CATCH would be a good fit for SOCI, so what
do you think about starting to use it?
And if not, using Boost.Test would still be much better than relying on
bad old assert()...
Finally, this is definitely not 3.2-critical but it would be nice to have
an agreement about this before the tests need to be modified the next time.
Regards,
VZ
[1] If you want to see the project activity, look at "integration" branch,
not the default "master".
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

[Vadim Zeitlin]

On Sat, 23 Mar 2013 15:54:54 +0100 Denis Arnaud <***@m4x.org> wrote:

DA> Hi Vadim,
DA>
DA> Boost.Test, in combination with CMake, works pretty well.

Hello Denis,

I have no doubt that Boost.Test works well, I used it in several projects
(CMake -- not so much, but it's a different story) and until recently it
was my preferred C++ unit testing framework. But then I discovered CATCH
and was really impressed by it, although, again, I didn't use it enough to
be sure that it scales well, and I also think it's a nice match for SOCI as
it doesn't need to be built and is also more lightweight/even less
restrictive than Boost.Test and so it should be easier to transition to it.
Also, it is still actively developed and improved, unlike Boost.Test which
hasn't changed for years. So I think it could be interesting to try using
CATCH. After all, why not?

DA> You can browse through a full example with the following project:
DA> http://github.com/airsim/stdair/. By default, (Boost.Test-based) test
DA> checking is activated; it may be de-activated thanks to the
DA> '-DENABLE_TEST:BOOL=OFF' option passed to the CMake command-line.

Thanks, this should definitely be useful if we go this way. But personally
I'd still be curious to try using CATCH.

Regards,
VZ

[Ricardo Fabiano de Andrade]

Hi all,

I also started working on soci recently and I would like to share my
opinion regarding the unit tests.

First, I already experienced a situation where the only available choice
for making a unit test was C assertions.
So I'm familiar with the issues and limitations of this solution and I
agree that there's great room for improvement.

Of course, using such solution was due political disputes but hey, an
assert-based test is better than no test at all.
If this is not the soci case (the political issue), there's still the case
of avoiding external dependencies.
Well, almost nothing can be done in this case with the exception of
reinventing the wheel (please no).

Currently, I'm working with googletest. It's suprisingly simple to get used
to and extremely produtive.
Besides that it has a great support for testing template classes what can
be extremely helpful with soci type support (integers?).
Even though googletest is being actively developed and heavly used, I'm
afraid it isn't the first option for soci.

AFAIK about Boost.Test, it does also support testing template
classes (although not that obvious) and so much more - points already
enumerated by others in this dicussion.
It really haven't changed much in a while but probably because it's already
very stable (I still miss the mocking support googlemock has).
As others have mentioned before, this would be one of the good options for
soci.

So being more dependent of the boost libraries can actually be beneficial.
If there's a goal of making soci to stick more with the future C++
versions, of course.
BTW, soci has been mentioned in the papers/proposals of the next C++
standard (see N3612).

Finally, the idea behind CATCH (a single header) is awesome but in my
opinion the framework is too immature right now.
I think soci could eventually be a good place for mutual colaboration
causing CATCH to become a stronger option.
It's a win-win for both soci (better tests) and CATCH (more collaboration).

That's my two cents.

Best regards,
Ricardo Andrade

Post by Vadim Zeitlin
Hello,
I've done my first modifications to the SOCI unit tests recently and it
wasn't a very pleasant experience. The 2 main reasons for this are the
rather unusual tests organization and the use of assert() instead of some
more advanced mechanism. I'm especially riled by the latter because
assert() is just not good enough: it doesn't show you which test failed, it
doesn't show you the values of the variables with which it failed and it
doesn't allow you to continue running past the first failure.
So I wonder what do you think about using some testing framework instead.
Considering SOCI existing dependencies on Boost it could make sense to use
Boost.Test (
http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/index.html)
but OTOH all the current dependencies are optional, while this one would be
semi-required as it's really nice to be able to run tests even if you don't
use Boost.
Hence an alternative solution: CATCH. This means "C++ AutomatedTest Cases
in Headers" and, as you can see at https://github.com/philsquared/Catch[1]
it is just a single header and all you need to do to start using it is to
just include it. It is also really nice in that it's enough to write
CHECK( ul == 4000000000ul );
to see the value of "ul" if the test fails, i.e. you have just a single
CHECK() which does everything, whereas with Boost.Test you need to use a
BOOST_CHECK_EQUAL( ul, 4000000000ul );
Personally I think this is a great idea but I admit that I haven't used
CATCH in really big test suites so I don't know what effect all the
meta-programming machinery necessary to make the above work has on the
compile times. Still, I believe CATCH would be a good fit for SOCI, so what
do you think about starting to use it?
And if not, using Boost.Test would still be much better than relying on
bad old assert()...
Finally, this is definitely not 3.2-critical but it would be nice to have
an agreement about this before the tests need to be modified the next time.
Regards,
VZ
[1] If you want to see the project activity, look at "integration" branch,
not the default "master".

[Alex Ott]

btw, we can use single-header variant of boost test in case if we
can't find boost installed:
http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/utf/user-guide/usage-variants/single-header-variant.html

On Sun, Mar 24, 2013 at 5:13 AM, Ricardo Fabiano de Andrade

Post by Ricardo Fabiano de Andrade
Hi all,
I also started working on soci recently and I would like to share my opinion
regarding the unit tests.
First, I already experienced a situation where the only available choice for
making a unit test was C assertions.
So I'm familiar with the issues and limitations of this solution and I agree
that there's great room for improvement.
Of course, using such solution was due political disputes but hey, an
assert-based test is better than no test at all.
If this is not the soci case (the political issue), there's still the case
of avoiding external dependencies.
Well, almost nothing can be done in this case with the exception of
reinventing the wheel (please no).
Currently, I'm working with googletest. It's suprisingly simple to get used
to and extremely produtive.
Besides that it has a great support for testing template classes what can be
extremely helpful with soci type support (integers?).
Even though googletest is being actively developed and heavly used, I'm
afraid it isn't the first option for soci.
AFAIK about Boost.Test, it does also support testing template classes
(although not that obvious) and so much more - points already enumerated by
others in this dicussion.
It really haven't changed much in a while but probably because it's already
very stable (I still miss the mocking support googlemock has).
As others have mentioned before, this would be one of the good options for
soci.
So being more dependent of the boost libraries can actually be beneficial.
If there's a goal of making soci to stick more with the future C++ versions,
of course.
BTW, soci has been mentioned in the papers/proposals of the next C++
standard (see N3612).
Finally, the idea behind CATCH (a single header) is awesome but in my
opinion the framework is too immature right now.
I think soci could eventually be a good place for mutual colaboration
causing CATCH to become a stronger option.
It's a win-win for both soci (better tests) and CATCH (more collaboration).
That's my two cents.
Best regards,
Ricardo Andrade

Post by Vadim Zeitlin
Hello,
I've done my first modifications to the SOCI unit tests recently and it
wasn't a very pleasant experience. The 2 main reasons for this are the
rather unusual tests organization and the use of assert() instead of some
more advanced mechanism. I'm especially riled by the latter because
assert() is just not good enough: it doesn't show you which test failed, it
doesn't show you the values of the variables with which it failed and it
doesn't allow you to continue running past the first failure.
So I wonder what do you think about using some testing framework instead.
Considering SOCI existing dependencies on Boost it could make sense to use
Boost.Test
(http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/index.html)
but OTOH all the current dependencies are optional, while this one would be
semi-required as it's really nice to be able to run tests even if you don't
use Boost.
Hence an alternative solution: CATCH. This means "C++ AutomatedTest Cases
in Headers" and, as you can see at https://github.com/philsquared/Catch [1]
it is just a single header and all you need to do to start using it is to
just include it. It is also really nice in that it's enough to write
CHECK( ul == 4000000000ul );
to see the value of "ul" if the test fails, i.e. you have just a single
CHECK() which does everything, whereas with Boost.Test you need to use a
BOOST_CHECK_EQUAL( ul, 4000000000ul );
Personally I think this is a great idea but I admit that I haven't used
CATCH in really big test suites so I don't know what effect all the
meta-programming machinery necessary to make the above work has on the
compile times. Still, I believe CATCH would be a good fit for SOCI, so what
do you think about starting to use it?
And if not, using Boost.Test would still be much better than relying on
bad old assert()...
Finally, this is definitely not 3.2-critical but it would be nice to have
an agreement about this before the tests need to be modified the next time.
Regards,
VZ
[1] If you want to see the project activity, look at "integration" branch,
not the default "master".

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

[Ricardo Fabiano de Andrade]

And once soci's unit tests are in single cpp file this solution sounds
perfect Ott.

Post by Alex Ott
btw, we can use single-header variant of boost test in case if we
http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/utf/user-guide/usage-variants/single-header-variant.html
On Sun, Mar 24, 2013 at 5:13 AM, Ricardo Fabiano de Andrade

Post by Ricardo Fabiano de Andrade
Hi all,
I also started working on soci recently and I would like to share my

opinion

Post by Ricardo Fabiano de Andrade
regarding the unit tests.
First, I already experienced a situation where the only available choice

for

Post by Ricardo Fabiano de Andrade
making a unit test was C assertions.
So I'm familiar with the issues and limitations of this solution and I

agree

Post by Ricardo Fabiano de Andrade
that there's great room for improvement.
Of course, using such solution was due political disputes but hey, an
assert-based test is better than no test at all.
If this is not the soci case (the political issue), there's still the

case

Post by Ricardo Fabiano de Andrade
of avoiding external dependencies.
Well, almost nothing can be done in this case with the exception of
reinventing the wheel (please no).
Currently, I'm working with googletest. It's suprisingly simple to get

used

Post by Ricardo Fabiano de Andrade
to and extremely produtive.
Besides that it has a great support for testing template classes what

can be

Post by Ricardo Fabiano de Andrade
extremely helpful with soci type support (integers?).
Even though googletest is being actively developed and heavly used, I'm
afraid it isn't the first option for soci.
AFAIK about Boost.Test, it does also support testing template classes
(although not that obvious) and so much more - points already enumerated

by

Post by Ricardo Fabiano de Andrade
others in this dicussion.
It really haven't changed much in a while but probably because it's

already

Post by Ricardo Fabiano de Andrade
very stable (I still miss the mocking support googlemock has).
As others have mentioned before, this would be one of the good options

for

Post by Ricardo Fabiano de Andrade
soci.
So being more dependent of the boost libraries can actually be

beneficial.

Post by Ricardo Fabiano de Andrade
If there's a goal of making soci to stick more with the future C++

versions,

Post by Ricardo Fabiano de Andrade
of course.
BTW, soci has been mentioned in the papers/proposals of the next C++
standard (see N3612).
Finally, the idea behind CATCH (a single header) is awesome but in my
opinion the framework is too immature right now.
I think soci could eventually be a good place for mutual colaboration
causing CATCH to become a stronger option.
It's a win-win for both soci (better tests) and CATCH (more

collaboration).

Post by Ricardo Fabiano de Andrade
That's my two cents.
Best regards,
Ricardo Andrade

Post by Vadim Zeitlin
Hello,
I've done my first modifications to the SOCI unit tests recently and it
wasn't a very pleasant experience. The 2 main reasons for this are the
rather unusual tests organization and the use of assert() instead of

some

Post by Ricardo Fabiano de Andrade

Post by Vadim Zeitlin
more advanced mechanism. I'm especially riled by the latter because
assert() is just not good enough: it doesn't show you which test failed, it
doesn't show you the values of the variables with which it failed and it
doesn't allow you to continue running past the first failure.
So I wonder what do you think about using some testing framework

instead.

Post by Ricardo Fabiano de Andrade

Post by Vadim Zeitlin
Considering SOCI existing dependencies on Boost it could make sense to

use

Post by Ricardo Fabiano de Andrade

Post by Vadim Zeitlin
Boost.Test
(http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/index.html)
but OTOH all the current dependencies are optional, while this one would be
semi-required as it's really nice to be able to run tests even if you don't
use Boost.
Hence an alternative solution: CATCH. This means "C++ AutomatedTest

Cases

Post by Ricardo Fabiano de Andrade

Post by Vadim Zeitlin
in Headers" and, as you can see at https://github.com/philsquared/Catch [1]
it is just a single header and all you need to do to start using it is

to

Post by Ricardo Fabiano de Andrade

Post by Vadim Zeitlin
just include it. It is also really nice in that it's enough to write
CHECK( ul == 4000000000ul );
to see the value of "ul" if the test fails, i.e. you have just a single
CHECK() which does everything, whereas with Boost.Test you need to use a
BOOST_CHECK_EQUAL( ul, 4000000000ul );
Personally I think this is a great idea but I admit that I haven't used
CATCH in really big test suites so I don't know what effect all the
meta-programming machinery necessary to make the above work has on the
compile times. Still, I believe CATCH would be a good fit for SOCI, so what
do you think about starting to use it?
And if not, using Boost.Test would still be much better than relying on
bad old assert()...
Finally, this is definitely not 3.2-critical but it would be nice to

have

Post by Ricardo Fabiano de Andrade

Post by Vadim Zeitlin
an agreement about this before the tests need to be modified the next time.
Regards,
VZ
[1] If you want to see the project activity, look at "integration"

branch,

Post by Ricardo Fabiano de Andrade

Post by Vadim Zeitlin
not the default "master".

------------------------------------------------------------------------------

Post by Ricardo Fabiano de Andrade
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

--
With best wishes, Alex Ott
http://alexott.net/
Twitter: alexott_en (English), alexott (Russian)
Skype: alex.ott
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

[Mateusz Loskot]

On 24 March 2013 17:03, Ricardo Fabiano de Andrade

Post by Ricardo Fabiano de Andrade
And once soci's unit tests are in single cpp file this solution sounds
perfect Ott.

Currently, we build one testing program per backend.
If we want, we can change it for use of a testing toolkit.

For example, we may decide to:
- build common tests separately as single program
common for all backends, which execution will be configurable
(i.e. list of backends and connection strings passed as command
line arguments);
- build backend-specific tests build as separate
one per backend.

Also, if we manage to write unit tests (for core) based on mocking,
then that will become a separate program too.

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

[Bruce Adams]

Hi,
    I confess to not having been following this discussion or SOCI development very closely but to throw my tuppence in
I would highly recommend cppunit as a unit test framework (based on the xunit framework). I've used it on a daily basis

for many years now (for test driven development)and it does its job very well. Having moved to cmake you can easily

drive cppunit via ctest.I find it useful to compliment unit tests with program level tests written as scripts

(each test program can be launched via ctest) but for libraries this generally isn't necessary.

When using cppunit to test templates we generally test just the significant specialisations (writing wrappers as necessary
to avoid duplication of code).

I should imagine the requirements for the boost testing framework are a little different from those of normal libraries as they
do more template meta-programming than most normal human beings care for.

Its worth giving the issue a little thought and experimentation before settling on one.There are a number of interesting comparisons between frameworks on the net. E.g.

http://gamesfromwithin.com/exploring-the-c-unit-testing-framework-jungle



Regards,

Bruce.

Post by Vadim Zeitlin
________________________________
Sent: Sunday, March 24, 2013 9:52 AM
Subject: Re: [soci-users] Using a unit testing framework?
btw, we can use single-header variant of boost test in case if we
http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/utf/user-guide/usage-variants/single-header-variant.html
On Sun, Mar 24, 2013 at 5:13 AM, Ricardo Fabiano de Andrade

Post by Ricardo Fabiano de Andrade
Hi all,
I also started working on soci recently and I would like to share my opinion
regarding the unit tests.
First, I already experienced a situation where the only available choice for
making a unit test was C assertions.
So I'm familiar with the issues and limitations of this solution and I agree
that there's great room for improvement.
Of course, using such solution was due political disputes but hey, an
assert-based test is better than no test at all.
If this is not the soci case (the political issue), there's still the case
of avoiding external dependencies.
Well, almost nothing can be done in this case with the exception of
reinventing the wheel (please no).
Currently, I'm working with googletest. It's suprisingly simple to get used
to and extremely produtive.
Besides that it has a great support for testing template classes what can be
extremely helpful with soci type support (integers?).
Even though googletest is being actively developed and heavly used, I'm
afraid it isn't the first option for soci.
AFAIK about Boost.Test, it does also support testing template classes
(although not that obvious) and so much more - points already enumerated by
others in this dicussion.
It really haven't changed much in a while but probably because it's already
very stable (I still miss the mocking support googlemock has).
As others have mentioned before, this would be one of the good options for
soci.
So being more dependent of the boost libraries can actually be beneficial.
If there's a goal of making soci to stick more with the future C++ versions,
of course.
BTW, soci has been mentioned in the papers/proposals of the next C++
standard (see N3612).
Finally, the idea behind CATCH (a single header) is awesome but in my
opinion the framework is too immature right now.
I think soci could eventually be a good place for mutual colaboration
causing CATCH to become a stronger option.
It's a win-win for both soci (better tests) and CATCH (more collaboration).
That's my two cents.
Best regards,
Ricardo Andrade

  Hello,
  I've done my first modifications to the SOCI unit tests recently and it
wasn't a very pleasant experience. The 2 main reasons for this are the
rather unusual tests organization and the use of assert() instead of some
more advanced mechanism. I'm especially riled by the latter because
assert() is just not good enough: it doesn't show you which test failed, it
doesn't show you the values of the variables with which it failed and it
doesn't allow you to continue running past the first failure.
  So I wonder what do you think about using some testing framework instead.
Considering SOCI existing dependencies on Boost it could make sense to use
Boost.Test
(http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/index.html)
but OTOH all the current dependencies are optional, while this one would be
semi-required as it's really nice to be able to run tests even if you don't
use Boost.
  Hence an alternative solution: CATCH. This means "C++ AutomatedTest Cases
in Headers" and, as you can see at https://github.com/philsquared/Catch [1]
it is just a single header and all you need to do to start using it is to
just include it. It is also really nice in that it's enough to write
        CHECK( ul == 4000000000ul );
to see the value of "ul" if the test fails, i.e. you have just a single
CHECK() which does everything, whereas with Boost.Test you need to use a
        BOOST_CHECK_EQUAL( ul, 4000000000ul );
Personally I think this is a great idea but I admit that I haven't used
CATCH in really big test suites so I don't know what effect all the
meta-programming machinery necessary to make the above work has on the
compile times. Still, I believe CATCH would be a good fit for SOCI, so what
do you think about starting to use it?
  And if not, using Boost.Test would still be much better than relying on
bad old assert()...
  Finally, this is definitely not 3.2-critical but it would be nice to have
an agreement about this before the tests need to be modified the next time.
  Regards,
VZ
[1] If you want to see the project activity, look at "integration" branch,
    not the default "master".

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

--
With best wishes,                    Alex Ott
http://alexott.net/
Twitter: alexott_en (English), alexott (Russian)
Skype: alex.ott
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

[Vadim Zeitlin]

On Mon, 25 Mar 2013 11:22:39 +0000 (GMT) Bruce Adams <***@yahoo.co.uk> wrote:

BA> I would highly recommend cppunit as a unit test framework (based on the
BA> xunit framework).

I have very strong feelings about cppunit and would love dearly to use
anything else but it. The best thing I can say about it is that it's better
than using plain assert() but that's all. I could go into the details if it
could be useful but the TL;DR version is that cppunit slavishly copies
xUnit approach which is perhaps good for Java but in C++ there are much,
much better -- and less verbose -- ways to do the same thing.

BA> I should imagine the requirements for the boost testing framework are a
BA> little different from those of normal libraries as they do more
BA> template meta-programming than most normal human beings care for.

You absolutely don't need to do any template meta-programming when using
Boost.Test. FWIW it doesn't contain much of it internally neither (unlike
CATCH) but as a user you really don't care about it. I'm genuinely curious:
did you use Boost.Test before standardizing on cppunit? Because if you did,
this would make you the first person I've ever heard about who preferred
cppunit to Boost.Test after trying them both...

Regards,
VZ

[Bruce Adams]

Hi,
    The question here is which library best meets the needs of SOCI. So apologies for helping roam off topic.

   cppunit was standardised on before I joined the company where I first used it. Also it was a company policy
not to use any of boost (for silly rather than sensible reasons that I've only recently managed to get overturned).
We make a point of having test classes separate from the class they are testing (e.g. class foo has a class fooTest).
Beyond that I've written very little additional boiler plate for cppunit but that could be a function of how we write our tests.
Certainly cppunit (and cxxunit) started out copying from junit but it has gradually evolved to fit better.
I'm very curious what features you feel are missing that make something else like boost.test
significantly better (assuming you aren't testing gui's or try to write multi-threaded tests for example).
I am aware of the competing school of thought that test methods should be part of the class under test itself but
generally I prefer having a clean separation between the two.

With regard to requirements for testing. I meant that the boost.test framework might have more facilities for testing
template meta-programming not that it did a lot of it itself (which would be just an implementation detail anyway).

I haven't actually had cause to try boost.test as cppunit meets my current needs and is a defacto standard where I work.
In boost I can see it from at least two sides.
 1. boost needs to be tested and therefore needs a test framework - anything outside boost could be a potentially unecessary external dependency
 2. the c++ community needs a unit test framework and boost aims to be "one" if not the "one" just as it does with other libraries.

I always assumed its main purpose was to serve boost (i.e. 1).
I wonder in that regard it how much it is like boost.build. Having seen boost.build in operation I am reasonably impressed
but the rest of the world seems to be moving towards cmake. I haven't run across many projects deciding to use bjam.    


Regards,

Bruce.

Post by Vadim Zeitlin
________________________________
Sent: Monday, March 25, 2013 11:41 AM
Subject: Re: [soci-users] Using a unit testing framework?
BA> I would highly recommend cppunit as a unit test framework (based on the
BA> xunit framework).
I have very strong feelings about cppunit and would love dearly to use
anything else but it. The best thing I can say about it is that it's better
than using plain assert() but that's all. I could go into the details if it
could be useful but the TL;DR version is that cppunit slavishly copies
xUnit approach which is perhaps good for Java but in C++ there are much,
much better -- and less verbose -- ways to do the same thing.
BA> I should imagine the requirements for the boost testing framework are a
BA> little different from those of normal libraries as they do more
BA> template meta-programming than most normal human beings care for.
You absolutely don't need to do any template meta-programming when using
Boost.Test. FWIW it doesn't contain much of it internally neither (unlike
did you use Boost.Test before standardizing on cppunit? Because if you did,
this would make you the first person I've ever heard about who preferred
cppunit to Boost.Test after trying them both...
Regards,
VZ
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

[Vadim Zeitlin]

On Mon, 25 Mar 2013 15:06:46 +0000 (GMT) Bruce Adams <***@yahoo.co.uk> wrote:

BA> I'm very curious what features you feel are missing that make something else like boost.test
BA> significantly better

There are many things and I probably won't be able to even remember all of
them right now but, just off the of my head:

1. You don't need to have classes to write code in C++. So with Boost.Test
you can write test functions (although you can organize them in classes
too, of course, if you want to but you rarely do because you can have
fixtures without classes). In cppunit you have to write test classes and
you have to register them with the test suite which adds a lot of boiler
plate, especially for simple tests.

2. Partly as a consequence of the above, with Boost.Test you write tests
and they run. With cppunit, you add a new test method to an existing
test class -- and nothing happens. Because you need to register it
separately, of course. So all your test methods must be repeated twice.

The above 2 are deep design problems due to copying rigid xUnit approach to
a much richer language which is C++ and I don't think they can be fixed
without turning cppunit into something that shouldn't have "unit" in its
name any more. The rest of the points below probably could be fixed, but I
think this is unlikely to happen because, AFAIK, there is no development
done on cppunit and also because why would anyone bother considering the
existence of many superior alternatives (even if you like xUnit, I think
xUnit++ is a better choice than cppunit)?

3. Non fatal assertions, i.e. BOOST_CHECK vs BOOST_REQUIRE. This is just
so convenient. With cppunit you basically need to strive to put one test
per function to work around their lack.

4. Easy control of the tests you run. It's quite incredible that cppunit
doesn't allow you to run individual tests. Of course, you can implement
this on top of it (and we did) but how could something like this be not
supported out of the box? With Boost.Test you can just specify the tests
to run on the command line, as expected.

5. Control of the output. Again, this is something you can do with cppunit
but why should you be forced to do it when you should just be able to
specify the output verbosity on the command line, as Boost.Test (and
everyone else) allows you to do?

6. Test timing. Boost.Test doesn't replace performance non-regression tests
but at least it gives you some useful information. cppunit? Not so much.

7. Greater choice of assert macros and easier possibility to specify
messages for failing checks (personally I think that all tests should
have them).


There are more points at

http://stackoverflow.com/questions/7922289/googletest-vs-cppunit-the-facts

although I don't think Boost.Test has all of them, so it could be unfair to
cppunit. OTOH if you look at

http://stackoverflow.com/questions/242926/comparison-of-c-unit-test-frameworks

you can see that all of Boost.Test, Google test and CATCH have all the
features above. And cppunit is probably the only one to have none. Hence my
original conclusion: do yourself a favour and choose anything but it.


BA> I wonder in that regard it how much it is like boost.build.

It absolutely isn't tied to testing Boost.

BA> Having seen boost.build in operation I am reasonably impressed but the
BA> rest of the world seems to be moving towards cmake. I haven't run
BA> across many projects deciding to use bjam.    

I fully agree that bjam is horrible and CMake is much better. Having said
this, I don't believe CMake is ideal neither but I'll leave this for
another discussion :-)

Regards,
VZ

[Bruce Adams]

Hi,
     I take your points but just for the record a number of the items listed on:

http://stackoverflow.com/questions/7922289/googletest-vs-cppunit-the-facts


are easily doable in cppunit. i.e. exceptions and adding information.
Others are not provided out of the box but doable as write once solutions.


The reason for having to register each test is, as you allude to, a consequence of porting from java.
junit avoids the boilerplate by using reflection and the original porters did not provide an alternative.

There are some features which are of low utility. For example, "test shuffling".

The big thing for me would be it ceasing to be actively maintained.
Still you have piqued my interest enough to want to do a more thorough review of the options now available
and where they stand with respect to each other when I get time.


I'm with you on cmake. Its like democracy, there must be a better system but we haven't found it yet.


Regards,

Bruce.

Post by Vadim Zeitlin
________________________________
Sent: Monday, March 25, 2013 3:26 PM
Subject: Re: [soci-users] Using a unit testing framework?
BA> I'm very curious what features you feel are missing that make something else like boost.test
BA> significantly better
There are many things and I probably won't be able to even remember all of
1. You don't need to have classes to write code in C++. So with Boost.Test
  you can write test functions (although you can organize them in classes
  too, of course, if you want to but you rarely do because you can have
  fixtures without classes). In cppunit you have to write test classes and
  you have to register them with the test suite which adds a lot of boiler
  plate, especially for simple tests.
2. Partly as a consequence of the above, with Boost.Test you write tests
  and they run. With cppunit, you add a new test method to an existing
  test class -- and nothing happens. Because you need to register it
  separately, of course. So all your test methods must be repeated twice.
The above 2 are deep design problems due to copying rigid xUnit approach to
a much richer language which is C++ and I don't think they can be fixed
without turning cppunit into something that shouldn't have "unit" in its
name any more. The rest of the points below probably could be fixed, but I
think this is unlikely to happen because, AFAIK, there is no development
done on cppunit and also because why would anyone bother considering the
existence of many superior alternatives (even if you like xUnit, I think
xUnit++ is a better choice than cppunit)?
3. Non fatal assertions, i.e. BOOST_CHECK vs BOOST_REQUIRE. This is just
  so convenient. With cppunit you basically need to strive to put one test
  per function to work around their lack.
4. Easy control of the tests you run. It's quite incredible that cppunit
  doesn't allow you to run individual tests. Of course, you can implement
  this on top of it (and we did) but how could something like this be not
  supported out of the box? With Boost.Test you can just specify the tests
  to run on the command line, as expected.
5. Control of the output. Again, this is something you can do with cppunit
  but why should you be forced to do it when you should just be able to
  specify the output verbosity on the command line, as Boost.Test (and
  everyone else) allows you to do?
6. Test timing. Boost.Test doesn't replace performance non-regression tests
  but at least it gives you some useful information. cppunit? Not so much.
7. Greater choice of assert macros and easier possibility to specify
  messages for failing checks (personally I think that all tests should
  have them).
There are more points at
http://stackoverflow.com/questions/7922289/googletest-vs-cppunit-the-facts
although I don't think Boost.Test has all of them, so it could be unfair to
cppunit. OTOH if you look at
http://stackoverflow.com/questions/242926/comparison-of-c-unit-test-frameworks
you can see that all of Boost.Test, Google test and CATCH have all the
features above. And cppunit is probably the only one to have none. Hence my
original conclusion: do yourself a favour and choose anything but it.
BA> I wonder in that regard it how much it is like boost.build.
It absolutely isn't tied to testing Boost.
BA> Having seen boost.build in operation I am reasonably impressed but the
BA> rest of the world seems to be moving towards cmake. I haven't run
BA> across many projects deciding to use bjam.    
I fully agree that bjam is horrible and CMake is much better. Having said
this, I don't believe CMake is ideal neither but I'll leave this for
another discussion :-)
Regards,
VZ
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

[Vadim Zeitlin]

On Sun, 24 Mar 2013 01:13:10 -0300 Ricardo Fabiano de Andrade <***@gmail.com> wrote:

RFdA> Currently, I'm working with googletest. It's suprisingly simple to
RFdA> get used to and extremely produtive.

What are the advantages of Google Test compared to Boost.Test? I know that
you mentioned testing template classes but I've never had any problems with
this when using Boost.Test so I'm not sure what is this about exactly. And
when in doubt, I'd rather choose a Boost library over a Google one for
several reasons.

RFdA> BTW, soci has been mentioned in the papers/proposals of the next C++
RFdA> standard (see N3612).

I didn't see this one yet, thanks! Here is the link for the lazy:

http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2013/n3612.pdf

RFdA> Finally, the idea behind CATCH (a single header) is awesome but in my
RFdA> opinion the framework is too immature right now.

As I said, I don't have much experience of using it in real projects yet
so it could definitely be the case, but could you please explain what
exactly do you mean, i.e. how does its immaturity manifest itself?

Thanks,
VZ

[Alex Ott]

Hi

I also don't see much benefits of Google Test vs. Boost.Test. The one
benefit of Google Test is that it's working with Google C++ Mocking
framework. But for Boost.Test there is turtle library with almost same
functionality

Post by Vadim Zeitlin
RFdA> Currently, I'm working with googletest. It's suprisingly simple to
RFdA> get used to and extremely produtive.
What are the advantages of Google Test compared to Boost.Test? I know that
you mentioned testing template classes but I've never had any problems with
this when using Boost.Test so I'm not sure what is this about exactly. And
when in doubt, I'd rather choose a Boost library over a Google one for
several reasons.
RFdA> BTW, soci has been mentioned in the papers/proposals of the next C++
RFdA> standard (see N3612).
http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2013/n3612.pdf
RFdA> Finally, the idea behind CATCH (a single header) is awesome but in my
RFdA> opinion the framework is too immature right now.
As I said, I don't have much experience of using it in real projects yet
so it could definitely be the case, but could you please explain what
exactly do you mean, i.e. how does its immaturity manifest itself?
Thanks,
VZ
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

[Mateusz Loskot]

Post by Alex Ott
I also don't see much benefits of Google Test vs. Boost.Test. The one
benefit of Google Test is that it's working with Google C++ Mocking
framework. But for Boost.Test there is turtle library with almost same
functionality

For me, that kind of combination is an important element,
as I explained briefly in comment [1] to your article.
I see advantages of using libraries from common sources/authors/projects
versus mixing and matching different ones.

[1] http://alexott.net/en/cpp/CppTestingIntro.html#comment-853141322

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

[Ricardo Fabiano de Andrade]

VZ> What are the advantages of Google Test compared to Boost.Test? I know
that
you mentioned testing template classes but I've never had any problems with
this when using Boost.Test so I'm not sure what is this about exactly. And
when in doubt, I'd rather choose a Boost library over a Google one for
several reasons.

I don't feel comfortable making a deep "vs." here once I didn't used
Boost.Test more than a few lines.
They seem pretty much equivalent in the number of features but personally
found googletest easier to learn/use.
You can try read the part relative to typed test (w/ templates) to see what
I mean and get some conclusions:
https://code.google.com/p/googletest/wiki/AdvancedGuide#Typed_Tests

Even though I have more time working with googletest and I could promptly
imagine uses for soci, I must agree there's no strong reasons for picking
it. But it's always cool bringing different experiences to the mix. :-)

VZ> http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2013/n3612.pdf

Thank you for complementing my comments by providing the link!
VZ> As I said, I don't have much experience of using it in real projects yet

so it could definitely be the case, but could you please explain what
exactly do you mean, i.e. how does its immaturity manifest itself?
CATCH has the reduced number of features when compared to

either Boost.Test or googletest, the documentation is very limited and the
maintainer is still inclined to make breaking changes in the interface what
caused him to consider CATCH a "developer preview" (see the last lines in
https://github.com/philsquared/Catch/wiki).
Its only initial appeal I could see was the single include
file, but another point to Boost.Test for doing that too.

BTW, shoud we move this discussion to the soci-devel mail list?

[Mateusz Loskot]

Post by Vadim Zeitlin
[...]
Personally I think this is a great idea but I admit that I haven't used
CATCH in really big test suites so I don't know what effect all the
meta-programming machinery necessary to make the above work has on the
compile times. Still, I believe CATCH would be a good fit for SOCI, so what
do you think about starting to use it?
And if not, using Boost.Test would still be much better than relying on
bad old assert()...
Finally, this is definitely not 3.2-critical but it would be nice to have
an agreement about this before the tests need to be modified the next time.

Using this place to reply as one-to-all posts, not specifically to Vadim's.

There has been a lot of interesting points discussed, some more general
some more SOCI-specific. I however would prefer to avoid attempt to
answer StackOverflow-like question "Which framework is best?".

I have looked at and learned to some extent about
CppUnit, xUnit++, C++TUT, CATCH, Boost.Test and GoogleTest.
I have only used Boost.Test and C++TUT, a lot.
This makes my personal inclinations failry obvious, apart other things like
preferences related to C++ coding style and project guidelines,
familiarity with one community vs others, etc.

Here are some of my more important points:

Using gtest + gmock will effectively add two new
dependencies to SOCI. Same for CATCH + Turtle (or other mocker).
Using Boost.Test + Turtle (Boost.Mock) will add, let's say, one (Turtle)
and 'half' (Boost). Currently Boost is optional but I suppose many if not
most users build SOCI with Boost, same for packes.
If someone has Boost installed, I think it's safe to assume it's complete
Boost distribution, so there are Boost.Test headers available
(if we are header-only-philic here, we'll be fine :-)).
Another important advantages of Turtle is that its interface, style and
conventions match Boost's. This promises seamless integration.

AFAIK, for gtest and gmock, there are no binaries for Windows provided.
Since BoostPro has closed down, situation with Boost binaries for Windows
is now tentative. So, no winners here.
For Linux, packages of both Boost and Google libraries are common.
Neither CATCH nor Turtle seems to be packaged, but Turtle integrates
seamlessly with Boost build configuration.

There are solid communities behind Google and Boost, seems promising.
(Boost.Test has got updates during 2012, several new features too,
Turtle is actively maintained and chances it will be reviewed for Boost
as mentioned in thread "Recommended mock framework to use with Boost?"
http://lists.boost.org/boost-users/2012/01/index.php)

I'm not sure about status of CATCH. I know its author Phil from ACCU.org
and remember there has been load of discussions at the time this library
was forming, but recently it seems slowed down, still not released (?).

As you can see, there is no clear winner from the points above,
except Google and Boost getting best scores for me.
I could draw more points, more pros and cons, but I honestly doubt it
would change anything for the discussion on which one to choose.

Unless a solid reason appears that will back up one option,
I think the choice will be made driven by
"self-regulated natural forces of community" (tm) :-),
that is at some point someone will start rewriting tests, will make no-brainer
decision choosing her/his preferred testing framework and complete certain
substantial amount of work in this area.

If I will start first, I probably will go for Boost.Test + Turtle (Boost.Mock).
Vadim may like to choose CATCH + Turtle|HippoMock|gmock|...
Someone else may go for gtest + gmock.

First contribute, First decide.

Why I mention mock every time?
I'd really like to add set of proper unit tests, along with integration tests
based on rewriting current tests. I'm also hoping to use this as
opportunity to master testing+mocking techniques myself.
So, I assume use of testing and mocking tools.


p.s. It looks this thread has grown a lot here, so moving it to soci-devel
may be too confusing. It also seems nobody complained on it here on
soci-users, so I don't mind to continue it here.
Although, let's try to discuss devel-specific stuff at soci-devel in future.

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

[Mateusz Loskot]

Post by Mateusz Loskot
There are solid communities behind Google and Boost, seems promising.
(Boost.Test has got updates during 2012, several new features too,
Turtle is actively maintained and chances it will be reviewed for Boost
as mentioned in thread "Recommended mock framework to use with Boost?"
http://lists.boost.org/boost-users/2012/01/index.php)
[...]
If I will start first, I probably will go for Boost.Test + Turtle (Boost.Mock).

Just reminded myself about the "What Should we do About Boost.Test?"
thread from Sept 2012
http://thread.gmane.org/gmane.comp.lib.boost.devel/234297/
So, I may want to revise my preferences.

We may decided to write our own, as John did for Boost.Multiprecision ;-)
http://svn.boost.org/svn/boost/trunk/libs/multiprecision/test/test.hpp

Best regards,

[Mateusz Loskot]

Post by Mateusz Loskot
Why I mention mock every time?
I'd really like to add set of proper unit tests, along with integration tests
based on rewriting current tests. I'm also hoping to use this as
opportunity to master testing+mocking techniques myself.
So, I assume use of testing and mocking tools.

While re-reading the Boost.Test thread, I stumbled upon this very curious
post by Gennadiy [1]. It turns out Boost.Test has got some mocking
facilities [2]. I will look into details and see what's the status, but if it's
basically usable, then Boost.Test makes a strong option for me.


[1] http://thread.gmane.org/gmane.comp.lib.boost.devel/234297/focus=234621
[2] http://svn.boost.org/svn/boost/trunk/libs/test/example/logged_exp_example.cpp

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

[Bruce Adams]

Probably teaching you to suck eggs but boost has a tool called bcp which lets you copy just the relevant subset of boost into your source tree.
If they are using it to test the rest of boost I imagine it has few dependencies and will therefore have a relatively small foot print.

Post by Vadim Zeitlin
________________________________
Sent: Sunday, April 7, 2013 9:27 PM
Subject: Re: [soci-users] Using a unit testing framework?

Post by Mateusz Loskot
Why I mention mock every time?
I'd really like to add set of proper unit tests, along with integration tests
based on rewriting current tests. I'm also hoping to use this as
opportunity to master testing+mocking techniques myself.
So, I assume use of testing and mocking tools.

While re-reading the Boost.Test thread, I stumbled upon this very curious
post by Gennadiy [1]. It turns out Boost.Test has got some mocking
facilities [2]. I will look into details and see what's the status, but if it's
basically usable, then Boost.Test makes a strong option for me.
[1] http://thread.gmane.org/gmane.comp.lib.boost.devel/234297/focus=234621
[2] http://svn.boost.org/svn/boost/trunk/libs/test/example/logged_exp_example.cpp
Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
------------------------------------------------------------------------------
Minimize network downtime and maximize team effectiveness.
Reduce network management and security costs.Learn how to hire
the most talented Cisco Certified professionals. Visit the
Employer Resources Portal
http://www.cisco.com/web/learning/employer_resources/index.html
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

[Mateusz Loskot]

Post by Bruce Adams
Probably teaching you to suck eggs but boost has a tool called bcp which
lets you copy just the relevant subset of boost into your source tree.
If they are using it to test the rest of boost I imagine it has few
dependencies and will therefore have a relatively small foot print.

Yes, I know bcp, but I see no need to maintain copy of boost libraries
for SOCI purposes. I have experience with such approach and
I'd prefer to avoid it.

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

[Vadim Zeitlin]

On Sun, 7 Apr 2013 04:52:48 +0100 Mateusz Loskot <***@loskot.net> wrote:

ML> Here are some of my more important points:
ML>
ML> Using gtest + gmock will effectively add two new
ML> dependencies to SOCI. Same for CATCH + Turtle (or other mocker).
ML> Using Boost.Test + Turtle (Boost.Mock) will add, let's say, one (Turtle)
ML> and 'half' (Boost). Currently Boost is optional but I suppose many if not
ML> most users build SOCI with Boost, same for packes.
ML> If someone has Boost installed, I think it's safe to assume it's complete
ML> Boost distribution, so there are Boost.Test headers available
ML> (if we are header-only-philic here, we'll be fine :-)).
ML> Another important advantages of Turtle is that its interface, style and
ML> conventions match Boost's. This promises seamless integration.

Hello,

I think the dependency point is pretty important for the testing
framework. Of course, for any developer of the library it's not difficult
to install either one of the libraries mentioned above (or all of them :-)
but this is not the only case where the dependencies matter. I'm thinking
more about the users who may be asked to run the SOCI tests, perhaps with
some small changes, to help debugging some problem. And also about
occasional contributors who'd have much higher probability of updating the

[Mateusz Loskot]

Post by Vadim Zeitlin
ML>
ML> Using gtest + gmock will effectively add two new
ML> dependencies to SOCI. Same for CATCH + Turtle (or other mocker).
ML> Using Boost.Test + Turtle (Boost.Mock) will add, let's say, one (Turtle)
ML> and 'half' (Boost). Currently Boost is optional but I suppose many if not
ML> most users build SOCI with Boost, same for packes.
ML> If someone has Boost installed, I think it's safe to assume it's complete
ML> Boost distribution, so there are Boost.Test headers available
ML> (if we are header-only-philic here, we'll be fine :-)).
ML> Another important advantages of Turtle is that its interface, style and
ML> conventions match Boost's. This promises seamless integration.
I think the dependency point is pretty important for the testing
framework.
[...]
And also about
occasional contributors who'd have much higher probability of updating the
tests if they didn't need to install anything extra to be able to run them.

[Vadim Zeitlin]

On Mon, 8 Apr 2013 16:56:20 +0100 Mateusz Loskot <***@loskot.net> wrote:

ML> > ML> Neither CATCH nor Turtle seems to be packaged,
ML> >
ML> > Again, CATCH is not packaged because it doesn't need packaging, this is
ML> > the appealing part.
ML>
ML> I meant packages deploying, like its common to have Boost headers packaged.

I see, sorry for the misunderstanding. I didn't realize you were already
using the same approach with TUT (which, BTW, I didn't spend much time on
just because the example on http://tut-framework.sourceforge.net/ is pretty
weird-looking, at least compared to the one at
https://github.com/philsquared/Catch/wiki/Tutorial), so I just wanted to
emphasize this, IMHO important, aspect.


ML> BTW, I know Phil is going to give talks about CATCH during a few upcoming
ML> conferences in UK, so that is a good indicator the project is active.

Also, knowing how these things happen, he'll probably have a few things
to fix/improve in CATCH too after these talks...

ML> Fair point. CATCH does not completely fit my preferred coding styles,
ML> but I'll be open minded :-)

Well, the combined header (catch.hpp) is definitely pretty horrible, but
browsing the code in their repository doesn't immediately show anything too
bad. The worst thing I noticed, style-wise, was the use of identifiers
starting with underscores, which should clearly be avoided but OTOH is not
quite a firing offense neither.

ML> > The main advantages of CATCH are the ease of installation (none needed)
ML>
ML> If we consider CATCH, I assume we will simply host copy of its headers
ML> inside SOCI, right?

Yes, I think we should put catch.hpp somewhere in our sources.

ML> > ML> Why I mention mock every time?
ML> >
ML> > Good question :-) I'm really not sure if we need it at all, what are we
ML> > going to mock exactly?
ML>
ML> Core. I am very keen in exercising mocking techniques against the core.

I must admit that I don't follow you at all here. For me mocking is used
to replace a component that can't be easily tested (e.g. because it uses an
external data source which is not always available) with a mock-up that
emulates this component in a predictable way and can also be used to verify
that it's used as intended. So if we mock the core itself, what are we
testing then?

ML> > A database backend?
ML>
ML> Nope, backends rely onDBMS specific features, so testing them in isolation from
ML> related DBMS is pointless.

My idea was to use mocking to check that the core code calls the
appropriate backend methods, as expected. I'm not sure if this really has
an enormous value but I at least can understand this, unlike the idea of
mocking the core which still has me scratching my head...


ML> So, to summarise my position, I'd be in favour to start rewriting
ML> tests using CATCH.
ML> Once we start, we should soon be able to confirm if it is lacking any
ML> features or not.

This would be perfect from my point of view. And the mocking discussion
can probably wait until later.

Thank you for driving this process!
VZ

[Mateusz Loskot]

Post by Vadim Zeitlin
ML> > ML> Neither CATCH nor Turtle seems to be packaged,
ML> >
ML> > Again, CATCH is not packaged because it doesn't need packaging, this is
ML> > the appealing part.
ML>
ML> I meant packages deploying, like its common to have Boost headers packaged.
I see, sorry for the misunderstanding. I didn't realize you were already
using the same approach with TUT

In two projects
http://svn.osgeo.org/geos/trunk/tests/unit/tut/
https://github.com/libLAS/libLAS/tree/master/test

Post by Vadim Zeitlin
(which, BTW, I didn't spend much time on
just because the example on http://tut-framework.sourceforge.net/ is pretty
weird-looking, at least compared to the one at
https://github.com/philsquared/Catch/wiki/Tutorial), so I just wanted to
emphasize this, IMHO important, aspect.

I started using it in 2006, given that generation of C++ libraries,
TUT is dead simple to use :)

Post by Vadim Zeitlin
ML> Fair point. CATCH does not completely fit my preferred coding styles,
ML> but I'll be open minded :-)
Well, the combined header (catch.hpp) is definitely pretty horrible, but
browsing the code in their repository doesn't immediately show anything too
bad. The worst thing I noticed, style-wise, was the use of identifiers
starting with underscores, which should clearly be avoided but OTOH is not
quite a firing offense neither.

I agree on both, the critique and relaxing (I personally don't like CamelCase or
class I* naming, but one can never have all personal prefs fulfilled :))

Post by Vadim Zeitlin
ML> > The main advantages of CATCH are the ease of installation (none needed)
ML>
ML> If we consider CATCH, I assume we will simply host copy of its headers
ML> inside SOCI, right?
Yes, I think we should put catch.hpp somewhere in our sources.

Alternative, at least in development version, is to use Git submodule.
In release, the file would be copied of course.

Post by Vadim Zeitlin
ML> > ML> Why I mention mock every time?
ML> >
ML> > Good question :-) I'm really not sure if we need it at all, what are we
ML> > going to mock exactly?
ML>
ML> Core. I am very keen in exercising mocking techniques against the core.
I must admit that I don't follow you at all here. For me mocking is used
to replace a component that can't be easily tested (e.g. because it uses an
external data source which is not always available) with a mock-up that
emulates this component in a predictable way and can also be used to verify
that it's used as intended. So if we mock the core itself, what are we
testing then?

I wasn't clear. I meant, to test Core with mock backend (i.e. similar to empty,
but covering more).
But, this idea is still forming, so I'd rather focus on rewriting
tests and not waste
time discussing details of that vague mocking idea, for the time being.
If I have time at some point, I may prepare a prototype, then it will
be easier to talk.

Post by Vadim Zeitlin
ML> > A database backend?
ML>
ML> Nope, backends rely onDBMS specific features, so testing them in isolation from
ML> related DBMS is pointless.
My idea was to use mocking to check that the core code calls the
appropriate backend methods, as expected. I'm not sure if this really has
an enormous value but I at least can understand this, unlike the idea of
mocking the core which still has me scratching my head...

My idea is exactly the same, I just expressed it badly.

Post by Vadim Zeitlin
ML> So, to summarise my position, I'd be in favour to start rewriting
ML> tests using CATCH.
ML> Once we start, we should soon be able to confirm if it is lacking any
ML> features or not.
This would be perfect from my point of view. And the mocking discussion
can probably wait until later.

OK, it looks we're set then. CATCH.

My short term plan then (details to be discussed on soci-devel or GH issues):
1. Release 3.2.1
2. Reorganise source tree a bit (shuffle stuff, remove unused stuff, etc.)
3. Buried headers
4. In parallel, kickstart tests rewrite and work on integers support

Post by Vadim Zeitlin
Thank you for driving this process!

Thanks great for help!

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

[Mateusz Loskot]

Post by Mateusz Loskot

Post by Vadim Zeitlin
ML> So, to summarise my position, I'd be in favour to start rewriting
ML> tests using CATCH.
ML> Once we start, we should soon be able to confirm if it is lacking any
ML> features or not.
This would be perfect from my point of view. And the mocking discussion
can probably wait until later.

OK, it looks we're set then. CATCH.

I have asked some basic questions about status of CATCH:

https://groups.google.com/d/msg/catch-forum/ogGWssRhk-g/zDnaMMiTH-oJ

If there is anything else we should ask about, let me know or
post questions to the CATCH group directly.

Best regards,

[philsquared]

I'm subscribed to this list now, too so will do my best to answer any
questions or concerns you might have.
Thanks for considering CATCH.

Please be assured it is very much in active development, but also fairly
stable.

Regards,

[)o
IhIL..



--
View this message in context: http://soci.6940.n7.nabble.com/soci-users-Using-a-unit-testing-framework-tp3439p3600.html
Sent from the soci-users mailing list archive at Nabble.com.

[Mateusz Loskot]

Post by philsquared
I'm subscribed to this list now, too so will do my best to answer any
questions or concerns you might have.

Phil,

That's great, thank you.

It's quite messy though that we'd stared discussing this development-only
thread here on soci-users, what in fact should happen on soci-devel [1].
So, any further discussions about SOCI testing with CATCH should
happen, I'm *hopeful*, on soci-devel.
Also, traffic volume on soci-devel is far smaller than here on soci-users,
so you may prefer to switch, if you are not particularly interested in
using SOCI :)

[1] https://lists.sourceforge.net/lists/listinfo/soci-devel
which is also also on Nabble at
http://soci.6940.n7.nabble.com/soci-devel-f3.html

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net