[soci-users] CQL support for Cassandra

Discussion:

Sebastian Lauwers

2013-03-31 18:35:42 UTC

Hi,

Has any research been done into bringing Cassandra/CQL support into
SOCI? Currently, Cassandra doesn't offer much in terms of C++ drivers.

Considering I've been using SOCI in a number of my projects, rather
than rolling my own driver for Cassandra, I was thinking about
bringing CQL support into SOCI. There are no C nor C++ client
libraries that could be used to build an abstraction from, so the
"native" code would be in SOCI.

Would anyone have an opinion on this?

Thanks,

-S.

Mateusz Loskot

2013-03-31 18:51:33 UTC

Permalink

Post by Sebastian Lauwers
Has any research been done into bringing Cassandra/CQL support into
SOCI? Currently, Cassandra doesn't offer much in terms of C++ drivers.
Considering I've been using SOCI in a number of my projects, rather
than rolling my own driver for Cassandra, I was thinking about
bringing CQL support into SOCI. There are no C nor C++ client
libraries that could be used to build an abstraction from, so the
"native" code would be in SOCI.
Would anyone have an opinion on this?

I have no experience with this technology myself, so I have no opinion really.

It is not clear to me what would be role of SOCI here.
Are you thinking of implementing complete backend and expose
access to it using SOCI API and syntax features?
Would it require any changes in public API or core?

I don't have time to learn about Cassandra/CQL, but personally
I'm generally very supportive to new features as long as 1) they
do not significantly affect existing codebase, 2) other developers
and major users don't object.

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

Sebastian Lauwers

2013-03-31 19:10:09 UTC

Permalink

Hi Mateusz,

Post by Mateusz Loskot
It is not clear to me what would be role of SOCI here.
Are you thinking of implementing complete backend and expose
access to it using SOCI API and syntax features?

Exactly, yes.

Post by Mateusz Loskot
Would it require any changes in public API or core?

Not that I'm currently aware of. It may require a few specifics when
it comes to connecting to the cluster (Cassandra is designed to be
distributed, so instead of one host, you'd list a number of different
hosts to which the queries can be dispatched, multiple connections
could be created, etc), but that part can already be back-end
specific, if I'm not mistaken.

Regardless of the above, if I can not find a way to make CQL support
in SOCI painless (as in, requiring changes to the core or the API),
then it completely defeats the purpose of introducing CQL support to
SOCI :). However, from what I can tell (I'm hardly a Cassandra expert
myself), having Cassandra support in SOCI would benefit me massively,
as moving data from one type of DB to another is exactly what I have
to do.

Post by Mateusz Loskot
I don't have time to learn about Cassandra/CQL, but personally
I'm generally very supportive to new features as long as 1) they
do not significantly affect existing codebase, 2) other developers
and major users don't object.

I don't intend to put any burden on anyone but myself (and my company,
if it is taken as a company project). I'd be glad to hear the feelings
of others.

-S.

Denis Arnaud

2013-03-31 20:01:47 UTC

Permalink

Good idea, if at all feasible :)

Now that you bring the issue about supporting NoSQL databases, we can think
of plenty other cases, *e.g.*, MongoDB, HBase, Hypertable, Impala, Drill.
Note that Hypertable may be easier to support, as it is being developed
natively in C++. As for Impala, a friend of mine is developing a JDBC
driver and a client wrapper interface (
https://github.com/pauldeschacht/impala-java-client), but it is in Java...

Kind regards

Denis

Post by Sebastian Lauwers
Hi Mateusz,

Post by Mateusz Loskot
It is not clear to me what would be role of SOCI here.
Are you thinking of implementing complete backend and expose
access to it using SOCI API and syntax features?

Exactly, yes.

Post by Mateusz Loskot
Would it require any changes in public API or core?

Not that I'm currently aware of. It may require a few specifics when
it comes to connecting to the cluster (Cassandra is designed to be
distributed, so instead of one host, you'd list a number of different
hosts to which the queries can be dispatched, multiple connections
could be created, etc), but that part can already be back-end
specific, if I'm not mistaken.
Regardless of the above, if I can not find a way to make CQL support
in SOCI painless (as in, requiring changes to the core or the API),
then it completely defeats the purpose of introducing CQL support to
SOCI :). However, from what I can tell (I'm hardly a Cassandra expert
myself), having Cassandra support in SOCI would benefit me massively,
as moving data from one type of DB to another is exactly what I have
to do.

I don't intend to put any burden on anyone but myself (and my company,
if it is taken as a company project). I'd be glad to hear the feelings
of others.
-S.
------------------------------------------------------------------------------
Own the Future-Intel(R) Level Up Game Demo Contest 2013
Rise to greatness in Intel's independent game demo contest. Compete
for recognition, cash, and the chance to get your game on Steam.
$5K grand prize plus 10 genre and skill prizes. Submit your demo
by 6/6/13. http://altfarm.mediaplex.com/ad/ck/12124-176961-30367-2
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

Mateusz Loskot

2013-03-31 21:01:27 UTC

Permalink

Post by Denis Arnaud
Now that you bring the issue about supporting NoSQL databases, we can think
of plenty other cases, e.g., MongoDB, HBase, Hypertable, Impala, Drill. Note
that Hypertable may be easier to support, as it is being developed natively
in C++.

The term of NoSQL has been dangling around for a while:
https://github.com/SOCI/soci/wiki/Roadmap

I suppose, any storage interface accessible with CRUD operations
should be "wrapable" as SOCI backend, especially those with
text-based queries.

Post by Denis Arnaud
As for Impala, a friend of mine is developing a JDBC driver and a
client wrapper interface
(https://github.com/pauldeschacht/impala-java-client), but it is in Java...

The fact that JDBC is used sounds encouraging, it should feasible then :)

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

Sebastian Lauwers

2013-03-31 21:35:33 UTC

Permalink

FWIW, there is also a JDBC driver for Cassandra/CQL:

https://code.google.com/a/apache-extras.org/p/cassandra-jdbc/

I'm not sure why this is a good thing, but I'm mentioning it as per
Mateusz's remark regarding the Impala JDBC driver.

In addition, Datastax is working on a binary protocol that enables
native CQL, instead of going over thrift. The current implementation
is java based:
https://github.com/datastax/java-driver/tree/master/driver-core

Post by Mateusz Loskot

https://github.com/SOCI/soci/wiki/Roadmap
I suppose, any storage interface accessible with CRUD operations
should be "wrapable" as SOCI backend, especially those with
text-based queries.

Post by Denis Arnaud
As for Impala, a friend of mine is developing a JDBC driver and a
client wrapper interface
(https://github.com/pauldeschacht/impala-java-client), but it is in Java...

The fact that JDBC is used sounds encouraging, it should feasible then :)
Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
------------------------------------------------------------------------------
Own the Future-Intel(R) Level Up Game Demo Contest 2013
Rise to greatness in Intel's independent game demo contest. Compete
for recognition, cash, and the chance to get your game on Steam.
$5K grand prize plus 10 genre and skill prizes. Submit your demo
by 6/6/13. http://altfarm.mediaplex.com/ad/ck/12124-176961-30367-2
_______________________________________________
soci-users mailing list
https://lists.sourceforge.net/lists/listinfo/soci-users

Mateusz Loskot

2013-03-31 21:57:43 UTC

Permalink

Post by Sebastian Lauwers
https://code.google.com/a/apache-extras.org/p/cassandra-jdbc/
I'm not sure why this is a good thing, but I'm mentioning it as per
Mateusz's remark regarding the Impala JDBC driver.

My general remark was based on the following reasoning:
if there is a JDBC driver for a storage engine, it means the storage
interface should fit the traditional notions of connection, statement
and data bindings that are also present in SOCI.
Thus, it should be possible to implement SOCI backend for it.

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

Mateusz Loskot

2013-03-31 20:55:30 UTC

Permalink

Post by Sebastian Lauwers

Post by Mateusz Loskot
It is not clear to me what would be role of SOCI here.
Are you thinking of implementing complete backend and expose
access to it using SOCI API and syntax features?

Exactly, yes.

Sounds good.

Post by Sebastian Lauwers

Post by Mateusz Loskot
Would it require any changes in public API or core?

Yes, that should be feasible.

Recently, Vadim has added connection_parameters feature that
may be helpful for this purpose:
https://github.com/SOCI/soci/pull/98
https://github.com/SOCI/soci/pull/99

Post by Sebastian Lauwers
Regardless of the above, if I can not find a way to make CQL support
in SOCI painless (as in, requiring changes to the core or the API),
then it completely defeats the purpose of introducing CQL support to
SOCI :).

It would be interesting to evaluate SOCI for that purpose anyway.
For example, I have been thinking of possible way to support NoSQL
databases with JSON-based queries.
SOCI core is just a proxy transporting query and data source/sink
objects to/from backend. So, backends hold majority of responsibilities
here, thus it should be possible to implement various flavours of backends.

Post by Sebastian Lauwers
However, from what I can tell (I'm hardly a Cassandra expert
myself), having Cassandra support in SOCI would benefit me massively,
as moving data from one type of DB to another is exactly what I have
to do.

I agree.

Post by Sebastian Lauwers

I don't intend to put any burden on anyone but myself (and my company,
if it is taken as a company project). I'd be glad to hear the feelings
of others.

I'll try to help as much as I can.

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net