Opened 10 years ago

Closed 10 years ago

#179 closed enhancement (fixed)

Enable support for MySQL and PostgreSQL plugins

Reported by: rudi Owned by:
Priority: major Milestone: Qt 4.6.3
Component: General Version: 4.6.2
Severity: low Keywords:
Cc:

Description

It turns out, that these two database plugins can be build quite nicely. We should at least offer "configure" support for them (if not even ship binary versions). Here is, what I did to compile them:

Added to src\plugins\sqldrivers\mysql\mysql.pro

    os2 {
        TARGET_SHORT = qmysql
        LIBS += -lmysqlclient_r -lpthread -llibssl -llibcrypto
    }

Then run:

qmake "INCPATH+=<mysql_path>\include\mysql" "LIBPATH+=<mysql_path>\lib"

make

I couldn't find a clear statement, if the "reentrant" version of the MySQL client library is really necessary when used in the context of a Qt plugin. If not, it would remove the dependency from pthread.

For PostgreSQL the change in src\plugins\sqldrivers\psql\psql.pro
could look like this:

   os2 {
       TARGET_SHORT = qpgsql
       LIBS += -llibpq -llibssl -llibcrypto
   }

And the qmake/make command being similar.

In both cases the resulting plugins depend on OpenSSL. I'm not sure, if we should make the selection of the SSL libs a config option so that the user can also build the plugins with SSL linked in statically. But as I wrote in #167, I would prefer the DLL approach.

BTW, I did a quick test only with MySQL. I don't have PostGreSQL running here.

Attachments (1)

sockname_fail.zip (957 bytes) - added by rudi 10 years ago.
testcase to show TCP/IP failure with high memory

Download all attachments as: .zip

Change History (21)

comment:1 Changed 10 years ago by Dmitry A. Kuminov

Could you please give links to MySQL/PostgreSQL development libs you used for that?

comment:2 Changed 10 years ago by rudi

I was using the client code from:

http://download.smedley.info/mysql-5.1.49-os2-20100724.zip

and

http://download.smedley.info/postgresql-8.4.3-os2-20100408.zip

However, I was not able to get the Postgres Plugin to work. It always sayed:

QSqlError(-1, "QPSQL: Unable to connect", "could not get client address from socket: Bad address") 

This appears to be a bad return code from libpq (function PQconnectdb()), but I don't feel like investigating this further...

comment:3 Changed 10 years ago by rudi

I have to add, that I always connected to a sever running on a Windows machine...

comment:4 Changed 10 years ago by rudi

That is *CRAZY* ! I've probably found the reason for the error with PostgrSQL: It's a problem with TCP/IP.

getsockname() will fail, if the variable that holds the length of "struct sockaddr" is in high memory. libpq uses the following code (conn is a structure that is malloc'ed).

    /* Fill in the client address */
    conn->laddr.salen = sizeof(conn->laddr.addr);
    if (getsockname(conn->sock,
                    (struct sockaddr *) & conn->laddr.addr,
                    &conn->laddr.salen) < 0)
    {
       ... error
    }

Question: Is it possible to force a plugin not to use high memory while the rest of Qt does ?

comment:5 Changed 10 years ago by Dmitry A. Kuminov

Aha, yet another faulty case. No, you can't cause a single DLL use low memory in malloc(). What you can do is to replace the respective malloc() calls with _lmalloc() by hand.

comment:6 Changed 10 years ago by rudi

As expected getpeername() suffers from the same disease. However, I don't know if the applies to all versions of the IP stack. I'm using a rather old one.

Finally I was able to query a PostgreSQL database successfully by forcing my test application to low memory. So the Qt plugin appears to work...

I have created a ticket on Paul's issue tracker with a link to this thread. If he updates PostgreSQL, he might consider the _lmalloc thing.

Just for reference: here is my a testcase that shows the problem.

Changed 10 years ago by rudi

Attachment: sockname_fail.zip added

testcase to show TCP/IP failure with high memory

comment:7 Changed 10 years ago by rudi

Thinking a bit more about this it appears to m that the right place to provide a workaround for this would be LIBC. The wrappers could pass the address of a temporary stack variable to TCPIP and copying the content back and forth.

comment:8 Changed 10 years ago by psmedley

Interesting - I've seen failures in getsockname() too but hadn't associated it with high-mem. What I had seen, is that some apps like Cups work when compiled with GCC 3.3.5 works ok, but fails with GCC 4.x.

Will look into the _lmalloc thing when I get some time.

comment:9 Changed 10 years ago by Dmitry A. Kuminov

So, for now I will only enable MySQL in the official build as Postgres is broken.

comment:10 Changed 10 years ago by Dmitry A. Kuminov

From what I see in the configure script for Linux, they prefer the reentrant version if it is available but get also satisfied with the non-reentrant if it's not. This looks strange, as the reentrancy is either needed or not. But anyway, since we already require pthread for CUPS, it's not a problem to require it here as well.

comment:11 Changed 10 years ago by Dmitry A. Kuminov

Resolution: fixed
Status: newclosed

Done in r792.

comment:12 Changed 10 years ago by rudi

Paul has just released a new version of Postgres. But don't know, if he fixed that issue. Furthermore it requires LIBC064X. No idea, if that can coexist wthe the LIBC version we are using for Qt.

comment:13 Changed 10 years ago by Silvan Scherrer

Resolution: fixed
Status: closedreopened

just discussed that with Paul on irc. he will do a test libpq right now. if it works we should also enable postgress

comment:14 Changed 10 years ago by psmedley

OK, I think I found the relevant malloc in makeEmptyPGconn()

Compiling a new libpq.a now, using libc063 - as afaik the libc064x fixes are needed only for the database server itself (due to UCONV breakage)

http://smedley.info/libpq.zip [] include libpq.a and pq.dll

comment:15 Changed 10 years ago by Dmitry A. Kuminov

Added PostgresSQL support in r803 (sorry, no description, Visual Slick editor's SVN support is ill).

Rudi or anyone else, please check that the new Paul's build fixes the socket problem. I don't have a Postgres database so I can't check. Once this is done, Paul will update his PostgresSQL distribution, I will put a link to it in README.OS2 and then we will close this ticket.

comment:16 Changed 10 years ago by psmedley

Please use http://download.smedley.info/libpq.zip for readme.os2 - the other link was intended only for private distribution as it's on my home DSL connection rather than download.smedley.info which is on a fast server at mensys

comment:17 Changed 10 years ago by Dmitry A. Kuminov

Well, I'm not releasing now. I hope I will put a link to a new single zip that you will create after someone tests the stuff :)

comment:18 Changed 10 years ago by rudi

Even though I'm a bit short on time right now, a quick test revealed that the new version works. I can successfully query data from a 8.4.4 server running on Windows.

A note about README.OS2: With the PSQL_LIBS statement in the sample we create a self-contained plugin. I.e. one that doesn't use PQ.DLL. Thus, for this to work we need a patched libpq.a, not not a patched "PostgresSQL DLL". The statement about the DLL would be correct, if we link against PQ.DLL directly (in which case we would not need the SSL and PTHREAD libs).

comment:19 Changed 10 years ago by Dmitry A. Kuminov

Hmm, right. Both MySQL and PostgresSQL plugins are actually self-contained. And as far as I see, this is the only way to go (at least I don't see any kind of import libraries in the Paul's distributions). I will correct README.OS2 once Paul has updated the zip so this inaccuracy will go away. Thank you for noticing. And many thanks for testing!

comment:20 Changed 10 years ago by Dmitry A. Kuminov

Resolution: fixed
Status: reopenedclosed

The link to the new Postgres is http://download.smedley.info/postgresql-9.0.1-os2-20101108.zip. Thank you, Paul! Corrected README.OS2 (r817).

Note: See TracTickets for help on using tickets.