Opened 9 years ago

Closed 9 years ago

#179 closed enhancement (fixed)

Enable support for MySQL and PostgreSQL plugins

Reported by: rudi Owned by:
Priority: major Milestone: Qt 4.6.3
Component: General Version: 4.6.2
Severity: low Keywords:
Cc:

Description

It turns out, that these two database plugins can be build quite nicely. We should at least offer "configure" support for them (if not even ship binary versions). Here is, what I did to compile them:

Added to src\plugins\sqldrivers\mysql\mysql.pro

    os2 {
        TARGET_SHORT = qmysql
        LIBS += -lmysqlclient_r -lpthread -llibssl -llibcrypto
    }

Then run:

qmake "INCPATH+=<mysql_path>\include\mysql" "LIBPATH+=<mysql_path>\lib"

make

I couldn't find a clear statement, if the "reentrant" version of the MySQL client library is really necessary when used in the context of a Qt plugin. If not, it would remove the dependency from pthread.

For PostgreSQL the change in src\plugins\sqldrivers\psql\psql.pro
could look like this:

   os2 {
       TARGET_SHORT = qpgsql
       LIBS += -llibpq -llibssl -llibcrypto
   }

And the qmake/make command being similar.

In both cases the resulting plugins depend on OpenSSL. I'm not sure, if we should make the selection of the SSL libs a config option so that the user can also build the plugins with SSL linked in statically. But as I wrote in #167, I would prefer the DLL approach.

BTW, I did a quick test only with MySQL. I don't have PostGreSQL running here.

Attachments (1)

sockname_fail.zip (957 bytes) - added by rudi 9 years ago.
testcase to show TCP/IP failure with high memory

Download all attachments as: .zip

Change History (21)

comment:1 Changed 9 years ago by dmik

Could you please give links to MySQL/PostgreSQL development libs you used for that?

comment:2 Changed 9 years ago by rudi

I was using the client code from:

http://download.smedley.info/mysql-5.1.49-os2-20100724.zip

and

http://download.smedley.info/postgresql-8.4.3-os2-20100408.zip

However, I was not able to get the Postgres Plugin to work. It always sayed:

QSqlError(-1, "QPSQL: Unable to connect", "could not get client address from socket: Bad address") 

This appears to be a bad return code from libpq (function PQconnectdb()), but I don't feel like investigating this further...

comment:3 Changed 9 years ago by rudi

I have to add, that I always connected to a sever running on a Windows machine...

comment:4 Changed 9 years ago by rudi

That is *CRAZY* ! I've probably found the reason for the error with PostgrSQL: It's a problem with TCP/IP.

getsockname() will fail, if the variable that holds the length of "struct sockaddr" is in high memory. libpq uses the following code (conn is a structure that is malloc'ed).

    /* Fill in the client address */
    conn->laddr.salen = sizeof(conn->laddr.addr);
    if (getsockname(conn->sock,
                    (struct sockaddr *) & conn->laddr.addr,
                    &conn->laddr.salen) < 0)
    {
       ... error
    }

Question: Is it possible to force a plugin not to use high memory while the rest of Qt does ?

comment:5 Changed 9 years ago by dmik

Aha, yet another faulty case. No, you can't cause a single DLL use low memory in malloc(). What you can do is to replace the respective malloc() calls with _lmalloc() by hand.

comment:6 Changed 9 years ago by rudi

As expected getpeername() suffers from the same disease. However, I don't know if the applies to all versions of the IP stack. I'm using a rather old one.

Finally I was able to query a PostgreSQL database successfully by forcing my test application to low memory. So the Qt plugin appears to work...

I have created a ticket on Paul's issue tracker with a link to this thread. If he updates PostgreSQL, he might consider the _lmalloc thing.

Just for reference: here is my a testcase that shows the problem.

Changed 9 years ago by rudi

testcase to show TCP/IP failure with high memory

comment:7 Changed 9 years ago by rudi

Thinking a bit more about this it appears to m that the right place to provide a workaround for this would be LIBC. The wrappers could pass the address of a temporary stack variable to TCPIP and copying the content back and forth.

comment:8 Changed 9 years ago by psmedley

Interesting - I've seen failures in getsockname() too but hadn't associated it with high-mem. What I had seen, is that some apps like Cups work when compiled with GCC 3.3.5 works ok, but fails with GCC 4.x.

Will look into the _lmalloc thing when I get some time.

comment:9 Changed 9 years ago by dmik

So, for now I will only enable MySQL in the official build as Postgres is broken.

comment:10 Changed 9 years ago by dmik

From what I see in the configure script for Linux, they prefer the reentrant version if it is available but get also satisfied with the non-reentrant if it's not. This looks strange, as the reentrancy is either needed or not. But anyway, since we already require pthread for CUPS, it's not a problem to require it here as well.

comment:11 Changed 9 years ago by dmik

  • Resolution set to fixed
  • Status changed from new to closed

Done in r792.

comment:12 Changed 9 years ago by rudi

Paul has just released a new version of Postgres. But don't know, if he fixed that issue. Furthermore it requires LIBC064X. No idea, if that can coexist wthe the LIBC version we are using for Qt.

comment:13 Changed 9 years ago by diver

  • Resolution fixed deleted
  • Status changed from closed to reopened

just discussed that with Paul on irc. he will do a test libpq right now. if it works we should also enable postgress

comment:14 Changed 9 years ago by psmedley

OK, I think I found the relevant malloc in makeEmptyPGconn()

Compiling a new libpq.a now, using libc063 - as afaik the libc064x fixes are needed only for the database server itself (due to UCONV breakage)

http://smedley.info/libpq.zip [] include libpq.a and pq.dll

comment:15 Changed 9 years ago by dmik

Added PostgresSQL support in r803 (sorry, no description, Visual Slick editor's SVN support is ill).

Rudi or anyone else, please check that the new Paul's build fixes the socket problem. I don't have a Postgres database so I can't check. Once this is done, Paul will update his PostgresSQL distribution, I will put a link to it in README.OS2 and then we will close this ticket.

comment:16 Changed 9 years ago by psmedley

Please use http://download.smedley.info/libpq.zip for readme.os2 - the other link was intended only for private distribution as it's on my home DSL connection rather than download.smedley.info which is on a fast server at mensys

comment:17 Changed 9 years ago by dmik

Well, I'm not releasing now. I hope I will put a link to a new single zip that you will create after someone tests the stuff :)

comment:18 Changed 9 years ago by rudi

Even though I'm a bit short on time right now, a quick test revealed that the new version works. I can successfully query data from a 8.4.4 server running on Windows.

A note about README.OS2: With the PSQL_LIBS statement in the sample we create a self-contained plugin. I.e. one that doesn't use PQ.DLL. Thus, for this to work we need a patched libpq.a, not not a patched "PostgresSQL DLL". The statement about the DLL would be correct, if we link against PQ.DLL directly (in which case we would not need the SSL and PTHREAD libs).

comment:19 Changed 9 years ago by dmik

Hmm, right. Both MySQL and PostgresSQL plugins are actually self-contained. And as far as I see, this is the only way to go (at least I don't see any kind of import libraries in the Paul's distributions). I will correct README.OS2 once Paul has updated the zip so this inaccuracy will go away. Thank you for noticing. And many thanks for testing!

comment:20 Changed 9 years ago by dmik

  • Resolution set to fixed
  • Status changed from reopened to closed

The link to the new Postgres is http://download.smedley.info/postgresql-9.0.1-os2-20101108.zip. Thank you, Paul! Corrected README.OS2 (r817).

Note: See TracTickets for help on using tickets.