TD-17330: TD v6 crash on a Select from Informix database

[nfg]

Hello,

I am no sure if it is a bug, because I don't know which version of Informix database is supported with TD 6.0 SP5,
therefore I inserted this problem in general discussion.

we have a problem with Select statements, which crahses the Team Developer if there are more than the half of
columnsize filled with data (i.e. colum size char 60 and ther are 45 chars saved in the row).

First I have a question which versions of setnet should be used, because there is no version number mentioned in
connectivity guide.

We have a Informix database version 11.7 FC 4 and we have tried setnet 3.5 TC1 and 3.7 TC3, but in both version the
team developer crashes.
We use TD 6.0 SP5, it also crashes with TD 6.0 SP4.

We are using unicode and the database is set on UTF8.
sql.ini and setnet configuration is in the zip-file.

Details:

In the database is a table "teilnehmer" which has a column name with length 60 and its from typ char.
There are also others columns like vorname, strasse and so on.
We inserted a string into database "123456789012345678901234567890123456789012345" with our team developer
application, this works.
After that we use a SELECT-command:
SELECT name, vorname, strasse FROM teilnehmer INTO :strName, :strVorname, :strStrasse

During the SqlPrepareAndExecute command Team Developer 6.0 crashes.

TD 6.0 crashes also on statement:
SELECT name, vorname FROM teilnehmer INTO :strName, :strVorname

But this statement will work:
SELECT name FROM teilnehmer INTO :strName

We guess that there is a problem in changing from utf8 (informix database) to utf16 (Team developer),
because we have switched on informix client logging with the environment variable SQLIDEBUG and
we could see the correct dataset in the logfile.

Thanks for help
Norbert Gugu

[Jeff Luther]

I don't know which version of Informix database is supported with TD 6.0 SP5

Let's clear that up first. Here's the compatibility matrix for v6
SP number doesn't matter.

which crahses the Team Developer if there are more than the half of columnsize filled with data (i.e. colum size char 60 and ther are 45 chars saved in the row)

Hmmm, Unicode/double-byte chars in TD v6 comes to mind immediately, because Unicode in TD is 2 bytes/char; TD v4.2 and earlier was 1 byte/char. Though wrote;

We are using unicode and the database is set on UTF8.

TD shouldn't crash, of course, but the first thing suggest is that you do is put together a small test case. You can probably go a long way toward answering some of your SELECT questions in a test. You don't show CREATE TABLE statements for those DB tables, but you can experiment with SELECT and INSERTs to see what gets inserted. Example tests:
VARCHAR( 100 ) -- INSERT string up to 50 OK, crash > 50?

SqlPrepareAndExecute command Team Developer 6.0 crashes

First, break that down to separate SqlPrepare and SqlExecute commands. Right now you can't tell where the problem is and this might matter. I'm surprised the crash is here somewhere and not in the fetch. Prep just resolves INTO list in TD, then rest of SELECT passed to DB. Exe just tells DB to build the result set for later fetching.

Where's the WHERE clause in your queries? Don't have one? How many rows in those tables?

But this statement will work:
SELECT name FROM teilnehmer INTO :strName

Interesting. What about: SELECT vorname FROM teilnehmer INTO :strVorname
That work OK too? Maybe it's the vorname column?

I suggest digging some more, write a test case with DB test table and digging around to find out more about the 'edges' to this crash. If you can get a small test case + script for CREATE TABLE, etc. then dev. can look at it. Again, this might be a Unicode issue, but of course TD should not crash.

[nfg]

Hello,

there is a small client application and the create table statement in the attachment file.

We have also changed the column from char to varchar, but the result is the same, Team Developer Crash.

The SqlPrepare command works fine, the crash is during the SqlExecute command.

I hope this will help you to analyse the problem

[Jeff Luther]

Thanks, and good test case! I'm trying to find/get a connection for INFORMIX at the moment, but I looked through your test case and at least let me verify or ask:

** This line I commented:

Set strNachname = '123456789012345678901234567890123456789012345' ! 45 chars - DB column: nachname CHAR(60)

looks to be the culprit. INSERT OK, but on select you get a crash. While this line fetches OK:

Set strNachname = '123456789012345678901' ! 21 chars - DB column nachname: CHAR(60)

I'm real curious about the 'edge' -- does INSERT/SELECT where strNachname = 30 chars work OK? But 31 would fail? Or if 30 crashes, does 29 work OK?

** "nachname CHAR(60)" -- 2nd thing that caught my eye is the CHAR datatype. Have you tried changing that to VARCHAR to see if the crash still occurs?

SqlExecute() should not crash, of course, but narrowing down the when it crashes can go a long way for development to finding out the 'why.'

Meanwhile, I'll work on getting an INFORMIX DB to test with.

[nfg]

Thanks for your answer:

We have changed the "nachname" column to VARCHAR(60), but the problem still exists.

You are right the 45 chars select crashed and the 21 chars select works.

We have tried several string length and its not exactly the half of the defined char length.
Char(60) works with stringlength of 31 chars and crashes on 32 chars, but a char(80) column
works correctly until 45 chars and crashes at 46 chars.

We have investigated the informix connect logfile (SQLIDEBUG switch) and could see the resultset
of the select statement which was correct (all 45 chars of the database column nachname).

I guess that its a pointer/memory problem, because if only the "nachname" column is in the select statement
it works. It will also work if you change the order of the columns in the SELECT statement and the "nachname" column
is the last one in the SELECT statement.

We have only installed ODBC and informix connectivity of Team Developer deployment an nothing else, maybe
its important.

[nfg]

Any new inspirations?

We need a solution within the next few (2 or 3) weeks.

[nfg]

Some additional information:

During some more testing the app crashed in the sqlifx32.dll at address 0x037940a8.

See the attached DrWatson log file and look there for the word "FAULT".

[Jeff Luther]

Dr. Watson logs aren't helpful, for me anyway. I do see the DLL in the right place: your TD v6 install folder.

I have an email out internally to find an Informix server, so cannot run your earlier test. It's out of date now as you said you changed CHAR to VARCHAR, Ithink. Also, you have SqlPrepareAndExecute calls. Didn't you change that to separate prep and exe?

I would suggest a really simple, simple test. 2 columns, ID for referencing and one nachname VARCHAR(60) column.

Questions:
* Is your INFORMIX on the TD v6 compatibility chart?

* "We are using unicode and the database is set on UTF8" -- so yes, the DB is configured for Unicode? I guess the test is that you can insert that short string, then fetch it back OK. If so, then sounds like Unicode both sides, front- and back-end.

* Running in v6 Win32?

* What about the Informix client? 10 or 11

* Once again, with a VARCHAR(60) column, INSERTing 30 is OK? 31 is not? Or is 29 OK but 30 and larger are not?

You are right the 45 chars select crashed and the 21 chars select works.

Problem with that test is that is doesn't test the 'edge,' since 21 isn't half of or one-off from half of 60. And 45 is way over half. If it's an edge issue, then sure sounds like Unicode, but if you have a Unicode DB and TD's Unicode, what's the problem? (I know, that's the question.)

I just looked in our TD defects DB and don't see anything for informix that's as fundamental as a crash in SqlExecute for a string. I'm stumped.

I wonder if this utility: https://www.dependencywalker.com/
might prove beneficial to you. It will list everthing that your TD v6 app is loading in. Can't think what would cause this crash, but if somehow an earlier sqlXXX.DLL file from earlier TD was getting in, it might be possible that this is it.

[nfg]

Dr. Watson logs aren't helpful, for me anyway.

Perhaps not helpful for you, but (one of) the programmer(s) of the sqlifx32.dll should be able to look what the code of that DLL does do at the position (address) given in the DrWatson log. And that was my hope: that a programmer of the DLL would have a look at this code position and the surroundings. Perhaps the reason for the crash would become obvious very quickly then.

. . . you changed CHAR to VARCHAR

We tried to change to VARCHAR, but as that made no difference regarding the crash, we remained with CHAR.

Also, you have SqlPrepareAndExecute calls. Didn't you change that to separate prep and exe?

Yes, we separated prepare and execute. The crash always occurs in the SqlExecute statement and only when reading from the DB, not when writing to it.

I wonder if this utility: https://www.dependencywalker.com/ might prove beneficial to you.

We already used DependencyWalker to proove that the correct versions of the required DLLs are loaded, esp. the sqlifx32.dll.

What about the Informix client? 10 or 11?

We use version 11.70.

Once again, with a VARCHAR(60) column, INSERTing 30 is OK? 31 is not? Or is 29 OK but 30 and larger are not?

I tested with different field sizes. The results:

Field size in the DB: 64.
Up to 35 characters OK, crash with 36 characters.

Field size in the DB: 60.
Up to 31 characters OK, crash with 32 characters.

Field size in the DB: 40.
Up to 23 characters OK, crash with 24 characters.

Field size in the DB: 10.
Up to 7 characters OK, crash with 8 characters.

The only common pattern I see is, that the crash always happens with the first multiple of 4 characters, which is greater than the half field size.

The problem is not with a certain column of the DB table, but it becomes noticeable (by the crash) when the field containing too many chars is followed by one (or more) other fields in the SELECT statement.

I just looked in our TD defects DB and don't see anything for informix that's as fundamental as a crash in SqlExecute for a string.

How many users of TD 6.0 employ an Informix DB configured for Unicode? Did you hear of one at least (besides us)?

[Jeff Luther]

Perhaps not helpful for you, but (one of) the programmer(s) of the sqlifx32.dll should be able to look what the code of that DLL does do at the position (address) given in the DrWatson log.... Perhaps the reason for the crash would become obvious
That can be helpful to find out where, but now necessarily why or how. Besides, dev. cannot debug a log file to find the problem. That's why a small test case is important.

How many users of TD 6.0 employ an Informix DB configured for Unicode? Did you hear of one at least (besides us)?

I don't know. And you might get a rough idea by searching this forum for Informix and see what you get. I see threads going all the way back to 2008. And yes, I'm sure at least one other user connects to INFORMIX.

I did just speak with our dev. router guy who will look at the test case and I'll include the LOG file. I'll enter this into our TD defects database.
New defect: TD-17330

QUESTIONS:
** Is v6 SP4 the first version you've used and found this problem?

** Did an earlier TD, either earlier v6 or another version like v5.2 run OK?

You wrote: "We inserted a string into database "123456789012345678901234567890123456789012345" with our team developer
application, this works."
** So, it's a SELECT where this issue is only? INSERT/UPDATE OK.

The can help development narrow the problem and determine whether this is a regression or not.

[nfg]

We jumped from TD 4.2 to TD 6.0 SP4, skipping the versions in between. Now we are using TD 6.0 SP5.

Together with porting our app from TD 4.2 to 6.0 we switched to Unicode. So we first experienced the problem with TD 6.0 in combination with Unicode. I can't tell if the same problem would exist with 5.2 and I think there would be no problem without Unicode.

Yes, the issue is only with SELECT while INSERT/UPDATE are OK.

[Jeff Luther]

I can't tell if the same problem would exist with 5.2 and I think there would be no problem without Unicode

Then you likely would have found this problem with v5.2. Since v5.1 TD is in Unicode.