String handling in 5.1 ?

[RogerArmstrong]

Having just installed 5.1 via CD - We are having a few problems and these seem to be related to inconsistent treatment of data fields and data field lengths. Can anybody help ???? explain this or am I missing something so obvious somewhere !



As background we have an app that reads an ini file, logs onto a database and also saves some parameters in global memory for other modules to use.



The following data items are retrieved from an ini file 'user' 'password'. The following data items are picked up from a screen dropdown 'dbprefix' and 'db suffix' .



The actual values involved are 'sysadm' 'sysadm' 'am' 'lib' .



So we are trying to connect to a database called amlib with a username and password of sysadm.



The user and password fields are retrieved using the function SalGetProfileString and these are retrieved correctly. At this stage each of the fields has a length of 6 using SalStrLength.

These two values are then stored in global memory as below but unless the length is multiplied by 2 only 3 chars are stored



Set nLength = SalStrLength ( sValue )

!** RAFIX 6.0 !!! line below added to actually get 6 chars saved away in memory

Set nLength = nLength * 2

If nLength

Set nLength = nLength + 1

!

! Compacts the buffer if it is full

If nOffset + nLength > SM_StrBuffer

Call MutexUnlock ( hStrMutex )

Set bRetVal = DLLStrCompact ( nType, sValue )

Else

If nOffset = 0

Set nOffset = 804

Call CStructPutLong ( sBuffer, 0, nOffset + nLength )

Call CStructCopyStrToMem ( pStrShared, 0, sBuffer, 0, 4 )

!

! Inserts the offset from the beginning of the buffer to the string

Call CStructPutLong ( sBuffer, 0, nOffset )

Call CStructPutLong ( sBuffer, 4, nLength )

Call CStructCopyStrToMem ( pStrShared, ( nType * 8 ) + 4, sBuffer, 0, 8 )

!

! Inserts the string at the specified offset

Call CStructCopyStrToMem ( pStrShared, nOffset, sValue, 0, nLength )

Call MutexUnlock ( hStrMutex )

Set bRetVal = TRUE





** eg Unless I have the line after RAFIX eg which multiples the length of the fields by 2 - then only 3 of the 6 characters get saved away. I presume this is because the cstructcopystrtomem thinks the fields are unicode or alternatively only works in unicode ? and therefore I need to give it a length of 13 to get 6 chars stored in memory.

I then thought - this is an easy fix - just make the ini file unicode and maybe the lengths will tie up - but no luck - salstrlength still returns a length of 6.



That is the first problem which i can get around.

---------------------------



The second problem i cant. The data saved in memory cannot be easily recovered. When recovering the saved data later using a call to cstructgetstring it doesnt matter what length you use in cstructgetstr as only 1 character is ever returned ! even tho' there are 6 chars there which can be obtained anyway using salstrleftx.



eg

Call SalStrSetBufferLength ( sBuffer, 8 )

Call CStructCopyMemToStr ( sBuffer, 0, pStrShared, ( nType * 8 ) + 4, 8 )

Set nLength = CStructGetLong ( sBuffer, 4 )

If nLength

** eg 13

Set nOffset = CStructGetLong ( sBuffer, 0 )

Call SalStrSetBufferLength ( sBuffer, nLength )

Call CStructCopyMemToStr ( sBuffer, 0, pStrShared, nOffset, nLength )

Call SalStrSetBufferLength ( sString, nLength )

! Call CStructGetString ( sBuffer, 0, nLength, sString )

! above doenst seem to work

Set sString = SalStrLeftX( sBuffer,nLength )

Set sValue = sString





So CStructGetString only ever returns 1 char

-------------------------



My final problem is the database suffix and prefix. Instead of being saved as AM and LIB , these are saved in memory as A and LI which I can get around in the same way as problem 1 above and then use the string left to retrieve so I can eventually end up with the values AM and LIB being restored from shared memory.

Note that these values do not originate from a file. They are actually set based from a screen drop field eg the user chooses the amlib database.

However a further complication is that these fields do not seem to be able to concantenated . eg when the user name, password , prefix and suffix are re-established a log in is then attempted as below



If DBSqlConnect ( hSqlConnect, sUserName, sPassword,

sPrefix || sSuffix, sagLibFileName[ngAppFileIndex], nError )





note sPrefix || sSuffix - even though these contain 'AM' and 'LIB' the results of the concatenation is just AM so the attempt to login fails. If before the call I set the length of the two fields to 2 and 3 or 5 and 7 the concatention also fails . These are just string fields in the call to the function. If I set the fields myself to AM and Lib on a breakpoint , the concatenation works ..



Any assistance would be gratefully received ......................





Roger Armstrong

[markus.essmayr]

Roger,



everything you describe seems to result from TD 5.1 beeing UNICODE.

That means, that every character in strings is not stored as one but as two bytes.

I didn't take a close look at the CStruct-functions, but it seems, that even they should work with "strings", work byte-based.



This will explain your problems.



For example, you wrote, that your string has 6 characters, but only the first 3 are stored.

Thats correct, but you have to remember, that your string consists of 12 bytes, each 2 bytes forming a character. So if you call the CStruct-function to store the string, you pass 6 as length, and it seems, that this makes 6 bytes beeing copied. And this in turn represent 3 characters.



The second thing, that you can only get the first character returned, when you read a string, looks like to be explained by this:

In UNICODE, every character is formed of 2 bytes. Letters are defined in the basic 7 bit area of the ANSI codepage, so it happens, that in UNICODE, the charactercode of the letter is stored in the first byte of the UNICODE-character, and the second byte is set to zero.

When reading this string with a method, that doesn't use UNICODE results in the fact, that it reads the first byte, the letter, and then the second byte which happens to be zero meaning '\0' meaning <end-of-string>.

BTW, end-of-string in UNICODE is formed of two zero-bytes.

So, the function believes that the string ends after the first character, so you only get one character returned.



Hmm ... i don't know, if this was in any way helpful to you!?



Maybe you should verify, which datatype is specified in the CStruct-functions declaration in "External Functions".

If the strings are defined as LPCWSTR/LPWSTR, you could try to change that to LPCSTR/LPSTR to force TD to convert from UNICODE to ANSI befor calling the function.



If this works, it means, that the CStruct-library strci51.dll is NOT UNICODE-aware which Unify should fix.



Max

[RogerArmstrong]

Hi Max and thanks for the reply.



Yes I assumed the unicode change would be the cause of these problems however I would have thought that the functions salstrleft , salstrlength and the pipe || should be able to handle unicode automatically. The fact that unicode cant be piped or concatenated and the cstrsctgetstring doesnt seem to be able to handle unicode properly either is a case for worry. If td treats every string as unicode then this is great but we still want to use existing functions otherwise we face a huge redevelopment cost



Roger

[JBoston]

At the risk of burying this post, I'll send it as a follow-up to RA's string handling question because it appears related.

We've relied on using a nifty pair of functions StringMessageSend and StringMessageReceive posted online by Christian Astor many moons ago. However, the same types of Unicode-related problems appear to occur when our application is ported to TD 5.1.

Code: Select all

Function: StringMessageSend

   Description: author:    C. Astor /adapted by tl

         version:    1.00

         

   ...         

   Returns

   Parameters

      Window Handle: whDestination

      String: sText

      Number: nMessageNumber

   Static Variables

      Number: nAddrString

      String: sBufferString

   Local variables

      String: sBuffer

      Number: nAddr

      Number: nStrLength

   Actions

      ! !!!! allocate 12-byte parameter buffer used in VisSendMsgString cal

      Call SalStrSetBufferLength( sBuffer, 12 )

      

      ! !!!! put 0 in first 4 bytes???

      Call CStructPutLong( sBuffer, 0 , 0 )

       

      ! !!!! put length of passed text in next 4 bytes  - NOTE: multiply by 2 since 5.1 now uses Unicode

      Set nStrLength = ( SalStrLength(sText) * 2  ) + 1

                                Call CStructPutLong( sBuffer, 4 , nStrLength )

    

      ! !!!! allocate 255-byte buffer to process text

      Call SalStrSetBufferLength( sBufferString, 255 )

      

      ! !!!!! put passed string into this buffer

      Call CStructPutString( sBufferString, 0, nStrLength, sText )

      

      ! !!!! get a handle to 255 bytes of far memory

      Set nAddrString = CStructAllocFarMem( 255 )

      

      ! !!!! copy the string in our text buffer to this memory address

      Call CStructCopyToFarMem( nAddrString, sBufferString, nStrLength )

      

      ! !!!! put the memory address pointer in final 4 bytes of our 12-byte parameter buffer

      Call CStructPutLong( sBuffer, 8,  nAddrString )

      

      ! !!!! send WM_COPY_DATA message passing 12-byte parameter buffer as lParam

      Call VisSendMsgString( whDestination, nMessageNumber, SalWindowHandleToNumber( hWndForm ), sBuffer )

      

      ! !!!! free the far memory handle

      Call CStructFreeFarMem( nAddrString )

Code: Select all

Function: StringMessageReceive

   Description: author:    C. Astor /adapted by tl

         version:    1.00

         

         desc:

         receives the strings on WM_COPY_DATA, sent by StringMessageSend(..).

         

         example:

         On WM_COPY_DATA

            Set sText =  StringMessageReceive(wParam,lParam)

   Returns

      String:

   Parameters

      Number: passed_wParam

      Number: passed_lParam

   Static Variables

   Local variables

      String: sText

      String: sBuffer

      Number: nAddr

      Number: nAddrString

      Number: nStringLength

   Actions

      ! !!!!! allocate a 12-byte string buffer

      Call SalStrSetBufferLength( sBuffer, 12 )

      

      ! !!!! copy the lParam into this buffer

      Call CStructCopyFromFarMem( passed_lParam, sBuffer, 12 )

      

      ! !!!! extract string length from bytes 5 - 8

      Set nStringLength = CStructGetLong( sBuffer, 4 )

      

      ! !!! extract memory address of string from bytes 9 - 12

      Set nAddrString = CStructGetLong( sBuffer, 8 )

      

      ! !!!! initialize local string variable - NOTE: this is important!

      Set sText = ""

      

      ! !!!!! allocate 255-byte buffer to hold longest possible string we might have received

      Call SalStrSetBufferLength( sText,  255 )

      

      ! !!!! copy string from far memory into our local String variable

      Call CStructCopyFromFarMem( nAddrString, sText, nStringLength  )

      

      Return sText

We've introduced the highlighted change into the StringMessageSend() method in an attempt to get this to work, but it appears that there must be more to this than just that modification since the receive method still seems to be unable to get back to the original string.

Any help and/or even heckling would be appreciated. Thank you.

[Jean-Marc Gemperle]

Hi,

There are known issues with concatenation that are fixed in the internal build we have, but I really like to test that to confirm it. I understand what you do with the CSTRUCT but if you have handy a repro case, I will test asap that issue and let you know, if not I will try to reproduce your issue.

Jean-Marc

[JBoston]

Thanks Jean-Marc.

It looks like in this case the problem was solved by replacing:

Call CStructPutString( sBufferString, 0, nStrLength, sText )

in the "send" function with:

Call CStructPutStringW( sBufferString, 0, nStrLength, sText )

which available in strci51.dll

[I'll leave it you and/or others to explain why this would work, but it appears that this was all it took to get these old functions to convert to 5.1]

[RogerArmstrong]

I have revisited this again after the latest patch - but I still have problems with piping fields . Piping any unicode fields just does not work , though normal fields do. eg two fields containing AM and Lib with lengths of 5 and 7 will not pipe . If I create the fields with lengths of 2 and 3 they will pipe together Does anyone else have this problem ?

[Jean-Marc Gemperle]

Hi,

Not sure what is left to answer on that post. Well when using CSTRUCT surely you need to be realy carfull, there is a Wide string function in cstruct.apl for string to be used in case and that seems to have solved the migration issue here. With the details in the post it is hard for me to figure out why that is.

Personally I feel, that is my opinion and I might be wrong, maybe other can share some words on this : If you use CSTRUCT in TD and migrate to TD 5.1 I would tend to believe the application does not really need to be migrated (widely modified). If string are used and you get a result from CSTRUCT that result is ANSI and should be proper (without modification of original source) except converting to UNICODE using SalStrToWideChar() at the end... Well it might of course depend but this is what I saw couple times. I have an example that I will post that retrieve version information from a WINAPI using relatively complex structue, there is different way to migrate it, initially I started the complex way diging in it, that did not require me to convert the final result to UNICODE, untill I figured out with the original source, that the garbage I had was in fact correct, just ANSI to be converted to UNICODE. So there is different way in migrating CSTRUCT depending the scenario I believe.... I myself are partisant of the least efforts but that's me.

As for the issue regarding concatenation I'm not sure I understand. You say you can't concatenate AM + LIB contents to AMLIB in a DATAFIELD DF3 when DF1, DF2 MAXSIZE ( in the property ) is respectively non Default, 5 and 7 in size? because I don't get that problem with SP1. Also if DF3 is of MAXSIZE 5 no problem

Please provide extra information, as we are busy with SP2 and that sort of problem are critical and need to be catched asap.

Thanks

JM

[RogerArmstrong]

HI JM

sorry I should have come back to this - the pipe seems to be working now. Once I changed the code that gets these fields to use CStructGetStringW as suggested then presumably the fields now had the correct lengths and now pipe ok.


cheers Roger