How to concatenate Binaries

Discussion forum about all things Gupta, OpenText and the community.
thomas.uttendorfer
Site Admin
Site Admin
Germany
Posts: 138
Joined: 05 Mar 2017, 17:19
Location: Munich Germany

How to concatenate Binaries

Post by thomas.uttendorfer » 24 May 2018, 15:14

Hi,
does anybody know how to concatenate two Binary-variables?
Use case is:
I use Win32-Api-function InternetReadFile(..) where I get portions of data as binary.
How do I put them together?

Code: Select all

Binary: binPortion1
Binary: binPortion2
Binary: binData

binData = binPortion1 || binPortion2
results in a compile-error.
Thomas Uttendorfer
[ frevel & fey ] Software-System GmbH
https://thomasuttendorfer.wordpress.com/

Jeff Luther
Site Admin
Site Admin
United States of America
Posts: 2070
Joined: 04 Mar 2017, 18:34
Location: Palm Springs, California

How to concatendate Binaries

Post by Jeff Luther » 24 May 2018, 17:37

Thomas: I have not done this before, but my first thought is to use the functionality of cstructl.apl to do what you need. Some of the functions in that APL that catch my eye for doing what you want might be these. Note that this is just a guess but write a small test appl. is how I would attack the problem:
CStructAllocFarMem
CStructCopyFromFarMem
CStructCopyToFarMem
CStructPutByte or CStructCopyBuffer ?
+ SAL functions SalBinaryToString & SalStringToBinary

Given bin1 and bin2 I see the possibility of 2 approaches:
(need to work in string so use SAL functions for converting binary <--> string)

1 - increase buffer length of bin1, return ptr to the first byte after bin and copy bin2 beginning at that offset in bin1

2 - Get buffer lens of bin1 and bin2, alloc. far mem to hold both binaries, copy both binaries
Jeff Luther @ PC Design
Palm Springs, California
TD info. & samples: http://www.jeffluther.net/TD/

Dave Rabelink
Founder/Site Admin
Founder/Site Admin
Netherlands
Posts: 1088
Joined: 24 Feb 2017, 09:12
Location: Gouda, The Netherlands

How to concatendate Binaries

Post by Dave Rabelink » 24 May 2018, 18:07

As addition to Jeffs answer.

The cstructl.apl library is the way to go, but by default the cstruct functions operate on strings and not on binary datatype.

The binary datatype is a bit awkward as a lot of Sal/VT and the cstruct functions can not be used on this datatype.
It is easier to stick with strings.

The second parameter of the Win32 API function InternetReadFile should be defined as

Receive String: LPVOID

So then, when called you will have the data in a string variable.

It also depends on the actual contents of the data. is it text (ANSI, UNICODE) or is it really binary data (image, soundfile etc)?

When it is binary data which probably could contain bytes having value 0 (zero), you are unable to concatenate the data using ||.

These steps should be taken:

1) Calculate the total size in bytes needed for all the pieces put together
2) Create a memory buffer buffer having this size using CStructAllocFarMem
3) Copy each part to the memory buffer. The first part will be copied to position 0 in the memory buffer. The second part at the offset given by the size of the first piece. Using CStructCopyToFarMem.
4) When all parts are placed in the buffer, copy all the data back into a local string variable.

When you insist in using binary datatype, you must be aware that you can not use SalStringToBinary and SalBinaryToString.
These functions actually encode the contents of the binary as ANSI/UNICODE which will destroy your data.

To convert between binary and string and visaversa, read this article:

http://wiki.tdcommunity.net/index.php/B ... ng_casting
Regards,
Dave Rabelink

Image
Articles and information on Team Developer Tips & Tricks Wiki
Download samples, documents and resources from TD Sample Vault
Videos on TDWiki YouTube Channel

thomas.uttendorfer
Site Admin
Site Admin
Germany
Posts: 138
Joined: 05 Mar 2017, 17:19
Location: Munich Germany

How to concatenate Binaries

Post by thomas.uttendorfer » 25 May 2018, 07:42

Hi Dave,
just for your information on:

Code: Select all

InternetReadFile
Receive String: LPVOID
I first used datatype String too. But when I received an uneven amount of bytes (it was data encoded as UTF8) I got a problem when decoding the received string.
There was always one character more than expected at the end.

I think the reason is that Gupta thinks that strings always contain an even amount of bytes.
E.g.: let's say I received 7 bytes. When decoding this with e.g SalStringToWideChar(sIn, sOut, ENC_UTF8) the function uses 8 Bytes for decoding.
When I switched to datatype binary the problem has gone.

Regards Thomas
Thomas Uttendorfer
[ frevel & fey ] Software-System GmbH
https://thomasuttendorfer.wordpress.com/

Dave Rabelink
Founder/Site Admin
Founder/Site Admin
Netherlands
Posts: 1088
Joined: 24 Feb 2017, 09:12
Location: Gouda, The Netherlands

How to concatenate Binaries

Post by Dave Rabelink » 25 May 2018, 08:49

So, to be sure. The original data you expect to receive using the external function is UTF8?

(UTF-8 is a variable width character encoding format, from 1 to 4 8bit bytes )

A string in TD can hold any size data.

You must be aware how the data is ending (NULL chars).
What I mostly do when I'm not sure how the data is packed into a memory buffer (string) I display it in a HEX editor.
There you can inspect all bytes separately and see how the data is structured and what the ending of the buffer is.

Having the info and the sample in this article:

http://wiki.tdcommunity.net/index.php/W ... ryToString

What I do is this:

In TD, when a string has data to inspect, I pass the string to PALCryptBinaryToString with flag CRYPT_STRING_HEXASCIIADDR.
It will return you a formatted string like HEX editors do, showing all the bytes with their addresses.
(see screenshot in the article).

Inspecting the result gives you clues what the data represents.

So, when getting the data from the external function, display the data using the PAL function immediately (so before manipulating/encoding it).
See what the actual data is.
Then encode it using SalStringToWideChar.
Inspect the resulting data using the PAL function to see what TD has produced and see where it fails and probably will get you the clue what to do with the data before encoding it.
It could be you need to subtract or add one byte (probably NULL byte) before encoding it.
Regards,
Dave Rabelink

Image
Articles and information on Team Developer Tips & Tricks Wiki
Download samples, documents and resources from TD Sample Vault
Videos on TDWiki YouTube Channel

Jeff Luther
Site Admin
Site Admin
United States of America
Posts: 2070
Joined: 04 Mar 2017, 18:34
Location: Palm Springs, California

How to concatenate Binaries

Post by Jeff Luther » 25 May 2018, 18:05

Thanks for your additional thoughts and WIKI link, Dave!

Thomas -- How about writing and attaching as a ZIP a small test case for this issue this Forum thread is discussing? That will help all of us see how Dave's suggestions and your coding can show how this issue is resolved for you. TIA!
Jeff Luther @ PC Design
Palm Springs, California
TD info. & samples: http://www.jeffluther.net/TD/

Return to “General Discussion”

Who is online

Users browsing this forum: [Ccbot] and 0 guests