TD 64bit and WinAPI32 declarations

Post your tools and samples or ask for them.
Igor Ivanovic
Site Admin
Site Admin
Croatia
Posts: 1282
Joined: 05 Mar 2017, 12:37
Location: Zagreb, Croatia

TD 64bit and WinAPI32 declarations

Post by Igor Ivanovic » 24 Jun 2020, 13:58

Hi,

I use the WinApi32 declarations a lot but just have found an error in WinApi32_Functions_W.apl ADVAPI32.dll registry function declarations.
Some of them don't work under TD64bit, RegQueryValueExW for example.
There is also a sample (in TDSamples) Dave made for registry manipulation (currently at version 2.1) and the declarations found there work for both TD32 and TD64.

I don't have the TD 5.x anymore, required to make the changes in the source files.
The fix is simple, I removed the registry functions from WinApi32_Functions_W.apl and just included the registryW.apl from Registy_v21.zip
Igor Ivanovic
Image

Dave Rabelink
Founder/Site Admin
Founder/Site Admin
Netherlands
Posts: 2956
Joined: 24 Feb 2017, 09:12
Location: Gouda, The Netherlands

Re: TD 64bit and WinAPI32 declarations

Post by Dave Rabelink » 26 Jun 2020, 13:19

Hi Igor,

Thanks for pointing this out. So I started to adapt the Win32 archive library.
But I found another issue which I do not understand. It could be a TD x64 defect or I'm having a black-out.

The Registry_v21.zip archive contains this API function:

RegCreateKeyExW

The first parameter is

Number: HANDLE

The API function is used in the class cBTRegistry, function OpenKey().

As it is now, the following code does not work on TD x64 within the OpenKey() function:

Code: Select all

Set m_ErrorCode = RegCreateKeyExW( GetBaseKey(Relative),  Key, 0, STRING_Null, REG_OPTION_NON_VOLATILE,  KEY_ALL_ACCESS | m_AlternateView, STRING_Null, TempKey, Disposition )
It always returns ErrorCode = 5

But it should work as all API parameters are of the correct type. My understanding is that we need to use HANDLE as parameter type.

I played around and changed the first parameter of RegCreateKeyExW from HANDLE to

Number: ULONGLONG

After this change, the function returns m_ErrorCode = 0, which is correct.
(The API can create the registry element or can open it when present)

It seems TD does not process the HANDLE parameter correctly. It only works with ULONGLONG.

To me this is a TD defect. But maybe I'm mistaken.
Regards,
Dave Rabelink

Image
Articles and information on Team Developer Tips & Tricks Wiki
Download samples, documents and resources from TD Sample Vault
Videos on TDWiki YouTube Channel

pascmeul
Belgium
Posts: 16
Joined: 09 Oct 2017, 08:50
Location: Oostkamp, Belgium

Re: TD 64bit and WinAPI32 declarations

Post by pascmeul » 29 Jun 2020, 13:25

Hi,

we have mayby a simular issue with fcSMTPMail.

We get no port or ip-adres from Wsock32.dl and CTD7.3 64bit:
- Set nPort = sock.GetServByName('mail', STRING_Null) => always = 0
- Set sHost = sock.GetHostByName(i_sSMTPServer) => always = STRING_Null
We can solve the base64-enoding-issue by using an other dll.

This class works fine in CTD6.3.9 (32 bit).
WSock32orig.zip
You do not have the required permissions to view the files attached to this post.

Dave Rabelink
Founder/Site Admin
Founder/Site Admin
Netherlands
Posts: 2956
Joined: 24 Feb 2017, 09:12
Location: Gouda, The Netherlands

Re: TD 64bit and WinAPI32 declarations

Post by Dave Rabelink » 30 Jun 2020, 07:41

pascmeul wrote:
29 Jun 2020, 13:25
We get no port or ip-adres from Wsock32.dl and CTD7.3 64bit:
- Set nPort = sock.GetServByName('mail', STRING_Null) => always = 0
- Set sHost = sock.GetHostByName(i_sSMTPServer) => always = STRING_Null
For all API functions using a pointer, use LPVOID. Both on x86 and x64 it will take the right datalength (32 vs 64 bit).

So for wsock functions, I see many API functions using ULONG for pointers, which is always 32 bit.
Change them to LPVOID.

eg

Code: Select all

Function: gethostbyname
	Description: see MSDN documentation
	Export Ordinal: 0
	Returns
		! memory pointer to struct hostent or zero (= error)
		Number: LPVOID
	Parameters
		! name of the host
		String: LPCSTR
Regards,
Dave Rabelink

Image
Articles and information on Team Developer Tips & Tricks Wiki
Download samples, documents and resources from TD Sample Vault
Videos on TDWiki YouTube Channel

pascmeul
Belgium
Posts: 16
Joined: 09 Oct 2017, 08:50
Location: Oostkamp, Belgium

Re: TD 64bit and WinAPI32 declarations

Post by pascmeul » 01 Jul 2020, 09:01

Hi,

thanks
sock.GetServByName is fixed, but not sock.GetHostByName.

Probably there are other issues with references of wsock32.dll, but I does not know enough of this.
Who can help me?

Dave Rabelink
Founder/Site Admin
Founder/Site Admin
Netherlands
Posts: 2956
Joined: 24 Feb 2017, 09:12
Location: Gouda, The Netherlands

Re: TD 64bit and WinAPI32 declarations

Post by Dave Rabelink » 01 Jul 2020, 11:02

pascmeul wrote:
01 Jul 2020, 09:01
sock.GetServByName is fixed, but not sock.GetHostByName.

Probably there are other issues with references of wsock32.dll, but I does not know enough of this.
Who can help me?
Difficulty is that we need a working sample to be used to track the specific issues.
As for GetServByName being fixed, I doubt it. Could be that the result accidentally is correct.

But maybe I can give you a direction where to look and how to detect issues on API functions like this, specially here on x64.

First you have to read this article which explains a "feature" of structures.
There I try to explain how structures are presented in memory.

https://wiki.tdcommunity.net/index.php/ ... 864_bit.29


So as an example, I will try to see if sock.GetHostByName is correct on x64.
Please follow the steps to investigate.

This is called in the function GetHostByName :

Code: Select all

Set nMem = gethostbyname(p_sName)
Having a look at the documentation of gethostbyname:

https://docs.microsoft.com/en-us/window ... hostbyname

There it says:
hostent *WSAAPI gethostbyname(
const char *name
);
So it has a parameter, a pointer to const char.
And returns a pointer to hostent structure.

Have a look at how this function is declared in TD:

Code: Select all

Function: gethostbyname
	Description: see MSDN documentation
	Export Ordinal: 0
	Returns
		! memory pointer to struct hostent or zero (= error)
		Number: ULONG
	Parameters
		! name of the host
		String: LPCSTR
The parameter is indeed declared as LPCSTR, which means a long pointer to a const string.
(correct)

But it returns a Number: ULONG.
A ULONG is a 32 bit number. Looking at the documentation it returns a pointer (to hostent).
A pointer on x86 is 32 bit long. So a ULONG will do the trick, but it is not really correct.
A pointer on x64 is 64 bit long.

This means that on x86 this function will work, but on x64 the function will fail. TD will treat the return value as a 32 bit value and not a 64 bit value.

So to fix it, we need to specify the correct type for the return value which is:

Code: Select all

Function: gethostbyname
	Description: see MSDN documentation
	Export Ordinal: 0
	Returns
		! memory pointer to struct hostent or zero (= error)
		Number: LPVOID
	Parameters
		! name of the host
		String: LPCSTR
LPVOID is a pointer.

So after this change, the function should work on both x86 and x64.

But we have to continue investigating what happens afterwards:

Code: Select all

If bOk
	Set nMem = gethostbyname(p_sName)
	Set bOk = (nMem != 0)
If bOk
	Set bOk = SalStrSetBufferLength(sBuffer, 16)
If bOk
	Set bOk = CStructCopyFromFarMem(nMem, sBuffer, 16)
If bOk
	Set nAddrType = CStructGetWord(sBuffer, 8)
	Set nLen = CStructGetWord(sBuffer, 10)
	Set nMem = CStructGetLong(sBuffer, 12)
	Set bOk = (nAddrType = AF_INET) AND (nLen = 4) AND (nMem != 0)
When looking at this functionality, it does roughly this:

- Get the memory pointer to a structure
- Copy the data from that memory structure to a local string variable using the total size of that structure
- Get the members of the structure as separate values

So the first action is now correct.
The second action, copy the data, lets check that if it is ok.

On the documentation you have a link to the documentation of the hostent structure:

https://docs.microsoft.com/en-us/window ... ck-hostent

The description of this structure is:

Code: Select all

typedef struct hostent {
  char  *h_name;
  char  **h_aliases;
  short h_addrtype;
  short h_length;
  char  **h_addr_list;
} HOSTENT, *PHOSTENT, *LPHOSTENT;
It describes all members which are placed in a memory block.
Each member has a size (in bytes).
Counting all sizes together will result in the total size of this memory block.

So lets do that. First for x86 (where pointers are 32 bit)

Code: Select all

  char  *h_name = 4
  char  **h_aliases = 4
  short h_addrtype = 2
  short h_length = 2
  char  **h_addr_list = 4
  
(pointers are 4 bytes, short is 2 bytes)

So in total we get : 16 bytes

Is it correct in the TD code, lets see:

Code: Select all

Set bOk = SalStrSetBufferLength(sBuffer, 16)
Set bOk = CStructCopyFromFarMem(nMem, sBuffer, 16)
Yes. That is correct. On x86 we have to copy 16 bytes

But now on x64. Because the pointer sizes are 64 bit instead of 32 bit, we need to recalculate the memory block:

Code: Select all

  char  *h_name = 8
  char  **h_aliases = 8
  short h_addrtype = 2
  short h_length = 2
  char  **h_addr_list = 8
  
The total sum is now : 28 bytes


BUT BUT BUT:

Be aware of the article about memory of structures on TDWiki I linked to:
The rule is, a pointer/handle MUST start on a 8 byte boundary on x64
(on x86 it MUST start on a 4 byte boundary).

Lets check the boundaries then on x64:

Code: Select all

  char  *h_name = boundary starts at 0 (zero)
  char  **h_aliases = boundary starts at 8
  short h_addrtype = boundary starts at 16
  short h_length = boundary starts at 18
  char  **h_addr_list = boundary starts at 20
  
(you start at zero, take the size of the member and this will be the boundary for the next item. Then take the next item and use the size and add it up to its own boundary)

So each member of the memory block have a position which can be written as:

0->8->16->18->20

Are all pointer members on a 8 byte boundary?

The first pointer starts at 0, which is correct
The second pointer starts at 8, yes on correct boundary
The last pointer starts at 20, this one is wrong. 20 is not a multiple of 8.

So you MUST place the last member on 24 which is a multiple of 8 after 20.
You need to place a "dummy" member between the last and second last member. And this dummy member should be a 4 byte value.
The value is not important. It is just a padding member which is not used by the API.

So this will be:

Code: Select all

  char  *h_name = boundary starts at 0 (zero)
  char  **h_aliases = boundary starts at 8
  short h_addrtype = boundary starts at 16
  short h_length = boundary starts at 18
  LONG dummy = boundary starts at 20
  char  **h_addr_list = boundary starts at 24
  
MInd the extra dummy member which it have set to LONG which is 4 bytes.

Now the size will be not 28 but because of this padding has an extra 4 bytes = 32 bytes.

So the code in TD has to be changed:

Code: Select all

! This works on x86
Set bOk = SalStrSetBufferLength(sBuffer, 16)
Set bOk = CStructCopyFromFarMem(nMem, sBuffer, 16)
and for x64:

Code: Select all

! This works on x64
Set bOk = SalStrSetBufferLength(sBuffer, 32)
Set bOk = CStructCopyFromFarMem(nMem, sBuffer, 32)

Now the memory is copied, the members are read from it in TD here:

Code: Select all

Set nAddrType = CStructGetWord(sBuffer, 8)
Set nLen = CStructGetWord(sBuffer, 10)
Set nMem = CStructGetLong(sBuffer, 12)
Set bOk = (nAddrType = AF_INET) AND (nLen = 4) AND (nMem != 0)
So, reading the docs on the structure, the 3rd member is the AddrType.
To get the value, we read a WORD (2 bytes) from location 8 now in TD.

Is that correct?

On x86, the member starts at byte 8. And this is correct.

But on x64, the sizes are different. There, nAddType starts at 16

Same for nLen. On x86 it starts at 10.
But on x64 it starts at 18.

Also for nMem. ON x86 it starts at 12 but on x64 it starts at 24.

So the positions must be changed to:

Code: Select all

Set nAddrType = CStructGetWord(sBuffer, 16)
Set nLen = CStructGetWord(sBuffer, 18)
Set nMem = CStructGetLong(sBuffer, 24)
Set bOk = (nAddrType = AF_INET) AND (nLen = 4) AND (nMem != 0)
Almost finished now.

The only thing which is incorrect is

Code: Select all

Set nMem = CStructGetLong(sBuffer, 24)
It reads a long (4 bytes) from position 24.
But that member is a pointer, which is 8 bytes.

So we have to change it to:

Code: Select all

Set nMem = CStructGetLongLong(sBuffer, 24)


These are the steps to take for any API investigation. I will stop at this point here. But if you understand what I did here, you can check all APIs yourself by checking the sizes of the members, if there are any dummy padding members and use the correct cstruct functions.
Regards,
Dave Rabelink

Image
Articles and information on Team Developer Tips & Tricks Wiki
Download samples, documents and resources from TD Sample Vault
Videos on TDWiki YouTube Channel

pascmeul
Belgium
Posts: 16
Joined: 09 Oct 2017, 08:50
Location: Oostkamp, Belgium

Re: TD 64bit and WinAPI32 declarations

Post by pascmeul » 01 Jul 2020, 14:02

This helps me a lot.
Thanks

Return to “Tools & Samples”

Who is online

Users browsing this forum: [Ccbot] and 0 guests