TD-23620: Create profiled TD applications out-of-the-box

Missing or incomplete features? Discuss it here!
Dave Rabelink
Founder/Site Admin
Founder/Site Admin
Netherlands
Posts: 296
Joined: 24 Feb 2017, 09:12
Location: Gouda, The Netherlands

TD-23620: Create profiled TD applications out-of-the-box

Post by Dave Rabelink » 03 Jul 2017, 13:01

I would like to know who is interested to have an option in TD to create profiled versions.

I talked about this last devcon with Gupta dev and they think it can be done.
I still have to create an official enhancement request, but it would be nice to see who else is interested in this and maybe come up with suggestions how this would look like.

This is the case:

For years now, I'm using TDProfiler tool. Originally created by Armin Pokorny.
I adapted (forked) it to be easily used in all TD versions and I'm currently changing it even further to support new TD7 features and other changes to get extra info from running applications.

What it does basically:

A profiled application, when executing, is writing a trace log file which writes the entry and exit points of functions/messages.
Along with this it records the time taken to execute these functions/messages.

Using the trace files, you can inspect what functions/messages were executed and the timings.
It is specially handy to detect which parts of the application is having performance issues, what functions have been called.
When you have a crashing application, the trace will show the last function action. In most cases pinpoint you to the issue.

As an example: suppose I get a complaint from the field a specific workflow on customer site causes the application to crash.
Trying to reproduce it at developers side fails.
When the customer approves, I can create a profiled version of the application. This "special" exe or apd is put on their system(s) and the users there let the application crash.
The trace files are then send back and using the profiling tools I can inspect what workflows have been executed and also the last functionality performed.
In fairly often cases I could pinpoint the cause of the crash, or at least see which parts of the application were touched by the user.

This saves a lot of time trying to reproduce the error and in some cases could solve the issue without being able to get a reproducing workflow at all.

Though the TDProfiler tool is very handy, it involves that the application is profiled at source code level.

What I envision is this, maybe two options to have this out-of-the-box:

1) When building a TD application (exe/apd), the build options include a profiler option.
The exe/apd is then "enabling" some kind of extra code in the runtime which traces all called functions into a file.

or

2) We can "somehow" instruct the runtime to trace without having to build a special profiled version.

Advantage of option 2) is that an existing setup at customer site can be switched to use profiling, without changing exe's or apd's.
But that can be a security risk. And maybe too much.

At least, a way to instruct the TD runtime to log everything (or even better, specify which levels to log) would be a huge help.

I was thinking that logging the API calls would be very nice. Specially API calls could lead to crashes.
If the log shows that a WinAPI function is being called but does not return (a crash) pinpoints to a possible wrong declaration.

Any thoughts on this?
Regards,
Dave Rabelink

Image
Articles and information on Team Developer Tips & Tricks Wiki
Download samples, documents and resources from TD Sample Vault
Videos on TDWiki YouTube Channel

Christof
Germany
Posts: 8
Joined: 06 Mar 2017, 07:27
Location: Frankfurt, Germany

Re: Create profiled TD applications out-of-the-box

Post by Christof » 04 Jul 2017, 06:42

I am also a huge fan of the TDProfiler Tool. It was several times of great help when we had to track down oddly crashes.
So a "native" integration into TD would be a great advancement.

I would prefer option 1, the build option. Even if the additional tracing code would be only present in binary code, it would blow up the whole thing. Also if you could start the executable with such a tracing option, the code would be always present. At each function entrance and exit, there would be a decision about to trace or not to trace, which must lead to performance losses.
So in my opinion, a special built version where we or our customers accept the loss of performance for debugging reasons would be the better way.

Anyway, having the opportunity to get a profiled application by just checking a build option, instead of the time consuming process including the current TDProfiler tool, would be just cool!

thomas.uttendorfer
Site Admin
Site Admin
Germany
Posts: 35
Joined: 05 Mar 2017, 17:19
Location: Munich Germany

Re: Create profiled TD applications out-of-the-box

Post by thomas.uttendorfer » 04 Jul 2017, 08:57

Hi Dave,
a native profiler would be a huge help.
I would like to have the timing feature to identify the hotspots and maybe optimize execution times.
It's a RPITA to do optimizing up front not knowing where the real bottleneck is

Regards Thomas
Thomas Uttendorfer
[ frevel & fey ] Software-System GmbH
https://thomasuttendorfer.wordpress.com/

EwaldP
Austria
Posts: 51
Joined: 07 Mar 2017, 08:00
Location: Austria

Re: Create profiled TD applications out-of-the-box

Post by EwaldP » 04 Jul 2017, 09:09

Hi Dave,

A native profile would be very welcome. I have built a profiler tool for our applications, but I appreciate if someone else does this work for me and I do not have to worry about it anymore.

Ewald
Ewald P. Palmetshofer
EDV-Hausleitner GmbH
4020 Linz
www.edv-hausleitner.at

H3PO4
Germany
Posts: 26
Joined: 10 Mar 2017, 07:53
Location: Rohlstorf, Germany

Re: Create profiled TD applications out-of-the-box

Post by H3PO4 » 04 Jul 2017, 10:46

Hi Dave,

a builtin profiler would be a great help.

For me option 1) is the best idea. Developers who like option 2) more, could deploy the 'normal' and the 'profiler' version and decide with a starter-program whether to profile or not to profile...

Best regards
Henning

efarina
France
Posts: 5
Joined: 09 Mar 2017, 07:47
Location: Paris, France

Re: Create profiled TD applications out-of-the-box

Post by efarina » 04 Jul 2017, 12:07

Hi Dave,

I fully support your request but does it have a chance to succeed with Open TEXT.
Best regards,

Eric Farina
Research and Development deputy Director
www.aareon.fr

Dave Rabelink
Founder/Site Admin
Founder/Site Admin
Netherlands
Posts: 296
Joined: 24 Feb 2017, 09:12
Location: Gouda, The Netherlands

Re: Create profiled TD applications out-of-the-box

Post by Dave Rabelink » 05 Jul 2017, 07:49

I created a ticket. I will keep you posted on the responses there.

Ticket # 3073387: Profiling support in TD
TD-23620: Profiling support in Team Developer
Regards,
Dave Rabelink

Image
Articles and information on Team Developer Tips & Tricks Wiki
Download samples, documents and resources from TD Sample Vault
Videos on TDWiki YouTube Channel

a_sivababu
India
Posts: 23
Joined: 10 Mar 2017, 08:46
Location: Chennai,India

Re: TD-23620: Create profiled TD applications out-of-the-box

Post by a_sivababu » 20 Sep 2017, 14:40

Thanks Dave for raising this request. this would be great help. as of now, I am just internally managing with extra code.

"At least, a way to instruct the TD runtime to log everything (or even better, specify which levels to log) would be a huge help."

It should basically emit
SalTrace(EVENT_SQL,'trace.txt') - Emit only SQL statements and bind variables are replaced with values. It should work like AppTools.
SalTrace(EVENT_SAL,'trace.txt') - Emit all Team Developer sal code including external functions
SalTrace(EVENT_SAL || EVENT_SQL ,'trace.txt') - Emit all Team Developer sal code including external functions


I think we can set this two events through our ini file or I am wrong.

Thanks
Siva A

Dave Rabelink
Founder/Site Admin
Founder/Site Admin
Netherlands
Posts: 296
Joined: 24 Feb 2017, 09:12
Location: Gouda, The Netherlands

Re: TD-23620: Create profiled TD applications out-of-the-box

Post by Dave Rabelink » 27 Nov 2017, 11:47

Ha, just had a look at Guptas TD roadmap and it seems this enhancement request will be available in TD 7.2 !!

viewtopic.php?p=152713#p152713
Regards,
Dave Rabelink

Image
Articles and information on Team Developer Tips & Tricks Wiki
Download samples, documents and resources from TD Sample Vault
Videos on TDWiki YouTube Channel

Igor Ivanovic
Site Admin
Site Admin
Croatia
Posts: 88
Joined: 05 Mar 2017, 12:37
Location: Zagreb, Croatia

Re: TD-23620: Create profiled TD applications out-of-the-box

Post by Igor Ivanovic » 28 Nov 2017, 13:37

The list looks really nice, except one detail...
All of this will not be available before Q2 (or Q3/4 as per this year delays) 2019. :o :o :o

I have no option but to stick with TD/TDM, but this are not good news for my current projects.
Igor Ivanovic
Image

Dave Rabelink
Founder/Site Admin
Founder/Site Admin
Netherlands
Posts: 296
Joined: 24 Feb 2017, 09:12
Location: Gouda, The Netherlands

Re: TD-23620: Create profiled TD applications out-of-the-box

Post by Dave Rabelink » 28 Nov 2017, 14:07

Igor Ivanovic wrote:
28 Nov 2017, 13:37
The list looks really nice, except one detail...
All of this will not be available before Q2 (or Q3/4 as per this year delays) 2019. :o :o :o

I have no option but to stick with TD/TDM, but this are not good news for my current projects.
Are there specific features you would need which are not possible now with the current TD 7 featureset?
Regards,
Dave Rabelink

Image
Articles and information on Team Developer Tips & Tricks Wiki
Download samples, documents and resources from TD Sample Vault
Videos on TDWiki YouTube Channel

Igor Ivanovic
Site Admin
Site Admin
Croatia
Posts: 88
Joined: 05 Mar 2017, 12:37
Location: Zagreb, Croatia

Re: TD-23620: Create profiled TD applications out-of-the-box

Post by Igor Ivanovic » 28 Nov 2017, 17:59

Hi Dave,

My main problem is with TDM, with TD I lived and can live (although I would really like to have the window resizing/anchoring feature and profiling and code diagnostics etc.).

TDM has limited client side controls, localization is only half way done, and sometimes I spent more time testing and making repro's then actual coding :-D
Although, I must not complain, I received really great support from the OT Gupta support team!

Some of the things I am missing are planned for TDM2.2, which was due this December, but it was postponed to Q2/2018 (and I am reading it late June or July 2018)
To mention only some: global JS functions, user defined events, and some bugs I found out which should be resolved in TDM2.2 as there are no minor versions and so on...

My problem is that I have to have a working prototype by the end of march, so I was hoping to use TDM2.2 to avoid some unnecessary coding, but...

Just ranting...
Igor Ivanovic
Image

a_sivababu
India
Posts: 23
Joined: 10 Mar 2017, 08:46
Location: Chennai,India

Re: TD-23620: Create profiled TD applications out-of-the-box

Post by a_sivababu » 29 Nov 2017, 04:55

I feel that OT will soon move cover below wishlist :-)

1. HTML based RTF Editor like CKEditor. this can be manually integrated now.
2. ElasticSearch. I feel this is important and should be integrated if TDM supports for Enterprise Apps.
3. Supporting MongoDB through Sqlxxx functions than just Data operations
4. Creating pagelets including Jquery UI interactions in TDM pages to have dashboard like pages
5. Report Designer to support Charting and HTML RTF Editor etc. There is no change happening in RD.
6. Creating deployment package instead of installing IIS, .NET framework, Database and then pushing TDM Apps. it can developed through Puppet.
7. metadata-SQL rather than writing complete SQL coding and should work as a Object Relationship Model (ORM)
8. Scheduling interfaces, reports through scheduler. Hope Multithreading might support this.
9. as of now, we can consume REST API but TD/TDM should also support developing REST API with all verbs.

aspurga
Lithuania
Posts: 8
Joined: 06 Mar 2017, 08:54
Location: Lithuania

Re: TD-23620: Create profiled TD applications out-of-the-box

Post by aspurga » 29 Nov 2017, 05:44

In my opinion the further TDM development is money wasting for OpenText. There are only some developers why use this hopeless tool.

Sorry...

Arunas

Igor Ivanovic
Site Admin
Site Admin
Croatia
Posts: 88
Joined: 05 Mar 2017, 12:37
Location: Zagreb, Croatia

Re: TD-23620: Create profiled TD applications out-of-the-box

Post by Igor Ivanovic » 29 Nov 2017, 08:18

a_sivababu wrote:
29 Nov 2017, 04:55
I feel that OT will soon move cover below wishlist :-)
Siva,

Is this already on OT official wishlist :D or something you wish for ;) ?
Igor Ivanovic
Image

Return to “Enhancement suggestions”

Who is online

Users browsing this forum: Ccbot [Crawler] and 0 guests