Note that there is still a pseudo-random issue of unknown origin under Windows 7, where the utility may or may not be able to gather profiling information. If that happens, close the application and launch it again. The issue doesn’t happen with Windows OSes before 7.

The main goals:

• Strong typing: this is a long term goal of dropping “Eval : Variant” methods, and ultimately, the reliance on Variant. Variants will stay in the scripting language though, but may become optional (see below). Changes will be incremental, with new methods and classes popping into existence, some as temporary glue, and other bits of code will fade away. What this means is that you shouldn’t rely on the internals classes, their methods, or even their still being there down the road.
• Expression tree: this is an architectural goal of keeping the expression tree “understandable” and maintainable. Even though specialized classes may come to abound, higher level classes will be maintained to keep the expression tree easy to understand and maintain. This is to allow using the expression tree as a specialized DOM, facilitate debugging, refactoring, static analysis, etc.
• Custom primitive types: allow optional primitive types to be plugged into the compiler, I’m thinking primarily of Variant, date-time, vectors and matrices at this time. From a language point of view, this could also mean optionally allowing operator overloading.
• Generics/Templates: provide some mechanisms for genericity, at least on the Delphi side, so that you can f.i. have a “TList<T>” in the scripts, even if you may not be able to design a generic containers in script.
• Local JITter: introduce a framework to simplify JITting portions of a script. The initial aim would be for math computations involving floats, vectors & matrices into SSE instructions. Complete JITting isn’t on the radar at this point, and probably shouldn’t be. This is unlikely to happen until a 64bit Delphi compiler is out.

If you want to help on the DWS compiler & runtime side, at this point the most needed things are unit tests. Can’t have too much of them .

Library, tools and utilities front:

DWS2 came with a rather vast set of companion libraries, tools and utilities, covering many things from HTML web page generation to auto-completion support functions. At this point I will not focus on those, but on the core aspects mentioned above.

There are however a few RTL-like sets of libraries still to be released, TStringList, TStringBuilder, TXxxxList, file access, etc. all these providing extra optional functionality for the scripts.

If some of you want to help on that front, or need some of the treasures from the old DWS2 chest, they can go ahead and bring the old code up to Unicode Delphi, things should be compatible for the most part on the DWS side apart from a few name changes.

Documentation:

A natural choice would be the google code wiki facility, but it’s very primitive to say the least. Another possibility is wikia, but I’ve no idea how easy it would be to export/backup a wikia wiki at this point, should wikia go the way of the Dodo. Another option would be a no-database, xcopy-installable, portable wiki like PmWiki. I’m still looking around, if you’ve suggestions.

http://code.google.com/p/dwscript/

The core compiler and some internal libraries are there, along with a few unit tests, but no demos just yet. It’s compatible with Delphi 2009 (and 2010?) only at this point.

Hopefully some demos will be added, ad interim, you’ll have to look at the unit test source to see how to compile your own scripts. More modules should also come in the next weeks/months as their dependencies get cleaned up, and the dust of many years shaken away.

Here is a quick summary of the most significant changes since DWS2:

What changed in the script language?

• DateTime type was deprecated, in favor of plain old Float for timestamps.
• Integer is now an Int64. For language simplicity, there is only one integer type, 64bit it is.
• Strong typing is more strictly enforced throughout, this can cause issues in previous scripts that abused compiler bugs.
• Support for “deprecated” warnings was added.
• for loop variables can no longer be altered within the loop.

What changed internally?

• Move to strong typing away from Variants, this is still a work in progress (and source of much speedups).
• Handling of break/continue/exit no longer relies on exceptions (it’s now orders of magnitude faster).
• Expression tree memory usage reduced significantly, this is also still a work in progress.
• Tokenizer was re-architectured for higher parsing speed.
• Naming prefix was simplified from “dws2″ to just “dws”, though the code is based on DWS2.

The PascalScript version was downloaded yesterday, though when compiling, I got many warnings hinting its code hasn’t been fully upgraded to Unicode and D2009, so maybe it wasn’t the latest version? I’m not a user of PascalScript, so if anyone want to chime in, feel free!

The script I used for testing was something like

var s, i: Integer;
for i:=1 to 1000000 do s:=s+i;

with only minimal syntax adaptations to get it running and compiling in the various environments. For PascalScript, I tweaked the “sample1″ project to run the script, so maybe there were some optimization options not active? If so, let me know!
The result timings on my machine were (including compilation and execution, except for Delphi):

Delphi: <1 msec (using 32bit integers), 1.5 msec (using 64bit integers)
DWS2 revival: 150 60 msec in D2009 (using 64bit integers)
DWS2 vanilla: 420 msec in D7 (using 32bit integers)
PascalScript: 1860 msec in D2009, 1490 msec in D7 (using 32bit integers)

Note that in DWS2 revival, integers are 64bit integers (more on that and other changes later).

I then redid the test using a double precision float for the “s” variable: Delphi goes up to 15 msec, DWS2 revival increases to 180 95 ms (vs 670 ms for vanilla), PascalScript results were unchanged.

Last test consisted in moving the “s:=s+i” to a function, to measure the call overhead. Delphi laughed at me and stayed with roughly the same figures, PascalScript went up to 3000 msec, and DWS2 revival took a hit to 1000 msec. Function calls are indeed a DWS area with still some simplifications and optimizations potential

As for the re-release time-frame, I’m waiting to hear from the original maintainers, as well as removing a few dependencies to make it into a standalone library (currently the compiler itself is standalone, but the DWS library equivalents of system & sysutils units aren’t).

edit 2010-09-01: following a round of cleanup and optimizations, figures for DWS revival are now 60 and 95 msec for 64bit integers and floats respectively.

DWS is a 100% Delphi-based Delphi-language script engines, that supports a fairly large subset of the Delphi language (including objects), with a performance high enough for use in real-time scenarios (simulators, game engine scripting…).

We’ve been making use of the library at work for years now, with it receiving updates covering a variety of things from Delphi compatibility, to language enhancements to speed or memory usage optimizations. On the other hand, the parts we didn’t use fell significantly behind (to the point of deprecation), and wouldn’t be part of the revival (unless some extra hands join in for the update).

However, most emails and websites from back then have gone the way of the Dodo, so this post is mostly a call to former DWS2 maintainers, contributors and users to make themselves known if they are interested… if they are still using DWS2 in some form or another, or if they are merely just still around to share memories of a bygone era

Why does it work?

The most well known case is “const String“, for which the compiler than takes advantage of the “const” to pass the String reference directly (as a pointer)… and without increasing the reference counter.

To illustrate what the reference counting implies, here are two screen-shots from the CPU disassembly view, taken in Delphi 6, but the recent compilers up to Delphi 2010 at least behave in a similar fashion (just with the Unicode version of the functions).
I’ve made a pseudo-function, with only one String parameter that calls a single function (here Length(), which isn’t inlined in Delphi 6, and corresponds to LStrLen), guess which CPU disassembly corresponds to “Test(const a : String)” and which to “Test(a : String)”

What you’re seeing on the left is an implicit try…finally construct which is used to protect the reference counting (LStrAddRef/LStrClr) on the implicit local variable used to store the String parameter. If you have such calls nested a few levels deep (not uncommon in object-oriented code), you could accumulate quite some overhead.
Also not see here but hidden within the AddRef and Clr calls are bus locks, which may hit you disproportionately in multi-threading scenarios.

And what if you’re modifying the passed String? Can you forego the “const“? Well no, the extra overhead is still there, and using a “const” still beneficial.

String isn’t the only type affected, there are similar gains for all reference counted types (interfaces, dynamic arrays, records holding reference-counted types). And when passing a record as “const“, there is an additional gain in the lack of defensive copying of the record (the larger the record, the greater the gain).

For ordinal and numeric types, there is no compiler optimization (yet), but it can be good practice to mark them as “const“, not in the optimistic hope that someday the compiler will optimize it, but to ease up debugging and code writing. Copies of those parameters (when you want to modify them in the function body) are typically handled adequately by the compiler (and often enough comes for free), so don’t let performance considerations hold you.

There is one case in which using “const” could have adverse effects, but it involves global variables which you would modify or release during the call… so you don’t really need to know more about it, do you?

How does it manifests itself in Profiling?

A missing “const” can manifests itself in different ways during profiling, for String, Interface and dynamic array types, you’ll usually see it via the relevant xxxAddRef or xxxClr functions (the obvious case) and via time spent in begin/end (the less obvious case). Depending on how many functions calls with the reference counting are involved in your inner loop, none of the above may come ahead, even if calls and reference counting are an issue: the time spent will be spread over the above functions and your various begin/end sections.
For record parameters, the lack of “const” could materialize itself in “Move“, or other data copying costs (large records will be moved, smaller ones can be copied via ad-hoc in-place code by the compiler).

In multi-threading, things can be a little less obvious, as rather than the reference counting and exception frame, it could be the bus locks hitting you, or sometimes worse, inter-CPU traffic to pass around the values of the reference counter.

Multi-threading’s worst case would be referring to the same string field from multiple threads, and passing that string around in functions without “const” parameters, thus repeatedly hitting the reference counter of that string from multiple threads at the same time, resulting in extra inter-CPUs traffic to share the value of that reference counter.

In such cases, “const” can help, but it will often not be enough, you’ll also have to look for functions/methods that return strings with an unnecessary reference counting, like when getting your strings from a TStringList.

Good Practice

Usually I tend to “const” just about every parameter that isn’t “var” or an object, this isn’t just  to pre-emptively alleviate possible performance issues, but also to gain something valuable when debugging and writing a function’s body: untouched input parameters, wherever you are in the function’s body. This means not just in the function’s body, but also when in the functions called by the function.

And that’s the key point to remember about “const” parameter: it’s here to state that a parameter will be left unchanged inside the body, something which allows the compiler to optimize it’s output, and the developer to optimize his thought process too. Qualifying all your parameters as either “const” or “var” makes your intentions obvious.

So IME, it’s not worth it to hoard your keystrokes, best spend them on “const” and explicit local variables when you really need to alter those input parameters.
Whoever has to maintain your code will be thankful.

Object parameters

There is one case I’ve not mentioned in all the above, the case of object parameters (and class references). It’s a special case because when you pass an object as “const“, you’re not making any promises of not keeping the object constant, but just keeping the underlying variable (pointer) constant. This is unlike other languages. There is also a risk of confusion when your code will be read by developers not really familiar with the underlying pointer-based mechanics of objects (and they are more common than you may think, even, and maybe more so in highly OO languages).

So for objects, unless you want to spend a lot of time educating, my rule of thumb is to avoid “const” or “var” for object parameters.

Beyond “const” parameters

Const’ing your parameters will eliminate some defensive copying, implicit try…finally, and reference counting overhead, but it leaves open another field of similar inefficiencies: that of return values.

Alas there is no “const Result” syntax, which would allow to state that your result can’t be altered in any way (a sort of strict immutability if you will). Thus whenever you have a function returning a String f.i., you’ll end up triggering the implicit reference counting and exception machinery if you’re not careful (like in this case)…

There are strategies to tackle this issue, but none of them are as simple as adding a “const“, and most of them come with sacrifices… so, let them be food for other articles.

Apparently along with Nick, more than a Spirit of Delphi award winner may have left Embarcadero.

Its purpose is to facilitate gathering device OpenGL ES support information, when the device you’re targeting or have issues with is in the hands of a non-developer… or in the hands of a lazy developer

It is named after from the glInfo utility that used to be hosted at Tom Nuydens’s Delphi3d site.

The old site has been resurrected there too, as the wiki engine lost most of the images added during the last years… I’ll be looking into setting up something to replace both with a more functional content manager.

The reason behind this dust being shaken is that I’ll likely be initiating a GLScene for Android project (for Java / Dalvik).