Partial classes allow to split the definition and implementation of a class over multiple files, or over multiple sections of the same file. Partial declarations must have the same modifiers (abstract, sealed, static…), must specify the same ancestor (or none), however each partial declaration is allowed to introduce new interfaces and their implementations.

Partial class can have multiple purposes, to quote wikipedia:

• Facilitates the implementation of code generators that have to “inject” code declarations/implementations that comes form other descriptions, such as a Visual UI designer, data and string resources, etc.
• Enables separation of concerns, in a way similar to aspect-oriented programming but without using any extra tools.
• Enables optionally composited functionality, which can be omitted or substituted depending on target or requirements..
• Enables separation of a class’s interface and implementation code in a unique way. (irrelevant for Pascal!)
• Enables multiple developers to work on a single class concurrently without the need to merge individual code into one file at a later time.
• Eases navigation through large classes within a editor.

The Problem

Effectively, “records with methods” treat all record “const” elements as “var“, even when they shouldn’t, and effectively ignores the compiler option “assignable constants”. Witness the following record:

type
   TRec = record
      Field : Integer;
      procedure IncMe;
   end;
...
procedure TRec.IncMe;
begin
   Inc(Field);
end;

Then the compiler will happily allow you to do:

const MyNullRec : TRec = (Field: 0);
...
MyNullRec.IncMe; // watch me, I'm magic!
WriteLn(MyNullRec.Field); // will write 1 !

…it will also accept…

procedure Myproc(const r : TRec);
begin
   r.IncMe; // modified whatever r as if it had been a var
end;

…and that as well…

type
   TMyClass = class
      private
         FRec : TRec;
         procedure SetRec(const aRec : TRec);
      public
         property RW : TRec read FRec write SetRec;
         property RO : TRec read FRec;
   end;
...
var myClass : TMyClass;
...
// everything's as it should be there
myClass.RO := aRec; // gives an error
myClass.RW := aRec; // uses SetRec
// but that compiles to...
myClass.RO.IncMe; // modifying a read-only property !
myClass.RW.IncMe; // modifies and bypasses SetRec !

It would “compile” even if the getter was a function returning a record… or just in about any case, really.

And of course you can cascade all the above in a real-world case, so you end up with weird side-effects and really hard-to-track down bugs.

Besides bug, that also raises the issue of usability of records like TPoint and other simple vector/matrix types, as properties of objects. As long as they did not have methods, you would have to use the setter to modify such a property, so the objects had guaranteed notification of a position change f.i., with methods being able to magically mutate the records, you no longer have that guarantee. And adding all the baggage so that a TPoint can be owned and notify its owner of a change would be over-the-top, wouldn’t it?

What’s Missing

Two syntax elements are missing:

• the ability to specify if a method is mutating the record or not, ie. if it should get “Self” as a “var” or as a “const“, so that you can make the above behavior explicit, and the compiler can emit errors when it’s not appropriate
• the ability to mark a record as mutable/immutable, to further remove the ambiguities

Ideas for such a syntax, in explicit form, and reusing existing keywords:

type
   TRec = record
      Field : Integer;
      var procedure IncMe; // mutating
      const function ToString : String; // non-mutating
   end;

Modifiers in Pascal (except “class“) usually follow the declaration, but “var” and “const” can’t, as they could be ambiguous with the declaration of a following constant/variant. So if reusing those, rather than introducing new keywords, they have to be in front.

Also you could mark a record as explicitly mutable or immutable, by prefixing it with “var” or “const”

type
   TImmutableRec = const record
      Field : Integer;
      var procedure IncMe; // mark as explicitly mutating ?
      function ToString : String; // non-mutating by default
   end;

   TMutableRec = var record
      Field : Integer;
      procedure IncMe; // implicitly mutating
      const function ToString : String; // mark as explicitly non-mutating
   end;

For backward compatibility with existing records with methods, the implicit default would have to be “var record” as the records are currently all mutable.

With that extra information, the compiler could then perform proper checks and proper parameter passing, so the side-effects mentioned could happen only if you explicitly went for them, rather than by mistake as is currently the case.

Additionally, immutable records would implicitly have their fields as read-only everywhere outside the record constructor (which would then become truly useful as language syntax element). It might also be possible to extend the syntax to be able to declare explicitly immutable classes.

The case of helpers

When generalizing helpers (as in DWScript), the above mutability issue also applies to helpers for value types, ie. records and static arrays, so a similar syntax would have to be introduced.

To minimize side-effects, helpers on record and static arrays currently pass their “Self” as const, this allows to use such helpers in all cases (including “const” parameters, function results, read-only properties…), but is restrictive, as it prevents mutation. So once the above syntax is decided upon, it could apply to them as well.

For helpers on reference types in DWScript (classes, interfaces, dynamic arrays), the issue doesn’t apply.

Note that in Delphi, helpers also have the design bug of being able to magically access private fields… but that’s another story.

The syntax is currently the following:

type
   TMyHelper = helper for some_type_here
      private
         ...private helper methods, class vars & class consts...
      public
         ...public helper methods, class vars & class consts...
   end;

You can have helpers for base types, classes, records, interfaces or arrays.

For instance if you write

type TFloatHelper = helper for Float
      const PI = 3.141592;
      function ToString : String;
   end;
...
function TFloatHelper.ToString : String;
begin
   Result := FloatToStr(Self);
end;

then it allows you to code things like ‘Float.Pi‘ or ‘myFloatVar.ToString‘.

The literal form is also accepted (‘TFloatHelper.Pi‘ and ‘TFloatHelper.ToString(myFloatVar)‘).

It’s possible to have class methods in helpers, for types with a meta-symbol, Self will be that meta-symbol (f.i. for a TObject helper, Self would be TClass), for others (like Float or records), Self will not be defined.

Current limitations of the experimental support:

• priority/scope when multiple helpers introduce the same name isn’t fully spec’ed yet and may evolve
• compiler may accept things it should reject, and vice-versa
• parser performance was slightly reduced, optimization will happen when the spec has been completed
• ‘inherited‘ isn’t yet supported to call a non-helped version of an helped member added
• suggestions class doesn’t yet support suggesting helpers

The scope resolution when there are multiple-helpers introducing the same method is at the moment not fully defined, currently the compiler will pick the first one it finds in scope. Other options are under considerations: using the same resolution logic as overloads, allowing helpers to be “sub-classed” and then using class-like resolution logic… if you have good ideas pilfered from other languages, or bad approaches that should be avoided, now is the time to chime in!

The syntax will also probably be extended to encompass the Delphi syntax (“class helper”) so you can share code, and maybe others variations (“interface helper”, “type helper”…) though which/how remains to be determined.

Once all the syntax and scoping elements have been finalized, the compiler will get proper optimizations, but for now, it’s expected to be somewhat less efficient.

Helpers on functions pointers/delegates types aren’t yet allowed, mostly because of ambiguity, f.i. in

type
   TFunc = function : Integer;
   TFuncHelper = helper for TFunc
      function ToString : String;
   end;
   TIntegerHelper = helper for Integer
      function ToString : String;
   end;
...
var f : TFunc;
f.ToString;

the “f.ToString” could be understood as meaning “TFuncHelper.ToString(f)” or “TIntegerHelper.ToString(f())” if helpers on function pointers were allowed. If a good resolution is found, they could be “helped” too, but helpers may not be very relevant to function pointers (?).

Language:

• dynamic arrays now support Pop(), Peek() and Push() and can be used as stacks.
• dynamic arrays Add() and Push() now accept a variable number of parameters, and in addition to elements, they now accept dynamic arrays too (which will be concatenated)
• Count() can now be used as an alternative to Length() for static and dynamic arrays.
• Class variables are now supported, both by classes and records.
• Oxygene-like syntax for explicitly scoped “enum”  and “flags” is now supported:

  MyEnum = enum (One, Two, Three)
  MyFlags = flags (Alpha, Beta, Gamma)

  Both forms must be prefixed, as in MyEnum.Two, the “enum” form is otherwise similar to classic enumerations. The flags form is a special case in which the elements values are powers of two (f.i. MyFlags.Beta = 2, MyFlags.Gamma = 4)

• class methods can now be marked as static.
  
• Class const syntax is now supported (had to be written as just ‘const’ before)
• multi-line indented strings can now be either #” or #’
• tweaked overload metric to give precedence to Float over Integer when Variants are involved
• added a new hint if you redeclare a variable with the same name in a local sub-scope (doesn’t apply across sub-procedures)
• improved some error locations, other minor fixes

Built-in functions and interfacing:

• Partial RTTI support for indexed properties (Stefan Glienke)
• StrSplit() splits a string on a delimiter and returns an array of string
  StrSplit(‘jan,feb,mar’, ‘,’) returns ['jan', 'feb', 'mar'] as a dynamic array
• StrJoin() takes an array of string and returns a string made by joining all the individual strings
  StrJoin(a, ‘,’) with a the above array would return ‘jan,feb,mar’

JavaScript CodeGen:

• various improvements to the output when optimizing for size (about 10% of size reduction)
• fixed an issue with var parameters of a parent procedure used in a local procedure or anonymous method
• smart-linker is now capable of eliminating some cases of unused interfaces
• fixed an issue with aliased types used as record fields

Support Tools:

• added fledgling code metrics classes

As a side note, the recent (and planned) improvements to dynamic arrays are currently tipping the generics vs templates balance towards templates, as they’re becoming capable enough “generic collections” on their own, meaning that “classic” generics wouldn’t bring much extra capability to the language for everyday usage.

Alas the optimalcode.com website and Robert Lee disappeared years ago without a trace, but the “Delphi Optimization Guidelines” (dating back from 2002-3003) has been safeguarded and preserved. Recently someone pointed to me that the mirror I had in my Links section had disappeared too…

I guess it’s my turn to host the sacred relics, so I’ve updated the link and placed not one, but three copies on this site:

• Delphi Optimization Guidelines, browsable version in html
• Delphi Optimization Guidelines, as a zip of html files
• Delphi Optimization Guidelines, as a PDF

The guide is, at the time of this writing, nearly 10 years old, so read it with that in mind!
That said many tips still apply to the 32bit Delphi XE2 compiler, and quite a few tips are valid regardless of compiler and programming language.

The controls are the same as in Delphi: arrow keys to turn and accelerate, space bar to fire.

Since the original code is under Embarcadero copyright, I can’t post the source here, but like with the Old School Maze, the amount of adaptations required was minimal, and the old code is running in near-vanilla form (with all its quirks). You can find DMeteor source code in the Samples\Delphi\VCL\Meteor directory that came with Delphi or RAD Studio.

The old Delphi code was pegged at 20 FPS and 600×400, and this version is too. Maybe once the PGD Challenge is over, I’ll try to find time to upgrade it to a 60 FPS, HD version, fix the difficulty and add Playtomic global leader-boards, just to show how simple it can be to leverage Pascal in the HTML5 era 

type
   TMyEnum = enum (Alpha, Beta, Gamma);
...
e := TMyEnum.Alpha; // works
e := Alpha; // syntax error

Since the scoping with this declaration syntax is explicit, you’re encouraged not to use prefixing for the enumeration elements, removal of the leading “T” being optional.

Classic enumerations (without keyword) stay globally scoped and unchanged for now (so this addition is fully backward-compatible).

Besides adding support for one more Oxygene syntax elements (alongside method, implies, contracts, etc.), this also provides a cleaner syntax than the {$SCOPEDENUMS} directive-based  approach introduced with Delphi XE2, with which you end up having to wrap enum declarations between directives (cf. VCL & FMX source).

TStringListCracker

The code of the function looks like:

procedure UnifyAssignString(const fromStr : UnicodeString; var toStr : UnicodeString);
var
   i : Integer;
   sl : TUnifierStringList;
begin
   if fromStr = '' then
      toStr := ''
   else begin
      i := Ord(fromStr[1]) and High(vCharStrings);
      sl := vCharStrings[i];
      sl.FLock.Enter;
      i := sl.AddObject(fromStr, nil);
      toStr := TStringListCracker(sl).FList[i].FString; // HACK HERE
      sl.FLock.Leave;
   end;
end;

The non-hacky variant would be to have that line just be

toStr := sl[i];

but if you use that simpler form, you could see a performance drop by up to 45% (Delphi XE and XE2 32bit) or 38% (Delphi XE2 64bit).

Ouch! Why is that?

The hacky version

The hacky version compiles to a single UStrAsg, the internal function for string assignment which takes care of the reference-counting. So it’s just:

dwsUtils.pas.487: toStr:=TStringListCracker(sl).FList[i].FString;
00511594 8B0424           mov eax,[esp]
00511597 8B532C           mov edx,[ebx+$2c]
0051159A 8B14F2           mov edx,[edx+esi*8]
0051159D E8A659EFFF       call @UStrAsg

i having been returned by AddObject, no range-check is required.

(note that TStrings.AddObject is called directly because TStrings.Add is just an indirection to AddObject)

What about the “toStr := sl[i]” version?

The property is just syntax sugar on a getter method, so what is actually compiled is  “toStr := sl.Get(i)“, which itself, being a virtual call, looks like

toStr := sl.VMT[ TStrings_Get ]( sl, i );

However the virtual call to TStringList.Get on its own costs only a tiny bitsy of extra CPU cycles, though it also prevents inlining (at least, until the compiler gets de-virtualisation optimizations).

But there are other factors at work, the virtual call on its own can’t come close to explaining the performance differential, as after all, the rest of the code UnifyAsstringString is far from trivial (with a critical section and a binary search…)

First, it’s a function returning a string, which is a managed type, so it’s actually compiled to something like:

try
   temp := sl.getValue( i );
   UStrAsg( toStr, temp );
finally
   UStrClr( temp );
end;

Second, TStringList.GetValue itself is compiled to

TStringList.Get:
00448204 53               push ebx
00448205 56               push esi
00448206 57               push edi
00448207 8BF9             mov edi,ecx
00448209 8BF2             mov esi,edx
0044820B 8BD8             mov ebx,eax
0044820D 3B7330           cmp esi,[ebx+$30]
00448210 720F             jb $00448221
00448212 8B1544EA5100     mov edx,[$0051ea44]
00448218 8BCE             mov ecx,esi
0044821A 8BC3             mov eax,ebx
0044821C E8C3E3FFFF       call TStrings.Error
00448221 8BC7             mov eax,edi
00448223 8B532C           mov edx,[ebx+$2c]
00448226 8B14F2           mov edx,[edx+esi*8]
00448229 E81AEDFBFF       call @UStrAsg
0044822E 5F               pop edi
0044822F 5E               pop esi
00448230 5B               pop ebx
00448231 C3               ret

That’s quite a lot of code, a fair share of it is spent juggling registers because TStringList.Get is implemented as

function TStringList.Get(Index: Integer): string;
begin
   if Cardinal(Index) >= Cardinal(FCount) then
      Error(@SListIndexError, Index);
   Result := FList^[Index].FString;
end;

If the range check fails, Error triggers an exception and the last line won’t ever be reached, but the compiler can’t know that, so it compiles everything assuming the last line can be reached even after an Error, thus has to save the parameters, hence the stack juggling.

(the fix to the juggling? use an “else”)

Apart from that issue, there remains in TStringList.Get a range check and an extra UStrAsg call.

And UStrAsg itself includes an atomic incrementation instruction (bus locking)

lock inc dword ptr [edx-$08]

Even though modern CPUs handle them far more efficiently than their ancestors, atomic instructions still cost quite a bit more than their non-atomic counterparts, and the more multi-threaded you are, the more expensive they get.

Summary

So when all is said and done, the hacky version has:

• 1 atomic instruction (1 UStrAsg)
• 1 call

While the non-hacky version has

• 3 atomic instruction (2 UStrAsg, 1 UStrClr)
• 4 calls (2 UStrAsg, 1 UStrClr, 1 TStringList.Get)
• 1 exception frame (free in 64bit, not free in 32bit)
• higher register pressure in UnifyAssignString
• 1 range check
• a lot of register juggling (especially in 32bit)

So while “cracking” TStringList isn’t safe, depending on the circumstances, you can use it to achieve a boost for very little extra complexity.

In a couple of weeks we are going to start the next PGD Challenge with a brand new theme. As promised it will be simple, yet fun for creating ideas. The development period will last about 4 weeks, plus a weekend to wrap things up. All are welcome to participate.

It’s going to be a lot of fun. There is lots of possibilities from the theme I’ve picked and I hope you will have fun with it.

All game entries must be made using a Pascal-based programming language or they will be disqualified. The game can take advantage of any game library or API written in another language, but the core of the game must be written in a Pascal-based language.

Allowed Pascal-based Languages:

• Pascal (aka Classic Pascal, Borland Pascal, etc)
• Object Pascal (called “Delphi language” by some)
• Objective-Pascal
• Oxygene (new language based on Object Pascal)

Key Development tools that support the above languages:

• Delphi
• Free Pascal
• Lazarus
• Prism (Oxygene for .NET)
• Oxygene for Java
• Smart Mobile Studio

I wondered how it would hold its own when compiled to JavaScript, and by that I mean compiling the old 2004 source… click on the picture to the right to run a version compiled with SmartMobileStudio. Works in all the usual desktop and mobile suspects. Click or touch the maze to solve between arbitrary locations.

The original source works on a large static array that is passed as var parameter and where the bytes are used in a bitwise fashion. All in all it does its job in a brute-force native-compiler friendly way, but is quite suboptimal for a JS execution. Anyway I compiled as close as possible to the original (update source zip, 6kB).

Very few changes were required to get it to compile:

• changed the TObjectList to an “array of TCell“
• replaced Random() with RandomInt()
• replaced the FillChar() with nested loops
• mapped TCanvas to TW3Canvas

But the bulk of the code compiled without changes, and runs in Android, Chrome, IE9, etc. (only FireFox has a glitch, which is not related to the Maze code, but to the VJL and a Mozilla event name).