Or in other words, before getting to the gory details, DWScript now works when compiled with {$HIGHCHARUNICODE ON} on a machine with Cyrillic code-page 1251.
DWScript was converted years ago to Unicode, and been working just fine.
But there was a leftover bug from that crossing of the Styx.
As noted in a recent post, Unicode is not so straightforward. Namely claims of utf-16 being simpler than utf-8, or that you do not have to care about Unicode complexities.
Maybe that was the case ten years ago, but The Unicode jungle is much closer to home these days.
Here are a few dangers lurking in the not-so-dark shadows.
(more…)
The FreePascal version of DWScript has been stalled for a little while on the incomplete UnicodeString (utf-16) support among other things.
It’s hard to blame the FreePascal team for that, given that Linux is primarily utf-8, and that utf-8 has quite a few advantages over utf-16.
In a now infamous and enormous thread I won’t name, Allen Bauer dropped a bomb:
<bomb>Oh, and strings may become immutable and 0-based
…</bomb>
Here is a summary of recent changes for DWScript SVN-side:
In Unicode Delphi, post-Delphi 2009, there are two ways of making case-insensitive string comparisons, CompareText, which only does case-insensitivity in the ASCII range (non-accentuated characters), and the judiciously misnommed AnsiCompareText, which works on the whole Unicode range by calling into the Windows API.