Search Results

Keyword: ‘json’

DWScript news roundup for May 2013

May 7th, 2013

dws-mirrorLanguage and Script Engine

  • Delphi XE4 is now supported, compiler hints for XE3 have been taken care of as well
  • compiler now supports “in” operator across strings, for for instance “if subString in myString then” is equivalent to “if myString.Contains(subString) then
  • added standard “helpers” for most built-in functions operating on String, Integer, Float and Boolean. These roughly follow the Delphi & .Net conventions, though standard Maths functions are accessible directly (f.i. you can do “angle.Cos” directly, rather than have to go through “Math.Cos(angle)“)
  • added some more string functions (DeleteLeft, DeleteRight, etc.)
  • added support for EmptyParam for the COM connector
  • added DivMod to the standard functions
  • improved performance of DecodeDate/Time functions
  • improvements and fixes for the JSON connector (now supports manipulating an existing JSON structure, creating, moving and deleting nodes)
  • misc. fixes and improvements

Note that there aren’t specifics DPKs for XE4, you can just copy/rename the XE2 or XE DPKs.

Changes for SmartMobileStudio / JavaScript Codegen

In addition to the  previous changes, the following will apply to the next version of SmartMS:

  • support “for str in variant” which allows enumerating members of a raw JavaScript object (compiles to “for (str in v)”)
  • improved code generation for functions with an “exit” statement
  • added specific optimizations for string copies and testing (startswidth/endwidth)
  • slightly faster code generation

 

News ,

Publishing pictures from an HTML5 app

March 15th, 2013
Comments Off

imgur-logoThis is a belated followup to the L-System Fiddle series from last year (more like a forgotten article), which ended with a Pascal-based HTML5 app being used to generate fractals and publish them to imgur.

Why imgur? Well because they have a well-documented API and they don’t require registration, authentication or other personal data for uploads, making it a good candidate for a demo anybody can try online, and where you may not want to enter your personal credits.

L-System Fiddle

You can try the L-System web app for yourself, it comes with a few sample L-System fractals, and it allows tweak the grammar, iterations and basic simple rendering parameters.

 

In SmartMobileStudio 1.1 final, the whole L-System Fiddle source is included, and there is an “ImgurUtils” unit which exposes a class that facilitates asynchronous uploading of images to Imgur. Below is an excerpt of the upload method:

...
   FReq : TW3HttpRequest;
...
procedure TImgurUpload.Upload(canvas : TW3Canvas);
begin
   var img := canvas.toDataURL('image/jpg');
   var imgParts := StrSplit(img, ',');

   FReq.Open('POST', 'http://api.imgur.com/2/upload.json');
   FReq.SetRequestHeader('Content-Type', 'application/x-www-form-urlencoded');

   FReq.Send( 'key='+APIKey
             +'&title='+EncodeURIComponent(Title)
             +'&caption='+EncodeURIComponent(Caption)
             +'&image='+EncodeURIComponent(imgParts[1]));
end;

The first two lines get the base64-encoded image data from the canvas, the next lines perform the imgur API call. They’re using inline variable declaration with type inference.

Imgur answers with a JSON (asynchronously), here is handler that retrieves the URL of the uploaded image:

procedure TImgurUpload.DoReadyStateChange(Sender: TW3HttpRequest);
begin
   if Sender.ReadyState<>rrsDone then exit;
   ...
   if Sender.Status in [200..299] then begin
      FResponse := JSON.Parse(Sender.ResponseText);
      FImgurPage := FResponse.upload.links.imgur_page;
   end;
   ...
end;

So as you can see, getting the basic behavior running is quite simple, and here is what it’ll look like after the upload has completed:

Picture uploaded to imgur

Picture uploaded to imgur

Tips , , , , , ,

Introducing dwsDatabase

January 23rd, 2013

dbDWScript now has database support classes built-in, these are based on a thin, interface-based layer, and can be used Delphi-side as well to get automatic memory management for simple DB access.

It currently supports mORMot SynDB and Universal InterBase, meaning it gets high performance native connectivity to SQLite, FireBird, Oracle, ODBC and OleDB (MySQL, MS SQLServer, MS Jet, AS400…).

Usage

Usage is very simple, for instance to connect to a local SQLite database and print two fields for a table:

var db := DataBase.Create('SQLite', ['d:\db\base.sql3']);
var query := db.Query('select fld1, fld2 from mytable where fld3=?', ['filter']);

while query.Step do
   PrintLn(query.AsString(0)+', '+query.AsString(1));
query.Close; // optional, only required if you need it closed ASAP

and to perform several queries in a transaction

db.BeginTransaction;
db.Exec('delete from mytable');
db.Exec('insert into mytable (fld1, fld3) values (?, ?)', ['hello', 'world']);
db.Commit;

The DataSet class supports classic EOF/Next iteration as well as the simpler Step iteration as in the previous example.

There are also JSON generation helpers, you can get a whole data set or a single record as JSON. This makes building ajax requests or even database “middle-ware” services simple.

Supporting other DB connectivity layers is quite simple, as you’ll see if you look in the source. SynDB & UIB were picked initially because together they offer high coverage of the usual suspects, and bring what are among, and maybe “the”, best in class performance and stability for Delphi DB connectivity these days.

Exemple: minimal web service

Those classes will be (are) used in leveraging the new “DWScript-returns-to-its-web-roots” Web Server (based on Synopse server). For instance in the DWS WebServer demo, you can make a minimal database “middle-ware” service with Windows domain authentication with just the following code:

case WebRequest.Authentication of
   WebAuthentication.None : 
      WebResponse.RequestAuthentication(WebAuthentication.Negotiate);
   WebAuthentication.NTLM .. WebAuthentication.Kerberos : 
      Print(DataBase.Create('UIB', ['dbServer:d:\db.fdb', 'login', 'password'])
                    .Query(WebRequest.QueryString)
                    .StringifyAll);
else
   WebResponse.StatusCode := 401;
end;

The above code will expect http requests with the sql as query string (ie. after the ‘?’ in the url). If the connection isn’t authenticated, it’ll ask for authentication. Note that Windows domain Single Sign On is supported by Chrome, Internet Explorer and FireFox (if you enabled it).

The query is run against a FireBird database hosted on another server, and the result returned as JSON. If the query fails for a reason or another, the client will get automatically get a 500 error code with the exception or error message. Oh, and the reply is compressed automatically if it’s larger than a couple hundred bytes.

And if you want to only serve on a secure connection, just add a check for WebRequest.Security, and you can then safely use extra or alternative authorization tokens or credentials.

News , , , ,

Thirteen features of DWScript

December 31st, 2012

2013After having survived the Mayan apocalypse and just before the year ends, and the US economy falls from the fiscal cliff into the pit of recession, here is a quick look at thirteen features of DWScript, so you don’t come into 2013 unaware :)

1. Supercharged memory management

ARC is trendy, but introduces cycles and weak references hell, GC is cool, but garbage collections stalls aren’t, deterministic memory management is fast and deterministic, but verbose and error-prone.

Why not have them all at the same time work together instead? That’s what you get in DWScript: ARC for immediate releases in simple cases, GC to collect cycles and avoid the weak reference, and manual deterministic release when you need it.

2. First-class dynamic arrays

Dynamic arrays are strongly typed, first-class dynamic arrays have methods to Add, Delete, etc. like a (T)List<T>.

And where automatic memory management absolves you from destructors most of the time, true dynamic arrays absolve you from requiring constructors for list and LIFO queues.

3. Type-inferenced variables

Why manually specify the type when the compiler can infer it for you and you get a strongly typed variable?

“var a := 2;” is the same as “var a : Integer; … a := 2;” would be in Delphi.

4. Combined field declaration and initialization

Also type inferenced, fields default values can absolve you from having to write a constructor in trivial cases.

5. Multiple helpers per type

Helpers aka extension methods can be a powerful way to add functionality to existing types. Having a limit of one-helper-per-type hamstrings them.

6. Multi-line strings

Whatever your strings or constants are used for, sometimes they need to span more than one line. When that happens, being able to absolve yourself from concatenating an obscure #13#10 is nice. Having a syntax that will allow you to properly indent those strings without introducing spurious indentation in the string is even better.

7. Lambda syntax

This applies to SmartPascal, where closures and anonymous methods are common, by leveraging type inference and a less verbose, yet not operator-soupy syntax, you can have a syntax far more compact, yet still statically and strongly typed. Enough to make one happy.

8. Custom language extensions

Sometimes it would be nice to introduce custom construct into your source code, with DWScript’s language extension mechanism you can do just that and take over the parser for a section of code, can be used to introduce asm or JSON capability.

9. Unicode-safe strings and iterations

Because sooner or later you’re going to end up parsing Chinese and your 16-bit-characters assumption are going to haunt you. Having Unicode-aware loops and no 8-bit or 16-bit Char type in the language avoids that.

10. Unified function pointers / delegate

Because life is too short for procedure vs procedure of object vs reference to procedure. Let the compiler figure it out.

11. Compile-time function evaluations and constant expressions

When something is constant, it’s constant, no? DWScript lets you use constant expressions to define constants, so that you can have a constant defined by just “Sin(PI/4)” rather than a magic number plus a comment to explain what that magic number is.

12. Partial classes

Partial classes allow you to support automatically-generated code more easily, including code generated from compile-time type information, or just make your classes more extensible, by design.

13. Deprecated

Feature #13 got deprecated and isn’t available anymore.

14. Being able to compile to HTML5+JavaScript

With SmartMobileStudio, Make your business logic cross-platform, cross-terrain and as ubiquitous as  it can get these days. Run in mobile or desktop browsers, run server-side with node.js, or client-side with node-webkit as runtime environment.

Tips , , ,

DWScript pre-Apocalypse news

December 17th, 2012

Logo DWScriptWith the end of the world in a few days, it’s time to flush the what’s new log for DWScript.

There are two major items that will be discussed in future posts:

  • the JSON connector, which gives direct, JS-like access to JSON-based data objects
  • the Gabelou, which aims to become to DWScripot what StyleCop is to C#.

Also the old Indy-based webserver demo is being replaced by one based on Synopse high-performance http server classes.

Language

  • Support for var to do loop syntax
  • Support unit namespace syntax
  • Support Low/High/Length methods on strings
  • Delphi compatibility: classic units (with interface/implementation) now treat private/protected visibility in the same way as Delphi, visibility rules are unchanged for scripts and consolidated units.
  • Delphi compatibility: tolerate (and ignore) Delphi call conventions (safecall, cdecl, etc.), these will now trigger a “strict”-level hint rather than an error.
  • Improved error messages when attempting to use instance methods in a class or static method
  • Support nil for function pointer default parameters
  • Fixes when using helpers on implicit Self reference inside methods
  • Fixes for partial classes support

Scripting

  • JSON Connector (mostly read-only at the moment, cf. future article)
  • Improvements when using the compiler to compile/test a standalone unit rather than a script or program
  • Support declaring internal static methods
  • Added facilities for subclassing TdwsProgramExecution
  • Fixed Guardian issue when guarded scripts are still running after the guardian is released
  • HexToInt now throws an exception when value is invalid
  • Improved unit tests coverage

Tools & Utilities

  • Introduced Gabelou (cf. future article)
  • Initial test server based on Synopse high-performance http server
  • Faster UnicodeCompareLen/UnicodeCompareText

News

Desktop HTML5 apps game changer? node-webkit

December 13th, 2012

icon-node-webkit-pre-releaseRecently stumbled upon node-webkit, an open-source from  Intel Open Source Technology Center, which is deliciously “simple”: combine a standalone webkit (Chromium) build and node.js into a runtime.

Webkit/Chromium brings a high-performance HTML5 environment with JavaScript JITting compiler, WebGL, Audio, hardware accelerated Canvas, camera access, etc.

Node.js brings high-performance access to the local machine resources, so that HTML5 apps can break out of the browser and get access to the file system, databases, server sockets, etc. Just like any other regular application. Check the wiki for more details.

Could it be a game changer?

You get many of the capability of managed frameworks ala .Net / Java with the deployment convenience of a native application.
Node.js asynchronous design means it will just run circles around anything else in a variety of I/O scenarios, and the JavaScript V8 engine can duke it out not just with Java & .Net JITters, but also many native compilers.

  • The runtime is available for Windows, OSX and Linux, so you’re instantly cross-platform.
  • Bundling Chromium means you’re no longer dependent on the browser available on the target machine, its updates… or lack of.
  • You just deploy an exe with a few (optional) companion DLLs (which you can just place alongside the exe).
  • No registry hacks needed: the apps are portable and can run just fine from an USB stick f.i.
  • Application options are controlled by a simple json manifest

Bundling an HTML5 SmartMobileStudio App

wartrail_smallI couldn’t resist the temptation to bundle WarTrail, my entry to the recent PGD Challenge as a node-webkit app. The node-webkit runtime stands at about 13 MB in an installer, which is a lot lighter than .Net or Java runtimes, and not so heavy compared to some native frameworks I won’t name whose executables start at 8-9 MB.
If you have a lot of resources and data, it becomes a quite acceptable overhead (f.i. media for WarTrail stands at nearly 17 MB compressed, which is light by modern game standards).

You can download the resulting installer (30 MB, built with node-webkit 0.3.4 & InnoSetup). The app app starts instantly and runs at high speed. Memory usage remains quite reasonable given the media resources involved.

For those that want to try it in Linux or OSX, grab the node-webkit precompiled binaries there (scroll down to “Downloads”), the get WarTrail.zip (17 MB), rename it to “WarTrail.nw” (sorry, server doesn’t recognize .nw extension), and feel free to report back on the bundling capabilities of node-webkit on those two systems.

So there you have it, with node-webkit, a SmartPascal app can look like a native app, behave like one, and access resources like one.

News , , , ,

DWScript – October 2012 – 2.3 preview 2

October 31st, 2012

A new DWScript 2.3 preview 2 7zip (422 kB) is available for download for those that don’t use the DWScript SVN. This version should be the last before a release candidate for 2.3. As the lead version is Delphi XE, users of Delphi 2009, 2010, XE2 and XE3 are especially encouraged to report any issues they encounter. FreePascal support is still considered experimental at this moment.

Language improvements

  • Introduce new root Object above TObject (see this article for details)
  • ‘_’ (underscore) is now allowed for formatting purposes in binary and hexadecimal literals, ie. you can write $DEAD_BEEF or 0b110_001_010
  • Script RTTI now supports published fields
  • Dynamic array parameters can now have [] (empty array) as default value
  • Added missing $HINTS PEDANTIC switch
  • Fixes for some cases of records holding sub-records
  • Fixed comparison of a meta-class against nil (previously only Assigned() was supported)
  • Fixed error message for HexToInt
  • Missing a semi-column at the end of a procedure prototype just before its implementation is now a warning rather than an error

Compiler and support libraries

  • language extension have a new StaticSymbols virtual method, which allows static symbols to be used even when language extensions are in use
  • dwsJSON Delphi 2009 compatibility fixes (thx Hallvard Vassbotn)
  • dwsJSON new AddValue() overloads, new jvtUndefined, minor speedups and other minor fixes
  • XE3 compatibility workaround (thx Steffen Binas)

Quick note about static symbols:

TdwsUnit has a StaticSymbols property, which when set to true will allow that unit’s symbols to be considered static by the compiler, ie. reused across multiple compilations.

When static symbols are not used (the default), each compiled scripts is self-sufficient, it holds its own copies of the symbol tables, and is sand-boxed from scripts compiled before or after it. When static symbols are used, the units with static symbols are shared across compiled scripts, which will speedup compilation and reduce memory usage.

When you enable static symbols, the TdwsUnit (and all it dependencies), is no longer allowed to change. You should also avoid manipulating the symbols and symbol tables (through reflections or symbol libs), unless you want those manipulations to affect all compiled scripts.

News ,

Spotlight on dwsJSON

September 17th, 2012

dwsJSON is a unit that supports JSON parsing/creating, it’s part of DWScript but relatively “standalone”, in that if you add it in your Delphi (or FPC) projects, it won’t pull the whole of DWScript library, and thus can be used anywhere you need.

The JSON parser in dwsJSON generates an object tree based on TdwsJSONValue (and its subclasses), you can also create such a tree from scratch and use it to spit out JSON. There is also a low-level JSON writer for fast, write-only streaming. All classes support full Unicode (in Delphi) as well as very long strings (like a base64 streamed image).

dwsJSON is rather fast, especially compared to Delphi’s DBXJSON, don’t be surprised if it parses data 15 to 30 times faster (f.i. on JSON.org examples). And the ratio can go even higher on larger JSON datasets. It is also more resilient to bad input, less buggy ( ;) ), and should use less memory in just about any usage scenario. It also is about twice faster than the last version (as of this writing) of SuperObject, which is itself quite efficient.

Parsing JSON

Let’s suppose you have the following JSON data

{
   "Hello" : [
      "here",
      {
         "There" : 456;
      }
   ],
   "World" : 123
}

and you loaded it in the rawJSON variable, you can parse it with

var json : TdwsJSONValue;
...
json := TdwsJSONValue.ParseString(rawJSON);

That’s all.

Accessing the JSON data

You can access the data using explicit methods and properties (see the source and unit tests), or use the short forms:

WriteLn('Hello has ', json['Hello'].ElementCount, ' elements');

WriteLn('The first element has a ValueType of ', GetEnumName(TypeInfo(TdwsJSONValueType), Ord(json['Hello'][0].ValueType)));
WriteLn('The second element class name is ', json['Hello'][1].ClassName);

WriteLn('The first element has a first field named "', json['Hello'][0].Names[0], '"');
WriteLn('and its JSON representation is ', json['Hello'][0]['here'].ToString);

WriteLn('The value of the DoesNotExists field is "', json['DoesNotExists'].ToString, '"');
WriteLn('and its type is ', GetEnumName(TypeInfo(TdwsJSONValueType), Ord(json['DoesNotExists'].ValueType)));

WriteLn('Also World = ', json['World'].ToString);

will give the following output:

Hello has 2 elements
The first element has a ValueType of jvtObject
The second element class name is TdwsJSONImmediate
The first element has a first field named "here"
and its JSON representation is {"There":456}
The value of the DoesNotExists field is ""
and its type is jvtUndefined
Also World = 123

One thing of note is that TdwsJSONValue allows to navigate in nodes that do NOT exist in the JSON (ie. nil objects in Delphi), they’ll have a ValueType of jvtUndefined, if you query their value, you’ll get an empty string, a zero, etc.

This can be particularly convenient when the JSON has optional portions (common in JSON), as it allows to drastically cut down on the “if” testing. For instance you can write

json['DoesNotExists'].ElementCount

even if json['DoesNotExists'] is nil, ie. if your array is missing in the JSON, ElementCount will be zero, and if you write

json['DoesNotExists'][2]['Field']

it will return a nil TdwsJSONValue, whose ValueType will be jvtUndefined.

Altering or writing JSON

You can alter an existing (or newly generated) JSON tree by using the AddXxxx methods and assign values using the AsXxxx. You remove elements through the DetachChild method, which can be used to delete elements, or to split a monolithic JSON into simpler sub-trees.

Another option to generate JSON is to use the lower-level TdwsJSONWriter, which is a simple streamer (so the output JSON will only be correct if you write everything needed). It has a TdwsJSONBeautifiedWriter subclass, which instead of generating a “tight” JSON will introduces tabs, spaces and new lines to make the output more palatable when it’s intended to be human-readable.

The TdwsJSONWriter provides a set of WriteXxxx methods, along with Begin/End Object/Array methods, so can be easily adapted to most serialization frameworks.

Licensing

Like the rest of DWScript, dwsJSON is under MPL 1.1, ie. free to use in both open-source and commercial software, as long as you honor the license terms: the DWScript project or dwsJSON must be “conspicuously” mentioned, and you have to publish modifications to the code (if you make any, you don’t have to publish anything if you use it verbatim). If you’re interested in using it under other terms, feel free to contact me.

Tips ,

Debugging PhoneGap/Cordova apps on the Desktop

September 11th, 2012

If you’ve tried building PhoneGap/Cordova apps, be it in Object Pascal via Smart Mobile Studio or directly in JS, you’ll quickly have realized that debugging on the devices, even with remote debuggers and emulators, isn’t always convenient.

Being able to debug in a desktop browser would be so much more convenient, eh?

The most commonly recommended way to do that is via Ripple, a Chrome plugin, which is indeed pretty, but IME proved quite buggy and heavy (meaning that Chrome debuggers would spend more time reporting stuff buried deep in ripple.js than in my own code).

Enter phonegap-desktop, an open source project by Jonathan Prince, which is essentially just a phonegap.js library that exposes the PhoneGap API, and allows you to provide fake device data just by writing in a json file. Simple, no-fuss.

To use it with a SmartMS project, just copy the “phonegap-desktop.js” file in the bin subfolder of a project, rename it to “phonegap.js“, et voilà! you can then run your app in Chrome or FireFox as usual. And when building the zip for Phonegap Build, just don’t include that file.

Tips , , , , ,

DWScript July 2012 news

July 13th, 2012

Here is a summary of recent changes since the last news roundup. These apply to the SVN version, which will likely become 2.4 soon, as it includes quite a bit of improvements and fixes over 2.3 (which was released in march).

Language

  • Added support for initialization and finalization for units, unlike with Delphi, you’re allowed to declare variables in those sections.
  • Added support for deprecated for properties.
  • Delphi compatibility; accept “class helper” syntax, also added “interface helper” and “record helper“, which restrict the allowable types for a helper. Unqualified helper declaration is unchanged and still allows helpers on any type.
  • Delphi compatibility: accept “reference to” and “of object” for function pointers/delegates declarations, the compiler just ignores them, as functions pointers are unified in DWScript.
  • Added Swap() internal magic function, Swap(a, b) will swap the values of a & b as efficiently as possible, and works for all types (arrays and records included).
  • Added StrToBool() to the standard functions.
  • Added hint about private functions that are marked “virtual” (likely error, since they aren’t overridable).
  • Added “pedantic” hints about parameters of class type that are passed as const.
  • Fixed resolution of “=” and “<>” overloaded operators when applied to classes.

Script Engine

  • Reduced compiled scripts memory usage (by 5 to 25%), also reduced script runtime objects memory usage slightly.
  • Added an explicit lock to TDelphiWebScript to prevent multi-threaded compilation (multithreaded execution of compiled programs is not affected). A particular instance of TDelphiWebScript can only perform one compilation at a time, but you can have multiple instances of TDelphiWebScript to perform multiple compilations at the same time in different threads.
  • Fixed a very rare bug in parameter passing that could occur for some combinations of parameters and memory allocations, it could result in spurious script errors that were almost impossible to reproduce.
  • Fixed a bug with temporary space allocation for some methods

Support tools and classes

  • Refactored and cleaned up the SourceFile list of compiled programs, fixed compiled line count for script involving includes and units.
  • Fixed parsing of Unicode literals in the JSON support classes (characters specified by code rather than in the text)
  • Fixed some methods in the SymbolDictionary that were case sensitive

Changes applying to SmartMS (for v1.0.1 and beyond)

Beyond v1.01

  • Devirtualization support, it allows a reduction of compiled JS size, and can give a small execution speedup. It also means it’s now harmless to mark methods as virtual even if they’ll only be very infrequently overridden.
  • Sourcemaps and “debugging” source files support is being tested (allows to debug a compiled application right in Chrome Dev Tools or FireBug on the Pascal source code, placing breakpoints and watches in your Pascal source, with JavaScript only appearing as the CPU view would in Delphi)
Included in v1.0.1
  • Date/Time functions are now supported on the JS side, offering a floating-point based date time which is compatible with TDateTime
  • Fixed publishing of record properties that map to class functions, class vars or class consts
  • Fixes and improvements for Integer() and Float() casts
  • Fixed constructing sub-classed externale classes

 

News