An utility that was added to the DWScript sample WebServer during the 4 TeraPixel Mandelbrot experiment is the BackgroundWorkers API.
This is a rather low-level threading facility that can work well with Global Queues to handle background tasks or serializing tasks.
BackgroundWorkers static class
It’s found in the dwsBackgroundWorkersLibModule module, which exposes a System.Workers unit to the script. It has only 4 methods:
- CreateWorkQueue( name ) : create a work queue of the given name
- DestroyWorkQueue( name ) : destroys the work queue of given name, aborting all queued work
- QueueWork( name, task, data ) : queues a work unit
- QueueSize( name ) : queries the number of work units for a given queue (size of the queue)
Each queue will process its work units sequentially, you can have as many queues as you want (or rather as memory and OS allows, but that’s a lot). The queues exist outside of the script contexts, and are shared by all scripts running in the same executable.
Under the hood queue rely on IOCP and are thus quite cheap both in terms of memory and overhead.
Usage
The dwsBackgroundWorkersLibModule offers an OnBackgroundWork event which you’re supposed to tie to whatever code you’ll use to process the queued work units.
In the context of the DWS sample WebServer, tasks are programs (scripts) that will be executed as if they had been invoked by an http request (ie. their WebRequest.ContentData will hold the data you passed)
Typically, you’ll start by creating your work queues in the startup script (the “.startup.pas” file default), then proceed to give them work. From that point on:
- asynchronous workers: other requests will either immediately queue work, or accumulate in a global queue, and then create a work unit
- background (polling) task: these will usually queue themselves, perform some work, sleep & terminate (note that the sleep() function in DWS is interruptible), rinse and repeat
You’re thus free to use the BackgroundWorkers as a basic MultiThreading API, to spread work over multiple threads, or as a serialization API, to ensure a strictly sequential execution.
This looks interesting, but trying to use it for multithreading looks like it would be unnecessarily complex due to the requirement for a named queue, which also seems a bit at odds with the concept of pooling, which is essentially that any one pool item is as good as another. Is there any way to fire off a background task and just let whichever background worker is currently available service it?
What do you mean unnecessarily complex? IOPC is lighter than most other multi-threading dispatch mechanisms I could test by at least one order of magnitude. Also, it allows to sand-boxing workers from consumers.
The rationale behind named queues/pools is that when work units are generated from an already multi-threaded environment (like a web server), you’ll commonly want to avoid some work units from pre-empting/overwhelming a common worker pool, and the simpler way to avoid that is to just have multiple pools. Typical case would be to have a pool for handling computations, another for updating statistics and a third for db, and want to guarantee that the computation work units won’t stall work units for statistics or db.
> Is there any way to fire off a background task and just let whichever background worker is currently available service it?
The idea here is that you just fire a background worker and have it pull/pop work from a global queue in a loop.
There are two things missing to ease that scenario, which are waitable pull/pop for global queues (currently you have to poll or trigger through a work unit), and being able to create more than one worker per queue. I haven’t encountered a scenario so far where these were required, so these haven’t been implemented yet.
(workloads for the Mandelbrot experiment f.i. benefited from polling, as it would allow to batch work units, which helped with the database and disk I/O)
Oh, I see. You provide the pool of background workers yourself as a response to items waiting in the queue, then? That makes more sense.
>There are two things missing to ease that scenario, which are waitable pull/pop for global queues (currently you have to poll or trigger through a work unit),
Didn’t you say this is IOCP-based? On the linked page, it says:
>The worker thread is just a loop around GetQueuedCompletionStatus calls, which will have the thread wait (in the kernel) for work units, and the execute them if the command (lpNumberOfBytesTransferred) is 1.
That sounds waitable to me. What am I missing?
The workers are IOCP based, but the global queues aren’t.
So what’s missing is to have your worker(s) do something like:
GlobalQueueWaitPop(‘whatever’, newWork)
and the client threads just doing GlobalQueuePush. What you can do is push work to workers through QueueWork(). While it sounds (and is) similar, the difference are that
* Workers API uses IOCP and restarts a script execution for each work unit
* GlobalQueues wouldn’t use IOCP but the script wouldn’t be restarted
Which one is going to be more effective will depend on workloads, states that need to be kept in memory, database connections (or transactions), etc. Also using queues would allow a worker to pull data and work from different queues (which may or may not make sense depending on the workloads).
Wile it sounds complicated, it’s a bit like wrenches, which come in various forms which all serve the same basic need, but will excel or fail in special cases.