Parallel Processing

CAESES provides extensive scripting capabilities for process automation and customization. While this flexibility enables advanced workflows, script complexity can increase rapidly as projects grow.

For scripts that perform repetitive or computationally intensive tasks, users may consider leveraging multiple CPU cores to improve performance and process the tasks in parallel.

caution

Parallel execution in CAESES is inherently non-deterministic. The order in which parallel operations are executed cannot be guaranteed. Users are therefore advised to apply the provided examples carefully and verify their scripts thoroughly before deploying them in production environments.

Parallel processing is limited to the Generic Types FList<>, FArray2D and FArray3D. Please read through the Generics in the FPL documentation, especially the chapters about FCommand to understand the syntax and the signature of the parallel functions. An FList offers three functions using the available CPU power you have given to CAESES (all by default. Settings -> Experimental Options -> Max. Concurrency). All these commands are available as sequential commands as well.

• fill_parallel()
• resize_parallel()
• process_parallel()

fill_parallel() and resize_parallel()

The sequential functions fill() and resize() fill an FList with content, while resize() resets the number of list items while filling them as well. The functions fill_parallel() and resize_parallel() are the parallel equivalents. The main purpose of these functions is to parallelize a simple loop.

Assuming you have a loop which is creating multiple similar objects and storing them into a list. If only the iteration index is used (or not even that) and if the creation of these objects is somehow expensive, parallelization might become interesting. The following example only creates points which is a little trivial, but the potential should become clear.

Assuming your loop looks like this:

Simple Loop

unsigned n(100)
// initialize list
objectlist pointList()
loop(n)
  // create point and add to list
  point p($$i,0,0)
  pointList.add(p)
endloop

A parallel version of that code would look like this:

Parallel Execution

function createPoint(unsigned idx):F3DPoint
  point p(idx,0,0)
  return(p)
endfunction

unsigned n(100)
// initialize List
FList<F3DPoint> pointList()

// fill and resize list
pointList.resize_parallel(n,createPoint(unsigned))

The return type of the function needs to match the type provider of the FList. Additionally the argument of the function must be unsigned as this is the type an FList can provide as argument type of an FCommand which is the main type of a function. No additional arguments are allowed in this context.

Following table shows the syntax needed to be matched:

Type	Function Syntax
FList<type>	function myFunc(FUnsigned idx):type
FArray2D<type>	function myFunc(FUnsigned row, FUnsigned col):type
FArray3D<type>	function myFunc(FUnsigned page, FUnsigned row, FUnsigned col):type

process_parallel()

If not only an index like counter is needed but a list of objects or values, process() would be the command to use the list items as argument to a function which is doing something.

caution

The parallel version of process() should only be used if no dependencies between the list items or between the returned items exist.

The following example shows how to parse a list of strings, e.g. from a file, containing vector data.

// input template used as argument list: 
FList<FString> strList()
strList.add("1, 0.9, 0")
strList.add("2.0, 0.8, 0")
strList.add("3, 0.7, 0")

// parsing function definition
function parse2Vec(FString str):F3DPoint
  objectlist strsplit(str.split())
  double x(toDouble(strsplit.at(0).castTo(string)))
  double y(toDouble(strsplit.at(1).castTo(string)))
  double z(toDouble(strsplit.at(2).castTo(string)))
  point p(x,y,z)
  return(p)
endfunction

// parallel function call. 
// -> strList hold the argument items
// -> pointList receives the function results
FList<F3DPoint> pointList(strList.process_parallel(parse2Vec(Fstring)))

Especially the last line, the call of process_parallel(), needs to get some attention because it is somehow nested and you need understand the type dependencies as well. From left to right, or from outer to inner:

• FList<F3DPoint> pointList() creates the target list of points.
• strList is from type FList<string> and is filled with data to be processed.
• .process_parallel() calls a command/function
• parse2Vec(FString) takes the item of the item from strList, but only the type of it should has to be set here.

As the function parse2Vec returns a F3DPoint, this type needs to match the provided type of pointList and the type-dependency-circle is closed.

tip

If you need multiple arguments, consider using a compound type like FTuple, FMap, FHash, FPair, FList, FObjectlist or FEntitygroup or others.