Adventures with LB2D and OGSI::Lite

Historical note: this article was written in 2003, before Facebook, Twitter, V8, or AWS existed, and in the very early days of what was termed "Web 2.0". The toolchain for writing web applications has changed and improved massively since then - now you can quite easily run a Lattice Boltzmann simulation in the browser if you want to.

With hindsight, you can sort of see that we were nudging in the right direction; towards the end of the project which gave rise to this article we were getting interested in describing everything in terms of RESTful services, and this is now a standard technique. SOAP was kind of big and clunky but still sees use in Java-heavy corporate environments. "The Grid" turned into/was overtaken by "The Cloud", which was more ad-hoc and owned by Jeff Bezos rather than academic consortia. In 2003 you still sort of expected Serious Applications to have a native-GUI interface, whereas now browser-based interfaces are completely standard. Web services are now commonplace: if anything, maybe too commonplace.

LB2D doesn't really get used today, but I believe its big brother LB3D still sees some use in Jens Harting's group. I've long since changed careers, but "write a dumb solver in C/C++ and do everything else in a higher-level language" has been a consistently productive approach.

Introduction

This is an informal page to describe some exploratory work which has been done using the LB2D lattice Boltzmann code and the OGSI::Lite perl Grid Services container. These two components are described, followed by details of how they were linked together to build a Grid Service for lattice Boltzmann simulations. This allowed LB simulations to be launched on a remote machine and steered, using a web browser as the only client software.

Dramatis personae

LB2D

LB2D is an implementation of the lattice Boltzmann algorithm for fluid dynamics. The LB method can be regarded as a way of solving the continuum Boltzmann equation on a discrete lattice, although historically it was derived as a simplification of the lattice gas cellular automata method. It is commonly used in fluid dynamical problems, and possesses several attractive features: LB2D is a lightweight implementation of the lattice Boltzmann algorithm in two dimensions, and can simulate flow of binary interacting mixtures under a wide variety of boundary conditions.

Evolution of LB2D

LB2D is written in ANSI C, and runs on a wide variety of platforms, such as Linux (on x86, IA64, and Sparc64 systems), Solaris, IRIX, AIX, Compaq Tru64, and Cygwin. Calculations are not parallelized, although it supports use of MPI as well as the UNIX fork(2) call to spawn large numbers of simulations in a task farm.

Originally, it ran in the "traditional" mode for scientific codes: it read and parsed an ASCII input file, constructed a simulation accordingly, with the appropriate initial and boundary conditions applied, wrote the appropriate output files as it ran, and then terminated. This allowed simulations to be submitted to a batch queue, but did not allow any interaction with the simulation after it was started.

As the code evolved, it was used to simulate many different systems, each with its own particular set of initial conditions, boundary conditions, output styles, and parameters. Each time a new simulation system was added to the code, the input file parser had to be modified, and more C code had to be bolted on to the solver to implement the relevant boundary conditions. This led to the code becoming quite complicated and bloated, requiring frequent rewrites or refactorings.

To work around these problems, a scripting layer was added to the code, to effectively permit Turing-complete input files. Then, the description of each new system to be simulated could be encapsulated entirely in the (ASCII) script used to drive the simulation; new systems could be simulated simply by writing new scripts without touching the internal solver code.

Structure of LB2D

The C level
Internally, the entire state of a simulation is encapsulated in a sim structure, which contains all of the model parameters, a description of the boundary conditions, and the actual lattice data. Most functions in the code take the form of "methods" which take a sim object as their first parameter, and operate upon it. A common test case for multicomponent fluid models like LB2D is Spinodal Decomposition, the process of separation of a mixture of two normally immiscible fluids (like oil and water at room temperature) which are initially mixed up (because they have been vigorously stirred together, or heated above a certain critical temperature and then allowed to cool). If the mixture is initially composed of an almost-homogeneous 50/50 mix, it will quickly separate into single-component regions. Simulation of this situation in LB2D is straightforward: The C code to do this is relatively straightforward, as shown below, with the (sometimes quite complicated) code to dump output files omitted for brevity. 

	sim *mySim;
	int nx=64,ny=64; /* Size of lattice */
	int t;
	int maxtime=1000;

	mySim = sim_new(64,64);		/* Create new simulation */
	sim_settodefaults(mySim); 	/* Use default values of parameters */
	sim_allboth(mySim,1.0); 	/* Initialize to a mixture */
	sim_perturb(mySim,0.1); 	/* Perturb the system */

	for (t=0;t<maxtime;t++) {
		sim_timesteps(20);	/* Run for 20 timesteps */
		/* Code to dump output goes here.. */
	}
The Perl level
The quasi-Object-Oriented design of the C code made it particularly easy to build a perl interface to LB2D. Most of the details of the C layer are described in a single file, lbe2d.h, which defines all important structures, and contains prototypes for most of the useful LB2D functions. Perl was chosen as the scripting language, mainly because of existing experience with the language, and because of CPAN, a repository of a very large number of open-source code modules. There is, however, no reason why LB2D could not be interfaced to other languages such as Python, Ruby, or Java. The interface between the Perl and C levels is described in XS, a form of heavily-preprocessed C; most of the XS code can be automatically generated from the lbe2d.h file. The end result is a Perl module, called LB2D, which makes all of the LB2D functionality available to a high-level scripting language. The entirety of the perl script to model phase separation is shown below. 
use LB2D;

my $lb2d = LB2D->new(              # Create a new simulation.
		nx => 64,
		ny => 64,
		g_cc => 2.0,
	);

$lb2d->allboth(1.0);               # Initialise to a perturbed mixture.
$lb2d->perturb(0.1);

for my $t (1..100) {

	$lb2d->timesteps(20);      # Run for 20 timesteps.

	$lb2d->dump_img(           # Create an image file.
		filename => "phi-$t.png",
		type => "png",
		data => "phi",
	);
}
(Phase separation image) (Phase separation image) (Phase separation image) (Phase separation image) (Phase separation image)

Images generated from an LB2D simulation of phase separation.
In this case, the corresponding code to produce output has been left in; the LB2D module has an interface to Perl's powerful Imager module, which allows it to generate image files directly from its internal data. Every 20 timesteps, a PNG-format file showing the distribution of oil and water in the simulation is written to disk, showing the separation of the mixture into separate regions of oil (black) and water (white); these slowly grow and coalesce with time.

OGSI::Lite

OGSI::Lite is a container to allow Grid Services to be created in Perl, written by Mark McKeown at Manchester. It allows a Perl class to be quite easily exported as a grid service, by inheriting from the OGSI base classes: remote procedure calls are dispatched to the Perl class using the well-known SOAP::Lite module.

Exporting LB2D as a grid service

It was possible to export LB2D as a grid service using just over a hundred lines of extra code. A factory service was constructed simply by copying a demonstration factory service supplied with OGSI::Lite, and changing a few lines to make it spawn LB services instead of demonstration services.

Consider a perl module which contains some data, and makes available several functions which operate on that data. The module can be turned into a grid service simply by placing it in a certain directory, so that the OGSI::Lite server can find and spawn copies of the module as requested. If there is a function called init() defined in the module, then it will be called at startup time to initialise state. Once spawned, all functions of the module are made available to external clients through the SOAP protocol: if a client attempts to call a function called foo() using the grid service, OGSI::Lite will decode the SOAP function call request, invoke the corresponding foo() call in the Perl module, collect the result of the call, encode it as a SOAP response, and send this back to the client. The underlying module needs to know very little, if any, of the details of the communication with the client.

A Perl module called LBService.pm was written to make LB2D available as a grid service. Initially, it contained a few debugging routines of its own, and passed most function calls over to LB2D.pm, which performed the actual calculations. This pass-through effect was easily implemented using Perl's AUTOLOAD feature. This meant that the debugging function dump_args() would be caught and processed by the LBService module; the boundary condition function allboth() would be sent to the LB2D module.

A simple command-line interface to remote simulations

The procedure for launching a remote LBService instance is as follows: A simple command-line client was written to perform this procedure, and then allow a remote user to interact with the simulation once it had been spawned. An example session with this client is shown below, with user commands and comments in blue, and the responses in black. 
> generate nx 64 ny 64 g_cc 2.0 # Create new LB instance
LB2D object generated.
> get_total_rhor                # Find total density of red component
0
>  allboth 1.0                  # Fill system with fluid mixture.

> get_total_rhor
2048
> perturb 0.1                   # Randomize densities

> get_total_rhor                # Total density is very slightly different.
2048.36624704789
> timesteps 100                 # Run for 100 timesteps.
0
>

Hence, it is possible to set up and run simulations remotely:- almost all of the functionality of the LB2D module is made available to remote clients. Moreover, steering comes for free: while it is possible to spawn and run simple simulations which simply run for a fixed number of timesteps, it is also possible to interact with a remote simulation as it runs. However, extraction of data is still a problem: the routines for extracting images or other complex simulation data in LB2D.pm return chunks of binary data, which require extra encoding at the SOAP level, and specific client-side processing; a client must save images to disk or display them.

A thin steering client for LB2D simulations

Once spawned, the OGSI::Lite grid services communicate with clients using the SOAP protocol. Recent versions of the mozilla web browser allow it to interact with remote systems using SOAP, via scripts written in the Javascript language. This allows a simulation/steering client to be constructed and embedded entirely in a web page containing Javascript code.
(Block diagram of FileStore)

Operation of a FileStore service.

The LBService module was modified so that, upon startup, it would generate an instance of another grid service, FileStore. This service can act as a remote shared filestore, allowing data from the simulation to be stored over the Grid, and requested by the end user as required.

A client system, such as an LBService instance, sends the FileStore service a block of binary data to be saved. The FileStore saves the data to a file which is visible to a web server, and returns a URL which can be used to retrieve the file.

This method of storing simulation data has several advantages:

The LBService module was modified to intercept calls to the get_img() method. Normally, when called on an LB2D object, this object returns binary data corresponding to an image generated from simulation data. When get_img() is called on an LBService module, it makes the corresponding LB2D call, and turns the data into a PNG format image, which is stored in a FileStore instance; the get_img() call then returns the corresponding URL.

This process can be driven from the command-line client, as shown below. 

>  generate nx 64 ny 64 g_cc 2.0
LB2D object generated.
>  allboth 1.0

>  perturb 0.1

>  timesteps 100
0
>  get_img data phi   # Retrieve composition data

http://un.earth.li/~jon/TempFile/temp-17920/image-0.png
>  timesteps 100
0
>  get_img data phi

http://un.earth.li/~jon/TempFile/temp-17920/image-1.png
>
The Javascript SOAP interface is not quite versatile enough to handle communication with OGSI::Lite to spawn new grid services, so a proxy SOAP service was written. This makes available a single SOAP call which will start an LBService instance, and return a GSR, with all necessary information for a client to talk to the instance using SOAP. It was then straightforward to write a small amount of Javascript code which would launch new simulations using the proxy service, allow steering of simulation parameters, and retrieve and display image data as shown below.
(Screenshot of browser client) (Block diagram of components)

A screenshot of a browser-based steering session, and a diagram showing the components involved.

Problems with OGSI::Lite

Further work

$Id: index.html,v 1.7 2003/09/29 13:51:24 jon Exp $