Summary:
Rheology is the study of the flow properties of liquids. The flow properties of interest in our research are viscosity and yield stress. Viscosity is a measure of resistance to flow, where a high viscosity indicates a high resistance (e.g. honey) and a low viscosity indicates a low resistance (e.g. water). Yield stress, a related property, is a measure of the amount of force needed to initiate flow. These properties are typically measured using a device called a rheometer.
Our research focuses on the rheology of dense suspensions, specifically cement, mortar, and concrete. Understanding the flow properties of these dense suspensions, as well as other complex fluids such as colloids and ceramic slurries, is of importance to industry and presents a significant theoretical challenge. The computational modeling of such systems is challenging due to the large range of length and time scales involved as well as the difficulty of tracking boundaries between the different fluid/fluid and fluid/solid phases.
The main problem that we are attempting to solve is that existing rheometers are not currently able to measure properties of dense suspensions, such as concrete, in fundamental SI units, such as viscosity in the units of Pa • s or N • s/m^{2}. The best that can be done currently is rank suspensions in proper order by viscosity. The goal of this project is to develop a model and a set of reference materials suitable for the calibration of rheometers in order to enable the accurate measurements of these materials, either in the field or in a laboratory.
This research project has required combined efforts in the theory of the rheological properties of dense suspensions, physical experiments, and computational experiments, in order to develop suitable reference cement paste, mortar, and concrete materials.
To date, we have released two Standard Reference Materials (SRMs)  one for cement paste (NIST SRM2492) and one for mortar (NIST SRM2493). A third SRM, for concrete, is under development.
In addition to the development of these SRMs, we are also studying the flow of cement paste, mortar, and concrete, within rheometers of various designs. The raw output from the rheometers we are studying is the angular velocity of the blades with a fixed applied torque. In particular, we are studying rheometers with 6blade vane spindles and rheometers with doublehelix spindles.
Additional Technical Details:
Simulating the Flow of Dense Suspensions
We employ two computational methods to simulate the flow of dense suspensions, with the choice depending on whether the matrix fluid of the suspension, that is, the fluid part of the suspension, is Newtonian or nonNewtonian. A fluid is Newtonian if it has a constant viscosity (for a given temperature), regardless of any shearing forces applied to it. A common example of a Newtonian fluid is water. A fluid is nonNewtonian if its viscosity (for a given temperature) is not constant but is a function of the shear rate. There are several categories of nonNewtonian fluids based on how their viscosity changes in response to shearing. For example, nonNewtonian fluids that exhibit a lower viscosity while being sheared, as opposed to at rest, are referred to as 'shear thinning fluids'. Conversely, nonNewtonian fluids that exhibit a higher viscosity when sheared are referred to as 'shear thickening fluids'.
For simulating suspensions comprising a Newtonian fluid matrix we use the computational method called Dissipative Particle Dynamics or DPD, invented in the early 1990s [Hoogerbrugge]. DPD is used, instead of the more direct models of Molecular Dynamics (MD) or Brownian Dynamics (BD), in order to study the physical behavior of systems of a larger size and for longer time intervals, by orders of magnitude, than would be possible with either MD or BD. Also, the tracking of boundaries between the different fluid/fluid and fluid/solid phases is made tractable with DPD.
Each of the particles in a DPD model is a mesoscopic particle that represents the molecules within a small volume of the fluid. We use the term 'inclusion' to refer to the objects that are suspended in the matrix fluid. For our cement and concrete simulations, these inclusions are typical construction aggregates such as sand, gravel, or crushed rock. Each inclusion is modeled as a group of DPD particles that are locked in relative position and move as a unit. In contrast to the DPD particles that comprise an inclusion, the DPD particles that comprise the matrix fluid are called 'free' particles.
DPD methods resemble MD in that the particles move according to Newton's laws, however in DPD the interparticle interactions are chosen to allow for much larger time steps. The original DPD algorithms used an Euler algorithm for time integration and updating the positions of the free particles, and a leap frog algorithm for updating the positions of the solid inclusions. Our simulator, QDPD, (for Quaternionbased Dissipative Particle Dynamics) is a modification of the original DPD algorithm that uses a modified velocityVerlet algorithm [Verlet, Allen] to update the positions of both the free particles and the solid inclusions. In addition, the solid inclusion motion is managed using a quaternion based scheme [Omelayan] (hence the Q in QDPD).
For simulating suspensions comprising a nonNewtonian fluid matrix we use the computational method called Smoothed Particle Hydrodynamics (SPH) introduced in 1977 [Lucy, Gingold and Monaghan], originally developed for the simulation of astronomical bodies. This method uses mesoscopic particles in a similar way as DPD, with free particles making up the matrix fluid and each inclusion made up of a set of particles that move as a unit. Computationally, the main difference between the DPD and SPH models is the computation of the interparticle forces. In the DPD model, the force between a pair of particles is only dependent on the distance between them and their velocities. In an SPH model, the interparticle force computation also takes into account the particles in the immediate vicinity.
The Parallel Implementation
The QDPD program (which includes both the DPD and SPH methods) is a parallel program, that is, it is designed to run a single simulation using many processors simultaneously. QDPD is written in the common single program/multiple data (SPMD) style using MPI, the well supported message passing library, to handle all of the required interprocess communications. For a simulation, we use this single program, QDPD, run simultaneously on all of the requested processors, with each processor handling a subset of the required computations and communicating with the other processors as needed to maintain a consistent state.
A simple 3dimensional domain decomposition is used in QDPD to manage the parallel computation. Logically, the processors are arranged in a 3dimensional grid with each processor assigned a small subvolume of the entire simulated system, with adjacent processors within the grid owning adjacent subvolumes.
The number of processors that we can efficiently utilize for a single simulations depends primarily on the size of the system being simulated (number of free particles and number of inclusions) as well as the speed of the processors and the speed of the interconnecting communications network between the processors. Our largest simulations have been run on approximately 2^{17} processors (i.e. 131,072 cores) of the IBM Blue Gene/Q machine Mira at the Leadership Computing Facility of Argonne National Laboratory, with each simulation taking on the order of 100 hours to complete.
References

Allen, M P, and D J Tildesley. 1989. Computer simulation of liquids. Oxford science publications. Oxford University Press.

Gingold, R. A., and J. J. Monaghan. 1977. Smoothed Particle Hydrodynamics: Theory and Application to NonSpherical Stars. Monthly Notices of the Royal Astronomical Society 181: 375389.

Hoogerbrugge, P. J, and J. M. V. A Koelman. 1992. Simulating Microscopic Hydrodynamic Phenomena with Dissipative Particle Dynamics. Europhysics Letters (EPL) 19 (3) (June 1): 155160.

Lucy, L. B. 1977. Numerical Approach to the Testing of the Fission Hypothesis. The Astronomical Journal 82 (12) (December): 1013.

Omelyan, Igor P. 1998. On the Numerical Integration of Motion for Rigid Polyatomics: The Modified Quaternion Approach. Computers in Physics 12 (1) (January): 97103.

Verlet, Loup. 1967. Computer Experiments on Classical Fluids. I. Thermodynamical Properties of LennardJones Molecules. Physical Review 159 (1) (July 5): 98103.
Return to High Performance Computing