17th IFAC Symposium on System Identification, Beijing, China, 2015.

Benchmarks from the invited session System Identification Benchmarks.

• MIMO Lowly Damped System: Free-Free Vibrations of an Aluminium Tooling Plate, R. Pintelon and Y. Rolain,  Vrije Universiteit Brussel, dept. ELEC, Brussel.

An aluminium tooling plate of size 30.4 cm x 61.8 cm x 6.7 mm is excited under free-free boundary conditions by two mini-shakers spaced 25.5 cm apart. The mini-shakers are connected to the tooling plate via an impedance head and a plexiglass stinger rod of 1.8 cm length glued into screws of 1 cm length with a bore hole of 4 mm deep (see Schoukens et. al. 2011 for a more detailed description of the experimental setup). Gaussian noise and random phase multisine excitations with signal energy in the band [120 Hz, 600 Hz] are applied as inputs and the corresponding responses are measured at a sampling rate of 2.44 kHz. N = 128 x 1024 data samples of the input (force) and output (acceleration) signals are measured. Although the data sets are ``large'', they are not large compared with the length of the impulse response of the system. Indeed, to estimate accurately the
frequency response function of a system, the experiment duration should be at least ten times the dominant time constant of this system (see Schoukens et al. 2013 and Geerardyn et. al. 2012.  Since the aluminium plate is lowly damped, this condition is not fulfilled for the data sets provided. Estimating nonparametrically the frequency response matrix from these ``short'' data sets is the main challenge here. Another challenge consists in estimating a parametric model over the whole frequency band [120 Hz, 600 Hz].

J. Schoukens R. Pintelon, G. Vandersteen and Y. Rolain. Improved (non-)parametric identication of dynamicsystems excited by periodic signals - The multivariate case. Mechanical Systems and Signal Processing, 25(8): 2892-2922, 2011.

J. Schoukens, G. Vandersteen, R. Pintelon, Z. Emedi, and Y. Rolain. Bounding the polynomial approximation errors of frequency response functions. IEEE Trans. Instrum. Meas., 62(5):1346-1353, 2013.

E. Geerardyn, Y. Rolain, and J. Schoukens. Quasi-logarithmic multisine excitations for robust broad fre-quency band measurements. IEEE Trans. Instrum. Meas., 62(5):1364-1372, 2012.

• Identification of High-Tech Motion Systems: An Active Vibration Isolation System Benchmark.
  Robbert Voorhoeve, Tom Oomen, Annemiek van Rietschoten, Maarten Steinbuch, TU Eindhoven. Egon Geerardyn, Johan Schoukens, dept. ELEC, Vrije Universiteit Brussel.

High-tech motion systems are often constructed to show highly reproducible linear time invariant system behavior. Despite the linear behavior, the identification of models for such mechanical systems is numerically and algorithmically challenging due to high order lightly damped dynamics, many inputs and outputs. In this contribution, we will take initial steps to make systematically obtained data (including a full nonlinearity analysis) to interested users. In this paper, a benchmark example of an industrial active vibration isolation system is presented and these numerical and algorithmic difficulties are showcased in a comparison of identification methods. The extensive dataset has been measured for this system and will be made publicly available. The particular system is chosen since it is open-loop stable, enabling a systematic and clear data collection, it has an interesting control goal, it has 8 inputs and 6 outputs, and it is available most of the time in our laboratory. The data is also well suited to test new (non-) parametric identification approaches and algorithms, identification for control techniques, and is well suited for uncertainty modeling and (robust) control.

• Benchmark Data on a Linear Time- and Parameter-varying system. J. Lataire, E. Louarroudi, R. Pintelon and Y. Rolain, dept. ELECm Vrije Universiteit Brussel.

This article describes the benchmark data for time- and parameter varying systems, from measurements conducted at the ELEC department, Vrije Universiteit Brussel. The system is an electronic bandpass filter, the resonance frequency of which can be varied in a controlled fashion. The signal which controls the resonance frequency is provided (and can be interpreted as a scheduling variable), along with the measured input and output signals, and the signal stored into the arbitrary waveform generator. The system is suitably modelled as a Linear Parameter Varying system, or as a Linear Time-Varying system if the scheduling variable is not used. The measurements are conducted in a low-noise environment, allowing for a Signal-to-Noise-Ratio of more than 60 dB. The system is mainly linear in its input-output relation, although some nonlinear effects are visible. The data includes different typical excitation scenarios, including band-limited noise, random phase multisines with sparse excited frequencies, piecewise constant scheduling, and slow, medium and fast varying scheduling. 

• Hydrogeological Experimental Site of Poitiers, France. Afzal Chamroo, Régis Ouvrard, Thierry Poinot,University of Poitiers, Laboratoire d'Informatique et d'Automatique pour les Systèmes (LIAS), Poitiers, France. Gilles Porel, Benoît Nauleau and Jacques Bodin, University of Poitiers, Institut de Chimie des Milieux et Matériaux de Poitiers (IC2MP), Poitiers, France.

Even if our planet's surface is recovered by water up to 70%, availability of potable water on our overcrowded Earth is today a major issue. For sure, aquifers which can be found at various areas, represent an important source of underground water which can be pumped out and processed for domestic, agricultural and industrial purposes. However, freshwater aquifers which benefit from a limited recharge by meteoric water can be over-exploited. Moreover, in some cases, depending on local hydrogeology, non-potable water (presence of pollutants, mineral poisons, etc.) may be drawn from hydraulically connected aquifers leading to serious health problems.

Understanding and thus modeling underground water transfers is an important issue for hydrogeologists. This modeling can for instance help in predicting future water availability and in understanding how pollutants are dispersed from one area to another through our precious liquid. For this sake, PDE-based analytical models of water flow in aquifers can be found in literature (Butler 1997,Theis 1935). However, those models require a lot of geometrical hypotheses which are not verified in real cases.

In view of understanding and forecasting underground water flow, researchers in hydrogeology have been considering experimental data coming from aquifers equipped with several sensors and actuators (pumps). And in order to promote long-term monitoring of ground water data, several water databases have been developed (Dreuzy 2006,Seyfried 2001). For instance, the ERO (French Experimental Research Observatory) has developed the H+ database (http://hplus.ore.fr) which is a network of hydrogeological sites (Ploemeur, Poitiers, Majorca, Le Durzon (Larzac), LSBB, Hyderabad (India)) capable of providing data (including long-term observations) relevant to the understanding of the water cycle and of the motion of solute elements in aquifers. This paper accounts for one particular site: the Hydrological Experimental Site (HES) of Poitiers (France). Close to the university campus, the HES of Poitiers covers an area of 12 hectares located over the north flank of the ``Seuil du Poitou'', a huge Mesozoic carbonate plateau marking the transition between the Aquitaine and Paris sedimentary basins. Supported by the WATER program of Poitou-Charentes region and financed by the ERO, research studies have been undertaken by hydrogeologists of the University of Poitiers, with the overall objective being the understanding of water flow and solute transport down to depths planned for drinking and/or agricultural water supply. Those studies focus mainly on one of the two underlying aquifers: the 100 m thick Dogger aquifer. Works carried out by the hydrogeologist team involved digging several wells so that by means of pumps and appropriate sensors, not only is it possible to observe how the aquifer behaves naturally, but also to enable experimental tests by using specific protocols. These works have now lead to 35 instrumented boreholes on the HES, meeting depths going up to 165 m underground. Spatially distributed as nested five-spots (an elementary square pattern made of one central well and four corner wells), most of them were drilled on a regular 210 m × 210 m grid. As a global system, the HES can be considered as a network of interconnected wells.

In view of characterizing an aquifer, hydrogeologists can consider the so-called pumping test, also known as the aquifer test. During this experiment, water is pumped out at a steady rate for a relatively long period of time at a well so that the response of the aquifer can be analyzed by the water level changes at neighboring observation wells. In fact, once water is pumped out of a well, a change of pressure is produced in surrounding wells, thus causing water level changes. The water level drop, often referred to as a drawdown by hydrogeologists, is measured by piezometers placed at each well of a site. This pumping test helps in analyzing such characteristics of an aquifer as its conductivity, its storativity and its transmissivity.

J.J. Butler Jr. The Design, Performance, and Analysis of Slug Tests. Lewis Publishers, 1997.
J.R. De Dreuzy, J. Bodin, H. Le Grand, P. Davy, D. Boulanger, A. Battais, O. Bour, P. Gouze, and G. Porel. General database for ground water site information. Ground Water, 44(5):743-748, 2006.

M.S. Seyfried, M.D. Murdock, C.L: Hanson, G.N. Flerchinger, and S. Van Vactor. Long-term soil water content database, reynolds creek experimental watershed, idaho, united states. Water Resources Research, 37(11):2847-2851, 2001.
C.V. Theis. The relation between the lowering of the piezometric surface and the rate and duration of discharge of a well using groundwater storage. American Geophysical Union, (16):519-524, 2012.

• A benchmark on the identification of a flexible serial manipulator using a camera. E. Laroche and L. Cuvillon.ICube lab., University  of Strasbourg  and CNRS. D.  Vizer, LIAS, University  of Poitiers, France. Intelligent  Robots Laboratory, Budapest  University  of Technology
  and Economics,  Hungary. G. Mercère, LIAS, University  of Poitiers, France.

This paper presents a benchmark dedicated to the identification of a flexible manipulator using a camera. Robotic systems are usually identified using joint-level data (torque and position). However, in the current framework of vision-based control, the model must be able to predict the position of the instrument provided by the camera. Joint positions are also provided as inner variables and can also be used to improve the accuracy of the identified model. One additional complexity source is the presence of flexible modes that need to be accounted for in order to obtain a high bandwidth in control. The proposed simplified case with two inputs and two  outputs is considered  as sufficient  to  deal  with the  non-linear flexible nature  of the system. 

• A Critical View on Benchmarks based on Randomly Generated Systems. Cristian R. Rojas, Patricio E. Valenzuela, School of Electrical Engineering, KTH, Stockholm. Ricardo A. Rojas, Departamento de Electrónica, Universidad Técnica Federico Santa María, Valparaíso.

In data-based modelling communities, such as system identification, benchmarks are essential for testing and comparing old and new techniques for the estimation of models. During the last years, it has become customary in system identification to rely on data sets built from randomly generated systems. In this article we discuss the implications of this practice, in particular when using data sets generated with the Matlab command drss, and advocate the cautious use of comparisons based on these benchmarks, providing some suggestions for alternatives.

• A crude unit dataset for industrial MPC identification. Y.C. Zhu, Department of Control Science and Engineering, Zhejiang University, Hangzhou.

A crude unit dataset from an industrial identification test is provided as a benchmark for studying system identification. The identified model was used in an industrial MPC (model predictive control) project. A brief description of the process and the variables are given. As a reference, models identified using the so-called BJSM method is presented. 

• A benchmark for data-based modeling: challenges related to CO2 dynamics. R.S. Risuleo,  M. Molinari,  G. Bottegal, H. Hjalmarsson and K.H. Johansson. ACCESS Linnaues Center, School of Electrical Engineering, KTH, Stockholm

This paper describes a benchmark consisting of a set of synthetic measurements relative to an office environment simulated with the software IDA-ICE. The simulated environment reproduces a laboratory at the KTH{EES Smart Building, equipped with a building management system. The data set contains measurement records collected over a period of several days. The signals correspond to CO2 concentration, room and outdoor temperature, ventilation inflow and occupancy. Information on door and window opening is also available. This benchmark is intended for testing data-based modeling techniques. The ultimate goal is the development of models to improve forecast and control of environmental variables. Among the numerous challenges related to this framework, we point out the problem of occupancy estimation using information on CO2 concentration. This can be seen as a blind identification problem. For benchmarking purposes, we present two different identification approaches: a baseline overparametrization method and a kernel-based method.