2022 Precision_Motion_Control_of_Robotized_Industrial_Hydraulic_Excavators_via_Data-Driven_Model_Inversion-서울대(이동준)
2022 Precision_Motion_Control_of_Robotized_Industrial_Hydraulic_Excavators_via_Data-Driven_Model_Inversion-서울대(이동준)
Precision Motion Control of Robotized Industrial Hydraulic Excavators via Data-Driven Model Inversion
Minhyeong Lee , Graduate Student Member, IEEE , Hyelim Choi , Graduate Student Member, IEEE , ChangU Kim , Jihyun Moon , Dongmok Kim , and Dongjun Lee , Member, IEEE
Abstract —This work proposes a novel precision motion control framework of robotized industrial hydraulic excavators via datadriven model inversion. Rather than employing a single neural network to approximate the whole excavator dynamics, including input delays and dead-zones, we construct a physics-inspired datadriven model with a modular structure. The data-driven model is then inverted in a modular fashion which benefits the training speed. The data-driven model and its inversion are trained offline in a supervised manner using the real operational data since online learning methods can damage the machine and surroundings. The entire motion control framework consists of the data-driven model inversion that compensates for the excavator dynamics and the proportional control that determines the input of the model inversion to enhance the robustness. The framework is experimentally validated with a commercial 38-ton class hydraulic excavator for digging and grading tasks, achieving a precise control performance (i.e., root-mean-square of the path following error under 2 [cm]) even under severe soil interactions.
Index Terms —Industrial hydraulic excavator, data-driven, model inversion, input delay, dead-zone, soil interaction.
I. INTRODUCTION
YDRAULIC actuator systems are widely employed in H various engineering domains due to their high power-toweight ratio, reliability, and affordability. In particular, hydraulic excavators are essential in construction, demolition, mining, and forestry where large operating forces are required. However, multiple joints of the excavator must be manipulated simultaneously while maximizing the operation efficiency, making the manipulation a demanding task that must be performed by skilled operators with many years of experience. In this regard, the automation of the excavators has drawn a great interest [1]
Manuscript received September 9, 2021; accepted December 25, 2021. Date of publication January 13, 2022; date of current version January 21, 2022. This letter was recommended for publication by Associate Editor Gianluca Garofalo and Editor Youngjin Choi upon evaluation of the reviewers’ comments. This work was supported by the Hyundai Doosan Infracore. (Minhyeong Lee and Hyelim Choi contributed equally to this work.) (Corresponding author: Dongjun Lee.)
Minhyeong Lee, Hyelim Choi, ChangU Kim, and Dongjun Lee are with the Department of Mechanical Engineering, IAMD and IER, Seoul National University, Seoul 08826, South Korea (e-mail: minhyeong@snu.ac.kr; helmchoi@snu.ac.kr; kcu11@snu.ac.kr; djlee@snu.ac.kr).
Jihyun Moon and Dongmok Kim are with the Hyundai Doosan Infracore, Incheon 22502, South Korea (e-mail: jihyun3.moon@hyundai-di.com; dongmok.kim@hyundai-di.com).
This letter has supplementary downloadable material available at https://doi.org/10.1109/LRA.2022.3142389, provided by the authors. Digital Object Identifier 10.1109/LRA.2022.3142389
to reduce human-associated costs (e.g., fatigue, safety, etc.). Central to this automation is a precise motion control, but it still remains a challenging problem in practical settings [2], particularly with complex soil interactions.
In this paper, we propose a novel precision motion control framework of robotized industrial hydraulic excavators based on a data-driven model inversion. The data-driven inversion is challenging for typical data-driven approaches (e.g., methods using a recurrentneuralnetwork(RNN),amultilayerperceptron(MLP), etc.), as it is expensive to represent the inverse of time-related behaviors and to train discontinuous relations. The hydraulic excavators, however, are under the effect of input delays and dead-zones, which intensify in the presence of complex hydrauliccircuits(e.g.,amaincontrolvalve(MCV)[3])commonly found in industrial hydraulic settings. To address these distinct features, we introduce a physics-inspired data-driven model with a modular structure composed of the following neural network modules: 1) an infinite impulse response (IIR) unit, which accommodates the input delays; 2) a piecewise linear (PL) map, which deals with the state-dependent dead-zones; and 3) MLP networks, which capture the remaining nonlinear and coupled dynamics. The environmental impacts (e.g., soil interaction forces) and the hydraulic states (e.g., hydraulic pressures) are taken into account by including the measurements in the network input.
Learning the data-driven model and its inversion online can endanger the excavator and the environments, thus the learning is done offline in a supervised manner using the operational data of the real machine. We then design our control to consist of the following two layers: 1) the data-driven model inversion control, constructed as an inversion of the data-driven model in a modular fashion that significantly enhances the training speed; and 2) the proportional (P) control, implemented on top of the data-driven model inversion control to enhance the robustness. The stability and robustness of the control framework are theoretically established. Even in the presence of intense soil interactions, the proposed control framework accomplishes a remarkable performance (i.e., the path following root-meansquare error (RMSE) less than 2 [cm]) for digging and grading operations of a commercial 38-ton class hydraulic excavator Doosan DX380LC.
Model-based methods have been proposed [3]–[6] for the control of hydraulic excavators, but they adopted simplifications in modeling which necessarily compromise the control
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
performance. To avoid the difficulties of deriving accurate mathematical models, data-driven methods were introduced for the hydraulic excavator control [7]–[10]. Reinforcement learning (RL) approaches were presented in [7], [8], where the dynamics were approximated by a single large MLP. However, the large number of trainable parameters, which arises from importing the data history as an input of the MLP to handle the delays, substantially slows down the learning speed (e.g., 57476 parameters leading to 10 hours of training with 0.72 million data for [7] as compared to 9160 parameters and 2 hours with 2.6 million data for our plant model). Further, they were presented only in slow operation speeds to commercialize (e.g., average speed of 10 [cm / s] to 20 [cm / s] for a 12-ton class excavator), limiting the practical usefulness of the control. On the other hand, an RNN was employed to learn the controller of hydraulic excavators online in [9], [10]. Their performances, however, exhibited rather large tracking errors (e.g., RMSE greater than 1 [m]) in digging operations. The works on data-driven methods did not consider soil interactions [7], [8], [10], and the dead-zone compensation was simply defined by constant control input offsets [8], [10], further limiting them from precision motion control.
For a single hydraulic actuator, a data-driven force control was proposedin[11],wherethecontrollerwasconfiguredasanMLP. The force controller network was fed with a large dimensional history of the actuator position and force, which again put a strain on the training, and the dead-zone compensation was not considered in the control learning. The dead-zone compensation was studied in [12], [13] with a trainable tailored map, but they could not represent the state-dependent nature of the dead-zones. In [14], a soil interaction model of the excavator was suggested without an examination of the hydraulics and various soil properties for the industrial applications. In contrast to these previous results, our proposed framework can address the complex hydraulic excavator dynamics including the input delays and the state-dependent dead-zones without especially increasing the network size due to the modular structure, while fully considering the interaction with the soil. We also believe that our proposed framework would be advantageous for other hydraulically-actuated robotic systems with multiple actuators and complex environmental interactions.
The rest of the paper is organized as follows. Section II describes the autonomous hydraulic excavator adopted for the experimental validation. Section III introduces the modular design for the proposed data-driven model inversion. The entire offline process that derives data-driven model inversion is depicted in Section IV. Experimental results are presented in Section V, and then Section VI concludes the paper.
==> picture [228 x 133] intentionally omitted <==
Fig. 1. Doosan DX380LC, a commercial 38-ton class industrial hydraulic excavator. The excavator is customized with IMUs, pressure sensors, and a Velodyne Puck VLP-16 LiDAR. The IMUs are attached to each link, and the pressure sensors are located in all pumps and cylinders.
==> picture [208 x 110] intentionally omitted <==
Fig. 2. Joint configuration of the excavator. The bucket tip position is observed in the frame {B} attached to the base of the excavator.
consider the hydraulic behavior. The pumps and the cylinders are connected through the MCV as detailed in [3], which consists of spool valves that distribute the pump flow rate to generate the cylinder velocity (i.e., the joint angular rate). The spool positions are controlled by electronic proportional pressure reducing (EPPR) valves commanded by the joystick signal. The joystick signal also affects the pump control provided by the manufacturer, which implies that the joystick signal must be regarded as the control input of our industrial excavator. Meanwhile, the soil interactions are evaluated using the momentum-based wrench estimator [15] since we cannot attach a force/torque sensor to the bucket joint due to reliability and cost concerns. A LiDAR sensor scans the point cloud data (PCD) of the terrain for the reference trajectory planning. The communication is via controller area network (CAN), where the sensing and control frequency is set to 100 [Hz].
II. SYSTEM DESCRIPTION
This work employs Doosan DX380LC, an industrial hydraulic excavator, to validate our data-driven control strategy. The excavator is customized using sensors to measure states and environmental impacts, as shown in Fig. 1. Inertial measurement unit (IMU) sensors are attached to the boom, arm, and bucket links to estimate the joint configuration. Swing angle is also measurable, but we only consider the motion within the sagittal plane as visualized in Fig. 2 because the swing action is not involved in the excavation. Apart from the joint configuration, hydraulic pressure sensors are located in pumps and cylinders to
III. DESIGNING DATA-DRIVEN MODEL INVERSION
This section introduces the concept of the data-driven model inversion illustrated in Fig. 3. First, we propose a data-driven, physics-inspired, and easy-to-control model with a modular structure that provides an approximate of the excavator dynamics. The data-driven model can cope with the distinct features of the excavator dynamics, including the input delays, the state-dependent dead-zones, and the soil interactions. Then, the inversion control of the data-driven model is configured to compensate for the excavator dynamics.
==> picture [482 x 252] intentionally omitted <==
Fig. 3. Framework of the data-driven model inversion and its offline learning schema. First, the excavator plant model (right) is proposed to approximate the excavator dynamics. Then, the inversion control (left) of the excavator plant model, constructed in a modular manner, compensates for the excavator dynamics including the input delays, the state-dependent dead-zones, and the soil interactions.
A. Excavator Plant Model
Assuming that the time-related behavior (i.e., the spool dynamicsandthehydraulicdelays)canbeapproximatedbyalinear time-invariant (LTI) system, the resulting network, namely the excavator plant model shown in the right-hand side of Fig. 3, predicts the joint angular rate by
==> picture [159 x 11] intentionally omitted <==
==> picture [177 x 11] intentionally omitted <==
==> picture [153 x 12] intentionally omitted <==
where t ∈ Z is the time step identified by the subscript of a time signal ⋆t := ⋆ ( t ), Z{⋆t} :=[�] [∞] t =0 [⋆][t][/z][t][ is the] [ z][-transform, and] P ( z ) is the delaying system, a stable z -domain nh × nf LTI transfer function matrix which captures the multiple and different delays of the hydraulic excavator. The nonlinear nature of the hydraulic circuit is accommodated in pre-delay map f Γ t : [ − 1 , 1][3] → R [n][f] and post-delay map h Γ t : R [n][h] → R[3] with a simplified expression of a Γ t -dependent map ⋆ Γ t ( · ) := ⋆ (Γ t, · ). There are two intermediate variables, the pre-delay state ηf,t ∈ R [n][f] and the post-delay state ηh,t ∈ R [n][h] , to integrate the LTI system and the nonlinear maps. The control input (i.e., joystick signal) is denoted by ut ∈ [ − 1 , 1][3] , the joint angular rate and its prediction are denoted by ωt, ˆ ωt ∈ R[3] , and the excavator state is denoted by
==> picture [149 x 13] intentionally omitted <==
where θt := ( θt[boom] , θt[arm] , θt[bucket] ) ∈ R[3] is the joint angle, Pt[cyl] ∈ R[6] is the pressure of head- and rod-side chambers of the cylinders, Pt[pump] ∈ R[2] is the pressure of two pumps that supply the hydraulic fluid, and Ft[ext] ∈ R[2] is the horizontal and vertical external force acting on the bucket tip which captures
the soil interactions. Refer to Section IV-A for neural network module architectures and offline learning methods for the proposed excavator plant model. We would like to comment that a state-dependent delaying system is available for (2), but the LTI transfer function matrix P ( z ) works satisfactorily in our application with and without large soil interactions.
B. Excavator Plant Model Inversion Control
From the command joint angular rate ωt[cmd] ∈ R[3] , the excavator plant model inversion control shown in the left-hand side of Fig. 3 computes the joystick signal as
==> picture [162 x 12] intentionally omitted <==
==> picture [179 x 12] intentionally omitted <==
==> picture [156 x 11] intentionally omitted <==
where CP ( z ) is the delay-tracking system, a stable z -domain nf × nh LTI transfer function, gh, Γ t : R[3] → R [n][h] is the precontrol map, and gf, Γ t : R [n][f] → [ − 1 , 1][3] is the post-control map. The pre-control state ζh,t ∈ R [n][h] and the post-control state ζf,t ∈ R [n][f] are intermediate variables. The reference (e.g., nr = 0 forthestep-referenceand nr = 1 fortheramp-reference) tracking condition of the delay-tracking system CP ( z ) is written as
==> picture [229 x 15] intentionally omitted <==
from the final value theorem, where Ia ∈ R [a][×][a] is an identity matrix and 0 a×b ∈ R [a][×][b] is a zero matrix. Two nonlinear maps gh, gf satisfy the pseudo-inverse relation s.t. ⋆ Γ t ◦ g⋆, Γ t is an identity function on the domain of g⋆, Γ t given Γ t . Note that the exact inverse of the pre- and post-delay maps (i.e., g⋆, Γ t ◦ ⋆ Γ t is also an identity function) may be out of existence because
of many-to-one relations such as the dead-zones. The inversion method for each module is illustrated in Section IV-B. The following Proposition 1 provides the properties of our data-driven inversion control (4), (5), and (6).
Proposition 1: Consider the excavator plant model (1), (2), and (3) under the data-driven inversion control (4), (5), and (6). Assume that a) errors of the model prediction δω,t := ω ˆ t − ωt ∈ R[3] and the errors of the pseudo-inverse relations δh,t := ( h Γ t ◦ gh, Γ t )( ωt[cmd] ) − ωt[cmd] ∈ R[3] , δf,t := ( f Γ t ◦ gf, Γ t )( ζf,t ) − ζf,t ∈ R [n][f] are bounded; b) the post-delay map h Γ t is a Lipschitz continuous function; and c) the pre-control map gh, Γ t is a bounded function. Then if the joint angular rate ωt 0 and its command ωt[ref] 0 at the initial time step t 0 ∈ Z are bounded, the difference between the joint angular rate and its command νω,t := ωt − ωt[cmd] ∈ R[3] is bounded ∀t ≥ t 0.
Proof: The triangle inequality provides two inequalities s.t.
==> picture [132 x 13] intentionally omitted <==
==> picture [183 x 11] intentionally omitted <==
where ∥δω,t∥ and ∥δh,t∥ are bounded from the first assumption. For the second inequality, see the definition of the post-delay map (3) and the pseudo-inverse error δh,t = h Γ t ( ζh,t ) − ωt[cmd] . From the Lipschitz continuity of h Γ t , there ∃L ∈ R ≥ s.t.
==> picture [171 x 11] intentionally omitted <==
where L is referred to as a Lipschitz constant. The delaying system P ( z ) and its tracking control CP ( z ) is rearranged as
==> picture [61 x 12] intentionally omitted <==
==> picture [193 x 11] intentionally omitted <==
where P ( z ) CP ( z ) − Inh is a stable linear system satisfying the reference tracking condition (7). From the bounded-input bounded-output (BIBO) property, the error converges as
==> picture [171 x 12] intentionally omitted <==
==> picture [198 x 13] intentionally omitted <==
where γ⋆ : [0 , a ) → [0 , ∞ ) is a class K function (i.e., γ⋆ is strictly increasing with γ⋆ (0) = 0), and β : [0 , a ) × [0 , ∞ ) → [0 , ∞ ) is a class KL function (i.e., β ( r, s ) for each fixed s belongs to class K and β ( r, s ) for each fixed r is decreasing with lim s→∞ β ( r, s ) = 0). The pre-control state ζh,τ and the pseudo-inverse error δf,τ are bounded due to the last and the first assumptions, respectively. Bounded properties of the initial conditions lead ∥ηh,t 0 − ζh,t 0 ∥ to be bounded, which implies that νω,t is also bounded. ■
On top of the excavator plant model inversion control, the joint angle P control added to the feedforward reference angular rate enhances the robustness of the entire framework with the command joint angular rate:
==> picture [183 x 13] intentionally omitted <==
where ωt[ref] ∈ R[3] is the reference joint angular rate, θt[ref] ∈ R is the reference joint angle, eθ,t := θt − θt[ref] ∈ R[3] is the joint angle error, and K ∈ R[3] [×][3] is the P gain which is a positivedefinite matrix. Theorem 1 then concludes the entire control framework. Note that the command joint angular rate (8) can be determined independently of the data-driven inversion control.
For instance, a velocity field control or a proportional-integral (PI) control can replace the P control in (8).
Theorem 1: Consider the excavator plant model (1), (2), and (3) under the data-driven inversion control (4), (5), and (6) with the P control (8). Following the assumptions of Proposition 1, the joint angle error eθ,t is ultimately bounded.
Proof: Let us first consider the following Lyapunov function:
==> picture [61 x 21] intentionally omitted <==
for the error convergence in continuous-time domain. The time derivative of the Lyapunov function yields
==> picture [153 x 14] intentionally omitted <==
˙ ˙ withis the eθ,t joint + Ke angular θ,t = νω,t rate whereerror. eθ,t From= eω,t the:= ω inequality t − ωt[ref] ∈V ˙ R ≤[3] −λ min( K ) ∥eθ,t∥[2] + ∥eθ,t∥∥νω,t∥ with the minimum eigenvalue operator λ min( · ), the joint angle error is ultimately bounded by a closed ball of radius ∥νω,t∥/λ min( K ) where ∥νω,t∥ is bounded by Proposition 1. ■
In Proposition 1, the first assumption stems from reliable learning performances, and the second assumption is based on the continuous and bounded dynamic behavior of the excavator. The last assumption can be enforced by choosing a bounded output activation, such as a hyperbolic tangent, an arc-tangent, or a logistic function, for the pre-control map gh . Theorem 1 theoretically establishes the robustness of the entire control system, which is not provided in other data-driven controls of hydraulic excavators (e.g., [7]–[10]).
IV. LEARNING DATA-DRIVEN MODEL INVERSION
As schematized in Fig. 3, constructing the data-driven model inversion consists of two steps. The first step is to learn the excavator plant model, made up of the delaying system P ( z ) and the pre- and post-delay maps f, h ; and the second step is to obtain the inversion of each component to constitute the excavator plant model inversion control. The learning steps are detailed in the following Section IV-A and IV-B.
For the offline learning process, we assemble the measurements of Doosan DX380LC to capture complex nonlinear dynamics and soil interactions. The data is collected from autonomous digging/grading operations (with various depths and bucket speeds) and sinusoidal joystick signals (at frequencies 0 . 25 [Hz] to 0 . 5 [Hz] and amplitudes 0.3 to 0.5) near the initial and final configurations of the operations. The reference path of theautonomous operationis obtainedbylength-scalingthenominal bucket configuration extracted from the pattern of human experts [15]. The reference joint angle is computed by inverse kinematics, and the trajectory is time-scaled by the bang-bang approach on joint angular rate considering the hardware limits (e.g., the workspace of the excavator, a rough range of the joint angular rate, and the maximum excavation volume). For the control during the data collection, we employ the manufacturerprovided control and the proposed control trained with a small amount of data. In this work, the main focus is digging/grading tasks, but we believe that the proposed framework can be easily extended to the entire workspace by collecting sufficient data.
We use the data of 2.6 million time steps at a frequency of 100 [Hz] (i.e., 7.2 hours of data) to train the controller. The
data is randomly split into training, validation, and test sets at a ratio of 80:15:5. Using the data sets, the offline learning is performed on a computer with an AMD Ryzen 5 3600X 3 . 8 [GHz] CPU, a 16 [GB] RAM, and an NVIDIA GeForce GTX 1660 Ti GPU. Note that the proposed controller requires at least 1.2 million time steps (i.e., 3.3 hours) of data to obtain good enough performance under the nominal operating condition. However, we include data as extensively as possible on various operating conditions (e.g., soil properties and weather conditions) to address the diverse circumstances of the machines commercialized by the manufacturer.
A. Learning Excavator Plant Model
The first step, a supervised learning of the excavator plant model, exploits the following loss function:
==> picture [87 x 13] intentionally omitted <==
where P ( z ), f , and h are all trainable. The preexisting neural network architectures, however, cannot effectively address the unique properties of the excavator dynamics. For this reason, we propose new neural network modules: an IIR unit for the delaying system P ( z ) and a monotonically non-decreasing PL map for the pre-delay map f .
Infinite Impulse Response Unit: The delaying system P ( z ) is a multi-input multi-output (MIMO) transfer function configured as a matrix of single-input single-output (SISO) transfer functions. To construct a neural network for the transfer function learning, let us first consider a z -domain SISO LTI transfer function written as
==> picture [158 x 26] intentionally omitted <==
where b 0 , b 1 , . . . , bnb ∈ R and a 0 , a 1 , . . . , ana ∈ R are constant coefficients with a 0 = 0. The transfer function is equivalent to a recursive filter in t -domain, described in terms of the difference equation yt = ([�] [n] i =0 [b][b][i][x][t][−][i][ −][�] [n] i =1 [a][a][i][y][t][−][i][)] [/a][0][ where] [ x][t][∈][R] is the input signal and yt ∈ R is the output signal. The difference equation then can be rearranged to
==> picture [243 x 28] intentionally omitted <==
where[¯] bi := bi/a 0 ∈ R ∀i ∈{ 1 , 2 , . . . , nb} and a ¯ i := ai/a 0 ∈ R ∀i ∈{ 1 , 2 , . . . , na} are normalized coefficients and DCH := H (1) ∈ R is the low-frequency (DC) gain. Now the IIR unit can be written as an nh × nf matrix s.t.
==> picture [183 x 21] intentionally omitted <==
where Pjk ( z ) ∀j, k is a SISO transfer function whose orders of the numerator and the denominator are denoted by n[jk] b[, n] a[jk][∈][Z] [≥][and normalized coefficients are denoted by][¯] [b][jk] i ∈ R ∀i ∈{ 1 , 2 , . . . , n[jk] b[}][ and][ ¯] [a] i[jk] ∈ R ∀i ∈{ 1 , 2 , . . . , n[jk] a[}][. The] IIR unit belongs to the recurrent neural network family with ¯ b the given network size [jk] i[,][ ¯] [a][jk] i ∀i, j, k . The DC nh, n gain f , n can [jk] b[, n] also a[jk][and trainable variables] be trainable, but here, we choose DCP := P (1) as a nh × nf matrix with ones on the main diagonal and zeros on the off-diagonal so that rank P (1) =
min( nh, nf ). Any nh × nf transfer function matrix whose DC gain rank is min( nh, nf ) can be transformed into the IIR unit with row and column matrix operations.
Piecewise Linear Map: The post-control map gf must deal withjumpdiscontinuitiesorlargeslopestocompensatethedeadzones. However, the compensation map is not well trainable with a vanilla MLP because ( ωt, ut ) pairs have one-to-many relations in the dead-zone intervals. For this reason, the pre-delay map f learning is conducted with a monotonically non-decreasing n -segment PL map PL( X,Y ) : [ X 0 , Xn ] → [ Y 0 , Yn ] s.t.
==> picture [242 x 60] intentionally omitted <==
where Xi, Yi ∈ R ∀i ∈{ 0 , 1 , . . . , n} are breakpoints of the map and X := {Xi}[n] i =0 [, Y][:=] [ {][Y][i][}][n] i =0[are the non-decreasing] sequences of the breakpoints. The PL map is continuous at x = Xi with PL( X,Y )( Xi ) = Yi if Xi− 1 < Xi , while the map can represent the jump discontinuity at x = Xi if Xi− 1 = Xi . To apply distinct PL maps to the boom, arm, and bucket joystick signals, a tuple of multiple PL maps is expressed as ( y 1 , y 2 , . . . , ym ) := PL( X , Y )( x 1 , x 2 , . . . , xm ) where Xi[k][, Y] i[k][∈][R] [∀][i][ ∈{][0] [,][ 1] [, . . . , n][k][}][are][the][breakpoints][of] the n[k] -segment k -th PL map ∀k ∈{ 1 , 2 , . . . , m} . Lists of the breakpoints are denoted by X := {X[k] := {Xi[k][}][n] i =0 [k][}][m] k =1[and] Y := {Y[k] := {Yi[k][}][n] i =0 [k][}][m] k =1[.][Here,][we][choose][the][boundary] breakpoints as X 0 [k][=] [ Y] 0 [k][=] [ −][1][ and] [ X] n[k][k][=] [ Y] n[k][k][= 1] [ ∀][k][, so that] PL( X , Y ) : [ − 1 , 1] [m] → [ − 1 , 1] [m] . Note that the pseudo-inverse of the PL map is defined as PL[+] ( X , Y )[= PL][(] [Y] [,] [X][)][owing][to][its] monotonicity. The non-decreasing sequences of an interval [ a, b ] can be trained with the custom activation function σ : R [n] → [ a,b ] [ a, b ] [n][+1] written as σ a,b = d s.t.
==> picture [209 x 29] intentionally omitted <==
where c := ( c 1 , c 2 , . . . , cn ) ∈ R [n] is the activation input and d := ( a, d 1 , . . . , dn ) ∈ [ a, b ] [n][+1] is the partition of the interval with a ≤ d 1 ≤ d 2 ≤· · · ≤ dn = b . The activation function can also be extended to a two-dimensional input (e.g., σ [ a,b ] : R [n][(] [×][2] [m][)] → [ a, b ][(] [n][+1)] [×][2] [m] for m distinct n -segment PL maps) by applying the function to every column of the input.
Employing the proposed neural network modules, the delaying system P ( z ) is configured as a 3 × 3 IIR unit whose order is chosen by n[jk] b[=] [ n] a[jk][= 3][ if] [ j][=] [ k][ and] [ n][jk] b[=] [ n] a[jk][= 0] if j = k ∀j, k (i.e., 3-rd order transfer functions on the diagonal and zeros on the off-diagonal). This is based on the assumption that the pre- and post-delay maps f, h can adjust the input coupling. Fig. 4 shows the characteristics of the delaying system trained with the pre- and post-delay maps. Here, we would like to mention that the trained controller without the IIR unit causes fatal oscillations in the real machine.
The pre-delay map f is replaced with PL maps as f Γ t := PLΦ f,t : [ − 1 , 1][3] → [ − 1 , 1][3] . Breakpoints of the PL map are assumed to depend on Γ t by an auxiliary network Φ f,t :=
==> picture [90 x 80] intentionally omitted <==
==> picture [118 x 95] intentionally omitted <==
Fig. 4. Visualization of the pre-delay map f Γ t (left) and the delaying system P ( z ) (right). The PL map landscape visualizes the state-dependent pre-delay map for the bucket joystick signal during a trial of grading operation. The IIR step response shows the input delays of the boom, arm, and bucket, with settling times to within 5% are (0 . 41 , 0 . 33 , 0 . 26) [s].
φf (Γ t ) ∈ [ − 1 , 1][9] [×][6] which consists of a hidden layer of 64 ReLU nodes and an output layer of 8 × 6 nodes with the custom output activation function σ [ − 1 , 1] : R[8] [×][6] → [ − 1 , 1][9] [×][6] , for the three distinct 8-segment PL maps. The trained PL map is visualized in Fig. 4. Notice that the PL map enables the dead-zone compensation, shown in Fig. 5 as jumps of joystick signals. The pre-delay map can be further customized with constraints or other designs (e.g., applying a constraint to pass through the origin, training boundary breakpoints, or combining the PL map with an additional MLP), but this is not necessary in our case.
To address the remaining nonlinear properties and couplings, the post-delay map h is expressed as an MLP, with a single hidden layer of 256 nodes with a ReLU activation and a linear output layer. Training the excavator plant model (1), (2), and (3) takes only around 2 hours to converge using the Adam optimizer. Fig. 5 visualizes two examples of the prediction, while the prediction RMSE of the test data set is (0 . 51 , 0 . 66 , 1 . 16) [deg / s] for each boom, arm, and bucket joint angular rate, which is small enough to justify the presented neural network architecture.
B. Learning Excavator Plant Model Inversion Control
The second step configures the modular inversion of the datadriven excavator plant model. The delay-tracking system CP ( z ), the inversion of the delaying system P ( z ), is a 3 × 3 MIMO LTI transfer function. Although we can train CP ( z ) with another IIR unit, we analytically compose the delay-tracking system since obtaining the tracking control for the diagonal transfer function matrix is relatively simple. A stable and exact inverse of the delaying system (i.e., P[−][1] ( z )) does not exist because the trained delaying system has unstable zeros characterized by inverse responses as shown in Fig. 4. Thus, the delay-tracking system is constructed to meet the reference tracking condition (7) ∀nr ∈{ 0 , 1 } where the poles are empirically optimized to 0.82 with multiplicity 2. We choose the delay-tracking system with the minimum numerator order, which is a proper transfer function matrix. The post-control map gf , the pseudo-inverse of the pre-delay map f , does not require any offline learning process as the post-control map can be easily computed as gf, Γ t := PL[+] Φ f,t[.]
On the other hand, the pre-control map gh (i.e., the pseudoinverse of the post-delay map h ) cannot be analytically obtained sincethepost-delaymapisanMLP.Fortheofflinelearningofthe
pre-control map, a distal learning approach [16] is introduced.
==> picture [80 x 13] intentionally omitted <==
where ω ˇ t := ( h Γ t ◦ gh, Γ t )( ωt ) ∈ R[3] to realize the pseudoinverse relation. The MLP network of the pre-control map gh has a hidden layer of 256 nodes with a ReLU activation and an output layer with a hyperbolic tangent activation. The pre-control map can also be customized with additional PL maps to tolerate large slopes or jump discontinuities. After some trials, however, we found that an MLP is enough for the pre-control map gh , implying that the dead-zones are all captured in the pre-delay map f while training the plant model. The pre-control map trainingtakeslessthan5minutes,owingtothemodularinversion method. Reconstruction of the joystick signal is compared to the recorded signal in Fig. 5.
V. EXPERIMENT
The proposed control framework is verified with the commercial 38-ton class excavator Doosan DX380LC in digging and grading tasks. The reference trajectories for both operations are generated using the same planning algorithm as described in Section IV. The control frequency is 100 [Hz], though the inversion control can be easily implemented with higher frequencies. The P gain is chosen as K = 1 . 5 I 3 to determine the model inversion input. We implemented PI feedback as well, yet, found that only P control suffices as the control error is already fairly small without the I feedback.
The bucket tip position pt := ( px,t, pz,t ) ∈ R[2] is calculated to evaluate the control performance, where px,t, pz,t ∈ R are the horizontal and vertical tip positions as shown in Fig. 2. The path following error is denoted by e[path] p,t := min t 0 ≤τ ≤tf ∥pt − p[ref] τ[∥∈][R][,][which][indicates][the][error][of][the][excavated][ground] geometry. The trajectory error, or the bucket tip position error, is written as e[traj] p,t[:=] [ ∥][p][t][−][p][ref] t[∥∈][R][.][Here,][we][calculate][the] RMSE from one second after the initial time. This is to assess the performance/precision of our proposed control in the steady-state as typically done in control literature. Note from Figs. 6, 7, and the supplementary video that, at the initial time, we have a non-zero initial error due to our using of the (less accurate) manufacturer-provided PI control until then. See to it also that our proposed control well-behaves (e.g., with no sudden dipping) during this transition. This initial performance can further be improved by turning on our proposed data-driven controller before the initial operation or by re-planning the path using the measured configuration as the initial condition (i.e., ensuringthecontrollerinitssteady-state).Thecontrolperformance is compared to the manufacturer-provided PI control, which determines the joystick signal by a ( ωt, ut ) pair look-up table with a joint angle PI feedback and an angular rate feedforward. The manufacturer manually fine-tuned the control gain and the look-up table using the air digging data (i.e., data without soil interactions).
Digging: The digging operation is the removal of soil from the current terrain to achieve the target ground shape. Due to the excavation capacity limit, multiple digging operations may be required to reach the final target ground geometry. Fig. 6
==> picture [479 x 149] intentionally omitted <==
Fig. 5.set. The predicted joint angular ratePrediction of the joint angular rate (top) and reconstruction of the joystick signal (bottom) for the digging (left) and grading (right) data within the test ω ˆ t is computed using the excavator plant model given the same joystick signal ut . The reconstructed joystick signal u ˇ t is the output of the model inversion taking the measured joint angular rate ωt as the inversion input. About the discrepancies between the recorded/reconstructed joystick signals, we would like to note that: 1) there is no exact inverse of the dead-zones, that is, different joystick signals in the dead-zones can lead to the same excavator response; and 2) perfect tracking control of the non-minimum phase delays does not exist.
==> picture [478 x 217] intentionally omitted <==
Fig. 6. Bucket tip position (top-left), external force (bottom-left), and performance evaluation (right) during the digging operations. Repeated experiments are visualized as thin lines where the reference trajectories are planned for every repetition considering the current/target terrain and the hardware limit. The bold lines are the averages of the trials, and the shaded bucket plots are the average bucket configuration for every 2 [s]. Histograms of the path following and trajectory RMSEs are also provided for the performance comparison.
visualizes bucket trajectories, soil interactions, and error distributions of repeated experiments on various excavation depths and volumes. The experimental results of the manufacturerprovided PI control have a path following RMSE of 5 . 79 [cm] and a trajectory RMSE of 25 . 0 [cm]. The PI control results have an RMS reference bucket tip velocity of 66 . 2 [cm / s] and an RMS external force of 4 . 31 × 10[4] [N]. The proposed control framework outperforms the manufacturer-provided PI control, where a path following RMSE is 1 . 99 [cm] and a trajectory RMSE is 5 . 21 [cm] with an RMS reference bucket tip velocity of 66 . 3 [cm / s] and an RMS external force of 6 . 41 × 10[4] [N]. The operation speed and the external force are large enough for industrial applications.
Grading: The grading operation is to level the ground surface after the digging operations, where its experimental results are shown in Fig. 7. The manufacturer-provided PI control has a path followingRMSEof 5 . 28 [cm],trajectoryRMSEof 15 . 1 [cm],an RMS reference bucket tip velocity of 87 . 2 [cm / s], and an RMS external force of 2 . 69 × 10[4] [N]. The excavator plant model inversion control with the P control attains a path following RMSE of 1 . 83 [cm] and a trajectory RMSE of 3 . 17 [cm] with an RMS reference bucket tip velocity of 87 . 3 [cm / s] and an RMS external force of 2 . 23 × 10[4] [N]. The errors are evenly small both with and without intense soil interactions since the inversion captures and compensates for the effect of the external force.
==> picture [453 x 207] intentionally omitted <==
Fig. 7. Experimental results of the repeated grading operations which is to level the ground surface after the digging operations shown in Fig. 6.
VI. CONCLUSION
This work presents a precision motion control of robotized industrial hydraulic excavators via data-driven model inversion. Considering distinct features that hinder the learning-based control methods (i.e., input delays and dead-zones), we propose a data-driven model with a physics-inspired modular structure to approximate the excavator dynamics. We then derive the inversion of the plant model in a modular manner which considerably promotes the learning speed. To prevent injuries of the machine and surroundings, the model and its inversion are trained offline in a supervised fashion using the measurements of the Doosan DX380LC, a 38-ton class industrial hydraulic excavator. Our proposed control framework is composed of the data-driven model inversion control, which compensates for the excavator dynamics, and a P control that computes the model inversion input and enhances the robustness. The stability and robustness of the control framework are theoretically proven, and experimental results are presented in comparison withthemanufacturer-providedPIcontrol.Theproposedcontrol framework significantly outperforms the PI control and shows a precise control performance (i.e., path following RMSE under 2 [cm]) even in the presence of intense soil interactions.
Some possible future research directions include: 1) generalization of the excavator plant model using the state-dependent delaying system; 2) incorporation of the expert-emulating planning [15]; 3) implementation of the over-the-air programming to effectively collect the measurements and update the control; 4) rigorous comparison with other control strategies (e.g., [17]); and 5) application of our framework to other systems with delays and dead-zones.
REFERENCES
-
[1] L. Zhang et al. , “An autonomous excavator system for material loading tasks,” Sci. Robot. , Amer. Assoc. Advancement of Sci., vol. 6, no. 55, 2021, Art. no. eabc3164.
-
[2] J. Mattila, J. Koivumäki, D. G. Caldwell, and C. Semini, “A survey on control of hydraulic robotic manipulators with projection to future trends,” IEEE/ASME Trans. Mechatronics , vol. 22, no. 2, pp. 669–680, Apr. 2017.
-
[3] K. Kim, M. Kim, D. Kim, and D. J. Lee, “Modeling and velocity-field control of autonomous excavator with main control valve,” Automatica , vol. 104, pp. 67–81, 2019.
-
[4] S. Tafazoli, S. E. Salcudean, K. Hashtrudi-Zaad, and P. D. Lawrence, “Impedance control of a teleoperated excavator,” IEEE Trans. Control Syst. Technol. , vol. 10, no. 3, pp. 355–367, 2002.
-
[5] Q. Ha, M. Santos, Q. Nguyen, D. Rye, and H. Durrant-Whyte, “Robotic excavation in construction automation,” IEEE Robot. Automat. Mag. , vol. 9, no. 1, pp. 20–28, May 2002.
-
[6] P. Chang and S. Lee, “A straight-line motion tracking control of hydraulic excavator system,” Mechatronics , vol. 12, no. 1, pp. 119–138, 2002.
-
[7] P. Egli and M. Hutter, “Towards RL-based hydraulic excavator automation,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. , 2020, pp. 2692–2697.
-
[8] P. Egli and M. Hutter, “A general approach for the automation of hydraulic excavator arms using reinforcement learning,” IEEE Robot. Automat. Lett. , early access. 2021, doi: 10.3929/ethz-b-000487440.
-
[9] J. Park, D. Cho, S. Kim, Y. Kim, P. Kim, and H. Kim, “Utilizing online learning based on echo-state networks for the control of a hydraulic excavator,” Mechatronics , vol. 24, no. 8, pp. 986–1000, 2014.
-
[10] J. Park, B. Lee, S. Kang, P. Kim, and H. Kim, “Online learning control of hydraulic excavators based on echo-state networks,” IEEE Trans. Automat. Sci. Eng. , vol. 14, no. 1, pp. 249–259, Jan. 2017.
-
[11] S. Kim, B. Cho, S. Shin, J. Oh, J. Hwangbo, and H. Park, “Force control of a hydraulic actuator with a neural network inverse model,” IEEE Robot. Automat. Lett. , vol. 6, no. 2, pp. 2814–2821, Apr. 2021.
-
[12] R. R. Selmic and F. L. Lewis, “Deadzone compensation in motion control systems using neural networks,” IEEE Trans. Autom. Control , vol. 45, no. 4, pp. 602–613, Apr. 2000.
-
[13] J. Nurmi, M. M. Aref, and J. Mattila, “A neural network strategy for learning of nonlinearities toward feed-forward control of pressure-compensated hydraulic valves with a significant dead zone,” in Fluid Power Systems Technology , New York, NY, USA: American Society of Mechanical Engineers, 2018.
-
[14] F. E. Sotiropoulos and H. H. Asada, “Dynamic modeling of bucket-soil interactions using Koopman-DFL lifting linearization for model predictive contouring control of autonomous excavators,” IEEE Robot. Automat. Lett. , vol. 7, no. 1, pp. 151–158, Jan. 2022.
-
[15] B. Son, C. Kim, C. Kim, and D. J. Lee, “Expert-emulating excavation trajectory planning for autonomous robotic industrial excavator,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. , 2020, pp. 2656–2662.
-
[16] M. I. Jordan and D. E. Rumelhart, “Forward models: Supervised learning with a distal teacher,” Cogn. Sci. , vol. 16, no. 3, pp. 307–354, 1992.
-
[17] M. Neumann-Brosig, A. Marco, D. Schwarzmann, and S. Trimpe, “Dataefficient autotuning with bayesian optimization: An industrial control study,” IEEE Trans. Control Syst. Technol. , vol. 28, no. 3, pp. 730–740, May 2020.
[!info] 원본 참조 그림/표/수식이 필요하면 원본 파일을 확인하세요: 2022 Precision_Motion_Control_of_Robotized_Industrial_Hydraulic_Excavators_via_Data-Driven_Model_Inversion-서울대(이동준)