Prediction for civil aero-engine performance after shop visit based on lazy support vector machine regression model

Through consideration of problems that the influence of the aero-engine state before shop visit and the adopted maintenance work scope on its performance after shop visit is complex and the sample size is small, we propose a lazy support vector machine regression (LSVMR) model for aero-engine performance prediction after shop visit based on the ε -support vector machine regression (ε - SVMR) model. Unlike the ε -SVMR, the insensitive loss function in LSVMR depends on the distance between the training sample and the predicted sample. The proposed model not only makes full use of the information of the predicted sample, but also seeks the best tradeoff between the model complexity and the learning ability. In this article, we give the solving process of LSVMR and collect the actual aero-engine maintenance samples from an airline to validate it. By comparing the prediction accuracy among LSVMR, ε -SVMR and k-nearest neighbor algorithm (k-NN), we find that LSVMR has the best prediction accuracy and can be seen as an effective method for the aero-engine performance prediction after shop visit.


Introduction
The aero-engine is an important core component of the aircraft. In order to ensure flight safety, the aero-engine must be sent to the workshop for maintenance when the airworthiness requirements are not met. The first question before aero-engine shop visit is how to determine the aero-engine maintenance work scope. Recently, the determination of maintenance work scope is mainly based Performance recovery, L4 Overhaul. on guidance documents issued by aero-engine manufacturers, rule-based decision method [1], or objective-oriented decision method [2]. Unreasonable maintenance work scope will lead to overmaintenance or under-maintenance [3]. Therefore, the prediction for the aero-engine performance after shop visit is of crucial importance to mastering its state and service life after shop visit in advance, and then judging whether the maintenance work scope is reasonable or not.
Performance prediction after shop visit plays an important role in the whole maintenance system [4]. The traditional performance prediction after shop visit is based on the aero-engine state before shop visit and the adopted maintenance work scope. However, it is difficult to directly propose a physical model to predict the aero-engine performance after shop visit, due to the complex influence of the aero-engine state before shop visit and the adopted maintenance work scope on its performance after shop visit. For these reasons, data-driven prediction methods are generally adopted. According to whether the mapping relationship between the input space and the output space is established before prediction, the data-driven prediction methods can be divided into eager prediction methods and lazy prediction methods. The eager prediction methods mainly include various regression algorithms in statistics [5], artificial neural networks (ANN) [6][7], support vector machine regression (SVMR) models [8][9] in machine learning, grey forecast models in grey theory [10][11], etc. The lazy prediction methods mainly include k-nearest neighbor algorithms (k-NN) [12], locally weighted regressions (LWR) [13][14], etc.
The prediction for the aero-engine performance after shop visit is usually with a small sample size due to the small aero-engine fleet size, the low shop visit frequency or incomplete maintenance records. The essence of the prediction problem with small sample size is the lack of information, and the training samples cannot fully reflect a mapping relationship between input space and output space. At present, there are three main ways to deal with the prediction problem with small sample size. The first is directly using eager prediction methods with good generalization performance such as SVMR [15][16] and extreme learning machines (ELM) [17], etc. However, these methods limited by the incomplete sample space may result in a large difference between the founded global mapping relationship and the actual mapping relationship, and lead to poor prediction accuracy. The second is using some methods to extend the problem with small sample size to large sample size and then making predictions [18][19][20][21]. However, this approach may produce wrong results depending on the actual characteristics of the real data because of the random data distribution. Moreover, when the data only has a small number of features, Virtual Sample Generation (VSG) may not be able to effectively improve the prediction accuracy. Therefore, these methods also fail to improve the prediction accuracy in essence. The third is using lazy prediction method to make predictions. When the distance between the predicted sample and the training sample is relatively close, the lazy prediction methods can usually achieve better results than the eager prediction methods. However, existing lazy prediction methods generally use the training sample information directly and simply, which still have room for improvement of the prediction accuracy.
For prediction problems with small sample size, the more information is used in building model, the better the prediction accuracy will be. Comparing with eager prediction method, the lazy prediction method uses not only the training sample information but also the predicted sample information, while the existing lazy prediction method cannot make full use of the training sample information. Based on the VC dimension theory and the structural risk minimization, the SVMR can seek the best tradeoff between the model complexity and the learning ability according to the limited sample information, thus it has good generalization ability. In this article, we combine SVMR and the lazy prediction method and then propose the lazy SVMR (LSVMR) model, which is suitable for the aero-engine performance prediction after shop visit. By deducing the LSVMR algorithm, the decision function and optimal solution of the model can be obtained. We apply this proposed model to the actual aero-engine performance prediction after shop visit and compare the prediction accuracy among LSVMR, ε-SVMR and k-NN. We find that LSVMR has the best prediction accuracy after experiments and determination of the aero-engine maintenance work scope can be supported.
This article is organized as follows. In section 2 and section 3, the LSVMR model and its solving process are described. In section 4, experiments on real aero-engine maintenance data are conducted to compare the effectiveness of our method and competing methods, the results and discussions are also presented. Finally, several conclusions are drawn.

Lazy Support Vector Machine Regression Model
Given a training sample set and a predicted sample ( )  , l is denoted as sample size, i x is given as input and i y is the corresponding output. The proposed LSVMR model is to predict * y based on T and * x .
In this article, we propose the LSVMR model based on the ε-SVMR model. In order to guarantee its good generalization performance, ε-SVMR uses the ε-insensitive loss function (1), denoted as where ( ) f ⋅ is the decision function, ε is a small positive number. When the error between i y and ( ) i f x does not exceed ε , the fitting of ( ) is considered to be error free.
( ) (2) fitting T and minimizing where ω is a normal vector, ( ) Φ ⋅ represents the mapping function, b is denoted as the intercept, C is called penalty constant and 0 C > .
In ε-SVMR, ε is a constant. This means that allowable lossless intervals of all training samples are equal, and that all training samples play exactly the same role in process of building model. For the lazy prediction method, its purpose is generally not to establish the global mapping relationship between input space and output space, but to establish a better local mapping relationship near the predicted sample. The closer the training sample is to the predicted sample, the better it can reflect the local mapping relationship in vicinity of the predicted sample and play a greater role in process of building model and vice versa. Thus, in LSVMR, the allowable lossless intervals should be smaller for the training samples closer to the predicted sample. In order to establish the LSVMR, we first introduce the Ε-insensitive loss function (4), denoted as (4) where ( ) Ε ⋅ is a function of the distance between the training sample i x and the predicted sample where ε is a positive number. According to Eq (5), when * * 1,2, , = max Similar to ε-SVMR, the LSVMR also aims at finding an appropriate decision function, denoted as ( ) ∑ ω x (7) In contrast with Eq (3) we can find that the LSVMR is exactly the same as ε-SVMR except for insensitive loss function.

Model Solution
The minimization of Eq (7) is equivalent to the solution of the constrained optimization problem Eq (8) -Eq (11): In order to solve this constrained optimization problem, we introduce the Lagrange function, denoted as Therefore, the minimax problem of Eq (12), which is the dual problem of Eq (8) The optimal solution of the primal problem and dual problem is respectively represented as x x x is denoted as positive definite kernel.
It can be easy to prove that Eq (24) -Eq (26) is a convex quadratic programming problem. The optimal solution α and * α can be obtained through interior point method, sequential minimal optimization method, etc.
The process of solving b is as follows. According to the KKT condition, we can get

Application Case
In order to validate the prediction accuracy of the LSVMR, aero-engines from an airline are selected for numerical experiments. 42 maintenance samples of PW4077D aero-engines between 2006 and 2013 are collected. The first 37 samples are taken as training samples, and the remaining 5 are used as test samples. The maintenance levels of all relevant modules and DEGT, DFF, DN1, DN2, EGTM before shop visit are taken as input, while these five performance indicators after shop visit are taken as output. The partial maintenance samples are shown in Table 1. Among them, the maintenance level of each modules is 0-1 variable. Number 0 indicates that this maintenance level is not adopted at the time of shop visit, while number 1 indicates that this maintenance level is adopted.  L2 is repair, L3 is performance recovery, L4 is overhaul In order to comprehensively compare the prediction accuracy of the LSVMR and the ε -SVMR, we carry out the following experiments. The radial basis function is selected for the kernel functions of the LSVMR and the ε -SVMR, denoted as where γ , penalty parameter C and ε in the insensitive loss functions are respectively taken as 0.00001, 0.0001, 0.001, 0.01, 0.1, 1, 10, 100, 1000, 10000 or 100000, including a total of 1331 combinations. The actual and optimal predicted values of five parameters of the test samples are shown in Fig.1. The hyperparameters combination ( ) , , C γ ε , corresponding to the optimal value of five performance indicators respectively predicted by the LSVMR and ε -SVMR, are listed into the Table 2. By the predicted results，the Mean Absolute Errors(MAE) in each experiments are calculated and listed into the Table 3 And then, k-NN is used for experiments, where k is respectively taken as 1, 2, ... , 37, including a total of 37 experiments. The optimal values of five performance indicators predicted by k-NN and corresponding hyperparatmer k are respectively listed into Fig.1 and Table 2. The predicted results and MAE are also calculated and listed into Table 3.

Fig. 1. Experimental results
Table2: Optimal hyperparameter. By analyzing MAE in Table 3, the following conclusion can be obtained. In LSVMR, the MAE of DEGT, DFF, DN1, DN2, EGTM is respectively 3.06℃, 0.57%, 0.1954%, 0.244%, 5.65℃. The LSVMR method proposed in this paper has obvious advantages in predicting DEGT, DFF and EGTM. In predicting DN2, the effect of LSVMR is as good as ε -SVMR and much better than k-NN. In predicting DN1, the effect of LSVMR is also as good as ε -SVMR, but its effect is not as good as k-NN. Overall, the prediction accuracy of LSVMR is better than ε -SVMR and k-NN in the case of small samples, and can meet the requirements of engineering application.

Conclusions
(1) In order to improve the predictive effect of the problem with small sample size, this article combines advantages of ε-SVMR and lazy prediction algorithm to propose the LSVMR, which can not only make full use of the information of the predicted sample, but also seek the best tradeoff between the model complexity and the learning ability. Therefore, the LSVMR proposed in this article effectively improve the prediction ability.
(2) DEGT, DFF, DN1, DN2, EGTM are the most important performance indicators of an aeroengine. When an aero-engine is sent for shop visit, five engine performance indicators after shop visit are predicted based on the adopted maintenance levels of each modules and performance parameters before shop visit, which contributes to mastering the state and service life of aeroengine after shop visit in advance, and then making decision on whether the maintenance work scope is reasonable or not. Aiming at solving the problem of insufficient training samples, we apply the LSVMR to the aero-engine performance prediction after shop visit, and then obtain a good result.