A Remaining Useful Life Prediction Method for Rolling Bearings Based on Broad Learning System - Multi-Scale Temporal Convolutional Network

With the continuous development of industrial equipment, rolling bearings, as key rotating components in mechanical systems, play an irreplaceable role in manufacturing, transportation, and aerospace industries [1]. However, during long-term operation, rolling bearings may be affected by overload, contamination, and impact, which can lead to more severe failure modes and accelerate bearing degradation [2]. Therefore, accurately extracting the features of rolling bearings and predicting their RUL [3] is of profound significance for ensuring the safety performance of equipment [4], [5], [6].

RUL prediction methods can be categorized into four types based on their fundamental techniques and approaches: physics-based models, statistical models [7], deep learning models, and hybrid methods [8], [9]. Firstly, the core idea of physics-based models is to predict RUL by establishing a physical model of the equipment's operation, but this approach requires extensive expert knowledge and is difficult to model. Secondly, statistical models often exhibit strong nonlinear behavior, which may lead to information loss and require relatively ideal assumptions [10]. In recent years, with the rapid development of deep learning and neural networks, deep learning and hybrid methods have gradually become research hotspots for RUL prediction of rolling bearings [11]. Schwendemann et al. [12] proposed a combination of convolutional neural networks (CNN) and long short-term memory (LSTM) networks for classification and estimation to predict the RUL of bearings. Qi et al. [13] proposed a bearing RUL prediction method based on a broad learning system-temporal convolutional network (BLS-TCN), which enhances the identification of multiple failure features through the feature extraction capability of the temporal convolutional network while reducing the complexity of the network model through the flattened structure of broad learning, thereby reducing computation time. However, ordinary temporal convolutions can only capture single-scale temporal features. Gao and Lu [14] proposed a method based on bidirectional temporal convolutional networks and long short-term memory (Bi-TCN-LSTM), where data is normalized and fused, followed by feature extraction and deep learning using Bi-TCN-LSTM for RUL prediction of rolling bearings. However, the convolution kernel size of TCN [15] is limited, making it difficult to capture long-term dependencies, which limits the model's ability to predict the RUL of bearings. Deng et al. [16] proposed a MsTCN-transformer network model for prediction, which can adaptively extract inherent temporal feature information from the full-life signals of rolling bearings. They then constructed a transformer network model based on the self-attention mechanism. However, the transformer model typically requires more computational resources, especially when dealing with long sequences, where the computational and memory demands can increase significantly. Additionally, the model's complexity is relatively high, making it less effective at capturing local features of time series data.

To solve the above problems, this paper proposes a hybrid model (BLS-MsTCN) combining BLS and MsTCN for RUL prediction of rolling bearings. BLS [17], proposed by Junlong Chen and his students, is a novel neural network structure that increases the model’s representation ability by expanding its breadth (width), and it can learn and update quickly, with significant advantages in computational efficiency. MsTCN has made several improvements based on TCN, widely using dilated convolutions with different dilation factors to expand the receptive field without increasing the number of parameters, enabling the network to capture long-term dependencies and multi-scale patterns in temporal data. Furthermore, residual connections [18] are introduced at each stage to alleviate the vanishing gradient problem.

The main innovations of BLS-MsTCN are as follows:

(1) The operational data of rolling bearings is usually high-dimensional and complex. BLS-MsTCN, through its broad structure, can process large amounts of data in a short time, improving the efficiency of data processing and the scalability of the model.

(2) The operational state and failure modes of rolling bearings often exhibit long-term dependencies. BLS-MsTCN, with its multi-stage dilated convolution structure, can capture features at different temporal scales, and the stepwise refinement between stages allows for better modeling of long-term dependencies.

(3) The operational environment of rolling bearings is complex and variable, and the data characteristics under different operating conditions may vary significantly. The efficient feature extraction and multi-scale temporal modeling capabilities of BLS-MsTCN enable the model to adapt to the data characteristics under different operating conditions, enhancing prediction performance in various complex environments.

The composition of the BLS-MsTCN modules is shown in Figure 1. As can be seen from the figure, BLS-MsTCN mainly consists of a multi-scale temporal convolution module and a broad learning module.

The MsTCN aims to enhance the modeling capability for time series data and better handle changes at different temporal scales. The basic module structure of MsTCN is shown in Figure 2.

As shown in Figure 2, compared with ordinary temporal convolutional networks, the biggest difference in MsTCN lies in the feature extraction method, specifically in its multi-scale feature extraction and fusion strategy.

Although ordinary temporal convolutional networks can also use different dilation factors to expand the receptive field, the convolution at each layer can only use a fixed dilation factor, so the receptive field can only be expanded by stacking layers. As the number of stacked convolutional layers increases, the network depth also increases, resulting in higher training complexity.

In contrast, MsTCN introduces multi-scale convolution at each layer, that is, applying convolutional kernels with different dilation factors in parallel within the same layer (such as d=1, d=2, d=3), and each layer adopts full pre-activation [19], thereby capturing short-term, medium-term, and long-term dependency features simultaneously within the same layer. This structure significantly reduces the network depth and computational complexity while enhancing the effectiveness of feature extraction.

In ordinary temporal convolutional networks, feature fusion is mainly achieved through simple connections between layers, and the integration of features at different temporal scales primarily relies on the stacking of deep networks. This approach may result in suboptimal performance when capturing complex dependencies and can lead to information loss.

In MsTCN, feature fusion after multi-scale convolution at each layer is more flexible: it extracts features at different temporal scales through parallel convolution and then integrates these features into a unified representation through a fusion layer. This fusion process typically incorporates activation functions (such as ReLU), thereby enhancing the model’s nonlinear representation capability.

Pooling operations are added to feature fusion in the MsTCN module, further improving the model's performance in complex scenarios. Pooling operations aggregate these multi-scale features, making the information more concentrated, which helps to integrate key information across different temporal scales and improves the overall performance of the model. Moreover, by using pooling operations, the model can avoid overfitting detailed information to a certain extent, thereby improving its generalization ability. The overall structure of MsTCN is shown in Figure 3.

As shown in Figure 3, the network structure of MsTCN effectively captures both short-term and long-term dependencies in sequences by introducing dilated causal convolution, residual connections, and multi-scale feature fusion. The residual block used in the residual connections is an improved structure by Bai et al. [20], which addresses the overfitting issue in residual structures.

The core part of MsTCN consists of multiple multi-scale dilated convolution residual blocks. These residual blocks use different dilation rates (such as d=1, d=2, d=4), thus extracting features in parallel across different time spans. The introduction of dilated convolution enables the network to have a larger receptive field while maintaining fewer parameters, allowing it to capture both short-term details and long-term trends simultaneously. Moreover, the use of residual connections ensures that the network can still be trained effectively as its depth increases while preserving the original input information.

For a one-dimensional input sequence $x=\left[x_1, x_2, \ldots, x_t\right]$, the output $F(s)$ of the dilated causal convolution at time step $t$ can be expressed as:

where $F(\cdot)$ represents the convolution operation function, $k$ represents the size of the convolution kernel $(k=3$ in the figure), $f(i)$ is the $i$-th data point in the convolution kernel, $s-d \cdot i$ represents the direction of historical time, and $d$ represents the dilation factor

In the residual block, by setting different dilation rates, the convolution operation is performed in parallel at different time scales, enabling the capture of both short-term and long-term dependencies. After the convolution operation, a normalization layer (such as Batch Normalization) is applied to help stabilize training, followed by activation functions such as ReLU to introduce non-linearity and enhance the model's expressiveness.

To prevent overfitting, a Dropout layer is introduced in the residual block, randomly deactivating some neurons and improving the model's generalization ability.

MsTCN captures information at different temporal scales through convolutions with different dilation rates and fuses these multi-scale features at each layer. By transmitting information across hidden layers, the model can gradually aggregate features from different time windows, thus capturing local details while maintaining an understanding of the overall trend. This multi-scale fusion design enables MsTCN to handle diverse time series data.

In the MsTCN structure, assuming the output of the n-th layer is denoted as $T(n)$, the output of an MsTCN with m layers can be expressed as:

where, $Outputs (\cdot)$ represents the final output of the MsTCN, and $Concate (\cdot)$ represents the concatenation operation, i.e., concatenating multiple feature matrices along the last channel dimension.

BLS is derived from a network structure called the Random Vector Functional-Link Neural Network (RVFLNN) [21]. It is an efficient neural network learning framework designed to improve learning capacity and efficiency by expanding the width rather than the depth of the network. Compared with traditional deep learning, broad learning builds the network by stacking multiple layers of feature nodes and enhancement nodes, thus avoiding the common issues of gradient vanishing and complex training processes in deep learning.

Broad learning consists of an input layer, a hidden layer, and an output layer, with its structure diagram shown in Figure 4.

As shown in Figure 4, the hidden layer of BLS is composed of two parts: one part consists of nodes obtained through neural network learning of the original features, called feature nodes, while the other part consists of new nodes obtained through incremental learning of the feature nodes, called enhancement nodes.

In BLS, after processing through the feature node layer, the input undergoes a linear transformation using specific weight matrices and biases and is processed non-linearly through activation functions, generating an initial feature representation. This process can be expressed as:

For a given training set $\left\{(X, Y) \mid X \in R^{N \times p}, Y \in R^{N \times q}\right\}$, where $N$ is the number of training samples, $p$ is the sample dimension, and $q$ is the label dimension, the output of the mapping layer is:

where, $\varphi(\cdot)$ represents the activation function from the input layer to the mapping layer, and $W_{\mathrm{ei}}$ and $\beta_{\mathrm{ei}}$ are the weights and biases of the $i$-th feature node. The final feature layer $Z$ is:

The feature nodes extract features from the input data and map the input data into the feature space using different weight matrices and nonlinear activation functions. In BLS, the feature node layer plays a role similar to the hidden layer in traditional neural networks but with a simpler structure and lower computational overhead.

The enhancement node layer combines the output of the feature nodes with a set of new weight matrices in a linear combination and applies a nonlinear function, producing the output:

By concatenating the feature nodes and enhancement nodes, the final output $Y$ is obtained.

The output from both the feature nodes and enhancement nodes is connected to the final output layer through output weights. The output layer combines the information from both feature and enhancement nodes and generates the final prediction result $Y$ by weighted summation:

where, $W$ is the output weight between the hidden layer and the output layer, then $W=X^+ Y$ can be obtained through pseudo-inverse calculation.

BLS expands the feature space through random feature mapping and enhancement nodes, analyzing the health state of bearings from multiple dimensions and enhancing the model's expressive capacity to improve the performance of RUL prediction.

The multi-scale temporal convolution module and the broad learning module mentioned earlier form the basic structure of BLS-MsTCN. The model structure of BLS-MsTCN is shown in Figure 5.

As shown in Figure 5, MsTCN replaces the mapping relationship between feature nodes and enhancement nodes in BLS, enhancing the learning capacity of the network without increasing its complexity.

Furthermore, BLS-MsTCN combines broad learning and multi-scale temporal convolution to capture multiple signal features simultaneously, enabling the model to integrate diverse information when processing large datasets, thereby improving prediction accuracy.

The multi-layer convolution structure in BLS-MsTCN can capture both short-term and long-term features by using convolution kernels of different sizes. Small convolution kernels can identify rapidly changing signal patterns, while large convolution kernels can extract slowly changing trends, enhancing the model's prediction precision.

Assume $X$ is the input data for rolling bearings, $x \phi, \zeta$. are activation functions from the input layer to the feature layer and from the feature layer to the enhancement layer, respectively, and $Y$ is the actual remaining life. The BLS-MsTCN network algorithm flow is shown in Algorithm 1.

Hyperparameters play a crucial role in the performance of the BLS-MsTCN model. Different configurations of hyperparameters not only affect the training speed but also directly impact the model’s generalization ability. For bearing RUL prediction, appropriate hyperparameter settings enhance the model's capability to capture complex temporal dependencies, thus improving prediction accuracy.

As shown in Table 1, the hyperparameter settings have been experimentally validated to achieve optimal prediction performance on the specific dataset used in this study.

The model uses the Adam optimizer to effectively handle training on large datasets, with a global learning rate set at 0.001 [22], [23]. Adam is a popular and widely-used optimizer, known for its adaptive learning rate adjustment mechanism, which performs well across many tasks. The combination of MSE and MAE loss functions is applied to comprehensively evaluate model performance. By setting dilation factors to 1/2/4, the model can capture features across different temporal scales, enabling the extraction of multi-scale temporal information. The use of 32 convolution kernels and a kernel size of 3×3 ensures effective feature extraction. Setting the learning rate to 0.001 enables the model to converge more stably, reduces fluctuations, and ensures stable updates of the model.

To enhance the model's nonlinear representation capabilities, the feature multiplier and number of enhancement nodes are both set to 5, with a total of 64 enhancement nodes. During training, a batch size of 512 and 200 epochs ensures that the model is sufficiently trained. These hyperparameter choices provide a solid foundation for the model's accuracy, stability, and generalization ability.

The combination of the Adam optimizer, a small learning rate, MSE and MAE loss functions is well-suited for complex time series prediction tasks. The dilation factors, multi-scale convolution kernels, a large batch size, and sufficient epochs grant the model adequate expressive power and training space.

The data used in this experiment comes from the bearing run-to-failure dataset provided by the PHM 2012 Data Challenge, collected from the PRONOSTIA accelerated degradation platform [24], as shown in Figure 6.

Two accelerometers are mounted horizontally and vertically on the experimental setup to collect vibration signals from two directions, namely the x and y directions. The data sampling rate is 25.6 kHz, and data is recorded for 0.1 seconds every 10 seconds, totaling 2560 samples. For safety, the experiment stops when the amplitude of the vibration data exceeds 20g (20 times the gravitational acceleration, where $1 \mathrm{~g} \approx 9.8 \mathrm{~m} / \mathrm{s}^2$). The specific test conditions are shown in Table 2.

In the prediction of RUL for bearings, the RUL prediction is treated as a regression problem. During modeling, the bearing's operating time is transformed into input features, and the remaining life percentage is used as the RUL prediction label. This label design intuitively reflects the degradation level of the bearing throughout its lifecycle, helping improve the performance of the prediction model. The formula is as follows:

where, $x_i$ represents the current operating time of the bearing, $x_{\text{min}},x_\text{max}$ are the bearing's initial and maximum operating times, respectively; $y_i$ represents the severity of bearing degradation, and the smaller its value, the more severe the degradation. When $y_i$=0, the bearing is fully failed.

In evaluating the experimental results, in addition to the commonly used MAE and Root Mean Squared Error (RMSE), a scoring function specifically adapted for the PHM 2012 dataset, referred to as “score”, is used to assess the performance of the developed method. The formulas are as follows:

In practice, underestimating or overestimating RUL leads to different consequences. Underestimating RUL might result in premature maintenance and unnecessary downtime, which increases costs but poses relatively minor risks. Overestimating RUL is more dangerous as it might allow the equipment to continue running without timely maintenance, increasing the risk of failures and accidents.

The score function gives higher penalties for overestimation, reflecting its severity, ensuring that the model is more cautious when predicting the remaining lifespan of the equipment. At the early stages of an equipment lifecycle, prediction errors have relatively little impact because the equipment is far from failure, and inaccurate predictions pose lower risks. In contrast, errors in the later stages of the lifecycle are more critical as they are closer to equipment failure, making the consequences of prediction errors more severe. Therefore, the score function assigns higher weights to prediction errors in the later stages of the lifecycle, ensuring more accurate predictions at critical stages, which helps reduce failure risks.

The score range is between (0,1), and the higher the score value, the better the prediction performance. However, in performance evaluation, it is common to encounter inconsistent results between the three scoring standards (MAE, RMSE, and score). This inconsistency arises because MAE and RMSE do not consider the sign of the error (overestimation or underestimation) or differentiate between the impacts of different errors, while score incorporates a penalty mechanism for overestimations and underestimations, with a heavier penalty for overestimation. This design aligns better with practical applications, making score a more suitable metric when selecting or evaluating models.

For data processing, the x-axis and y-axis signals are stacked to form a new array of dimension (2660, 2), which is used as input to the model.

The data is first divided into training and test sets, followed by preprocessing, and then fed into the BLS-MsTCN model for prediction. The hyperparameters used are shown in Table 1.

The preprocessing step primarily involves normalizing the vibration data and RUL. The raw RUL values can vary widely, making it difficult for the model to effectively process during training. Normalization compresses the data into the (0,1) range, ensuring smoother weight updates and faster convergence, improving training efficiency. The normalized RUL is defined by Eq. (10).

This experiment employs a comparative validation method using the data from Condition 1 and Condition 2. For both conditions, the data from the first six bearings is used to train the model, and the last set of data is used to test the results. Predictions are then compared across the selected bearings. Figure 7 shows the RUL prediction results for bearings 1_3, 2_3, 1_4, and 2_5.

As shown in Figure 7, the RUL prediction results of bearings using the BLS-MsTCN model can effectively track the actual RUL of the bearings.

The experiment selected two bearings from each of Condition 1 and Condition 2, and predictions were made using four different models. Figure 8 provides a comparison of the RUL prediction results for bearings 1_6, 2_6, 1_7, and 2_7 across the four models.

As shown in Figure 8, compared with the other three models, the BLS-MsTCN model tracks the actual RUL of the bearings more effectively, with relatively smaller fluctuations.

The 14 sets of bearing data from Condition 1 and Condition 2 were tested across the four prediction models, and the final prediction results were compared. The scores of the seven bearings from Condition 1 are displayed as a bar chart. The results are shown in Table 3 and Figure 9.

From Table 3 and Figure 9, it can be observed that the score of the BLS-MsTCN model is significantly better than that of other models. Additionally, the BLS-MsTCN model demonstrates some advantages in terms of MAE and RMSE compared to the other three models. Although the prediction results for certain bearings may not show a significant advantage in score and may even be slightly lower than those of other models, the overall comparison in Figure 9 clearly indicates that the BLS-MsTCN model achieves better predictions.

Table 4 presents the parameter quantities and computation times for the four models in predicting the RUL of the bearings.

Table 4 shows that the BLS-MsTCN model processes a larger volume of data in a shorter time compared to Bi-TCN-LSTM and MsTCN-transformer. Though its parameter count is slightly lower than BLS-TCN, the BLS-MsTCN model takes 0.12 seconds longer due to the multi-scale operations it performs. However, as indicated in Figure 8, the BLS-MsTCN model provides more accurate RUL predictions, making it a better choice for tracking the actual RUL of bearings.

To address the limitations of current RUL prediction models—such as insufficient model expressiveness and difficulties in feature selection and extraction—this paper proposes a novel model combining BLS and MsTCN, named BLS-MsTCN. The BLS component enhances the model’s feature representation through random feature mapping and expansion with enhanced nodes, while the MsTCN component improves prediction accuracy by extracting features at different time scales using various convolution kernel sizes. The experimental results show that the BLS-MsTCN model not only increases the model’s expressiveness but also effectively captures complex nonlinear relationships within the data. In the future, research will focus on improving the model's performance across different operating conditions to maintain high accuracy for bearing RUL predictions.