Improved Relay Selection Strategy for 5G Non-Orthogonal Multiple Access Cooperative Communication Systems

In recent years, NOMA has been widely applied in various emerging communication scenarios. By transmitting superimposed coded signals, NOMA has significantly enhanced network performance [1], [2], [3]. As one of the most promising multiple access technologies, NOMA improves spectrum efficiency and system capacity, addressing several challenges in 5G wireless systems [4], [5], [6]. It has been recognized as a promising candidate for next-generation wireless networks, as it supports large-scale networks and achieves high spectral efficiency. By employing multiplexing techniques, multiple users' superimposed coded information is transmitted at the sender over the same time-frequency resources, greatly increasing spectrum utilization and user access capacity. It is anticipated that NOMA will meet the stringent performance requirements of future mobile communications [7], [8], [9]. The core principle of NOMA is to serve multiple users on the same resource block, which not only exponentially increases network transmission rates but also significantly reduces access latency, thereby enabling the connection of massive devices. Compared to orthogonal frequency division multiplexing (OFDM), NOMA can accommodate a larger number of users while ensuring fairness and improving spectral efficiency [10], [11], [12]. For low-traffic or cost-free applications, the grant-free NOMA uplink has been demonstrated to effectively reduce latency, communication overhead, and terminal power consumption. NOMA outperforms orthogonal multiple access (OMA) by supporting a relatively larger number of users. In OMA, each user is allocated a distinct sub-band, whereas in NOMA, multiple users simultaneously utilize the same sub-band with different power levels. Consequently, NOMA achieves higher spectral efficiency than OMA [13], [14], [15]. This study primarily focuses on power-domain NOMA.

Multiple access technology constitutes a fundamental component of wireless communication networks. Based on the principles and characteristics of various multiple access techniques, extensive research has been conducted on NOMA-related networks. Zhang and Ge [16] explores the integration of NOMA and energy harvesting (EH) in a relaying network to improve spectral efficiency and user fairness in 5G downlink systems. The proposed NOMA-EH framework employs an EH-enabled relay to connect a base station (using transmitting antenna selection) with multiple users (equipped with maximal ratio combining). The study evaluates outage performance, derives a closed-form outage probability expression, and validates through simulations that the NOMA-EH scheme outperforms conventional orthogonal access methods, demonstrating enhanced reliability and energy efficiency for sustainable wireless communication networks [17]. Lee et al. [18] investigated the impact of partial relay selection on the outage performance of two users in NOMA cooperative systems. However, in this approach, only the channel state information between the source node and the relay was considered, without comprehensively incorporating the state information of all communication links. Tang et al. [19] explores integrating device-to-device communication into a NOMA system. To handle complex co-channel interference from dense spectral reuse, it aims to maximize the sum proportional bit rate by jointly optimizing mode selection and power allocation. Given the original problem's high complexity and the wireless environment's dynamics, the authors propose an online double-layer mechanism combining machine learning with optimization theory. When the mode selection scheme is set, the remaining non-convex power allocation problem can be transformed into a convex one. Based on this optimum, a deep reinforcement learning-based online mechanism is designed, refining the deep neural network-generated MS scheme using recent historical experiences. Simulations show the proposed mechanism's superiority in balancing performance and online computational time tradeoff. Liu et al. [20] examines user clustering and power control in uplink MISO-NOMA networks, aiming to minimize system transmit power. It proposes a two-step algorithm: first, a K-means-based method clusters users considering channel gain and correlation to reduce interference; second, a semi-orthogonal user selection algorithm determines optimal cluster numbers and centers. It derives a closed-form expression for intra-cluster power control and solves the inter-cluster power control problem via an iterative algorithm. Simulation results show the proposed scheme outperforms reference methods, achieving near-optimal power consumption and energy efficiency with lower computational complexity.

Poposka et al. [21] explores federated learning in a multi-user wireless network, focusing on minimizing latency. wireless stations (WS) use NOMA to simultaneously transmit local model parameters to the base station. the resource allocation scheme considers users' maximum CPU frequency, transmit power, and available energy. it ensures fair resource sharing by restricting only one WS to use maximum allowable energy or transmit power, with others using less. the closed - form analytical solution for optimal resource allocation parameters enables efficient online implementation with low computational complexity. NOMA enables multi-user access by allowing users to transmit superimposed signals over the same time-frequency resources while leveraging non-orthogonality in the power or code domain. The core principle involves dynamic power allocation at the transmitter based on channel conditions, while at the receiver, successive interference cancellation (SIC) is employed to iteratively decode and separate user signals. This technique significantly enhances spectral efficiency and system capacity, making it particularly suitable for high-density user scenarios such as 5G, the Internet of Things (IoT), and visible light communication (VLC) systems. Additionally, dynamic power adjustment is supported to accommodate diverse business requirements. However, several challenges associated with NOMA have been identified. The high complexity at the receiver, which relies on SIC and multi-user detection algorithms, poses a significant implementation challenge. Moreover, the performance of NOMA is highly sensitive to power difference, as improper power distribution may lead to signal separation failures. Furthermore, channel estimation errors have been shown to substantially impact system performance. Despite these challenges, NOMA demonstrates considerable potential for applications in unmanned aerial vehicle (UAV) communications, industrial IoT, and future 6G networks. Its integration with multiple-antenna technologies and cooperative communication has been recognized as a promising approach to further enhance coverage and transmission reliability.

The cooperative NOMA network considered in this study is illustrated in Figure 1. The system consists of a base station, a far user $D_1$, a near user $D_2$, and N relay nodes. The focus is placed on a small-scale communication scheme within a specific time-frequency resource block, where communication occurs between a base station and two users. It is assumed that no direct link exists between the base station and the users; thus, communication with the NOMA users is established exclusively through relay nodes.

All channels in the system are assumed to experience Rayleigh flat fading, satisfying $h_{S R_n} \sim C N\left(0, \lambda_{S R_n}\right)$, $h_{R_n D_1} \sim C N\left(0, \lambda_{R_n D_1}\right)$ and $h_{R_n D_2} \sim C N\left(0, \lambda_{R_n D_2}\right)$, where the noise follows white Gaussian noise with zero mean and variance $\sigma^2$. The decode-andforward (DF) protocol is adopted at the relay nodes. The distances between the base station and the relay, as well as between the relay and the two users, are denoted as $d_{\mathrm{SR}_{\mathrm{n}}}, d_{\mathrm{R}_{\mathrm{n}} \mathrm{D}_1}$, and $d_{\mathrm{R}_{\mathrm{n}} \mathrm{D}_2}$, respectively. Let $X$ represent the path loss factor, then the variances of the respective channels are given by $\lambda_{S R_n}=$ $d_{S R_n}^{-X}, \lambda_{R_i D_1}=d_{R_i D_1}^{-X}$ and $\lambda_{R_n D_2}=d_{R_n D_2}^{-X}$. The base station, relay, and users are all equipped with a single antenna. The channel gains of the two users satisfy $\left|h_{R_n D_1}\right|^2<\left|h_{R_n D_2}\right|^2$. The signal sent by the base station transmitter satisfies $x_s=\sqrt{a_1 P_S} x_1+\sqrt{a_2 P_S} x_2$, where $P_s$ and $P_g$ denote the transmission power of the base station and relay node, respectively, and $a_1$ and $a_2$ are the power allocation factors, satisfying $a_1>a_2$, with $a_1+a_2=1$. The transmission power of the base station and relay satisfies the condition $P_S+P_R \leq P_T=2 P$, and the noise satisfies $\sigma_{R_n}^2=\sigma_{D_1}^2=\sigma_{D_2}^2=\sigma^2$. The system signal-to-noise ratio (SNR) is defined as $\gamma=\frac{\mathrm{P}}{\sigma^2}$, with the SNRs at the base station and relay given by $\gamma_S=\frac{P_S}{\sigma^2}$ and $\gamma_R=\frac{P_R}{\sigma^2}$, respectively.

In the first time slot, the received signal at the relay $R_n$ can be expressed as:

The relay begins to decode the information of $D_1$ using SIC and then demodulates the information of $D_2$. The signal-to-interference-plus-noise ratio (SINR) for $D_1$ can be expressed as $\gamma_{R_n}^1=\frac{a_1 \gamma\left|h_{S R_n}\right|^2}{a_2 \gamma\left|h_{S R_n}\right|^2+1}$, and the SINR for $D_2$ can be represented as $\gamma_{R_n}^2=a_2 \gamma\left|h_{S R_n}\right|^2$.

In the second time slot, after successfully decoding the information of both users, the relay forwards the information. The received signals at $D_1$ and $D_2$ can be expressed as:

The SINR of the signal received by $D_1$ can be expressed as:

The SINR of the base station-relay- $D_1$ link can be expressed as:

For $D_2$, the information of $D_1$ must first be demodulated. The SINR for demodulating the information of $D_1$ is given by:

Following the successful demodulation of the information of $D_1$, the received SINR for the intrinsic information of $D_2$ can be represented as:

Similarly, the SINR of the base station-relay- $D_2$ link is given by:

In cooperative communication, information is forwarded by relays, and signals from both the source and relay are processed at the receiving end. Due to its ability to mitigate signal fading, cooperative communication enhances spatial diversity gain. Recent research on relay selection has primarily focused on the relay selection algorithms based on location information [22], [23], instantaneous channel information [24], [25], statistical channel information [26], [27], and partial channel information [28], [29].

The random relay selection strategy involves selecting a relay randomly within the communication system. This approach is characterized by its randomness and offers advantages such as low complexity, ease of implementation, and the elimination of the need for feedback. However, since the selected relay may not always have favorable channel conditions, suboptimal communication performance may result [30].

In NOMA cooperative transmission networks [31], the partial relay selection strategy considers only the instantaneous channel gain between the source node and relay nodes. The relay node with the highest instantaneous SNR at the relay is selected as the optimal relay. The partial relay selection strategy can be expressed as:

The max-min relay selection strategy differs from conventional instantaneous channel gain-based opportunistic relay selection strategies in that all communication links are considered. The relay exhibiting the maximum value among the minimum channel gains is identified as the optimal relay. The max-min relay selection criterion can be formulated as:

During the first time slot, the signal of $D_1$ is initially decoded by the relay and subsequently canceled through SIC, enabling the signal decoding of $D_2$. The decoding conditions for the $n$-th relay to successfully decode both signals are given by:

where, $R_1$ and $R_2$ denote the target data rates of the users.

In the second time slot, the near user $D_2$ must first decode the signal of the far user $D_1$. SIC is then applied to remove the signal of $D_1$, after which $D_2$ decodes its own signal. The condition for $D_2$ to successfully decode the signal of $D_1$ is given by:

where, $\varepsilon_1=\frac{2^{2 R_1}-1}{\gamma\left(a_1-\left(2^{2 R_1}-1\right) a_2\right)}$ and $\varepsilon_2=\frac{2^{2 R_2}-1}{\gamma a_2}$ represent $|h|^2$ corresponding to the target data rates of $D_1$ and $D_2$, respectively.

The enhanced max-min relay selection strategy utilizes a weighted coefficient method. Since all signals must be decoded at both the relay node and $D_2$, while $D_1$ is only required to decode its own signal, weighting coefficients $\varepsilon_1$ and $\varepsilon_2$ are introduced to ensure channel fairness. The expression for the enhanced max-min relay selection strategy is formulated as:

For the far user $D_1$, the end-to-end link rate is expressed as:

For the near user $D_2$, the end-to-end link rate is given by:

In this section, the closed-form outage probability expressions for both the max-min relay selection strategy and the enhanced max-min relay selection strategy are derived. When a specific relay participates in cooperative transmission, the system outage probability can be expressed as:

where, $\left|h_{S R_n}\right|^2 \geq \max \left(\varepsilon_1, \varepsilon_2\right)$ denotes that both users at relay $n$ remain in a non-outage state. The term $\left|h_{R_n D_1}\right|^2 \geq \varepsilon_1$ represents that $D_1$ is not in an outage, where data decoding of $D_2$ is not required. Additionally, $\left|h_{R_n D_2}\right|^2 \geq \max \left(\varepsilon_1, \varepsilon_2\right)$ corresponds to the situation that both users' information remains in a non-outage state at $D_2$. In this case, the signal of $D_1$ must first be decoded and removed through SIC before $D_2$ can decode its own information.

(a) Outage probability of the enhanced max-min relay selection

Let $O_1=\left\{\underset{n^* \in N}{\arg \max }\left\{\min \left\{\varepsilon_1\left|h_{S R_n}\right|^2, \varepsilon_2\left|h_{R_n D_1}\right|^2, \varepsilon_1\left|h_{R_n D_2}\right|^2\right\}\right\}\right\}$ represent the relay node $n^*$ selected using the enhanced max-min relay selection strategy. The system outage probability is then given by:

Let $Y_n=\min \left\{\varepsilon_1\left|h_{S R_n}\right|^2, \varepsilon_2\left|h_{R_n D_1}\right|^2, \varepsilon_1\left|h_{R_n D_2}\right|^2\right\}$ and all channels be independent and identically distributed (i.i.d.) Rayleigh fading channels. The probability density function (PDF) is denoted by $f(x)=\frac{1}{\sigma^2} e^{\frac{x}{\sigma^2}}$, and the cumulative distribution function (CDF) of $Y_n$ is expressed as:

Let $Y_{n^*}=\max \left\{Y_1, Y_2, K, Y_N\right\}$. The CDF of $Y_{n^*}$ can then be expressed as:

Decoding begins with $D_1$, where the conditions of $R_1 \leq R_2$ and $\varepsilon_1 \leq \varepsilon_2$ must be satisfied. The system outage probability can be formulated as:

(b) Outage probability of the max-min relay selection

Let $O_2=\left\{\arg \max _{n^* \in N}\left\{\min \left\{\left|h_{\mathrm{SR}_{\mathrm{n}}}\right|^2,\left|h_{\mathrm{R}_{\mathrm{n}} \mathrm{D}_1}\right|^2,\left|h_{\mathrm{R}_{\mathrm{n}} \mathrm{D}_2}\right|^2\right\}\right\}\right\}$ represent the relay node $n^*$ selected using the max-min relay selection strategy. The system outage probability can then be expressed as:

Let $X_n=\min \left\{\left|h_{S R_n}\right|^2,\left|h_{R_n D_1}\right|^2,\left|h_{R_n D_2}\right|^2\right\}$ and $X_{n^*}=\max \left\{X_1, X_2, \ldots, X_N\right\}$. The CDFs of $X_n$ and $X_{n^*}$ are given by:

By applying Eq. (21), the system outage probability for the max-min relay selection strategy can be computed using the following expression:

Figure 2 illustrates the throughput performance of the near and far users under the enhanced max-min relay selection strategy. The source, relay, $D_1$, and $D_2$ are positioned at coordinates (0,0), (0.5,0), (1.5,0), and (1,0), respectively. The target data rates are $R_1=0.5 \mathrm{bps} / \mathrm{Hz}$ and $R_1=0.8 \mathrm{bps} / \mathrm{Hz}$, while the power allocation coefficients are given by $a_1=\frac{3}{4}$ and $a_2=\frac{1}{4}$. It can be observed that in the NOMA cooperative system, the near user successfully eliminates interference from the far user's information using SIC. In contrast, the far user continuously experiences interference from the near user's information. As the throughput increases, a threshold is eventually reached. Additionally, as the number of available relays in the system increases, the throughput of both users also improves. This result confirms that NOMA cooperative communication fundamentally enhances diversity gain by forming a virtual antenna array.

Figure 3 presents the variation in the near user’s throughput as a function of the average SNR under four relay selection strategies: random relay selection, partial relay selection, max-min relay selection, and enhanced max-min relay selection. Among these strategies, the enhanced max-min relay selection strategy demonstrates the best performance in terms of throughput.

Figure 4 illustrates the system outage probability under the enhanced max-min relay selection strategy for different numbers of relays. It can be observed that as the number of available relays increases, the system outage probability decreases correspondingly. This result indicates that in scenarios with poor channel conditions, deploying a larger number of relays—provided that sufficient resources are available—can effectively reduce communication interruptions and ensure stable communication. Consequently, the system outage condition can be expressed as:

For the overall system, the near user is required to decode the far user's information first due to the requirement of SIC. This requirement serves as a critical factor affecting the system outage probability. Therefore, the system outage probability can also be expressed as:

where, $\gamma_{R_n D_2}^1=a_1 \gamma\left|\mathrm{~h}_{\mathrm{R}_{\mathrm{n}} \mathrm{D}_2}\right|^2 /\left(a_2 \gamma\left|\mathrm{~h}_{\mathrm{R}_{\mathrm{n}} \mathrm{D}_2}\right|^2+1\right)$ represents the SNR of the far user's signal at the near user's receiver.

Figure 5 illustrates the outage probability under different cooperative strategies. It can be observed that the enhanced max-min relay selection strategy achieves the lowest outage probability. Meanwhile, the outage probabilities of random relay selection and partial relay selection strategies exhibit minimal differences. This result further validates the effectiveness of the enhanced max-min relay selection strategy.

Figure 6 illustrates the system outage probability under the enhanced max-min relay selection strategy for different Rayleigh fading parameters. The number of available relays ($K$=5), target data rates, and power allocation coefficients were set as in the previous configurations. By comparing the different curves of $\sigma^2$, it can be observed that when $\sigma^2=10$, the outage performance improves, particularly in the low-SNR region, where the performance gain is more significant. This improvement occurs because larger $\sigma^2$ values correspond to better channel conditions $\left(\mathrm{E}\left(|h|^2\right)=\sigma^2\right)$.

NOMA technology significantly increases the number of connected devices, effectively mitigating the challenges posed by limited bandwidth resources and the rapid growth of terminal devices. In this study, an enhanced max-min relay selection strategy was designed for NOMA cooperative communication systems based on the conventional max-min relay selection strategy. Unlike the traditional max-min relay selection strategy, the proposed approach incorporates a weighting mechanism that adjusts each channel according to the target data rates of the users, ensuring fairness across all channels. Mathematical derivations and simulation analyses were conducted to analyze and validate the proposed relay selection strategy in terms of user throughput and system outage probability. To verify the effectiveness of the proposed relay selection strategy, a comparative analysis was performed against other relay selection strategies. The results demonstrate that the proposed strategy achieves a lower outage probability and improved throughput, highlighting its superiority in enhancing system reliability and performance.