Main Content

Constellation Shaping for 6G

R2026b
Since R2026b

Conventional digital communication systems use uniform quadrature amplitude modulation (QAM), where the transmitter selects every constellation point in a regular square grid with equal probability. This approach leaves a performance gap relative to the theoretical channel capacity because the capacity-achieving input distribution of a Gaussian channel is Gaussian, not uniform. As 6G systems target higher spectral efficiencies with large constellation sizes, closing this gap becomes increasingly important.

Constellation shaping closes this gap. There are two complementary approaches:

  • Geometric shaping — Optimizes the positions of constellation points to improve performance while transmitting each point with equal probability.

  • Probabilistic shaping — Keeps the regular QAM grid but transmits lower-energy points more frequently than higher-energy points.

Both approaches reduce the shaping gap, and they can be combined. 3GPP has studied both geometric and probabilistic shaping for potential inclusion in 6G. [1] [2] The AI-Native Differentiable 6G Link using Autoencoder example shows how to implement an AI-native differentiable pre-6G link with an autoencoder that uses geometric shaping. In that example, an AI encoder and decoder learn jointly. The encoder learns the optimal choice of constellation and the decoder learns the optimal choice of decision boundaries.

Shaping Gap in Uniform QAM

For an additive white Gaussian noise (AWGN) channel, the capacity-achieving input distribution is Gaussian. A uniform QAM constellation, where each point transmits with equal probability, does not match this distribution. Historically, communication systems did not try to approximate Gaussian input distributions because the implementation complexity was too high. Uniform QAM was preferred because it offers a good trade-off between performance, spectral efficiency, and implementation complexity. Modern computational power enables experimentation with more complex modulation schemes and bit-to-symbol mappings.

The performance loss that results from using uniform QAM instead of a Gaussian distribution is called the shaping gap. At high signal-to-noise ratios (SNRs), the shaping gap approaches approximately 1.53 dB [3]. This means that a uniform QAM system requires approximately 1.53 dB more transmit power than a Gaussian-distributed input to achieve the same data rate.

The shaping gap increases in practical significance as the modulation order grows. For small constellations such as QPSK, the gap is negligible because few distinct amplitude levels exist. For higher-order schemes such as 4096-QAM, many amplitude levels are available, and the potential gain from shaping is larger.

Geometric Constellation Shaping

Geometric shaping (GS) optimizes the positions of constellation points to improve link performance. The transmitter chooses each point with equal probability, but the points are repositioned from the regular QAM grid to an irregular layout. The design criterion determines how the points are arranged:

  • Minimum-distance optimization — Under a transmit power constraint, the problem is equivalent to 2-D sphere packing: maximize the minimum Euclidean distance between any pair of points. This reduces the symbol error rate under a minimum-distance detector.

  • Mutual-information optimization — Maximize the bit-wise mutual information between transmitted bits and received symbols. When LDPC codes are used, this is equivalent to approaching the capacity of each bit channel. An autoencoder that learns both the constellation and the demapper using binary cross-entropy loss performs this optimization.

The two criteria produce different constellations. Minimum-distance designs focus on error-rate performance at a specific operating point. Mutual-information designs target the broadest achievable rate region for coded systems.

This figure illustrates the concept. In a geometrically shaped 16-QAM constellation, the points are shifted from the uniform grid positions to optimize mutual information. In this example, the points form an irregular, rotated pattern, and the outer points expand beyond the boundary of the uniform grid. All points are transmitted with equal probability.

Because GS changes only the constellation geometry and not the symbol probabilities, it uses a traditional transmit-receive chain without additions or alterations. The transmitter maps bits to a lookup table of optimized point coordinates, and the receiver uses the same lookup table for demapping. Standard FEC encoders and decoders operate unchanged.

The optimal point placement depends on the operating SNR. A constellation optimized for one SNR does not achieve the full shaping gain at a different SNR. Practical systems address this by designing a small set of constellations, each targeting a different SNR range, and switching between them as channel conditions change.

Probabilistic Constellation Shaping

Probabilistic shaping (PS) modifies the probability distribution of transmitted symbols while keeping the regular QAM grid. The transmitter assigns higher probability to constellation points with lower amplitude and lower probability to points with higher amplitude.

The probability assignment that maximizes the data rate for a QAM constellation over an AWGN channel follows the Maxwell-Boltzmann distribution:

P(x)=exp(−ν|x|2)Z,

where x is a constellation point, ν ≥ 0 is the shaping parameter that controls how non-uniform the distribution is, and Z is a normalization constant. When ν = 0, the distribution is uniform and no shaping occurs. As ν increases, probability concentrates on lower-energy points, and the constellation approaches a Gaussian amplitude profile.

This figure illustrates the concept. In a shaped 64-QAM constellation, inner points (low amplitude) appear with higher probability than outer points (high amplitude). The point size and color represent relative transmission probability.

PS requires a distribution matcher at the transmitter to convert uniformly distributed input bits into amplitude symbols that follow the target distribution, and an inverse distribution matcher at the receiver to recover the original bits. Because PS retains the QAM grid, standard demappers and equalization algorithms operate unchanged. The receiver needs only knowledge of the symbol probabilities for optimal log-likelihood ratio (LLR) computation. You can simulate PS in a pre-6G link using this Probabilistic Shaping Link-Level Simulator by Qualcomm®, which is based on 5G Toolbox™.

Benefits for 6G Systems

Both shaping approaches have properties that align with 6G system requirements:

  • Shaping gain — Both GS and PS reduce the required SNR for a given target information rate. The theoretical upper bound is approximately 1.53 dB in AWGN. In fading channels, the gain varies with channel conditions but remains significant at high modulation orders.

  • Fine-grained rate adaptivity — Conventional systems adapt rate by switching between discrete modulation and coding scheme (MCS) combinations. PS adds a continuous rate dimension: adjusting the shaping parameter changes the information rate without changing the constellation size or code rate. GS achieves rate adaptation by switching between constellation designs optimized for different SNR points.

  • Reduced peak-to-average power ratio (PAPR) — PS concentrates probability on lower-amplitude symbols, reducing the average transmitted power and PAPR. GS can also reduce PAPR depending on the constellation design.

Comparison of Shaping Approaches

This table summarizes the key differences between uniform QAM, geometric constellation shaping, and probabilistic constellation shaping.

CharacteristicUniform QAMGeometric ShapingProbabilistic Shaping
Constellation geometryRegular QAM gridOptimized irregular gridRegular QAM grid
Symbol probability distributionUniformUniformNon-uniform (Maxwell-Boltzmann)
SNR dependency of designNoneConstellation optimized per SNR rangeShaping parameter adjustable per SNR
Rate adaptationDiscrete (MCS switching)Discrete (constellation switching)Continuous (shaping parameter adjustment)
Transmitter modificationsN/AConstellation lookup tableDistribution matcher
Receiver modificationsN/AConstellation lookup table for demappingLLR calculation uses symbol priors; inverse distribution matcher

References

[1] Qualcomm, Modulation, joint channel coding and modulation, 3GPP TSG RAN WG1 Meeting #126, R1-2606372, Maastricht, Netherlands, Aug. 2026.

[2] Ericsson®, Modulation for 6GR air interface, 3GPP TSG RAN WG1 Meeting #126, R1-2605203, Maastricht, Netherlands, Aug. 2026.

[3] G. David Forney Jr., "Trellis Shaping," IEEE Transactions on Information Theory 38, no. 2 (March 1992): 281–300.

See Also

Topics

External Websites