ModulationPy: M-PSK and M-QAM implementation

[kirlf]

Hello!

This issue relates to #99 and #60. One of my pet-projects is implementation of the M-PSK and M-QAM modems: ModulationPy. Actually, this is a little faster than your implementation :) (results).

I've tested M-PSK case more or less successfully:

However, I did not have time to do something similar for M-QAM (only unit-tests are done).

Actually, you can just import my module into your project, or we can discuss the optimal way if you are interested!

Have a nice day! :)

[BastienTr]

The results are very good for M-PSK! You have already implemented the M-QAM and only the test is missing, isn't it?

Anyway, importing your code seems the right short-term fix to #99 and #60. I see that you also use BSD 3-clauses licence so we could merge the projects on a mid-term basis.

[veeresht]

I would suggest importing ModulationPy as a dependency and using it in the functions in https://github.com/veeresht/CommPy/blob/master/commpy/modulation.py so that there is no API change for CommPy.

[kirlf]

@BastienTr, yes, there are no simulations for M-QAM in my project. Only unit-tests (decoded message is equal to uncoded) and signal constellation looks right:

@veeresht, I guess it will be the most straightforward way.

[BastienTr]

@kirlf, if you only miss BER vs. SNR curves, I can try to find some time to make them. It's not the most challenging part.

Regarding the embedding of ModulationPy, I see two options to add @kirlf's codes without API changes. On the first hand, we can add ModulationPy in the dependency tree as suggested by @veeresht. On the other hand, we can merge the code directly into the modulation module of CommPy.

Pros for the dependency:

• @kirlf keep full control on his codes ;
• if improvements are made on ModulationPy without API changes, this will automatically improve CommPy.

Cons for the dependency:

• makes the dependency tree more complicated for a piece of code that would have fit in commpy ;
• Put some constraints on ModulationPy since an API change would break CommPy.

I guess that the right choice depends mainly on @kirlf's plans for his repo.

[edsonportosilva]

@BastienTr

Hello guys,

I have quickly written three functions that could be used to implement the gray mapping for M-QAM in your package. You just need to include them as methods in the current class QAMModem and it should work. I have tested them with BER vs EbN0 and the results are in agreement with the theory. The demodulate function assumes only hard decision, though. I believe your code for soft-decision could probably be easily adapted to receive a Gray mapping (or any mapping) as input and return the corresponding LLRs, but I did not look into that. I am attaching a jupyter notebook file with the functions and a quick test. Let me know if it could be of any help to you.

GrayMapping.zip

[kirlf]

@BastienTr @veeresht actually, I won't plan to change API. The interfaces for M-PSK (https://github.com/kirlf/ModulationPy#1-m-psk) and M-QAM (https://github.com/kirlf/ModulationPy#2-m-qam) were designed according to MatLab style and opportunities. I guess to add something is not necessary (and we always can use default values or specify version of the library).

So, the opportunity to make the decision is yours! :)

[BastienTr]

I just had a quick review of your suggestions. First of all, thanks for your inputs!

I see two rather independent parts in this discussion: enhancing modem performance and the implementation of Gray coding for QAM modems.

In the first phase, I feel that we should focus on implementing Gray code for QAM since it is the most requested feature (Cf. #99 and #60). The present modem object supports any constellation and maintains its coherency (hard and soft decoding, average power calculation, etc.). Therefore, I'm considering the function proposed by @edsonportosilva as a complementary function to our code. Below is a simple demonstration to show how simple it is.

from sympy.combinatorics.graycode import GrayCode
from numpy.matlib import repmat
import numpy as np
from commpy.modulation import QAMModem


def GrayMapping(M):
    L = int(np.sqrt(M) - 1)
    bitsSymb = int(np.log2(M))

    PAM = np.arange(-L, L + 1, 2)
    PAM = np.array([PAM])

    # generate complex square-QAM constellation
    const = repmat(PAM, L + 1, 1) + 1j * repmat(np.flip(PAM.T, 0), 1, L + 1)
    const = const.T

    for ind in np.arange(1, L + 1, 2):
        const[ind] = np.flip(const[ind], 0)

    code = GrayCode(bitsSymb)
    a = list(code.generate_gray())

    const_ = np.zeros((M, 2), dtype=complex)
    const = const.reshape(M, 1)

    for ind in range(0, M):
        const_[ind, 0] = const[ind, 0]  # complex constellation symbol
        const_[ind, 1] = int(a[ind], 2)  # mapped bit sequence (as integer decimal)

    # sort complex symbols column according to their mapped bit sequence (as integer decimal)
    const = const_[const_[:, 1].real.argsort()]

    return const

QAM16 = QAMModem(16)
QAM16.constellation = GrayMapping(16)[:, 0]
assert QAM16.Es == QAMModem(16).Es
QAM16.plot_constellation()

Since GrayMapping is called only once, there is no point of optimizing it to much but it may still be useful to have a closer look. This function could be included in QAMModem and called by QAMModem.__init__. I like this solution all the more because it only requires adding to the dependencies a quite standard library (sympy).

After this quick fix, we could optimize the code by borrowing and/or embedding @kirlf's code, but the work is a bit heavier since we need to interface both codes.

What do you think about it?

[edsonportosilva]

From my point of view, that would also be the simplest and the best solution, in the sense that you keep the basic code structure of Commpy. In that case, @BastienTr, you may consider adding this modified version with support to 'qam' and 'psk' constellations:

def GrayMapping(M, constType):
    
    L   = int(np.sqrt(M)-1)
    bitsSymb = int(np.log2(M))
    
    code = GrayCode(bitsSymb)
    a    = list(code.generate_gray())
    
    if constType == 'qam':
        PAM = np.arange(-L, L+1, 2)
        PAM = np.array([PAM])

        # generate complex square M-QAM constellation
        const = repmat(PAM, L+1, 1) + 1j*repmat(np.flip(PAM.T,0), 1, L+1)
        const = const.T
    
        for ind in np.arange(1,L+1,2):
            const[ind] = np.flip(const[ind],0)        
        
    elif constType == 'psk':
        pskPhases = np.arange(0,2*np.pi,2*np.pi/M)
        
        # generate complex M-PSK constellation
        const     = np.exp(1j*pskPhases) 
    
    const    = const.reshape(M,1)
    const_   = np.zeros((M,2),dtype=complex)
    

    for ind in range(0,M):    
        const_[ind,0]   = const[ind,0]   # complex constellation symbol
        const_[ind,1]   =  int(a[ind],2) # mapped bit sequence (as integer decimal)
        
    # sort complex symbols column according to their mapped bit sequence (as integer decimal)                 
    const = const_[const_[:,1].real.argsort()] 
    
    return const

[kirlf]

@BastienTr yes, I am agree that the simplest way means the best way in this case.

[BastienTr]

Ok.
The last question is : do we make Gray coding the new default or do we add a new argument? On one hand, changing the default behaviour is a retrocompatibility issue. On the other hand, Gray coding is assumed by most users.

I feel like this is one of the rare situation when we should break the retrocompatibility but I'm not sure...

[edsonportosilva]

I think it's better to make Gray mapping default because e.g. many people who will try to use Commpy for the first time will look for some way to test if the code is doing what it's supposed to do by basically comparing its results with a theoretical reference. If you don't have Gray mapping as default, the mismatch between simulations and theory will make people suspicious that there is a bug somewhere. There is a gap of less than 0.5 dB from the current mapping to the Gray mapping performance for some QAM formats. Because this penalty is small in some cases, it's not straightforward to find from where it's coming from if you e.g. have already built a reasonable complex simulation. Until you realize it's the mapping, if you do, you may have lost a long time debugging stuff. So, making it default would avoid that.

[kirlf]

P.S.

@BastienTr I'm writing to inform that I've finalized BER simulation and performance testing for M-QAM in ModulationPy project!

Have a nice day!

[edsonportosilva]

@BastienTr @veeresht

Hello guys, just a heads up: I tried to update the Commpy version I use, to get the code after the update to implement Gray mapped constellations, and the script I had before with QAMModem was still not using Gray mapping as default. I don't know why. I didn't try to look into the code in detail, but I have checked that the modifications made in the last pull request are there. Maybe something went missing while updating the modems.

[BastienTr]

Hi @edsonportosilva,

Thanks for the feedback. Could you please share minimal working example (MWE) or an executable piece of codes that show the bug? I have some free time to have a look 😄

Did you update using pip or did you downloaded the github version? The pip repo is currently outdated. Do you think that I should make a new release just for the Gray coding?

[edsonportosilva]

Hello @BastienTr

I have updated the code manually downloading the files from GitHub. Check the notebook attached with a simple implementation to test BER vs EbN0. If you run it, you will see that the BER curve yet does not agree with the theory, which is the very same problem the pull request was supposed to fix.

CommpyTest_BERvsEbN0_March2021.zip

[BastienTr]

I just run your code on my laptop and everything work as expected. Here is the output from your code.

Moreover, the unit tests from the library match the theory for both m-PSK and m-QAM.

CommPy/commpy/tests/test_modulation.py

Lines 132 to 162 in 15eef39

	@dec.slow
	class TestModulateHardDemodulate(ModemTestcase):
	@staticmethod
	def check_BER(modem, EbN0dB, BERs_expected):
	seed(8071996)
	model = LinkModel(modem.modulate,
	SISOFlatChannel(fading_param=(1 + 0j, 0)),
	lambda y, _, __, ___: modem.demodulate(y, 'hard'),
	modem.num_bits_symbol, modem.constellation, modem.Es)
	BERs = model.link_performance(EbN0dB + 10 * log10(log2(modem.m)), 5e5, 400, 720)
	assert_allclose(BERs, BERs_expected, atol=1e-4, rtol=.1,
	err_msg='Wrong BER for a standard modulation with {} symbols'.format(modem.m))
	def do_qam(self, modem):
	EbN0dB = arange(8, 25, 4)
	nb_symb_pam = sqrt(modem.m)
	BERs_expected = 2 * (1 - 1 / nb_symb_pam) / log2(nb_symb_pam) * \
	Qfunc(sqrt(3 * log2(nb_symb_pam) / (nb_symb_pam ** 2 - 1) * (2 * 10 ** (EbN0dB / 10))))
	self.check_BER(modem, EbN0dB, BERs_expected)
	def do_psk(self, modem):
	EbN0dB = arange(15, 25, 4)
	SERs_expected = 2 * Qfunc(sqrt(2 * modem.num_bits_symbol * 10 ** (EbN0dB / 10)) * sin(pi / modem.m))
	BERs_expected = SERs_expected / modem.num_bits_symbol
	self.check_BER(modem, EbN0dB, BERs_expected)
	def do(self, modem):
	for bits in product(*((0, 1),) * modem.num_bits_symbol):
	assert_array_equal(bits, modem.demodulate(modem.modulate(bits), 'hard'),
	err_msg='Bits are not equal after modulation and hard demodulation')

Could you double check that you use the last Commpy version from github? I'll try to use a fresh install on my desktop to see if I can reproduce the bug on my laptop.

[BastienTr]

I just checked with a fresh intall using Python 3.9 and pip install -r requirements.txt. Everything work as expected.

[edsonportosilva]

Hi @BastienTr,

Funny, then It must be something went wrong when I tried to update the code in my notebook. I remember I tried to use pip and it didn't work, so I move to manually install everything, then I checked the version of Commpy and tried to run some tests...

But still, right now, with another fresh install, following the same steps as you mention, and using a new conda environment I get this:

I am using GitHub Desktop to download the files, let me download them directly to see what happens...

[edsonportosilva]

Ok, after downloading the files from GitHub, if I try to install using python setup.py install, I get this error:

(py39) C:\Users\edson\Documents\GitHub\edsonportosilva\CommPy>python setup.py install
Traceback (most recent call last):
  File "C:\Users\edson\Documents\GitHub\edsonportosilva\CommPy\setup.py", line 9, in <module>
    LONG_DESCRIPTION = open('README.md').read()
  File "C:\Users\edson\AppData\Roaming\SPB_Data\.conda\envs\py39\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 5410: character maps to <undefined>

This error is fixed if you do: LONG_DESCRIPTION = open('README.md', encoding="utf8").read()

[edsonportosilva]

All right, finally:

So, the problem was that some updates were missing and Anaconda was messing up with the environments in my notebook...
I guess the whole installation became a bit trickier than simply using pip. Perhaps, it would be nice to have a new release for the pip repo.

[BastienTr]

Cool that you figured it out. I'll try to update on pip when I have some time.

[BastienTr]

Just to let you know that pip should be updated to include Gray coding @edsonportosilva

[edsonportosilva]

Just to let you know that pip should be updated to include Gray coding @edsonportosilva

Thanks, @BastienTr! Right on time. I have just started teaching a course this semester where I plan to use Commpy, and pip will ease the way for the students to install it.