Need Advise,micropython gb2312 encoding bug? Thank you

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
stategrid
Posts: 5
Joined: Fri Feb 07, 2020 6:57 am

Need Advise,micropython gb2312 encoding bug? Thank you

Post by stategrid » Fri Feb 07, 2020 7:36 am

Hi guys,I‘m a newbie of micropython。
I have ssd1306 OLED display with GT20L16S1Y spi font chip,since micropython officially don’t support GT20L16S1Y 。so I begin to wite driver for font chip 。Accroding to GT20L16S1Y data sheet,GT20L16S1Y is a spi chip。Chinese gb2312 encoded data store in it。

gb2312 = text.encode('gb2312')
# add this line
print("len(gb2312):",len(gb2312))


on pc the rusult is:
"len(gb2312):",2

on micropython the rusult is:
"len(gb2312):",3

Is it a bug of micropython?How can i fix it?Thank you for your help
#############################################
following guide:
https://github.com/pengfexue2/printPlay ... intPlay.py

code run perfect on pc

Code: Select all

def printPlay(textStr,line,background):
    for text in textStr:

        # get chinese  gb2312 encode,each chinese  gb2312 character contains 2 bytes
        gb2312 = text.encode('gb2312')

      # add this line
        print("len(gb2312):",len(gb2312))
on pc the rusult is:
"len(gb2312):",2
#############################################
then modified code and run on ESP32 micropython 1.12 version 20200206 ,

Code: Select all

    def getAddr2(self,chn_str):
        global  Address 

         #encoding chines in 'gb2312'

        print("input is:",text)

        gb2312 = text.encode('gb2312')
        print("gb2312:",gb2312)
        print("len(gb2312)",len(gb2312))

###on  micropython  the rusult is:
####"len(gb2312):",3

        #get hex of gb2312
        hex_str = ubinascii.hexlify(gb2312)
        print("hex_str:",hex_str)
        print("len(hex_str)",len(hex_str))
 
        # get area


 +---+---+---+---+---+---+
 | P | y | t | h | o | n |
 +---+---+---+---+---+---+
 0   1   2   3   4   5   6
-6  -5  -4  -3  -2  -1   0

        k= hex_str[:2]
        kkk=str(k,'utf8')
        l=hex_str[2:]
        lll=str(l,'utf8')
        print("kkk: ",kkk)
        print("lll: ",lll)
        hb = eval('0x' + bytes.decode(hex_str[:2])) - 0xA0
          #get index 
        lb = eval('0x' + bytes.decode(hex_str[2:])) - 0xA0

        print("HB:")
        print(hb)
        print("LB:")
        print(lb)
        #predefined,or it go wrong
        Address = 0x00

        if (hb > 0xA0  and  lb > 0xA0): ##is GB2312
            BaseAdd = 0
        #/*chines standard(GB2312)15x16 in font IC ,to calculate Address of each character ,
        #following this formula:*/
        #/*Address = ((MSB - 0xa1) * 94 + (LSB - 0xA1))*32+ BaseAdd BaseAdd=0*/
        #/*8bit mcu overlap,so in 3 step*/
        if (hb == 0xA9  and  lb >= 0xA1):
            Address = (282 + (lb - 0xA1 )) ##8 bit chip run whith error,so in 3 step and work well
            Address = Address * 32
            Address += BaseAdd
        elif (hb >= 0xA1  and  hb <= 0xA3  and  lb >= 0xA1):
            Address = ((hb - 0xA1) * 94 + (lb - 0xA1))
            Address = Address * 32
            Address += BaseAdd
        #16~87  (0xb0~0xf7) :chinese characters
        elif (hb >= 0xB0  and  hb <= 0xF7  and  lb >= 0xA1):
            Address = ((hb - 0xB0) * 94 + (lb - 0xA1) + 846)
            Address = Address * 32
            Address += BaseAdd
        #01~09 :symbol、numers
        else: ##is ASCII
            BaseAdd = 0x03b7c0
            if (lb >= 0x20  and  lb <= 0x7E):
                Address = (lb - 0x20 ) * 16 + BaseAdd
        print("Address is:",Address)
        return Address
#########################
on micropython the rusult is:
"len(gb2312):",3
Last edited by stategrid on Wed Feb 12, 2020 7:25 pm, edited 1 time in total.

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Need Advise,micropython gb2312 encoding bug? Thank you

Post by jimmo » Wed Feb 12, 2020 3:23 am

I'm not sure I quite understand the question you're asking...

Can you provide a simple case (and a link to the libraries you're using) that shows the error you're seeing?

stategrid
Posts: 5
Joined: Fri Feb 07, 2020 6:57 am

Re: Need Advise,micropython gb2312 encoding bug? Thank you

Post by stategrid » Wed Feb 12, 2020 8:39 pm

Thank you for your help,here explan what‘s GB_2312
https://en.wikipedia.org/wiki/GB_2312

Bcause chinese has so much words than english,so in microchip system,engineer use
Dot matrix font to store, read and display chinese。The font can be stored in SPI flash chip,or font chip(which also use SPI interface but more cheeper)。
Each chinese word has an 2 bytes address in font chip,the higher bytes is Index and the lower is area.
For example,chinese word ”啊“(which means oh,pronunciation ”Ah“).
In chinse font chip, data of the word ”啊“ is:
0000b040 00 04 2f 7e f9 04 a9 04 aa 14 aa 7c ac 54 aa 54

It is the 16*16 Dot matrix of chinese word.
16*16 Dot matrix font of ”啊“ is:

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ■ ○ ○
○ ○ ■ ○ ■ ■ ■ ■ ○ ■ ■ ■ ■ ■ ■ ○
■ ■ ■ ■ ■ ○ ○ ■ ○ ○ ○ ○ ○ ■ ○ ○
■ ○ ■ ○ ■ ○ ○ ■ ○ ○ ○ ○ ○ ■ ○ ○
■ ○ ■ ○ ■ ○ ■ ○ ○ ○ ○ ■ ○ ■ ○ ○
■ ○ ■ ○ ■ ○ ■ ○ ○ ■ ■ ■ ■ ■ ○ ○
■ ○ ■ ○ ■ ■ ○ ○ ○ ■ ○ ■ ○ ■ ○ ○
■ ○ ■ ○ ■ ○ ■ ○ ○ ■ ○ ■ ○ ■ ○ ○
■ ○ ■ ○ ■ ○ ■ ○ ○ ■ ○ ■ ○ ■ ○ ○
■ ○ ■ ○ ■ ○ ○ ■ ○ ■ ○ ■ ○ ■ ○ ○
■ ■ ■ ○ ■ ○ ○ ■ ○ ■ ■ ■ ○ ■ ○ ○
■ ○ ■ ○ ■ ■ ○ ■ ○ ■ ○ ■ ○ ■ ○ ○
○ ○ ○ ○ ■ ○ ■ ○ ○ ○ ○ ○ ○ ■ ○ ○
○ ○ ○ ○ ■ ○ ○ ○ ○ ○ ○ ○ ○ ■ ○ ○
○ ○ ○ ○ ■ ○ ○ ○ ○ ○ ○ ■ ○ ■ ○ ○
○ ○ ○ ○ ■ ○ ○ ○ ○ ○ ○ ○ ■ ■ ○ ○

The address of ”啊“ is 1601(decimal system),which means Index==16 and Area==01.
IF we write in Hexadecimal,Index==10 and Area==01.

And chinese word ”啊“ after gb2312 encoding is 0xB0A1,that is ,”啊“.encode('gb2312')==0xB0A1.
the higer of gb2312 encoded ==B0, the lower of gb2312 encoded==A1.

so transition between gb2312 encoding and address is:
the higer of gb2312 encoded - Index == B0-10==A0H
the lower of gb2312 encoded - Area == A1-01== A0H

A0H is a constant.

offset= ( 94*(Index-1) + (Area -1) ) * 32 ==0xb040

After the begin Address+offset,we read font chip for 2 bytes,
then get data of the word ”啊“:

0000b040 00 04 2f 7e f9 04 a9 04 aa 14 aa 7c ac 54 aa 54

We now can get a conclusion ,if we get the correct gb2312 encoding result ,we can calculate the Address of each chinese word in font chip

stategrid
Posts: 5
Joined: Fri Feb 07, 2020 6:57 am

Re: Need Advise,micropython gb2312 encoding bug? Thank you

Post by stategrid » Wed Feb 26, 2020 5:30 pm

Micropython not support gb2312 encoding now,yet python do。
Maybe I get a solution, step 1,encoding a chinese word at unicode;step 2,find gb2312 encoding from a list;step 3,read font chip and display on OLED。

find gb2312 encoding just by unicode2gb list,each list is a dictionary。

Code: Select all

#unicode_gb
#CODE_TABLE_SIZE 6768
#unicode, gb2312
#origin by  lyaohe,https://github.com/lyaohe/UTF-8toGB2312
#part of list,full list can acess form above,an modified

unicode2gb = [
{0x4E00,0xD2BB},
{0x4E01,0xB6A1},
{0x4E03,0xC6DF},
{0x4E07,0xCDF2},
{0x4E08,0xD5C9},
{0x4E09,0xC8FD},
{0x4E0A,0xC9CF},
{0x4E0B,0xCFC2},
{0x4E0C,0xD8A2},
{0x4E0D,0xB2BB},
{0x4E0E,0xD3EB},
{0x4E10,0xD8A4},
{0x4E11,0xB3F3},
{0x4E13,0xD7A8},
{0x4E14,0xC7D2},
{0x4E15,0xD8A7},
{0x4E16,0xCAC0},
{0x4E18,0xC7F0},
{0x4E19,0xB1FB},
{0x4E1A,0xD2B5},
{0x4E1B,0xB4D4},
{0x4E1C,0xB6AB},
{0x4E1D,0xCBBF},
{0x4E1E,0xD8A9},
{0x4E22,0xB6AA},
{0x4E24,0xC1BD},
{0x4E25,0xD1CF},
{0x4E27,0xC9A5},
{0x4E28,0xD8AD},
{0x4E2A,0xB8F6},
{0x4E2B,0xD1BE},
{0x4E2C,0xE3DC},
{0x4E2D,0xD6D0},
{0x4E30,0xB7E1},
{0x4E32,0xB4AE},
{0x4E34,0xC1D9},
{0x4E36,0xD8BC},
{0x4E38,0xCDE8},
{0x4E39,0xB5A4},
{0x4E3A,0xCEAA},
{0x4E3B,0xD6F7},
{0x4E3D,0xC0F6},
{0x4E3E,0xBED9},
{0x4E3F,0xD8AF},
{0x4E43,0xC4CB},
{0x4E45,0xBEC3},
{0x4E47,0xD8B1},
{0x4E48,0xC3B4},
{0x4E49,0xD2E5},
{0x4E4B,0xD6AE},
{0x4E4C,0xCEDA},
{0x4E4D,0xD5A7},
{0x4E4E,0xBAF5},
{0x4E4F,0xB7A6},
{0x4E50,0xC0D6},
{0x4E52,0xC6B9},
{0x4E53,0xC5D2},
{0x4E54,0xC7C7},
{0x4E56,0xB9D4},
{0x4E58,0xB3CB},
{0x4E59,0xD2D2},
{0x4E5C,0xD8BF},
{0x4E5D,0xBEC5},
{0x4E5E,0xC6F2},
{0x4E5F,0xD2B2},
{0x4E60,0xCFB0},
{0x4E61,0xCFE7},
{0x4E66,0xCAE9},
{0x4E69,0xD8C0},
{0x4E70,0xC2F2},
{0x4E71,0xC2D2},
{0x4E73,0xC8E9},
{0x4E7E,0xC7AC},
{0x4E86,0xC1CB},
{0x4E88,0xD3E8},
{0x4E89,0xD5F9},
{0x4E8B,0xCAC2},
{0x4E8C,0xB6FE},
{0x4E8D,0xD8A1},
{0x4E8E,0xD3DA},
{0x4E8F,0xBFF7},
{0x4E91,0xD4C6},
{0x4E92,0xBBA5},
{0x4E93,0xD8C1},
{0x4E94,0xCEE5},
{0x4E95,0xBEAE},
{0x4E98,0xD8A8},
{0x4E9A,0xD1C7},
{0x4E9B,0xD0A9},
{0x4E9F,0xD8BD},
{0x4EA0,0xD9EF},
{0x4EA1,0xCDF6},
{0x4EA2,0xBFBA},
{0x4EA4,0xBDBB},
{0x4EA5,0xBAA5},
{0x4EA6,0xD2E0},
{0x4EA7,0xB2FA},
{0x4EA8,0xBAE0},
{0x4EA9,0xC4B6},
{0x4EAB,0xCFED},
{0x4EAC,0xBEA9},
{0x4EAD,0xCDA4},
{0x4EAE,0xC1C1},
{0x4EB2,0xC7D7},
{0x4EB3,0xD9F1},
{0x4EB5,0xD9F4},
{0x4EBA,0xC8CB},
{0x4EBB,0xD8E9},
{0x4EBF,0xD2DA},
{0x4EC0,0xCAB2},
{0x4EC1,0xC8CA},
{0x4EC2,0xD8EC},
{0x4EC3,0xD8EA},
{0x4EC4,0xD8C6},
{0x4EC5,0xBDF6},
{0x4EC6,0xC6CD},
{0x4EC7,0xB3F0},
{0x4EC9,0xD8EB},
{0x4ECA,0xBDF1},
{0x4ECB,0xBDE9},
{0x4ECD,0xC8D4},
{0x4ECE,0xB4D3},
{0x4ED1,0xC2D8},
{0x4ED3,0xB2D6},
{0x4ED4,0xD7D0},
{0x4ED5,0xCACB},
{0x4ED6,0xCBFB},
{0x4ED7,0xD5CC},
{0x4ED8,0xB8B6},
{0x4ED9,0xCFC9}
]

Post Reply