Need help, mp_lexer_new_from_str_len returning null

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
cduran
Posts: 80
Joined: Thu Mar 17, 2016 4:52 pm

Need help, mp_lexer_new_from_str_len returning null

Post by cduran » Fri Apr 28, 2017 11:03 pm

I'm trying to compile and run a script stored in flash. I'm not using a file system, the script is just stored as raw ascii. The problem I'm having is that mp_lexer_new_from_str_len() is returning NULL. Seems to be having trouble allocating memory for something.

There are many other things going on in my firmware and for me to narrow down whats causing it, I need to know what conditions would cause a lexer creation to fail so I know where to start looking in my firmware.

Thanks in advance.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Need help, mp_lexer_new_from_str_len returning null

Post by dhylands » Sat Apr 29, 2017 12:10 am

It looks like mp_lexer_new_from_str_len should never return NULL. If it runs out of memory, it should raise a MemoryError exception.

cduran
Posts: 80
Joined: Thu Mar 17, 2016 4:52 pm

Re: Need help, mp_lexer_new_from_str_len returning null

Post by cduran » Wed May 10, 2017 11:04 pm

I'm still having problems with this. I'm using the following code, which is used in the Minimal build as well as other builds.

Code: Select all

mp_lexer_t *lex = mp_lexer_new_from_str_len(MP_QSTR__lt_stdin_gt_, src, strlen(src), 0);
Seems to be almost at random. The syntax of the script is correct and there is no way I could be having a memory problem, my heap is set to 16kb.

There are two def's. The first one call() is just a wrapper for a C function. The second one just calls the call function to do different things, sometimes it returns an int other times it returns a list.

Code: Select all

import c_sample
def call(arg0,arg1,arg2,arg3,arg4):
 return c_sample.call_callback(arg0, arg1, [arg2,arg3,arg4])

def test():
 t=call(0x23,0x04,0x16,0x00,0x00)
 if len(t) < 1
  return
 t[10]=0x06
 t[14]=0x63
 call(0x20,t,0,0,0)
 return
test()
But like I said, my error is coming from the lexer, when it does execute it runs fine. My mp engine is part of my firmware and running as a separate function.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Need help, mp_lexer_new_from_str_len returning null

Post by dhylands » Thu May 11, 2017 12:11 am

Is src a null-terminated string? strlen requires it.

cduran
Posts: 80
Joined: Thu Mar 17, 2016 4:52 pm

Re: Need help, mp_lexer_new_from_str_len returning null

Post by cduran » Thu May 11, 2017 12:33 pm

dhylands wrote:Is src a null-terminated string? strlen requires it.
Yes, for testing I'm passing it a constant string like this:

"def test():\n t=call(0x23,0x04,0x16,0x00,0x00)\n if len(t) < 1\n return\n t[10]=0x06\n t[14]=0x63\n call(0x20,t,0,0,0)\n return\ntest()"

The call() function never gives an error, that one is compiled as right after mp is initialized. The test() function is compiled and executed whenever an event happens in my firmware (no interrupts are used for the event). Sometimes the script runs without problems a handful of times, but then starts giving a lexer error.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Need help, mp_lexer_new_from_str_len returning null

Post by dhylands » Thu May 11, 2017 5:45 pm

What error?
Did you setup your heap?
Are you running out of memory?

cduran
Posts: 80
Joined: Thu Mar 17, 2016 4:52 pm

Re: Need help, mp_lexer_new_from_str_len returning null

Post by cduran » Mon May 15, 2017 2:58 pm

- No error codes per se, just returns NULL.
- My heap is currently set at 16kb:
static char heap[16384]

I can't imagine that with a heap that large and a script that small that I would be running out of memory.
dhylands wrote:What error?
Did you setup your heap?
Are you running out of memory?

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Need help, mp_lexer_new_from_str_len returning null

Post by dhylands » Mon May 15, 2017 4:51 pm

You'll probably need to sprinkle some prints through the code then, because I don't see anyway that mp_lexer_new_from_str_len returns NULL except via a failure to allocate from the heap.

cduran
Posts: 80
Joined: Thu Mar 17, 2016 4:52 pm

Re: Need help, mp_lexer_new_from_str_len returning null

Post by cduran » Tue May 16, 2017 3:38 pm

Actually, I forgot to mention that the heap is NOT declared in main(). Neither is the stack. My init function, heap, stack and gc callbacks are all in a separate file. Would this cause any issues?

Code: Select all

static volatile char *stack_top;
static char heap[16384] = {0xff};

void mp_initialize(void)
{
	mp_stack_set_top(stack_top);
	gc_init(heap, heap + sizeof(heap));
	mp_init();
}

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Need help, mp_lexer_new_from_str_len returning null

Post by dhylands » Tue May 16, 2017 4:17 pm

I wouldn't expect that to cause any problems related to what you're seeing, provided your initialize function is being called.

Post Reply