[uClinux-dev] Excluding one of n Cores

[Michael Schnell]

Hi Experts.

Is there a kind of "official" way to set aside one of the available
cores in an SMP system from the Linux OS to do deeply embedded
extremely-low-latency stuff in a kind of single task "main loop" type
environment ? I.e. creating a true coprocessor from an SMP hardware.

Some of the problems that come in ind here include:

- how to make the Linux initialization ignore one of the available
cores or free a core later on ?
- how to have a Linux task start the free running main loop ?
- how to assign certain interrupts to that core and have ISRs run
there only dedicatedly interrupting the "main loop" and not ever being
blocked by any Linux activity ?
- what about MMU issues ?

For example I (e.g.) would like a (now rather cheap) standard quadcore
ARM Cortex A9 processor chip and modify a Debian distribution in a way
that support this stuff.

Thanks,
-Michael

[Lennart Sorensen]

Post by Michael Schnell
Hi Experts.
Is there a kind of "official" way to set aside one of the available
cores in an SMP system from the Linux OS to do deeply embedded
extremely-low-latency stuff in a kind of single task "main loop"
type environment ? I.e. creating a true coprocessor from an SMP
hardware.
- how to make the Linux initialization ignore one of the available
cores or free a core later on ?
- how to have a Linux task start the free running main loop ?
- how to assign certain interrupts to that core and have ISRs run
there only dedicatedly interrupting the "main loop" and not ever
being blocked by any Linux activity ?
- what about MMU issues ?
For example I (e.g.) would like a (now rather cheap) standard
quadcore ARM Cortex A9 processor chip and modify a Debian
distribution in a way that support this stuff.

Something like this:

http://lwn.net/Articles/464391/

[Michael Schnell]

Post by Lennart Sorensen
http://lwn.net/Articles/464391/

While this is really interesting stuff and closely related (defining a
standard protocol between Linux and coprocessors), I understand that
this article is about using coprocessors that are dedicated by hardware
(e.g. because they don't have an MMU and/or use a different architecture
than the main Linux processors).

My question was about how to tell Linux that it should leave alone one
of the processors, it _could_ use for SMP, and on top of that start a
non-Linux-enabled "very hard realtime" program on same, that of course
needs hardware interrupts. On top of this, the article will help to
implement communication between Linux and this processor/program.

Thanks,
-Michael

[Lennart Sorensen]

Post by Michael Schnell
While this is really interesting stuff and closely related (defining
a standard protocol between Linux and coprocessors), I understand
that this article is about using coprocessors that are dedicated by
hardware (e.g. because they don't have an MMU and/or use a different
architecture than the main Linux processors).
My question was about how to tell Linux that it should leave alone
one of the processors, it _could_ use for SMP, and on top of that
start a non-Linux-enabled "very hard realtime" program on same, that
of course needs hardware interrupts. On top of this, the article
will help to implement communication between Linux and this
processor/program.

I thought it talked about AMP in the case where you did have an SMP
system but wanted to use some cores for other stuff.

Maybe I didn't read enough of it.

[Paul Chavent]

Post by Michael Schnell
Hi Experts.

I'm not an expert. But i give you my ("standard" linux kernel) point of view (don't know if it works with uClinux).

Post by Michael Schnell
Is there a kind of "official" way to set aside one of the available cores in an SMP system from the Linux OS to do deeply embedded extremely-low-latency stuff in a kind of single task "main loop" type environment ? I.e. creating a true coprocessor from an SMP hardware.
- how to make the Linux initialization ignore one of the available cores or free a core later on ?

See the isolcpu kernel parameter.

Post by Michael Schnell
- how to have a Linux task start the free running main loop ?
- how to assign certain interrupts to that core and have ISRs run there only dedicatedly interrupting the "main loop" and not ever being blocked by any Linux activity ?

Something like interrupt balancing ?
http://subversion.ffado.org/wiki/IrqPriorities

I got this link from the https://rt.wiki.kernel.org/index.php/RT:Tips page.

Post by Michael Schnell
- what about MMU issues ?

I never tried to enable mmu with uclinux. But on a "standard" linux i use mlock to deal with mmu, caches etc.

Post by Michael Schnell
For example I (e.g.) would like a (now rather cheap) standard quadcore ARM Cortex A9 processor chip and modify a Debian distribution in a way that support this stuff.
Thanks,
-Michael
_______________________________________________
uClinux-dev mailing list
uClinux-dev at uclinux.org
http://mailman.uclinux.org/mailman/listinfo/uclinux-dev
This message was resent by uclinux-dev at uclinux.org
http://mailman.uclinux.org/mailman/options/uclinux-dev

[Michael Schnell]

Post by Paul Chavent
I'm not an expert. But i give you my ("standard" linux kernel) point
of view (don't know if it works with uClinux).

GREAT ! As I have Debian or something similar in mind, this might not
really be "uCLinux", anyway. :-)
In fact, is "uClinux" still defined as noMMU ? I understand that
regarding the great success of ARM Cortex A8+ chips noMMU is more and
more obsolete an "uClinux" might get a synonym for "small embedded",
with "small" now going up to "1GHz 1GByte".

Post by Paul Chavent
See the isolcpu kernel parameter.

I'll take a look.

Post by Paul Chavent
Something like interrupt balancing ?

Not really.
I am not sure how SMP hardware usually is done. I understand that when a
hardware interrupt happens, exactly one of the cores gets the interrupt
and starts the ISR. But which one ? Is the assignment "Interrupting
hardware -> Core Number" fix ? Is it programmable at system start ? Is
it done dynamically e.g. according to the performance done by the
Kernels (i.e. a rather idle Kernel gets interrupted). Is there hardware
support for this ?

When doing AMP with such SMP-enable hardware, some interrupts need to be
fixed assigned to the "special purpose" core to trigger the appropriate
low latency actions there.

Post by Paul Chavent
I never tried to enable mmu with uclinux. But on a "standard" linux i
use mlock to deal with mmu, caches etc.

I see.
(I just don't know where to ask embedded "non uCLinux" questions, so I
thought here would be the best place for strictly embedded Linux
questions, even with MMU involved.)

Thanks a lot !
-Michael

[Geert Uytterhoeven]

Post by Michael Schnell
(I just don't know where to ask embedded "non uCLinux" questions, so I
thought here would be the best place for strictly embedded Linux questions,
even with MMU involved.)

linux-embedded at vger.kernel.org

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert at linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

[Michael Schnell]

Post by Geert Uytterhoeven
linux-embedded at vger.kernel.org

Hi Geert,

Thanks a lot for the pointer.

Astonishingly that list a a lot less busy than this here :-(

Just to let you (and others) know:

We found out that the idea I had (dedicating one of multiple CPUs in a
chip for AMP-non-Linux use, while the other run SMP Linux) seems to be
viable and doable, but seemingly not very widely supported by the
Kernel developers.

You can easily exclude a dedicated CPU from SMP, but seemingly the
scheduler might need some tweaking to prevent performance degrading. (I
failed to understand why this should be the case.)

In fact dedicated AMP even is fully supported by ARM Cortex A9 hardware:
- The interrupt sources seemingly can be dynamically assigned to any
core. So setting aside a core for AMP with dedicated interrupts for
embedded Hardware should be no problem.
- Each CPU features an "AMP" bit in a configuration register. With
that the 1st level cache synchronization is switched off.

The cache synchronization would be the major latency risk imposed by the
other processors to the dedicated AMP CPU. By switching it off, you can
calculate the max latency by just considering the bus scheduling latency
(besides the "local" issues). For communication between the Linux- and
the AMP- systems you would need to use a non-cached memory region
similar to DMA and of course Linux can't modify the other AMP's memory
regions on the fly.

Thanks again,
-Michael

[Michael Schnell]

I found that this

https://access.redhat.com/site/solutions/15482

answers a lot of questions on that behalf.

It's about Readhat. Maybe something similar is available for Debian, too...

-Michael