-
Notifications
You must be signed in to change notification settings - Fork 651
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Interrupt Priority for the Scheduler #3576
base: master
Are you sure you want to change the base?
Conversation
358c943
to
42b68ad
Compare
// Licensed under the Apache License, Version 2.0 or the MIT License. | ||
// SPDX-License-Identifier: Apache-2.0 OR MIT | ||
// Licensed under the Apache License, Version 2.0[0] or the MIT License. | ||
// SPDX-License-Identifier: Apache-2.0[0] OR MIT |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Find and replace gone wrong?
I know this was a quick PR for tockworld to illustrate, but at some point I at least need more description and comments to understand this change. |
I added a detailed description. |
Kind reminder to review and discuss this 😄 |
If I'm reading this correctly this PR would allow the scheduler to learn which interrupts are pending, and specify which should run. What I don't understand is how is the scheduler going to know what interrupt, say, 24 is, and whether it should request that the chip service interrupt 24 instead of, say, 13? I'm guessing that with this PR there would be a very custom scheduler that would know the meanings of each interrupt number. The tension appears to be that deciding how interrupts should be prioritized is (or may be) a board-level/application-level decision, yet the code for deciding how to prioritize interrupts lives in the very shared arch/ crates. Right now Tock doesn't specify how chips prioritize interrupts, it just so happens that (I believe) all of our arch/ implementations prioritize the lowest number. I see two or three options (assuming I'm understanding the problem correctly):
I don't think this has anything to do with scheduling, unless the scheduler needs to be able to say "if pending interrupt is 24, run apps, but if it is 13, service the interrupt". However, I don't think that is the case. |
NB: #1181 |
I think it is still a problem of the scheduler. Here are some examples:
The scheduler would be written in the board crate, so it would know the hardware that it runs on. I think options 2 and 3 proposed by @bradjc are worth exploring. |
This is an interesting use case that would directly affect the scheduler. There are two pieces:
It seems like the design question is around how general of a mechanism do we want? A rough sketch, from specific to general:
This should happen if the app subscribes to that interrupt, right? |
Even with this change, Tock cannot guarantee interrupt latencies because the kernel is non-preemptible: if the kernel thread is already executing some longer-running task when a CAN interrupt arrives, the CAN interrupt will not be serviced until the kernel task finishes. I understand that this PR would improve the status quo of interrupt latencies by a lot, but I do wonder how many real systems that require short interrupt handling latencies can actually be ok with that limitation. @alexandruradovici do you anticipate that the systems you care about are ok with that limitation, or are you looking at that as "the next problem to solve"? I am trying to figure out whether this is the first step down a path that will ultimately require a preemptible kernel to be useful, because if so I think we need to have that conversation first. |
Pull Request Overview
This pull request adds awareness about interrupt priority to the
Chip
trait to be used by the scheduler.The issue
The current implementation of the does not take into account the semantic meaning of the interrupts. For the scheduler trait, interrupts and deferred tasks are grouped into
kernel work
. The scheduler can only decide to instruct the kernel to perform kernel work, but is not able to instruct it about the order in which this work needs to be performed. This behavior leads to potential interrupt starvation.For example, on the STM32 boards, take interrupts:
As can be observed, all DMA interrupts have numbers lower than the CAN interrupts. The STM32F4 users DMA to read and write to the serial port. Due to the way in which the scheduler handles kernel work all together, as long as there is UART traffic, CAN interrupt handling will be delayed. The kernel referrers to the
cortex-m
crate for interrupt handling, which always handles the lower interrupts first due to:The CAN interrupt handling works great, as long as there are no
debug!
messages and no application is writing to the console. As soon as there is any reading or writing to the serial port, CAN interrupts are delayed a few milliseconds, enough to drop packets.Proposed solution
This PR introduces the concept of interrupts to the scheduler.
As interrupts can be represented in different on different platforms, we add an
Interrupts
trait to theChip
. This trait represents an interrupt set. The actual implementation of the set is left to thearch
crate.The
Interrupts
trait is added to theChip
trait as an associated type. Theservice_pending_interrupts
receives an interrupt set that it should handle and thehas_pending_interrupts
returns an interrupt set.The
For the
cortex-m
crate, this is implemented as a bit map composed out of a tuple([2; u128])
, as the cortex-m defines a maximum of 240 interreupts.The same idea applies to the deferred tasks, except that the task set can be represented as a
u32
, as there are a maximum of 32 deferred tasks.The default implementation of the
Scheduler
trait changes for theunsafe
functions:This allows the scheduler implementation to inform the
arch
crate about the order in which to handle the interrupts. Thearch
crate can still execute them in any arbitrary order, just that it is constrained to a masked set. To handle interrupts in a specific order, scheduler can than issue severalservice_pending_interrupts
calls with a set of one single interrupt.The same applies to deferred tasks.
This allows the scheduler to instruct the kernel about which tasks to handle as
kernel work
.Testing Strategy
N/A
TODO or Help Wanted
Feedback
Documentation Updated
/docs
, or no updates are required.Formatting
make prepush
.