Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Common hardware to run benchmarks #671

Open
rfuest opened this issue Apr 16, 2022 · 13 comments
Open

Common hardware to run benchmarks #671

rfuest opened this issue Apr 16, 2022 · 13 comments

Comments

@rfuest
Copy link
Member

rfuest commented Apr 16, 2022

Up until now most of the testing and benchmarking has been done on the development machine. Each time I've tried to use e-g on real hardware I quickly discovered problems with performance and/or code size, which weren't apparent during development. The next revision of e-g will need some changes to the DrawTarget trait and to get this right it is IMO necessary to do some testing on real hardware.

I think it would be a good idea to define a set of target hardware configurations we can use for testing, but which could later also used for hardware examples and tutorials. These could range from a Blue Pill and a display module on a breadboard to some evaluation board.

To make this useful the core team should have a copy of each of the selected configurations available to test code and help users if we later provide examples/tutorials. I would like to have a few test hardware configurations which are always ready to quickly test some code and which don't require fiddling with breadboards before I can do the actual work.

Here is a short list of possible ideas of target architectures/configurations:

  • STM32F0 (Cortex M0)
    This is about the weakest µC family I would use for e-g. I did recently use a STM32F0DISCOVERY with 64K flash and 8K RAM for some testing with a SPI 320x240 pixel RGB565 display. With the 48 MHz clock and the limited memory this was an interesting platform to test a lower end configuration. A more realistic paring would be monochrome OLED like a SSD1306 based one.
  • Raspberry PI Pico (RP2040, Cortex M0+)
    Much faster clock rate than the STM32F0 and a lot more RAM makes this a better platform for higher resolution color display. As part of the Raspberry PI family there are also compatible displays available. And as an additional bonus you can actually buy it without having to worry about fakes, as with the Blue Pill.
  • Wio Terminal (ATSAMD51P19, Cortex M4F)
    I haven't used this, but at first glance it looks like a nice self contained unit for a reasonable price. With the IMU and butons it would make an interesting platform for GUI tests. Unfortunately it requires some SMD soldering to access the SWD port, which makes it less useful for uses how are only interested in testing software.
  • Sipeed Longan Nano (GD32VF103, RISC-V)
    For all the RISC-V fanboys. Affordable and has an on board connector for a 160x80 color LCD.
  • Some ESP-32
    I haven't yet tried to use an ESP-32 with Rust, but the support seems to come along nicely. ESP32 modules are readily available and I'm pretty sure there will also be some display options (haven't looked into this).
  • STM32F7 or STM32H7 development board
    The other end of the performance spectrum compared to the STM32F0. A higher end development board with 2D hardware acceleration and a parallel or MIPI DSI TFT would be a good platform to test advanced features like alpha blending.
  • ...
    Any other suggestions are welcome.

On the display side it might be interesting to also test less common display types like 4 BPP OLED displays (e.g. SSD1327 based) or 3 color EPDs. While these are less commonly used, testing on these displays would make sure they are actually working and we don't make any assumptions in our code that breaks these types of display.

But unfortunately I don't have unlimited funds to buy development hardware for all these configurations. A STM32F7/STM32H7 development board with a big display would be one of the more useful items IMO, but also the most expensive. Perhaps we can pick a few options, which are targets that are actually used by e-g users.

I hope this rambling made any sense.

@jamwaffles
Copy link
Member

Amazingly (or perhaps not), my hoarding tendencies over the last few years mean I actually have quite a lot of this hardware.

I have two questions on MCU selections:

  1. Some ESP-32

    Do you mean the Xtensa architecture, or the newer ones using RISC-V? I think I'd prefer to use the newer ESP-32s as a RISC-V target instead of the GD32 unless you had a specific reason to use the latter in mind.

  2. STM32F7 or STM32H7

    Are there any major/relevant differences between the two or do you have a preference?

I'd be happy to leverage my Github Sponsors contributions to purchase some test hardware to fill any gaps in our collections (within reason; please don't invoice me for 8000€ :D) so if we can come up with a concrete list of dev boards and displays to get hold of, I can look at how much it all costs.

As for the Wio Terminal, we may want to look at the Feather/Metro/Gemma/QTPy/Trinket M4 instead, although we would have to plug an external display into it.

I like your display recommendations. It's certainly worth testing as many edge cases as we can. If we don't someone will eventually find issues with them anyway. Do you have a more complete list of target displays in mind?

An ideal solution would be a custom dev board with all this hardware soldered to it. That sounds like a fun project, but a time/money expensive one...

I hope this rambling made any sense.

This was eloquent rambling, so yes it did ;)

@jamwaffles
Copy link
Member

As for performance, I remember hearing some murmurs from someone that the Knurling project is looking into being able to run some kind of cargo bench on target hardware, which would be a fantastic addition to the e-g test suite.

@rfuest
Copy link
Member Author

rfuest commented Apr 17, 2022

Some ESP-32

Do you mean the Xtensa architecture, or the newer ones using RISC-V? I think I'd prefer to use the newer ESP-32s as a RISC-V target instead of the GD32 unless you had a specific reason to use the latter in mind.

I didn't specify this intentionally, because I haven't looked at ESP-32 and Rust in detail. While I also prefer RISC-V over Xtensa I'm not sure if the availability for Xtensa based modules is better.

STM32F7 or STM32H7

Are there any major/relevant differences between the two or do you have a preference?

I don't have a preference. The H7 comes in more powerful variants, with up to 550 MHz and also in dual core variants.

I'd be happy to leverage my Github Sponsors contributions to purchase some test hardware to fill any gaps in our collections (within reason; please don't invoice me for 8000€ :D) so if we can come up with a concrete list of dev boards and displays to get hold of, I can look at how much it all costs.

That would be awesome.

As for the Wio Terminal, we may want to look at the Feather/Metro/Gemma/QTPy/Trinket M4 instead, although we would have to plug an external display into it.

I'll take a look at these.

I like your display recommendations. It's certainly worth testing as many edge cases as we can. If we don't someone will eventually find issues with them anyway. Do you have a more complete list of target displays in mind?

An ideal solution would be a custom dev board with all this hardware soldered to it. That sounds like a fun project, but a time/money expensive one...

Maybe we could start with designing a simple base board that connects a Raspberry Pi Pico to some common Chinese display modules. I would use modules instead of designing a PCB for raw LCDs/OLEDs to make it easier to design and solder.

These displays should cover a wide array of color types.

  1. SSD1306 OLED (1 BPP)
  2. SSD1327 OLED (4 BPP)
  3. ILI9341 LCD (16 BPP or 18 BPP)
  4. A three color EPD

And maybe add a few buttons and a rotary encoder for some interactive examples.

As for performance, I remember hearing some murmurs from someone that the Knurling project is looking into being able to run some kind of cargo bench on target hardware, which would be a fantastic addition to the e-g test suite.

That would be really useful. While I tested the new color conversions I simply started a timer at the beginning of the test and printed the count via RTT after the test was finished. Even this very basic test setup made it really easy to test the performance.

@jamwaffles
Copy link
Member

I didn't specify this intentionally, because I haven't looked at ESP-32 and Rust in detail. While I also prefer RISC-V over Xtensa I'm not sure if the availability for Xtensa based modules is better.

The ESP 32-C3 (C3 meaning RISC-V core) seems readily available on Aliexpress. Purely by coincidence I ordered this one a few weeks ago. I'm honestly not that concerned with supporting Xtensa as 1. Espressif seem to be moving to RISC-V anyway and 2. Xtensa requires a much more involved custom toolchain than the RISC-V chips currently do. Shall we narrow the specification of "Some ESP chip" to an ESP-32-C3?

I don't have a preference. The H7 comes in more powerful variants, with up to 550 MHz and also in dual core variants.

The F7 development boards with builtin display are, from a cursory search on Farnell, significantly cheaper than the H7 stuff so I think the F series is a better way to go, purely to keep costs down. I think we can get a pretty good idea of how performance scales without dropping £400 on a dev board ;)

That would be awesome.

I'm happy to take this private in an Element DM if you prefer, but how would you like to sort this out? I'm happy to either order and send some hardware, or send funds if that makes life easier for you.

Maybe we could start with designing a simple base board that connects a Raspberry Pi Pico to some common Chinese display modules.

How about making something that's compatible with Arduino shields? It would hold the 4 different displays, some sort of mux between them, and the arduino pinout would mean it plugs straight into the inexpensive (and available!) STM32 Nucleo dev boards. It would also be quite simple to make adapters for e.g. the rPi Pico or a Feather form factor, etc. There's also an ATSAMD21 (or 51?) Arduino.

And yes, I very much agree it should use the readily available Chinese modules. I don't feel like hand soldering flat flex much.

These displays should cover a wide array of color types.

Sounds good to me. Keeping the list of displays small is a good idea as the test matrix is already quite large considering the number of different CPUs we want to test on.


Now that we have the list of displays in order, I'd like to start nailing down a list of actual boards to plug them into. At least right now, I'm worried that availability issues will mean an unobtanium custom board.

I think targetting the Arduino pinout is a good idea; the ST Nucleo dev boards all have it present, as does the ATSAMD Arduino (obviously). They're also easy enough to make adapters for. I'm happy to explore other solutions of course, so consider this as throwing my hat into the ring and nothing more.

@rfuest
Copy link
Member Author

rfuest commented Apr 20, 2022

Shall we narrow the specification of "Some ESP chip" to an ESP-32-C3?

SGTM.

The F7 development boards with builtin display are, from a cursory search on Farnell, significantly cheaper than the H7 stuff so I think the F series is a better way to go, purely to keep costs down. I think we can get a pretty good idea of how performance scales without dropping £400 on a dev board ;)

£400 would be much more than I expected and I would feel bad to take so much of your well earned sponsorship money. There are also cheaper H7 boards, still not in the china display module price range, but not to far off from the F7 counterparts.

This would be a high end option with capacitive touch and 800x480 MIPI DSI display:
https://www.mouser.de/ProductDetail/STMicroelectronics/STM32H747I-DISCO for 91.81€ + VAT.
A similar F7 model also exists 32F769IDISCOVERY and is ~10€ cheaper, but seems to be out of stock everywhere: https://www.mouser.de/ProductDetail/STMicroelectronics/STM32F769I-DISCO

The STM32F7508-DK might be a good lower cost option at 51.10€ + VAT. In this case the H7 variant is significantly more expensive and the F7 model is also in stock.

This ST document has a good list of all available discovery kits if you want to take another look.
https://www.st.com/content/ccc/resource/sales_and_marketing/presentation/product_presentation/group0/fe/b7/f3/0f/2f/f5/4a/60/stm32_eval-tools_portfolio/files/stm32_eval-tools_portfolio.pdf/jcr:content/translations/en.stm32_eval-tools_portfolio.pdf

I'm happy to take this private in an Element DM if you prefer, but how would you like to sort this out? I'm happy to either order and send some hardware, or send funds if that makes life easier for you.

It might be easier if I order the hardware myself to avoid problems with customs, since the UK isn't part of the EU anymore 😞. We can discuss the details in DM, but I would suggest that I pay the items myself at first and you refund me later. This way you can be sure that I actually bought the hardware and didn't use the money for something else 🍺 🍕.

How about making something that's compatible with Arduino shields? It would hold the 4 different displays, some sort of mux between them, and the arduino pinout would mean it plugs straight into the inexpensive (and available!) STM32 Nucleo dev boards. It would also be quite simple to make adapters for e.g. the rPi Pico or a Feather form factor, etc. There's also an ATSAMD21 (or 51?) Arduino.

That sounds like a good idea. I always forget about the Arduino stuff, because I haven't used it a lot. But even the higher end boards, which I mentioned above, have an Arduino Uno compatible header.

@jamwaffles
Copy link
Member

The STM32F7508-DK might be a good lower cost option at 51.10€ + VAT. In this case the H7 variant is significantly more expensive and the F7 model is also in stock.

Sounds like a more reasonable price. Shall we go with this one? There's also a Rust HAL for it so we should be good there.

It might be easier if I order the hardware myself to avoid problems with customs, since the UK isn't part of the EU anymore 😞. We can discuss the details in DM, but I would suggest that I pay the items myself at first and you refund me later.

Yes, that's probably the best way to go about this what with all the silly import fees, etc 🙄

Can you create a Mouser (or other) shopping list and post it here/in a DM? I can use that to build up a list of what I need to get, as well as work out how much to pay you once you order on your side.

You may have seen I just pushed the hardware-bench repo. I am by no means an EE, but it's a start on an arduino-like shield to mount a pile of displays on.

@rfuest
Copy link
Member Author

rfuest commented Apr 21, 2022

... Shall we go with this one? ...

I did take another look at it and noticed one issue. The controller itself only has 64 kB of flash. This means that we need to deal with an external flash memory for all but the simplest tests.

Under normal circumstances the 32F746GDISCOVERY would be a equally priced better alternative with 1 MB flash in the µC. But because buying STM32 related stuff is so much fun at the moments it is obviously not in stock anywhere.

There are the options I would consider viable at the moment:

I would prefer to not use the STM32F7508-DK, because IMO the hassle of having to deal with external memory (setting up XIP, linker settings, using a flash loader to program the external flash...) from the beginning isn't worth the saved 20€. The STM32H7 also seems to have an actively developed HAL.

The STM32F723E-DISO would be an option if we want to save some money now and upgrade later, when the availability of devboards has improved. It lacks the hardware accelerators, but with the fast parallel display interface it still has very different characteristics from the slow SPI/I2C interfaces we otherwise use.

Can you create a Mouser (or other) shopping list and post it here/in a DM? I can use that to build up a list of what I need to get, as well as work out how much to pay you once you order on your side.

I'm not really sure what to order at the moment. But as soon as we have decided what we want I can create a list.

You may have seen I just pushed the hardware-bench repo. I am by no means an EE, but it's a start on an arduino-like shield to mount a pile of displays on.

Great, let me know if I can help you in any way. It would be good if we can decide on which exact display modules we want to use soon, because I want to order them from AliExpress to keep the cost down and they will take a while to arrive.

@jamwaffles
Copy link
Member

STM32H7B3I-DK looks good as it has some more graphics hardware which might help guide some e-g development. Let's go with that and I'll reimburse you :). I think it makes sense to invest a little bit now.

It would be good if we can decide on which exact display modules we want to use soon, because I want to order them from AliExpress to keep the cost down and they will take a while to arrive.

Sorry if I wasn't clear earlier in the thread; I think your list of 4 displays is a good one:

  • SSD1306 OLED (1 BPP)
  • SSD1327 OLED (4 BPP)
  • ILI9341 LCD (16 BPP or 18 BPP)
  • A three color EPD

As for specifics (as close to those as you can get on Aliexpress...):

  • SSD1306: Any I2C 0.96" OLED module, e.g. https://www.aliexpress.com/item/32638662748.html (any colour)
  • SSD1327: I don't have much experience with these modules but this looks like a common form factor for the modules. Shall we go with the I2C one? We can jumper between this and the SSD1306.
  • ILI9341: this module is the most common form factor and supports SPI.
  • EPD: The 2.13" and 2.9" modules here look like a common form factor. The connectors appear the same, so making a layout to accept both sizes with two sets of mounting holes would be easy enough.

The only real requirement I have is that they have through hole pins so they can be either socketed or easily soldered on to the carrier board.

The links I posted aren't from any particular store. It's probably a good idea to find a store that stocks all the above kinds. If you do and make an order, please send me links so I can order the same modules :)

@rfuest
Copy link
Member Author

rfuest commented Apr 21, 2022

Sorry if I wasn't clear earlier in the thread;...

No you were clear, but I wasn't. I only meant that we need to agree on form factor, sizes and interfaces.

But you already provided a nice list with all that information, thanks.

SSD1327: ... Shall we go with the I2C one? We can jumper between this and the SSD1306.

OK. Some modules also have an address selection resistor. The most flexible solution would be to have two jumpers per display, to either connect one of the displays or connect both displays, which would require one to use another address.

ILI9341: this module is the most common form factor and supports SPI.

They are available in different size. I think 2.8" is the largest in that form factor. Should we use the version with or without touch?

The only real requirement I have is that they have through hole pins so they can be either socketed or easily soldered on to the carrier board.

Do we want to add mounting holes for the display or rely on the socket friction? For the larger displays it would be beneficial to have some additional mechanical support. The ILI9431 displays have an additional unpopulated header on the other side of the PCB, which gives them a lot more stability and standoffs aren't required IMO.

I'm not sure if any additional mounting is required for the smaller displays. If they are soldered in they won't move anyway, but even with a socket they should be reasonably stable. Getting the mounting holes correct might also be difficult. The shop page for the SSD1306 display mentions two different sizes with different hole pattern and has no mechanical drawing.

@jamwaffles
Copy link
Member

Some modules also have an address selection resistor.

That's true. If the addresses don't work or overlap, I was thinking of using either jumpers or, if signal integrity is terrible, a mux chip. We can cross that bridge when we come to it, let's decide on some hardware first...

They are available in different size. I think 2.8" is the largest in that form factor. Should we use the version with or without touch?

I'm fine getting one without touch. They're resistive touch screens which are nasty IMO. I think we'd be better off adding a rotary encoder and/or some directional buttons to the board for interface. That said, it's not difficult to hook up the touch interface so I'll add provisions to the board to cater for that.

Do we want to add mounting holes for the display or rely on the socket friction?

Ideally mounting holes as the KiCAD footprints already have them, but yeah the socket friction should be enough if the holes don't line up. If it's not, one could always shove some foam or other support material under the display. I don't think soldering them on is a great idea. The longevity of these modules isn't great for one, and I often like reusing the same display across multiple widgets.

@rfuest
Copy link
Member Author

rfuest commented Apr 21, 2022

I'm fine getting one without touch. They're resistive touch screens which are nasty IMO. I think we'd be better off adding a rotary encoder and/or some directional buttons to the board for interface. That said, it's not difficult to hook up the touch interface so I'll add provisions to the board to cater for that.

I'm also OK with not using a touch screen. I wasn't sure if the pinout for the touch and no touch versions would be the same, but it seems like they have the same number of pins and some pins aren't used in the no touch version. If we have pins left over we can connect the touch IC, but I wouldn't make this a priority.

Ideally mounting holes as the KiCAD footprints already have them, but yeah the socket friction should be enough if the holes don't line up. If it's not, one could always shove some foam or other support material under the display. I don't think soldering them on is a great idea. The longevity of these modules isn't great for one, and I often like reusing the same display across multiple widgets.

OK.

@rfuest
Copy link
Member Author

rfuest commented Apr 21, 2022

I've created embedded-graphics/hardware-bench#1 to discuss the exact modules we'll want to use on that PCB.

@rfuest
Copy link
Member Author

rfuest commented Apr 26, 2022

Now that the display selection is finished we need to decide which µC development boards we should use. The STM32H7B3I-DK can also be used for our display board, but we should also have at least one lower end µC (e.g. without FPU) to test on.

Do you already have some Nucleo boards which we could use so that we don't need to both order hardware? All STM32 I own are Discovery boards, which don't have a Arduino compatible pinout.

A possible low end board from ST, which is in stock, is the NUCLEO-F091RC (Cortex-M0 @ 48 MHz, 256 kB FLASH, 32 kB RAM).
Or the the blue pill as a Nucleo: NUCLEO-F103RB (Cortex-M3 @ 72 MHz, 128 kB Flash, 20 kB SRAM). IMO the additional RAM and Flash would be more important than the additional processing power for our low end target and I would choose the F0 over the F1.

If we don't want to be ST fanboys there is also a similar board with a Microchip SAM D21: https://www.digikey.de/de/products/detail/seeed-technology-co-ltd/102010248/10451901 Very similar to the NUCLEO-F091RC, but doesn't include an on board SWD adapter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants