Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: shard testing crates across multiple jobs #8612

Merged
merged 2 commits into from
May 16, 2024

Conversation

fitzgen
Copy link
Member

@fitzgen fitzgen commented May 14, 2024

No description provided.

@fitzgen fitzgen requested a review from a team as a code owner May 14, 2024 01:40
@fitzgen fitzgen requested review from alexcrichton and removed request for a team May 14, 2024 01:40
@fitzgen fitzgen force-pushed the ci-shard-tests branch 8 times, most recently from 6001f01 to 945a556 Compare May 14, 2024 03:15
@fitzgen
Copy link
Member Author

fitzgen commented May 14, 2024

Successful default PR run in ~8 minutes now: https://github.com/bytecodealliance/wasmtime/actions/runs/9073195225

Going to do a full run in a second.

@fitzgen
Copy link
Member Author

fitzgen commented May 14, 2024

Unfortunately, it looks like cargo nextest's CI partitioning isn't exposed/supported by cargo miri.

@fitzgen fitzgen force-pushed the ci-shard-tests branch 2 times, most recently from 6500cb5 to 5d6949b Compare May 14, 2024 05:14
@fitzgen fitzgen requested review from a team as code owners May 14, 2024 05:14
@fitzgen fitzgen requested review from abrown and removed request for a team May 14, 2024 05:14
@fitzgen fitzgen force-pushed the ci-shard-tests branch 5 times, most recently from 56807b6 to fa50840 Compare May 14, 2024 07:43
@fitzgen
Copy link
Member Author

fitzgen commented May 14, 2024

Successful full test run in 14 minutes: https://github.com/bytecodealliance/wasmtime/actions/runs/9075621159?pr=8612

@github-actions github-actions bot added cranelift Issues related to the Cranelift code generator cranelift:area:machinst Issues related to instruction selection and the new MachInst backend. cranelift:area:aarch64 Issues related to AArch64 backend. cranelift:area:x64 Issues related to x64 codegen isle Related to the ISLE domain-specific language labels May 14, 2024
@fitzgen
Copy link
Member Author

fitzgen commented May 15, 2024

I'll also point out that this is quite a lot of sharding, more than I've ever seen done before on github actions, so while I have no reason to think things will go wrong there's always the possibility we uncover something strange in the future. Most projects I don't think approach anywhere near this concurrency. The micro_checks job is also very heavily sharded where commands that might take ~5 seconds now take ~1 minute due to the overhead of a runner, downloading rustc, downloading deps, etc. That's not going to break the bank with our concurrency limit though so I don't have any reason to not do this.

Yeah I actually wanted to group that CI job into buckets of ~10 checks each or something but I couldn't figure out an easy way to do that ¯\_(ツ)_/¯

@fitzgen
Copy link
Member Author

fitzgen commented May 15, 2024

Okay I think I figured out how to avoid having a billion check jobs

@fitzgen fitzgen requested a review from alexcrichton May 15, 2024 23:24
@fitzgen fitzgen force-pushed the ci-shard-tests branch 5 times, most recently from 3404ee6 to 5f0a476 Compare May 16, 2024 00:08
@fitzgen fitzgen requested a review from a team as a code owner May 16, 2024 00:08
@fitzgen fitzgen force-pushed the ci-shard-tests branch 8 times, most recently from 978157a to 0d7a972 Compare May 16, 2024 01:12
@github-actions github-actions bot added the fuzzing Issues related to our fuzzing infrastructure label May 16, 2024
Copy link

Subscribe to Label Action

cc @fitzgen

This issue or pull request has been labeled: "fuzzing"

Thus the following users have been cc'd because of the following labels:

  • fitzgen: fuzzing

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

@elliottt elliottt removed the request for review from a team May 16, 2024 06:16
.github/workflows/main.yml Outdated Show resolved Hide resolved
@fitzgen fitzgen enabled auto-merge May 16, 2024 16:08
@fitzgen fitzgen added this pull request to the merge queue May 16, 2024
Merged via the queue into bytecodealliance:main with commit 934bf9d May 16, 2024
115 checks passed
@fitzgen fitzgen deleted the ci-shard-tests branch May 16, 2024 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cranelift:area:aarch64 Issues related to AArch64 backend. cranelift:area:machinst Issues related to instruction selection and the new MachInst backend. cranelift:area:x64 Issues related to x64 codegen cranelift Issues related to the Cranelift code generator fuzzing Issues related to our fuzzing infrastructure isle Related to the ISLE domain-specific language wasmtime:api Related to the API of the `wasmtime` crate itself
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants