How to determine required dependencies for new packages, and why and pyodide build needs metadata files for existing packages? #4632
-
Hey everyone, I'm running a local Pyodide build on my Ubuntu box, Charizard, to create a new package for pyiceberg:
The local unchanged build w/Docker runs like a champ! Followed these steps to build my meta.yaml file for PyIceberg:
With that context, I imagine I'm missing something regarding how to connect dependencies for the ones that should already be included, but I would have thought those would be created and linked in some way during the main build.
I promise I will submit some docs on this for the next poor uneducated fool like me if you help me. I was very careful to craft this question as I will use this to kind of guide the docs I will add. Thanks for listening to my TED squalk! |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments 4 replies
-
Okay, already answered question 1.i:
So the just pull everything answer would be to run the following but it's currently 230 packages which would take some time to build:
Still not sure how I determine if something is pure python once I run this to determine the rest of the projects I would actually need to include. Note: add a bold header over this section of the docs "Building existing Pyodide Packages". The current docs for this makes it look a bit optional and I grazed right over that. |
Beta Was this translation helpful? Give feedback.
-
Just figured out that you shouldn't put the version ranges in meta.yaml of Pyodide. As mentioned in the docs conda-forge meta.yaml has more syntax and Pyodide currently only requires the package name/tag. So This was specifically what was causing the error with referencing package name that didn't exist. |
Beta Was this translation helpful? Give feedback.
-
After removing the version ranges and uncommenting the package dependencies that do exist in Pyodide already, I have a successful build. Here's what my meta.yaml looks like now:
I have also split the packages into The files that are commented out are the files that pyodide doesn't currently support. Once we answer question 1.ii and 1.iii, then I can figure out which ones I likely need to also add support for to at least get the build working. Which if all my assumtions are correct about the build, would be, |
Beta Was this translation helpful? Give feedback.
-
So regarding to tell if something is a pure python function, I found this community SO answer that seems relatively smart and over my head so I'm just trusting it: isitc.py
Then run this against the libraries in question:
So in theory, I should only need to support the So I'm still not sure about the answer to 1.ii, but this seems to be a decent way to determine if it's a pure python package. I'm sure there's an easier way, but this comes from someone who knows little about python packaging stuff. |
Beta Was this translation helpful? Give feedback.
-
There are multiple ways to check the dependencies for each package.
Looking at the METADATA file is useful as you can check which packages are mandatory and which ones are optional. For instance, the package
You don't have to list python, pip, etc. Though Pyodide's meta.yaml file is inspired by conda's meta.yaml, but it's not identical. You only need to list Python package dependencies (+ some libraries if the package requires it). |
Beta Was this translation helpful? Give feedback.
There are multiple ways to check the dependencies for each package.
pip download <package>
: this command will download all dependencies of the package recursively so you can check which packages are dependencies of the package.METADATA
file inside the wheel file of the package. Each wheel file has a file namedMETADATA
in it, and it lists the dependencies of the package. For instance,pyic…