I'm hesitant to pin numpy in pyproject.toml because the right version sort of depends on your Python version. However, there's a potential stumbling block here that needs to be addressed. If anybody has more experience and wisdom to share about the new features of pip, I'm all ears.
You can (nowadays) specify the numpy version depending on the python version in pyproject.toml, which should make this possible. Something like:
In principle we should always pin the numpy version to the oldest supported version (if using build isolation), as otherwise using the built package in an environment with another numpy version breaks, as you explained.
Last year, we added a pyproject.toml to pandas, but it gave a lot of problems (https://github.com/pandas-dev/pandas/issues/20775
), related to the fact that the above python-dependent numpy version was not possible yet (environment markers in pyproject.toml files were not yet supported at that time) + build dependencies needed to be installed as wheels instead of from source (which can give problems on certain platforms for which there are no wheels; this is also fixed in the meantime in pip I think). So in the end we decided to remove it again for the next release.
But it seems we should maybe look into adding it again.