Using a local python package in pip_parse()

[adityapande-1995]

Is it possible to use a local python package in a pip_parse() rule ?

My workspace is structured as follows :

|-- WORKSPACE
|-- BUILD.bzl
|-- requirements.txt
|-- my_custom_python_package
     |-- __init__.py
     |-- setup.py
|-- app.py

The workspace file contains the pip_parse() rule, and is used as a dependency to app.py in BUILD.bzl, using the requirement('my_custom_python_package') macro.

I keep getting an error saying The repository '@pip_deps_my_custom_python_package' could not be resolved: Repository '@pip_deps_my_custom_python_package' is not defined and referenced by '//:my_app'

Is there a correct guide on how to use a local package using pip parse ? I'd be great to have an example to show that. Any help would be appreciated :)

[meteorcloudy]

/cc @rickeylev

[rickeylev]

pip parse doesn't really support local packages. It's not really needed -- just depend on the targets directly instead of going through packaging step.

[adityapande-1995]

My apologies for not clarifying this in the original question. My package/target contains an entry_point in its setup.py:

from setuptools import setup

setup(
    name = "roslaunch_util",
    entry_points={
        'launch.frontend.launch_extension': [
            'roslaunch_util = roslaunch_util',
        ],
    }
)

I might be wrong here, but the way to get this entry point registered and trigger setuptools is by doing a pip install . on the package (outside of bazel). What the best way to invoke setuptools and get the entry point registered in bazel ?

[groodt]

You can register it in userspace via pytest
conftest plugins https://docs.pytest.org/en/stable/how-to/writing_plugins.html

Or by setting the PYTEST_PLUGINS environment variable.

Plugin discovery for pytest is described here https://docs.pytest.org/en/stable/how-to/writing_plugins.html

But not all those methods should be expected to work in bazel.

You can also try this as your test runner.

https://pypi.org/project/pytest-bazel/

[maxbartel]

Thanks for the answer! I am not looking for pytest specifically, but for its plugin implementation called pluggy. It discovers plugins via entrypoints https://pluggy.readthedocs.io/en/latest/api_reference.html#pluggy.PluginManager.load_setuptools_entrypoints
I basically have a local wheel that defines an entry point to be discovered by another package. Can my py_test just depend on that wheel and the entrypoints are discovered? I also don't seem to find any mentioning on local wheel files. Maybe you can give me more context? Thanks!

[groodt]

For in-tree code, depend on the code. You don't need a wheel.

For testing, using an in-tree pytest plugin, you can use conftest.py as described.

Sounds like you're making 2 errors:

• For in-tree code, don't use a wheel or setup.py
• For your testing code (py_test not pytest) import your in-tree plugin manually in conftest.py

[maxbartel]

Ok, I will try this for the unit tests. However what about integration tests? One would want to create all local wheels to test the packaging, install them and run integration tests. We would now have a mixture between local wheels and dependencies from pypi. Is that possible at the moment? Thanks for the help so far!

[groodt]

You can indeed round-trip through wheels or setup more elaborate test harnesses to ensure that your package works when installed outside a bazel environment.

When doing this, you need to carefully consider how you isolate yourself from bazel and create the non-bazel environment. You can use things like testcontainers or even a non-bazel test that installs a wheel from bazel-out/* and mounts it into a container.

It's also possible to do all the container orchestration etc via bazel, but thats beyond the scope of rules_python.

I think the message from me would be:

• First get your in-tree tests working. So you can unit test via conftest.py and ensure the plugin works from a functional POV
• Then worry about how to test end-to-end that goes through a wheel archival and installation step into a non-bazel environment