Introduce Sphinx doctest builds#1406
Conversation
- enable the plugin - disable default console block behavior - new tox env - new CI config
Improve our doctest config with - a wrapped version of `mock.patch` which does auto-rollback - pre-set imports for all doctests
| extensions = [ | ||
| # sphinx-included extensions | ||
| "sphinx.ext.autodoc", | ||
| "sphinx.ext.doctest", |
There was a problem hiding this comment.
I'm concerned about how maintainers discover what different sphinx directives mean in this repository as we expand the number of internally-maintained & dependency extensions.
With the context of reviewing this PR which adds the sphinx.ext.doctest extension in a separate file, I was eventually able to find documentation for what testsetup:: doctest::, and testcode:: meant as symbols. If I encountered those in the context of browsing through a file though, I don't think I would've been able to backtrace why those symbols are meaningful (maybe with the exception of doctest because the extension happens to be named the same).
I think it's time that we have some in-repository documentation (a DOCUMENTATION.md or similar) that provides a central reference for which directives we're importing, which we've specialized, which we've built ourselves, etc. linking out to relevant docs like https://www.sphinx-doc.org/en/master/usage/extensions/doctest.html
There was a problem hiding this comment.
I agree with this comment: that information belongs in our contribution docs. I do not think we should block this addition on these grounds, in a case of "the perfect is the enemy of the good".
Can we describe the follow-up we would like in the form of an issue or story? (I think in-repo issue tracking would be appropriate in this case.)
There was a problem hiding this comment.
I was able to find doctest trivially with a search for "python sphinx testcode". I think we need to discuss what level of detail is needed in our contributor documentation. 🙂
There was a problem hiding this comment.
Can we describe the follow-up we would like in the form of an issue or story?
Yeah sure.
I was able to find doctest trivially with a search for "python sphinx testcode". I think we need to discuss what level of detail is needed in our contributor documentation. 🙂
I wouldn't expect a contributor to know that the thing they encountered in a random docstring in the form .. SomethingSomething:: <customer-symbol> is a sphinx directive so I think that assumes too much contextual knowledge.
It took me a while to figure out where to find documentation around these directives while I was reviewing the PR that introduced the extension which they use. If I were more familiar with exploring the extension space of custom sphinx directives, I probably could've gotten there quicker, but I am not & I don't think our average contributor (internal or external) will be.
There was a problem hiding this comment.
Yep, that's fair! Our contributor documentation should provide that necessary context.
We should not try to document every directive exhaustively ourselves.
2 participants
In order to allow for doctest usage, but without mandating it everywhere that examples are written (today and into the future), add Sphinx config which allows for use of the
doctestdirective but does not automatically run all examples as doctests.A pass over existing examples found a few items worth trying to turn into doctests, but this effort may not have been perfectly complete.