Bug#1090912: ITP: python-grpclib -- Pure-Python gRPC implementation for asyncio

[Simon Josefsson]

Hi Python Team,

I have prepared packaging of 'grpclib' and I'm new to Python packaging,
so I would appreciate open minded review of how this is packaged:

https://salsa.debian.org/python-team/packages/python-grpclib

Please bring up stylistic concerns, for me to learn. I plan to upload
this to NEW shortly.

Some of the questions/concerns I have are:

- Do the /usr/bin binaries have to go in a separate package, or could
they be in the python3-grpclib binary package? I believe they are
developer-only tools of limited applicability.

- Naming - right now it uses this style:

Source: python-grpclib
Package: python3-grpclib
Package: protoc-grpclib

Does that make sense? Upstream calls itself 'grpclib' but using
'python-grpclib' in Debian seems nicer. I see there is a
'rustc-protoc' in the archive, maybe this should be
'python-grpclib-protoc' instead? Or 'protoc-python-grpclib'? Or
'protoc-grpclib-python'?

- This uses tarballs from pypi.debian.net, which isn't identical to
upstream's GitHub git archive. It seems tests/ are missing. I've
approached upstream about including them in the PyPI upload --
https://github.com/vmagamedov/grpclib/issues/200 -- but I'm not sure
what the best practices are here. Is using the pypi.debian.net
tarball recommended? Or do people package the github.com tarball
instead? Or directly from git? What do we do when some stuff is
missing from the PyPI archive?

- Man pages are missing - the tools doesn't respond to -h or --help, I
suppose this is an upstream request, or are there any other ideas on
that?

- Others?

/Simon

Package: wnpp
Severity: wishlist
* Package name : python-grpclib
Version : 2.0.0b7
Upstream Author : Vladimir Magamedov, et al
* URL : https://github.com/vmagamedov/grpclib
* License : BSD-3-Clause
Programming Lang: Python
Description : Pure-Python gRPC implementation for asyncio
This is a Pure-Python gRPC implementation for asyncio.
https://salsa.debian.org/python-team/packages/python-grpclib
/Simon

[Soren Stoutner]

Simon,

Post by Simon Josefsson
Hi Python Team,
I have prepared packaging of 'grpclib' and I'm new to Python packaging,
https://salsa.debian.org/python-team/packages/python-grpclib
Please bring up stylistic concerns, for me to learn. I plan to upload
this to NEW shortly.
- Do the /usr/bin binaries have to go in a separate package, or could
they be in the python3-grpclib binary package? I believe they are
developer-only tools of limited applicability.
Source: python-grpclib
Package: python3-grpclib
Package: protoc-grpclib
Does that make sense? Upstream calls itself 'grpclib' but using
'python-grpclib' in Debian seems nicer. I see there is a
'rustc-protoc' in the archive, maybe this should be
'python-grpclib-protoc' instead? Or 'protoc-python-grpclib'? Or
'protoc-grpclib-python'?
- This uses tarballs from pypi.debian.net, which isn't identical to
upstream's GitHub git archive. It seems tests/ are missing. I've
approached upstream about including them in the PyPI upload --
https://github.com/vmagamedov/grpclib/issues/200 -- but I'm not sure
what the best practices are here. Is using the pypi.debian.net
tarball recommended? Or do people package the github.com tarball
instead? Or directly from git? What do we do when some stuff is
missing from the PyPI archive?

I am not experienced enough with Python packaging to respond to your other
questions, but I do know that many Python packages pull from github or gitlab
when thinks like tests are missing from PyPI, so you should feel free to do so
when needed.

Post by Simon Josefsson
- Man pages are missing - the tools doesn't respond to -h or --help, I
suppose this is an upstream request, or are there any other ideas on
that?
- Others?
/Simon

Package: wnpp
Severity: wishlist
* Package name : python-grpclib
Version : 2.0.0b7
Upstream Author : Vladimir Magamedov, et al
* URL : https://github.com/vmagamedov/grpclib
* License : BSD-3-Clause
Programming Lang: Python
Description : Pure-Python gRPC implementation for asyncio
This is a Pure-Python gRPC implementation for asyncio.
https://salsa.debian.org/python-team/packages/python-grpclib
/Simon

[Simon Josefsson]

Post by Soren Stoutner

Post by Simon Josefsson
- This uses tarballs from pypi.debian.net, which isn't identical to
upstream's GitHub git archive. It seems tests/ are missing. I've
approached upstream about including them in the PyPI upload --
https://github.com/vmagamedov/grpclib/issues/200 -- but I'm not sure
what the best practices are here. Is using the pypi.debian.net
tarball recommended? Or do people package the github.com tarball
instead? Or directly from git? What do we do when some stuff is
missing from the PyPI archive?

I am not experienced enough with Python packaging to respond to your other
questions, but I do know that many Python packages pull from github or gitlab
when thinks like tests are missing from PyPI, so you should feel free to do so
when needed.

Thanks for insight! This is good to know. I prefer to ask upstreams to
include the relevant stuff into the PyPI tarball instead. However if it
turns out upstreams don't want to do that (or that doing so is against
PyPI best practices somehow) AND there is significant advantage in using
upstream's github as the tarball source, then at least I know this isn't
considered a big no-no for Debian packages.

So far I've only noticed that some self-tests and auxilliary files are
missing from the PyPI tarballs, and I don't consider that to be
important enough to deviate from using the pypi.debian.net sources yet.

/Simon

[Simon Josefsson]

I have uploaded python-grpclib to NEW. I settled for including the
command-line tools even though they may be rarely used developer tools,
under the 'protoc-grpclib' name (I tried 'python-grpclib-protoc and
'python-protoc-grpclib' but then lintian complained about using python-*
namespace). It still uses the PyPI tarball, it seems to work fine and
I've opened upstream reports about including self-tests and man pages.
I added Testsuite:autopkgtest-pkg-pybuild to be prepared for when
upstream adds self-tests.

/Simon

[Julian Gilbey]

Post by Simon Josefsson
So far I've only noticed that some self-tests and auxilliary files are
missing from the PyPI tarballs, and I don't consider that to be
important enough to deviate from using the pypi.debian.net sources yet.

Hi Simon,

There is no Debian Python policy to use PyPI versus GitHub versions as
far as I am aware. However, the PyPI versions of packages frequently
do not include the test suites, as many developers regard those as
part of the development environment, not the release environment (and
having test suites included in all packages could cause significant
bloat). You are, of course, welcome to ask them to reconsider for
this package, but you may not have any success, and I wouldn't rely on
them agreeing to your request.

Whatever the outcome of your negotiations, the Debian Python team is
very keen on having autopkgtests for as many packages as possible:
doing so catches all sorts of bugs and can prevent your package from
mysteriously breaking. So given that upstream have written a test
suite that you have easy access to (via GitHub), unless you have a
very strong reason *not* to include it in the Debian sources, you
really ought to do so. "It isn't included in the PyPI source tarball"
is not a strong reason. So it seems that you have a few choices:

(1) Use the GitHub sources in place of the PyPI sources.

(2) Use the GitHub sources, patched in those few places where the PyPI
version differs to match the PyPI version. (This can be done via
debian/patches.)

(3) Use the PyPI sources and include the test suite from GitHub; this
could be done via the dpkg-source components mechanism, for example.
It would complicate the packaging a bit, and make upgrading harder
work, but it would certainly be feasible.

(4) Decide on principle that you're going to stick with the PyPI
version come what may, and as upstream haven't included a test suite
in that version, the Debian package won't either.


I am sure I'm not alone on the Debian Python Team in strongly
discouraging you from following option (4). My personal preference
would be (1) or (2), depending on how significantly different the
GitHub and PyPI versions are. (And this does also assume that the
developers have tagged their Git repository with appropriate tags.)

Best wishes,

Julian

[Simon Josefsson]

Post by Julian Gilbey

Post by Simon Josefsson
So far I've only noticed that some self-tests and auxilliary files are
missing from the PyPI tarballs, and I don't consider that to be
important enough to deviate from using the pypi.debian.net sources yet.

Hi Simon,
There is no Debian Python policy to use PyPI versus GitHub versions as
far as I am aware. However, the PyPI versions of packages frequently
do not include the test suites, as many developers regard those as
part of the development environment, not the release environment (and
having test suites included in all packages could cause significant
bloat). You are, of course, welcome to ask them to reconsider for
this package, but you may not have any success, and I wouldn't rely on
them agreeing to your request.
Whatever the outcome of your negotiations, the Debian Python team is
doing so catches all sorts of bugs and can prevent your package from
mysteriously breaking. So given that upstream have written a test
suite that you have easy access to (via GitHub), unless you have a
very strong reason *not* to include it in the Debian sources, you
really ought to do so. "It isn't included in the PyPI source tarball"
(1) Use the GitHub sources in place of the PyPI sources.
(2) Use the GitHub sources, patched in those few places where the PyPI
version differs to match the PyPI version. (This can be done via
debian/patches.)
(3) Use the PyPI sources and include the test suite from GitHub; this
could be done via the dpkg-source components mechanism, for example.
It would complicate the packaging a bit, and make upgrading harder
work, but it would certainly be feasible.
(4) Decide on principle that you're going to stick with the PyPI
version come what may, and as upstream haven't included a test suite
in that version, the Debian package won't either.
I am sure I'm not alone on the Debian Python Team in strongly
discouraging you from following option (4). My personal preference
would be (1) or (2), depending on how significantly different the
GitHub and PyPI versions are. (And this does also assume that the
developers have tagged their Git repository with appropriate tags.)

Thank you! I'm happy to adopt 1) but I didn't want to do that without
some +1 on doing so since

https://wiki.debian.org/Python/LibraryStyleGuide?action=show&redirect=Python%2FPackaging#debian.2Fwatch

says 'Thus, it is highly recommended that you update your existing
packages to the new debian/watch format described here'

and 'Since probably for most of our packages, the original tarball comes
from the Python Package Index (a.k.a. PyPI or the Cheeseshop)'

which give me the impression that using PyPI tarballs is generally
preferred, so it is hard for me as newbie to understand the preferences.

I do agree that self-tests are important, and I was surprised
maintainers don't include them in the PyPI tarballs. Maybe it is not
worth our time to open bug reports about changing the PyPI tarballs, and
switching to the GitHub tarballs is fine.

Another question is about which version to package -- latest GitHub tag
or PyPI version? Have a look at this project:

https://pypi.org/project/sigstore-protobuf-specs/#history

I suppose I may run into whatever bug that prompted this yanking too...

/Simon