Make the extraordinary leap from Sphinx 3.4.3 to Sphinx
7.2.6. Simplifications and cleanup will follow in subsequent patches.
Note that CentOS Stream 9 only ships a version of Sphinx that runs on
Python 3.9, so regardless of the version of Sphinx we require, if we
require Python 3.10+, CentOS Stream 9 will be unable to build QEMU
documentation in an isolated offline build environment.
CentOS Stream 9 continues to be able to build documentation so long as
PyPI access is permitted during configure, or if the requisite PyPI
packages are installed prior to configure time.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-ID: <20260825215851.1780608-15-jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
With Python3.9 EOL and Python3.10 soon to be EOL (October 2026), and
with ... no other supported build platform actually using Python 3.11,
require the use of Python3.12 or later.
Additionally, as part of this upgrade, remove the use of tomllib from
mkvenv as it is now standard in the Python stdlib: leaving this
transitional code in causes a regression with mypy which becomes
confused about the overloaded use of 'tomllib'.
Our current minimum requirements as determined by standard repository
packages on supported distributions is as follows:
distro EOS python3 pip setuptools sphinx wheel
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
-------------------------------------------------------------------------
centos_stream_9 2026-12-12 3.9.25 21.3.1 53.0.0 3.4.3 0.36.2
centos_stream_9+ 2026-12-12 3.12.13 23.2.1 68.2.2 --- 0.41.2
freebsd -- 3.12.14 23.3.2 63.1.0 9.0.4 0.48.0
ubuntu_24_04 2028-04-23 3.12.3 24.0 68.1.2 7.2.6 0.42.0
centos_stream_10 2030-01-01 3.12.13 23.3.2 69.0.3 7.2.6 0.41.2
alpine_3_21 2026-11-01 3.12.14 24.3.1 70.3.0 8.1.3 0.43.0
debian_13 2028-08-09 3.13.5 25.1.1 78.1.1 8.1.3 0.46.1
fedora_43 2026-12-09 3.14.7 25.1.1 78.1.1 8.2.3 0.45.1
ubuntu_26_04 2031-05-29 3.14.4 25.1.1 78.1.1 8.2.3 0.46.3
fedora_44 2027-06-02 3.14.7 26.0.1 80.10.2 8.2.3 0.45.1
macports -- 3.14.7 26.0.1 80.10.2 9.1.0 0.48.0
alpine_3_22 2027-05-01 3.12.14 25.1.1 80.9.0 8.2.3 0.46.3
alpine_3_23 2027-11-01 3.12.14 25.1.1 80.9.0 8.2.3 0.46.3
openbsd -- 3.14.6 26.2.1 80.9.0 9.1.0 0.48.0
alpine_3_24 2028-06-01 3.14.7 26.1.2 82.0.1 9.1.0 0.47.0
pkgsrc_current -- 3.13.15 26.2.1 84.0.0 9.1.0 0.48.0
CentOS stream 9 note: python3.12, python3.12-pip,
python3.12-setuptools and python3.12-wheel are available as optional
packages in the standard repository. Sphinx is not available, but can
be side-loaded from PyPI using pip3.12.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20260825215851.1780608-10-jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Replace human-monitor-command "info usernet" calls with the structured
x-query-usernet QMP command in get_usernet_hostfwd_port() and the VM
test runner boot path.
Since x-query-usernet returns a list of UsernetInfo entries, callers
now iterate over the result and extract the port from each entry's
"info" field. Also switch get_info_usernet_hostfwd_port() from
split('\r\n') to splitlines(), since QMP strings use \n rather than
the HMP monitor's \r\n line endings.
Reviewed-by: Thomas Huth <th.huth+qemu@posteo.eu>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Marc-Andre Lureau <marcandre.lureau@redhat.com>
Message-ID: <20260828-qemu-no-hmp-v5-11-9227de146347@redhat.com>
Meson 1.11.0 brings support for parsing Cargo.toml inside QEMU's source
tree, and Meson 1.12.0 brings support for cross-compilation of Cargo
subprojects.
Together, these two features allow QEMU to remove hundreds of lines of
manual compilation scripts.
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
There are argument we add because we want the test harness to control
QEMU and arguments we default for handling the display and machine
type. Split the obvious ones between base_args and a new list called
harness_args.
We will leave the complexity of the serial ports for now.
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20260619155657.944220-2-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
The newest versions of mypy do not support targeting Python 3.9, which
we still support. I want to address that soon, but in the meantime
it's nice if the tests pass.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20260611042332.482979-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Following suit with the rest of this repository, drop avocado and
replace it with the Python standard "pytest" package.
Our ultimate goal is to merge these python tests with the meson test
suite, so the use of 'pytest' here is only a stop-gap solution to get
the GitLab CI 'check-python-tox' passing again following recent Python
packaging ecosystem changes.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20260226213400.1254014-3-jsnow@redhat.com>
[ran black autoformatter. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
With recent python packaging ecosystem updates, the latest 'wheel' is no
longer compatible with older packages - and was somehow still compatible
with our Python 3.9 environment. Pin wheel to an older version (Version
based on Debian 11's available wheel package) to remove warnings during
the minreqs test.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20260226213400.1254014-2-jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
In anticipation of deleting the python/qemu/qmp source from the tree,
add a vendored version of the qemu.qmp package to ensure that "make
check" can be run in isolated build environments, offline.
Acked-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20260218213416.674483-13-jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Currently, we "implicitly" install the local 'qemu' python package for
'make check-venv' with some logic inside tests/Makefile.include. I would
like to make this installation explicit in pythondeps.toml instead.
This patch adds a path constraint that can be used in lieu of version
constraints to specify that a package should be installed from the
source tree instead of from PyPI or vendored packages. This is done to
allow us to install the python packages hosted inside of the tree while
also processing dependencies; i.e. so that our "qemu" package can
specify that it needs "qemu.qmp", which soon will not be included in
qemu.git.
This also has the benefit of being able to specify in a declarative
configuration file that our pyvenv environment *will* have our local
python packages installed and available without any PYTHONPATH hacks,
which should simplify iotests, device-crash-test and functional tests
without needing to manage local inclusion paths in environment
variables.
On the downsides, installing packages through mkvenv/ensuregroup means
that there are extra steps we need to take in order to install a local
package *offline*; namely we must disable build isolation (so we have
access to setuptools) and we must also include python3-wheel in QEMU's
build dependencies in order for "make check" to run successfully when in
an offline, isolated environment. These extra dependencies are handled
in a forthcoming commit; for now, nothing is utilizing this new pathway.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20260218213416.674483-6-jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Each group ensured by the mkvenv script will create an empty timestamp
file named {groupname}.group which can be used to conditionally trigger
dependency installation from various scripts and build machinery.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20260218213416.674483-3-jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
pkg_resources has been fully dropped from modern pip/setuptools
distributions and we should phase out its use. This patch is enough to,
by itself, repair most GitLab CI tests upstream; with the exception of
tox tests which are still making use of avocado - which will be dropped
in a separate series to restore functionality there.
Signed-off-by: John Snow <jsnow@redhat.com>
Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Message-ID: <20260211195804.135144-3-jsnow@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Pylint 4.x has refined checking for variable names that behave as
constants vs ones that do not; unfortunately our tricky import machinery
is perceived as these variables being re-assigned.
Add a temporary variable with an underscore and assign to the global
constants precisely once to alleviate this new nag message. Add an
ignore for this name for older versions of pylint that developers may
have installed locally.
(In other words: there is no solution that will cater to both pre- and
post- 4.x versions, so we target 4.x here and silence older versions.)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20251118200657.1043688-2-jsnow@redhat.com>
While QEMUQtestMachine closes the socket that was passed to
QEMUQtestProtocol, the python resource leak manager still
believes that the copy QEMUQtestProtocol holds is open. We
must explicitly call close to avoid this leak warnnig.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This patch collects comments and documentation changes from many commits
in the python-qemu-qmp repository; bringing the qemu.git copy in
bit-identical alignment with the standalone library *except* for several
copyright messages that reference the "LICENSE" file which is, for QEMU,
named "COPYING" instead and are therefore left unchanged.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This commit is two backports squashed into one to avoid regressions.
python: *really* remove get_event_loop
A prior commit, aa1ff990, switched away from using get_event_loop *by
default*, but this is not good enough to avoid deprecation warnings as
`asyncio.get_event_loop_policy().get_event_loop()` is *also*
deprecated. Replace this mechanism with explicit calls to
asyncio.get_new_loop() and revise the cleanup mechanisms in __del__ to
match.
python: avoid creating additional event loops per thread
"Too hasty by far!", commit 21ce2ee4 attempted to avoid deprecated
behavior altogether by calling new_event_loop() directly if there was no
loop currently running, but this has the unfortunate side effect of
potentially creating multiple event loops per thread if tests
instantiate multiple QMP connections in a single thread. This behavior
is apparently not well-defined and causes problems in some, but not all,
combinations of Python interpreter version and platform environment.
Partially revert to Daniel Berrange's original patch, which calls
get_event_loop and simply suppresses the deprecation warning in
Python<=3.13. This time, however, additionally register new loops
created with new_event_loop() so that future calls to get_event_loop()
will return the loop already created.
Reported-by: Richard W.M. Jones <rjones@redhat.com>
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@21ce2ee4f2df87efe84a27b9c5112487f4670622
cherry picked from commit python-qemu-qmp@c08fb82b38212956ccffc03fc6d015c3979f42fe
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This method was deprecated in 3.12 because it ordinarily should not be
used from coroutines; if there is not a currently running event loop,
this automatically creates a new event loop - which is usually not what
you want from code that would ever run in the bottom half.
In our case, we do want this behavior in two places:
(1) The synchronous shim, for convenience: this allows fully sync
programs to use QEMUMonitorProtocol() without needing to set up an event
loop beforehand. This is intentional to fully box in the async
complexities into the legacy sync shim.
(2) The qmp_tui shell; instead of relying on asyncio.run to create and
run an asyncio program, we need to be able to pass the current asyncio
loop to urwid setup functions. For convenience, again, we create one if
one is not present to simplify the creation of the TUI appliance.
The remaining user of get_event_loop() was in fact one of the erroneous
users that should not have been using this function: if there's no
running event loop inside of a coroutine, you're in big trouble :)
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@aa1ff9907603a3033296027e1bd021133df86ef1
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Based on the discussion at https://github.com/pypa/pip/issues/9726 -
even though the setuptools documentation implies that it is possible to
guard script execution with optional dependency groups, this is not true
in practice with the scripts generated by pip.
Just do the simple thing and guard the import statements.
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@df520dcacf9a75dd4c82ab1129768de4128b554c
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@9c889dcbd58817b0c917a9d2dd16161f48ac8203
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This is not strictly needed functionality-wise, but doing this allows
sphinx to see which decorated methods are async. Without this, sphinx
misses the "async" classifier on generated docs, which ... for an async
library, isn't great.
It does make an already gnarly function even gnarlier, though.
So, what's going on here?
A synchronous function (like require() before this patch) can return a
coroutine that can be awaited on, for example:
def some_func():
return asyncio.task(asyncio.sleep(5))
async def some_async_func():
await some_func()
However, this function is not considered to be an "async" function in
the eyes of the abstract syntax tree. Specifically,
some_func.__code__.co_flags will not be set with CO_COROUTINE.
The interpreter uses this flag to know if it's legal to use "await" from
within the body of the function. Since this function is just wrapping
another function, it doesn't matter much for the decorator, but sphinx
uses the stdlib inspect.iscoroutinefunction() to determine when to add
the "async" prefix in generated output. This function uses the presence
of CO_COROUTINE.
So, in order to preserve the "async" flag for docs, the require()
decorator needs to differentiate based on whether it is decorating a
sync or async function and use a different wrapping mechanism
accordingly.
Phew.
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@40aa9699d619849f528032aa456dd061a4afa957
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Expose the limit parameter of the underlying StreamReader and StreamWriter
instances.
This is helpful for the use case of transferring files in and out of a VM
via the QEMU guest agent's guest-file-open, guest-file-read, guest-file-write,
and guest-file-close methods, as it allows pushing the buffer size up to the
guest agent's limit of 48MB per transfer.
Signed-off-by: Adam Dorsey <adam@dorseys.email>
cherry picked from commit python-qemu-qmp@9ba6a698344eb3b570fa4864e906c54042824cd6
cherry picked from commit python-qemu-qmp@e4d0d3f835d82283ee0e48438d1b154e18303491
[Squashed in linter fixups. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@20a88c2471f37d10520b2409046d59e1d0f1e905
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This removes a non-idiomatic use of a "coroutine callback" in favor of
something a bit more standardized.
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@commit 97f7ffa3be17a50544b52767d14b6fd478c07b9e
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Now that the minimum version is 3.7, drop some of the 3.6-specific hacks
we've been carrying. A single remaining compatibility hack concerning
3.6's lack of @asynccontextmanager is addressed in the following commit.
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@3e8e34e594cfc6b707e6f67959166acde4b421b8
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The client name is mutable, so the logging name should also change to
reflect it when it changes.
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@e10b73c633ce138ba30bc8beccd2ab31989eaf3d
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This corrects an oversight in qmp-shell operation where new events will
not accumulate in the event queue when pressing "enter" with an empty
command buffer, so no new events show up.
Reported-by: Jag Raman <jag.raman@oracle.com>
Signed-off-by: John Snow <jsnow@redhat.com>
cherry picked from commit python-qemu-qmp@0443582d16cf9efd52b2c41a7b5be7af42c856cd
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
