Just like aarch64's prepare_host_addr(), x86_64 should use
MO_ATOM_WITHIN16 for the memop when it's capable. Unlike aarch64,
which needs to check a CPU feature, x86 has been capable since P6
family processors and newer (see Intel SDM Vol. 3 §11.1.1).
Since a 16-byte aligned region always fits within a 16-byte multiple
sized cache line (x86_64 implementations always have cache lines of
at least 64 bytes), then this enables riscv cpu models with Zama16b
to use the fast path, just as cpu models without Zama16b do.
Cc: LIU Zhiwei <zhiwei_liu@linux.alibaba.com>
Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
[rth: Update both atom_and_align_for_opc calls]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20260707122819.114105-1-andrew.jones@oss.qualcomm.com>
Zero operands produce a zero high and low product. One operands produce
a copy of the other operand and a zero or sign extension in the high
half.
Fold those cases during TCG optimization so wide-multiply idioms used by
target translators can collapse before code generation.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20260520125139.13352-3-philmd@linaro.org>
Add a helper for multi-limb 64-bit addition. The helper emits native
carry-chain TCG ops when they are available and falls back to explicit
carry propagation otherwise.
This lets target translators build wider integer accumulators inline
without open-coding the same add-with-carry sequence at each use site.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20260520172313.23777-17-philmd@linaro.org>
In next commit, we'll apply same helper pattern for base helpers
remaining.
Our new helper pattern always include helper-*-common.h, which ends up
including include/tcg/tcg.h, which contains one occurrence of
CONFIG_USER_ONLY.
Thus, common files not being duplicated between system and target
relying on helpers will fail to compile. Existing occurrences are:
- target/arm/tcg/arith_helper.c
- target/arm/tcg/crypto_helper.c
This occurrence of CONFIG_USER_ONLY is for defining variable
tcg_use_softmmu, and we rely on dead code elimination with it in various
tcg-target.c.inc.
Thus, move its definition to tcg/tcg-internal.h, so helpers can be
included by common files. Also, change it to a define, as it has fixed
values for now.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Message-id: 20260219040150.2098396-8-pierrick.bouvier@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Since we only support 64-bit hosts, there's no real need
to parameterize TCG_TARGET_REG_BITS. It seems worth holding
on to the identifier though, for documentation purposes.
Move one tcg/*/tcg-target-reg-bits.h to tcg/target-reg-bits.h
and remove the others.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
This requires renaming several directories:
tcg/riscv, linux-user/include/host/riscv, and
common-user/host/riscv.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
This requires renaming several directories:
tcg/mips, linux-user/include/host/mips, and
common-user/host/mips.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Remove instances of __i386__, except from tests and imported headers.
Drop a block containing sanity check and fprintf error message for
i386-on-i386 or x86_64-on-x86_64 emulation. If we really want
something like this, we would do it via some form of compile-time check.
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Remove tcg/arm.
Remove instances of __arm__, except from tests and imported headers.
Remove arm from supported_cpus.
Remove linux-user/include/host/arm.
Remove common-user/host/arm.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
TCG_REG_TMP0 may be used by set_vtype* to load the vtype
parameter, so delay any other use of TCG_REG_TMP0 until
the correct vtype has been installed.
Cc: qemu-stable@nongnu.org
Fixes: d4be6ee111 ("tcg/riscv: Implement vector mov/dup{m/i}")
Reported-by: Zhijin Zeng <zengzhijin@linux.spacemit.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
We inadvertently disabled affected bits optimizations on operations
that use fold_masks_zosa. These happen relatively often in x86 code
for extract/sextract; for example given the following:
mov %esi, %ebp
xor $0x1, %ebp
the optimizer is able to simplify the "extract_i64 rbp,tmp0,$0x0,$0x20"
produced by the second instruction to a move.
Cc: qemu-stable@nongnu.org
Fixes: 932522a9dd ("tcg/optimize: Fold and to extract during optimize")
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20251223163720.985578-1-pbonzini@redhat.com>
In computing a_mask, for or, we remove the bits from t1->o_mask
which are known to be zero. For orc, the bits known to be zero
are the inverse of those known to be one.
Cc: qemu-stable@nongnu.org
Fixes: cc4033ee47 ("tcg/optimize: Build and use zero, one and affected bits in fold_orc")
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Since d182123974, the number of bits in a MemOpIdx tops out at 17.
which won't fit in the TCI rrm format, thus an assertion failure.
Introduce new opcodes that take the MemOpIdx from a register, as
we already do for qemu_ld2 and qemu_st2.
Fixes: d182123974 ("include/exec/memopidx: Adjust for 32 mmu indexes")
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
For native code generation, zero-extending 32-bit addresses for
the slow path helpers happens in tcg_out_{ld,st}_helper_args,
but there isn't really a slow path for TCI, so that didn't happen.
Make the extension for TCI explicit in the opcode stream,
much like we already do for plugins and atomic helpers.
Cc: qemu-stable@nongnu.org
Fixes: 24e46e6c9d ("accel/tcg: Widen tcg-ldst.h addresses to uint64_t")
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
The else after the TCG_TARGET_HAS_extract2 test is exactly
the same as what tcg_gen_extract2_i32 would emit itself.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
