[prev in list] [next in list] [prev in thread] [next in thread]
List: oss-security
Subject: [oss-security] Xen Security Advisory 398 v1 - Multiple speculative security issues
From: Xen.org security team <security () xen ! org>
Date: 2022-03-08 18:16:43
Message-ID: E1nReO3-0003Ap-3Q () xenbits ! xenproject ! org
[Download RAW message or body]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Xen Security Advisory XSA-398
Multiple speculative security issues
ISSUE DESCRIPTION
=================
Note: Multiple issues are contained in this XSA due to their interactions.
1) Researchers at VU Amsterdam have discovered Spectre-BHB, pertaining
to the use of Branch History between privilege levels.
ARM have assigned CVE-2022-23960. Intel have assigned CVE-2022-0001
(Branch History Injection) and CVE-2022-0002 (Intra-mode BTI). AMD
have no statement at the time of writing.
For more details, see:
https://vusec.net/projects/bhi-spectre-bhb
https://developer.arm.com/support/arm-security-updates/speculative-processor-vulnerability
https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00598.html
2) Researchers at Open Source Security, Inc. have discovered that AMD
CPUs may speculate beyond direct branches.
AMD have assigned CVE-2021-26341.
For more details, see:
https://grsecurity.net/amd_branch_mispredictor_part_2_where_no_cpu_has_gone_before
https://www.amd.com/en/corporate/product-security/bulletin/amd-sb-1026
3) Researchers at Intel have discovered that previous Spectre-v2
recommendations of using lfence/jmp is incomplete.
AMD have assigned CVE-2021-26401.
For more details, see:
https://www.amd.com/en/corporate/product-security/bulletin/amd-sb-1036
IMPACT
======
An attacker might be able to infer the contents of arbitrary host
memory, including memory assigned to other guests.
VULNERABLE SYSTEMS
==================
Systems running all versions of Xen are affected.
Whether a CPU is potentially vulnerable depends on its
microarchitecture. Consult your hardware vendor.
Xen does not have a managed runtime environment, so is not believed to
be vulnerable to CVE-2022-0002 irrespective of any hardware
susceptibility.
Xen does not have any known gadgets vulnerable to Direct Branch Straight
Line Speculation. Therefore, no changes for CVE-2021-26341 are being
provided at this time.
The AMD BTI (Spectre v2) protections do not depend on isolating
predictions between different privileges, so the fact that Branch
History is shared (just like the Branch Target Buffer) is not believed
to be relevant to existing mitigations. Therefore, there is no believed
impact from Spectre-BHB on AMD hardware.
Patches to mitigate CVE-2022-23960 on affected ARM CPUs are provided.
Intel have recommended not making any changes by default for
CVE-2022-0001. Existing Spectre-v2 mitigations on pre-eIBRS hardware
are believed to be sufficient. On eIBRS capable hardware, there is
uncertainty over the utility of Branch History Injection to an
adversary. However, the risk can be removed by using eIBRS in
combination with retpoline.
For CVE-2021-26401, AMD have recommended using retpoline in preference
to lfence/jmp as previously recommended to mitigate Spectre-v2. This
recommendation also mitigates any risk from Branch History Injection.
For both CVE-2022-0001 on Intel, and CVE-2021-26401 on AMD, the
suggestion to use retpoline is incompatible with CET Shadow Stacks as
implemented in Xen 4.14 and later. The security team has decided that
disabling CET Shadow Stacks to work around speculation problems is not a
reasonable option for downstreams and end users.
Therefore, patches are also provided to:
* Use IBRS on capable AMD hardware. This also mitigates
CVE-2021-26401.
* Use CET Indirect Branch Tracking on capable Intel hardware. CET-IBT
has architectural guarantees about halting speculation, on top of
being a hardware mechanism to protect against Call/Jump Oriented
Programming attacks.
Both provide CET Shadow Stack compatible mitigations to these issues. A
practical consequence of this decision is that CET Shadow Stacks are now
considered security supported, upgraded from Tech Preview previously.
Note: CET-IBT patches are incomplete and will be backported at a later date.
MITIGATION
==========
On AMD systems, CVE-2021-26401 can be mitigated by specifying:
With CET-SS, `spec-ctrl=bti-thunk=jmp,ibrs`
Without CET-SS, `spec-ctrl=bti-thunk=retpoline`
on Xen's command line, and rebooting.
RESOLUTION
==========
Applying the appropriate attached patch resolves this issue.
Note that patches for released versions are generally prepared to
apply to the stable branches, and may not apply cleanly to the most
recent release tarball. Downstreams are encouraged to update to the
tip of the stable branch before applying these patches.
xsa398/xsa398-*.patch xen-unstable
xsa398/xsa398-4.16-*.patch Xen 4.16.x
xsa398/xsa398-4.15-*.patch Xen 4.15.x
xsa398/xsa398-4.14-*.patch Xen 4.14.x
xsa398/xsa398-4.13-*.patch Xen 4.13.x
xsa398/xsa398-4.12-*.patch Xen 4.12.x
$ sha256sum xsa398* xsa398*/*
9219c48d103a7eeda0fa9cbb5fc5b2265713589e29a9a483d0f3fb6523859903 xsa398.meta
32e7a7627609de2273fe474979e339f6a578cbcf7ce007b6a047954a31aec135 \
xsa398/xsa398-1-xen-arm-Introduce-new-Arm-processors.patch \
ef701fd64cfdd838299391cd736749db70ac3b18251d17768d42f4a610dda1be \
xsa398/xsa398-2-xen-arm-move-errata-CSV2-check-earlier.patch \
4d574bc40555f068608a595ade23ecdc224f8c0af86f447cba6e765d4ccde3ad \
xsa398/xsa398-3-xen-arm-Add-ECBHB-and-CLEARBHB-ID-fields.patch \
29a2880ab4fa492deecd2f3dc590609d0df5e9210565ab4121be0d731c4140b0 \
xsa398/xsa398-4.12-1-xen-arm-Introduce-new-Arm-processors.patch \
b81eb6a0f8ecde53318eeff1ec8bf1b3fd5f1b211a499317f6c596e831a90101 \
xsa398/xsa398-4.12-2-xen-arm-move-errata-CSV2-check-earlier.patch \
a9f5adc44eeeaf5a694f94c91a32e714c765bbcf61066a03e3c52d79d28a3366 \
xsa398/xsa398-4.12-3-xen-arm-Add-ECBHB-and-CLEARBHB-ID-fields.patch \
e70d4c06f789c8f5f45c7e27289f8c7aa4c448a6e33f67fb113630ed79382fd9 \
xsa398/xsa398-4.12-4-xen-arm-Add-Spectre-BHB-handling.patch \
6766c0b0d89f3be90046c05358e8b7c43c87b3e1012118af013faa098e783e74 \
xsa398/xsa398-4.12-5-xen-arm-Allow-to-discover-and-use-SMCCC_ARCH_WORKARO.patch \
fd047878fd53e130cd7d8cfd1d50334a958e7e962606afaacd5aa1da186f6341 \
xsa398/xsa398-4.12-6-x86-spec-ctrl-Cease-using-thunk-lfence-on-AMD.patch \
1ef4fae89d2bc75e33eb6c8e5f55d0b6f5ba45a274f6d3b5ea7e2eef4c08ad63 \
xsa398/xsa398-4.13-1-xen-arm-Introduce-new-Arm-processors.patch \
b0c25a34055dd5401dff1686f4f7ab978c6a449a76aa0e1b369f483fa184851a \
xsa398/xsa398-4.13-2-xen-arm-move-errata-CSV2-check-earlier.patch \
c6ffa2818480740dc30e232215531ab69c252e564df365c466e759886b207450 \
xsa398/xsa398-4.13-3-xen-arm-Add-ECBHB-and-CLEARBHB-ID-fields.patch \
a272621b1f03b2096a41d675b3ed46ff2c737cd2afcb3e2156a7ec2f8c31748b \
xsa398/xsa398-4.13-4-xen-arm-Add-Spectre-BHB-handling.patch \
8df9f4d3e7bd154246ebe7cd1bc0908ead1076aa35c0a183cd95359aa2173ad0 \
xsa398/xsa398-4.13-5-xen-arm-Allow-to-discover-and-use-SMCCC_ARCH_WORKARO.patch \
02c3a3c45bf3c2592bbc809ce4a8eb24d0b9d31856e9641d5566af68ebf2b476 \
xsa398/xsa398-4.13-6-x86-spec-ctrl-Cease-using-thunk-lfence-on-AMD.patch \
59edd0b8303a39451893d425b8e7ab8aeacf3e6d0bf460ba66a3a323dc0e3145 \
xsa398/xsa398-4.14-1-xen-arm-Introduce-new-Arm-processors.patch \
60bd3003759404b60fd8a7dcf0de87a13463bf64c3724f8fe6570e07c515cecb \
xsa398/xsa398-4.14-2-xen-arm-move-errata-CSV2-check-earlier.patch \
138511c69d00ef1dc0dfe5432af06d744e7b66945bada78024e343943fc001f2 \
xsa398/xsa398-4.14-3-xen-arm-Add-ECBHB-and-CLEARBHB-ID-fields.patch \
9c1b338511422629c98f11c42da27b1cd82435decc0531bca6b8a51218909101 \
xsa398/xsa398-4.14-4-xen-arm-Add-Spectre-BHB-handling.patch \
1a212de641ac1cebfc1aee32c55e9f8bfac6b059f5419ed62589eed99cc0dea5 \
xsa398/xsa398-4.14-5-xen-arm-Allow-to-discover-and-use-SMCCC_ARCH_WORKARO.patch \
da382a5baee60ecdf8b4cb0da2c1901b23f324b03dbbe33018fb825e70f78446 \
xsa398/xsa398-4.14-6-x86-spec-ctrl-Cease-using-thunk-lfence-on-AMD.patch \
296e5fdd53328e768908a4e790959841264a410548b4f52f7ccdcf793e9aca7a \
xsa398/xsa398-4.15-1-xen-arm-Introduce-new-Arm-processors.patch \
4498957a1f91c69e2a72cfcfb88804537ee0c05f05fa5d898f452a4dc8205f9e \
xsa398/xsa398-4.15-2-xen-arm-move-errata-CSV2-check-earlier.patch \
2f2b9ec3945283e48486cfd32d5b4343892040d48adc105a89e15953a128df3d \
xsa398/xsa398-4.15-3-xen-arm-Add-ECBHB-and-CLEARBHB-ID-fields.patch \
6d0ade4dfb59fc87c7ae22e4faa333fb5ccef5ecc595de58ef9bcf35f4e3eb26 \
xsa398/xsa398-4.15-4-xen-arm-Add-Spectre-BHB-handling.patch \
77a0a93cd9617c8f0ec0bab1b79f6ed60cab20f5b6ea76a9b6158c4d3a1d0d89 \
xsa398/xsa398-4.15-5-xen-arm-Allow-to-discover-and-use-SMCCC_ARCH_WORKARO.patch \
7df5b320c5887c72c8ed4ffe5b4bcdce9263fde76fe6a67e0876933f8d1ebcff \
xsa398/xsa398-4.15-6-x86-spec-ctrl-Cease-using-thunk-lfence-on-AMD.patch \
92bdba8102f88a2c9d71b46df4db43176fbf0082f9d438101407dbb7e6d458c6 \
xsa398/xsa398-4.16-1-xen-arm-Introduce-new-Arm-processors.patch \
0c9a6fbebc13a0dee288d67a94562fd76e3c6aec20b543c66ac2c16a812973ee \
xsa398/xsa398-4.16-2-xen-arm-move-errata-CSV2-check-earlier.patch \
4f084857ed79af49d2814c02ff6e090a14d77bb0f0d29ac6ddce3576fdb98c68 \
xsa398/xsa398-4.16-3-xen-arm-Add-ECBHB-and-CLEARBHB-ID-fields.patch \
8032757effe8dbc5ef8479403461e604b1520f007489620eace3857b467a4fe2 \
xsa398/xsa398-4.16-4-xen-arm-Add-Spectre-BHB-handling.patch \
3763998bb62d9b251b9358edff220fd22847729768c98dbd46362c290041025b \
xsa398/xsa398-4.16-5-xen-arm-Allow-to-discover-and-use-SMCCC_ARCH_WORKARO.patch \
114f07da2d79f45e0fa45c826c308b273e8c29b6d458bac10fe1aa231a3c2748 \
xsa398/xsa398-4.16-6-x86-spec-ctrl-Cease-using-thunk-lfence-on-AMD.patch \
1bedca674ecee5437e492e2f71275cd32e799d839d26a8f0d75ddee44db2e4d2 \
xsa398/xsa398-4-xen-arm-Add-Spectre-BHB-handling.patch \
6d63089af3eca863599bbe20e26f1f12d2d9c9b637317e7af44fc59750b09f77 \
xsa398/xsa398-5-xen-arm-Allow-to-discover-and-use-SMCCC_ARCH_WORKARO.patch \
f79e357079744bbee3e1f7d99d93196e925739297a16fdd8bc1cc86d3b846ce3 \
xsa398/xsa398-6-x86-spec-ctrl-Cease-using-thunk-lfence-on-AMD.patch $
-----BEGIN PGP SIGNATURE-----
iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAmInnKUMHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZ6XQIAIpVmnShgCYyb51BWYzZJ1yG2elg2sVPByc7NF2C
5VcIVOpE3QHRxmJzMIM01peHrfIbY61ZFfD76pKaBaSxUjkvWII+9Q7Qir+q9I4+
X6Kwmf7pbjX2AsFR08TQoYyNMWKFwf0fhc4AK0BtDe83FuEu0wg3EY0sThzS32jf
WcBSVf29KOlh7dVEbBHKQsaGGjVJsgdloIK3z8XX4ACRpku+eUrl/7O7lJBtT0Zo
BxBPObMteh9IA3Pt0UqHqB8XZWLP95XDAq/FvLGts/EqxxKQAlfIm9hN8MJeDQUl
4Sh0dAE1Ab2eGdQSl5OeqXdvs+ZTBqlDbbLD/grIfJcYHUg=
=qv9D
-----END PGP SIGNATURE-----
["xsa398.meta" (application/octet-stream)]
["xsa398/xsa398-1-xen-arm-Introduce-new-Arm-processors.patch" (application/octet-stream)]
From 35d1b85a6b43483f6bd007d48757434e54743e98 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Tue, 15 Feb 2022 10:37:51 +0000
Subject: xen/arm: Introduce new Arm processors
Add some new processor identifiers in processor.h and sync Xen
definitions with status of Linux 5.17 (declared in
arch/arm64/include/asm/cputype.h).
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <julien@xen.org>
diff --git a/xen/arch/arm/include/asm/processor.h b/xen/arch/arm/include/asm/processor.h
index 8ab2940f688e..852b5f3c24b8 100644
--- a/xen/arch/arm/include/asm/processor.h
+++ b/xen/arch/arm/include/asm/processor.h
@@ -65,6 +65,7 @@
#define ARM_CPU_PART_CORTEX_A17 0xC0E
#define ARM_CPU_PART_CORTEX_A15 0xC0F
#define ARM_CPU_PART_CORTEX_A53 0xD03
+#define ARM_CPU_PART_CORTEX_A35 0xD04
#define ARM_CPU_PART_CORTEX_A55 0xD05
#define ARM_CPU_PART_CORTEX_A57 0xD07
#define ARM_CPU_PART_CORTEX_A72 0xD08
@@ -72,11 +73,20 @@
#define ARM_CPU_PART_CORTEX_A75 0xD0A
#define ARM_CPU_PART_CORTEX_A76 0xD0B
#define ARM_CPU_PART_NEOVERSE_N1 0xD0C
+#define ARM_CPU_PART_CORTEX_A77 0xD0D
+#define ARM_CPU_PART_NEOVERSE_V1 0xD40
+#define ARM_CPU_PART_CORTEX_A78 0xD41
+#define ARM_CPU_PART_CORTEX_X1 0xD44
+#define ARM_CPU_PART_CORTEX_A710 0xD47
+#define ARM_CPU_PART_CORTEX_X2 0xD48
+#define ARM_CPU_PART_NEOVERSE_N2 0xD49
+#define ARM_CPU_PART_CORTEX_A78C 0xD4B
#define MIDR_CORTEX_A12 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A12)
#define MIDR_CORTEX_A17 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A17)
#define MIDR_CORTEX_A15 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A15)
#define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
+#define MIDR_CORTEX_A35 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A35)
#define MIDR_CORTEX_A55 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A55)
#define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57)
#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72)
@@ -84,6 +94,14 @@
#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75)
#define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A76)
#define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N1)
+#define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A77)
+#define MIDR_NEOVERSE_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V1)
+#define MIDR_CORTEX_A78 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78)
+#define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1)
+#define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710)
+#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2)
+#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)
+#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
/* MPIDR Multiprocessor Affinity Register */
#define _MPIDR_UP (30)
["xsa398/xsa398-2-xen-arm-move-errata-CSV2-check-earlier.patch" (application/octet-stream)]
From 599616d70eb886b9ad0ef9d6b51693ce790504ba Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Tue, 15 Feb 2022 10:39:47 +0000
Subject: xen/arm: move errata CSV2 check earlier
CSV2 availability check is done after printing to the user that
workaround 1 will be used. Move the check before to prevent saying to the
user that workaround 1 is used when it is not because it is not needed.
This will also allow to reuse install_bp_hardening_vec function for
other use cases.
Code previously returning "true", now returns "0" to conform to
enable_smccc_arch_workaround_1 returning an int and surrounding code
doing a "return 0" if workaround is not needed.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <julien@xen.org>
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index b398d480f113..00f9ebe9cee0 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -103,13 +103,6 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry,
printk(XENLOG_INFO "CPU%u will %s on exception entry\n",
smp_processor_id(), desc);
- /*
- * No need to install hardened vector when the processor has
- * ID_AA64PRF0_EL1.CSV2 set.
- */
- if ( cpu_data[smp_processor_id()].pfr64.csv2 )
- return true;
-
spin_lock(&bp_lock);
/*
@@ -167,6 +160,13 @@ static int enable_smccc_arch_workaround_1(void *data)
if ( !entry->matches(entry) )
return 0;
+ /*
+ * No need to install hardened vector when the processor has
+ * ID_AA64PRF0_EL1.CSV2 set.
+ */
+ if ( cpu_data[smp_processor_id()].pfr64.csv2 )
+ return 0;
+
if ( smccc_ver < SMCCC_VERSION(1, 1) )
goto warn;
["xsa398/xsa398-3-xen-arm-Add-ECBHB-and-CLEARBHB-ID-fields.patch" (application/octet-stream)]
From 4b68d12d98b8790d8002fcc2c25a9d713374a4d7 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Wed, 23 Feb 2022 09:42:18 +0000
Subject: xen/arm: Add ECBHB and CLEARBHB ID fields
Introduce ID coprocessor register ID_AA64ISAR2_EL1.
Add definitions in cpufeature and sysregs of ECBHB field in mmfr1 and
CLEARBHB in isar2 ID coprocessor registers.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <julien@xen.org>
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 6e51f530a80e..a58965f7b9bf 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -122,6 +122,7 @@ void identify_cpu(struct cpuinfo_arm *c)
c->isa64.bits[0] = READ_SYSREG(ID_AA64ISAR0_EL1);
c->isa64.bits[1] = READ_SYSREG(ID_AA64ISAR1_EL1);
+ c->isa64.bits[2] = READ_SYSREG(ID_AA64ISAR2_EL1);
c->zfr64.bits[0] = READ_SYSREG(ID_AA64ZFR0_EL1);
diff --git a/xen/arch/arm/include/asm/arm64/sysregs.h b/xen/arch/arm/include/asm/arm64/sysregs.h
index d7e4772f217f..eac08ed33f53 100644
--- a/xen/arch/arm/include/asm/arm64/sysregs.h
+++ b/xen/arch/arm/include/asm/arm64/sysregs.h
@@ -84,6 +84,9 @@
#ifndef ID_DFR1_EL1
#define ID_DFR1_EL1 S3_0_C0_C3_5
#endif
+#ifndef ID_AA64ISAR2_EL1
+#define ID_AA64ISAR2_EL1 S3_0_C0_C6_2
+#endif
/* ID registers (imported from arm64/include/asm/sysreg.h in Linux) */
@@ -139,6 +142,9 @@
#define ID_AA64ISAR1_GPI_NI 0x0
#define ID_AA64ISAR1_GPI_IMP_DEF 0x1
+/* id_aa64isar2 */
+#define ID_AA64ISAR2_CLEARBHB_SHIFT 28
+
/* id_aa64pfr0 */
#define ID_AA64PFR0_CSV3_SHIFT 60
#define ID_AA64PFR0_CSV2_SHIFT 56
@@ -232,6 +238,7 @@
#define ID_AA64MMFR0_PARANGE_52 0x6
/* id_aa64mmfr1 */
+#define ID_AA64MMFR1_ECBHB_SHIFT 60
#define ID_AA64MMFR1_ETS_SHIFT 36
#define ID_AA64MMFR1_TWED_SHIFT 32
#define ID_AA64MMFR1_XNX_SHIFT 28
diff --git a/xen/arch/arm/include/asm/cpufeature.h b/xen/arch/arm/include/asm/cpufeature.h
index 8a5afbaf0baf..db126508f159 100644
--- a/xen/arch/arm/include/asm/cpufeature.h
+++ b/xen/arch/arm/include/asm/cpufeature.h
@@ -243,14 +243,15 @@ struct cpuinfo_arm {
unsigned long lo:4;
unsigned long pan:4;
unsigned long __res1:8;
- unsigned long __res2:32;
+ unsigned long __res2:28;
+ unsigned long ecbhb:4;
unsigned long __res3:64;
};
} mm64;
union {
- register_t bits[2];
+ register_t bits[3];
struct {
/* ISAR0 */
unsigned long __res0:4;
@@ -286,6 +287,12 @@ struct cpuinfo_arm {
unsigned long dgh:4;
unsigned long i8mm:4;
unsigned long __res2:8;
+
+ /* ISAR2 */
+ unsigned long __res3:28;
+ unsigned long clearbhb:4;
+
+ unsigned long __res4:32;
};
} isa64;
["xsa398/xsa398-4.12-1-xen-arm-Introduce-new-Arm-processors.patch" (application/octet-stream)]
From f1346b2cfdbeb468b50be7b6f7aa38ce3c1acf2a Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Tue, 15 Feb 2022 10:37:51 +0000
Subject: xen/arm: Introduce new Arm processors
Add some new processor identifiers in processor.h and sync Xen
definitions with status of Linux 5.17 (declared in
arch/arm64/include/asm/cputype.h).
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 35d1b85a6b43483f6bd007d48757434e54743e98)
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index 0f35ec59d15e..cd45fba9786f 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -48,23 +48,43 @@
#define ARM_CPU_PART_CORTEX_A17 0xC0E
#define ARM_CPU_PART_CORTEX_A15 0xC0F
#define ARM_CPU_PART_CORTEX_A53 0xD03
+#define ARM_CPU_PART_CORTEX_A35 0xD04
+#define ARM_CPU_PART_CORTEX_A55 0xD05
#define ARM_CPU_PART_CORTEX_A57 0xD07
#define ARM_CPU_PART_CORTEX_A72 0xD08
#define ARM_CPU_PART_CORTEX_A73 0xD09
#define ARM_CPU_PART_CORTEX_A75 0xD0A
#define ARM_CPU_PART_CORTEX_A76 0xD0B
#define ARM_CPU_PART_NEOVERSE_N1 0xD0C
+#define ARM_CPU_PART_CORTEX_A77 0xD0D
+#define ARM_CPU_PART_NEOVERSE_V1 0xD40
+#define ARM_CPU_PART_CORTEX_A78 0xD41
+#define ARM_CPU_PART_CORTEX_X1 0xD44
+#define ARM_CPU_PART_CORTEX_A710 0xD47
+#define ARM_CPU_PART_CORTEX_X2 0xD48
+#define ARM_CPU_PART_NEOVERSE_N2 0xD49
+#define ARM_CPU_PART_CORTEX_A78C 0xD4B
#define MIDR_CORTEX_A12 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A12)
#define MIDR_CORTEX_A17 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A17)
#define MIDR_CORTEX_A15 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A15)
#define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
+#define MIDR_CORTEX_A35 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A35)
+#define MIDR_CORTEX_A55 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A55)
#define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57)
#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72)
#define MIDR_CORTEX_A73 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A73)
#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75)
#define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A76)
#define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N1)
+#define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A77)
+#define MIDR_NEOVERSE_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V1)
+#define MIDR_CORTEX_A78 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78)
+#define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1)
+#define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710)
+#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2)
+#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)
+#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
/* MPIDR Multiprocessor Affinity Register */
#define _MPIDR_UP (30)
["xsa398/xsa398-4.12-2-xen-arm-move-errata-CSV2-check-earlier.patch" (application/octet-stream)]
From 35164a1704fe13e1f83dbd4b5b79838f07d564c6 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Tue, 15 Feb 2022 10:39:47 +0000
Subject: xen/arm: move errata CSV2 check earlier
CSV2 availability check is done after printing to the user that
workaround 1 will be used. Move the check before to prevent saying to the
user that workaround 1 is used when it is not because it is not needed.
This will also allow to reuse install_bp_hardening_vec function for
other use cases.
Code previously returning "true", now returns "0" to conform to
enable_smccc_arch_workaround_1 returning an int and surrounding code
doing a "return 0" if workaround is not needed.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <julien@xen.org>
(cherry picked from commit 599616d70eb886b9ad0ef9d6b51693ce790504ba)
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index b254b9865783..9e1ecd071470 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -102,13 +102,6 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry,
printk(XENLOG_INFO "CPU%u will %s on exception entry\n",
smp_processor_id(), desc);
- /*
- * No need to install hardened vector when the processor has
- * ID_AA64PRF0_EL1.CSV2 set.
- */
- if ( cpu_data[smp_processor_id()].pfr64.csv2 )
- return true;
-
spin_lock(&bp_lock);
/*
@@ -167,6 +160,13 @@ static int enable_smccc_arch_workaround_1(void *data)
if ( !entry->matches(entry) )
return 0;
+ /*
+ * No need to install hardened vector when the processor has
+ * ID_AA64PRF0_EL1.CSV2 set.
+ */
+ if ( cpu_data[smp_processor_id()].pfr64.csv2 )
+ return 0;
+
if ( smccc_ver < SMCCC_VERSION(1, 1) )
goto warn;
["xsa398/xsa398-4.12-3-xen-arm-Add-ECBHB-and-CLEARBHB-ID-fields.patch" (application/octet-stream)]
From 2e519fd8c1e3e7ae5370a6638615d2a52169db28 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Wed, 23 Feb 2022 09:42:18 +0000
Subject: xen/arm: Add ECBHB and CLEARBHB ID fields
Introduce ID coprocessor register ID_AA64ISAR2_EL1.
Add definitions in cpufeature and sysregs of ECBHB field in mmfr1 and
CLEARBHB in isar2 ID coprocessor registers.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 4b68d12d98b8790d8002fcc2c25a9d713374a4d7)
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 44126dbf0723..13dac7ccaf94 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -117,6 +117,7 @@ void identify_cpu(struct cpuinfo_arm *c)
c->isa64.bits[0] = READ_SYSREG64(ID_AA64ISAR0_EL1);
c->isa64.bits[1] = READ_SYSREG64(ID_AA64ISAR1_EL1);
+ c->isa64.bits[2] = READ_SYSREG64(ID_AA64ISAR2_EL1);
#endif
c->pfr32.bits[0] = READ_SYSREG32(ID_PFR0_EL1);
diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
index 08585a969ebd..5f1e9b998f33 100644
--- a/xen/include/asm-arm/arm64/sysregs.h
+++ b/xen/include/asm-arm/arm64/sysregs.h
@@ -57,6 +57,10 @@
#define ICH_AP1R2_EL2 __AP1Rx_EL2(2)
#define ICH_AP1R3_EL2 __AP1Rx_EL2(3)
+#ifndef ID_AA64ISAR2_EL1
+#define ID_AA64ISAR2_EL1 S3_0_C0_C6_2
+#endif
+
/* Access to system registers */
#define READ_SYSREG32(name) ({ \
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 60e677d84200..c748fc17fe66 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -182,12 +182,26 @@ struct cpuinfo_arm {
unsigned long lo:4;
unsigned long pan:4;
unsigned long __res1:8;
- unsigned long __res2:32;
+ unsigned long __res2:28;
+ unsigned long ecbhb:4;
};
} mm64;
- struct {
- uint64_t bits[2];
+ union {
+ uint64_t bits[3];
+ struct {
+ /* ISAR0 */
+ unsigned long __res0:64;
+
+ /* ISAR1 */
+ unsigned long __res1:64;
+
+ /* ISAR2 */
+ unsigned long __res3:28;
+ unsigned long clearbhb:4;
+
+ unsigned long __res4:32;
+ };
} isa64;
#endif
["xsa398/xsa398-4.12-4-xen-arm-Add-Spectre-BHB-handling.patch" (application/octet-stream)]
From d340fad8be324e1760ea29d7c25658a8aec83306 Mon Sep 17 00:00:00 2001
From: Rahul Singh <rahul.singh@arm.com>
Date: Mon, 14 Feb 2022 18:47:32 +0000
Subject: xen/arm: Add Spectre BHB handling
This commit is adding Spectre BHB handling to Xen on Arm.
The commit is introducing new alternative code to be executed during
exception entry:
- SMCC workaround 3 call
- loop workaround (with 8, 24 or 32 iterations)
- use of new clearbhb instruction
Cpuerrata is modified by this patch to apply the required workaround for
CPU affected by Spectre BHB when CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR is
enabled.
To do this the system previously used to apply smcc workaround 1 is
reused and new alternative code to be copied in the exception handler is
introduced.
To define the type of workaround required by a processor, 4 new cpu
capabilities are introduced (for each number of loop and for smcc
workaround 3).
When a processor is affected, enable_spectre_bhb_workaround is called
and if the processor does not have CSV2 set to 3 or ECBHB feature (which
would mean that the processor is doing what is required in hardware),
the proper code is enabled at exception entry.
In the case where workaround 3 is not supported by the firmware, we
enable workaround 1 when possible as it will also mitigate Spectre BHB
on systems without CSV2.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 62c91eb66a2904eefb1d1d9642e3697a1e3c3a3c)
diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
index d8743d955c4a..4e6382522048 100644
--- a/xen/arch/arm/arm64/bpi.S
+++ b/xen/arch/arm/arm64/bpi.S
@@ -58,16 +58,42 @@ ENTRY(__bp_harden_hyp_vecs_start)
.endr
ENTRY(__bp_harden_hyp_vecs_end)
-ENTRY(__smccc_workaround_1_smc_start)
+.macro mitigate_spectre_bhb_loop count
+ENTRY(__mitigate_spectre_bhb_loop_start_\count)
+ stp x0, x1, [sp, #-16]!
+ mov x0, \count
+.Lspectre_bhb_loop\@:
+ b . + 4
+ subs x0, x0, #1
+ b.ne .Lspectre_bhb_loop\@
+ sb
+ ldp x0, x1, [sp], #16
+ENTRY(__mitigate_spectre_bhb_loop_end_\count)
+.endm
+
+.macro smccc_workaround num smcc_id
+ENTRY(__smccc_workaround_smc_start_\num)
sub sp, sp, #(8 * 4)
stp x0, x1, [sp, #(8 * 2)]
stp x2, x3, [sp, #(8 * 0)]
- mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
+ mov w0, \smcc_id
smc #0
ldp x2, x3, [sp, #(8 * 0)]
ldp x0, x1, [sp, #(8 * 2)]
add sp, sp, #(8 * 4)
-ENTRY(__smccc_workaround_1_smc_end)
+ENTRY(__smccc_workaround_smc_end_\num)
+.endm
+
+ENTRY(__mitigate_spectre_bhb_clear_insn_start)
+ clearbhb
+ isb
+ENTRY(__mitigate_spectre_bhb_clear_insn_end)
+
+mitigate_spectre_bhb_loop 8
+mitigate_spectre_bhb_loop 24
+mitigate_spectre_bhb_loop 32
+smccc_workaround 1, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
+smccc_workaround 3, #ARM_SMCCC_ARCH_WORKAROUND_3_FID
/*
* Local variables:
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index 9e1ecd071470..d70d1e16e946 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -144,7 +144,16 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry,
return ret;
}
-extern char __smccc_workaround_1_smc_start[], __smccc_workaround_1_smc_end[];
+extern char __smccc_workaround_smc_start_1[], __smccc_workaround_smc_end_1[];
+extern char __smccc_workaround_smc_start_3[], __smccc_workaround_smc_end_3[];
+extern char __mitigate_spectre_bhb_clear_insn_start[],
+ __mitigate_spectre_bhb_clear_insn_end[];
+extern char __mitigate_spectre_bhb_loop_start_8[],
+ __mitigate_spectre_bhb_loop_end_8[];
+extern char __mitigate_spectre_bhb_loop_start_24[],
+ __mitigate_spectre_bhb_loop_end_24[];
+extern char __mitigate_spectre_bhb_loop_start_32[],
+ __mitigate_spectre_bhb_loop_end_32[];
static int enable_smccc_arch_workaround_1(void *data)
{
@@ -176,8 +185,8 @@ static int enable_smccc_arch_workaround_1(void *data)
if ( (int)res.a0 < 0 )
goto warn;
- return !install_bp_hardening_vec(entry,__smccc_workaround_1_smc_start,
- __smccc_workaround_1_smc_end,
+ return !install_bp_hardening_vec(entry,__smccc_workaround_smc_start_1,
+ __smccc_workaround_smc_end_1,
"call ARM_SMCCC_ARCH_WORKAROUND_1");
warn:
@@ -192,6 +201,93 @@ static int enable_smccc_arch_workaround_1(void *data)
return 0;
}
+/*
+ * Spectre BHB Mitigation
+ *
+ * CPU is either:
+ * - Having CVS2.3 so it is not affected.
+ * - Having ECBHB and is clearing the branch history buffer when an exception
+ * to a different exception level is happening so no mitigation is needed.
+ * - Mitigating using a loop on exception entry (number of loop depending on
+ * the CPU).
+ * - Mitigating using the firmware.
+ */
+static int enable_spectre_bhb_workaround(void *data)
+{
+ const struct arm_cpu_capabilities *entry = data;
+
+ /*
+ * Enable callbacks are called on every CPU based on the capabilities, so
+ * double-check whether the CPU matches the entry.
+ */
+ if ( !entry->matches(entry) )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].pfr64.csv2 == 3 )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].mm64.ecbhb )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].isa64.clearbhb )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_clear_insn_start,
+ __mitigate_spectre_bhb_clear_insn_end,
+ "use clearBHB instruction");
+
+ /* Apply solution depending on hwcaps set on arm_errata */
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_8) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_8,
+ __mitigate_spectre_bhb_loop_end_8,
+ "use 8 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_24) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_24,
+ __mitigate_spectre_bhb_loop_end_24,
+ "use 24 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_32) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_32,
+ __mitigate_spectre_bhb_loop_end_32,
+ "use 32 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_SMCC_3) )
+ {
+ struct arm_smccc_res res;
+
+ if ( smccc_ver < SMCCC_VERSION(1, 1) )
+ goto warn;
+
+ arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FID,
+ ARM_SMCCC_ARCH_WORKAROUND_3_FID, &res);
+ /* The return value is in the lower 32-bits. */
+ if ( (int)res.a0 < 0 )
+ {
+ /*
+ * On processor affected with CSV2=0, workaround 1 will mitigate
+ * both Spectre v2 and BHB so use it when available
+ */
+ if ( enable_smccc_arch_workaround_1(data) )
+ return 1;
+
+ goto warn;
+ }
+
+ return !install_bp_hardening_vec(entry,__smccc_workaround_smc_start_3,
+ __smccc_workaround_smc_end_3,
+ "call ARM_SMCCC_ARCH_WORKAROUND_3");
+ }
+
+warn:
+ printk_once("**** No support for any spectre BHB workaround. ****\n"
+ "**** Please update your firmware. ****\n");
+
+ return 0;
+}
+
#endif /* CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR */
/* Hardening Branch predictor code for Arm32 */
@@ -451,19 +547,77 @@ static const struct arm_cpu_capabilities arm_errata[] = {
},
{
.capability = ARM_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ MIDR_RANGE(MIDR_CORTEX_A72, 0, 1 << MIDR_VARIANT_SHIFT),
.enable = enable_smccc_arch_workaround_1,
},
{
- .capability = ARM_HARDEN_BRANCH_PREDICTOR,
+ .capability = ARM_WORKAROUND_BHB_SMCC_3,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .enable = enable_smccc_arch_workaround_1,
+ .enable = enable_spectre_bhb_workaround,
},
{
- .capability = ARM_HARDEN_BRANCH_PREDICTOR,
+ .capability = ARM_WORKAROUND_BHB_SMCC_3,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- .enable = enable_smccc_arch_workaround_1,
+ .enable = enable_spectre_bhb_workaround,
+ },
+ /* spectre BHB */
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_8,
+ MIDR_RANGE(MIDR_CORTEX_A72, 1 << MIDR_VARIANT_SHIFT,
+ (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A76),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A77),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_X1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_X2),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
+ .enable = enable_spectre_bhb_workaround,
},
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+
#endif
#ifdef CONFIG_ARM32_HARDEN_BRANCH_PREDICTOR
{
diff --git a/xen/include/asm-arm/arm64/macros.h b/xen/include/asm-arm/arm64/macros.h
index 9c5e676b3773..a13ad8e2b133 100644
--- a/xen/include/asm-arm/arm64/macros.h
+++ b/xen/include/asm-arm/arm64/macros.h
@@ -21,5 +21,10 @@
ldr \dst, [\dst, \tmp]
.endm
+ /* clearbhb instruction clearing the branch history */
+ .macro clearbhb
+ hint #22
+ .endm
+
#endif /* __ASM_ARM_ARM64_MACROS_H */
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index c748fc17fe66..87989eac6fc2 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -45,8 +45,12 @@
#define ARM_SSBD 7
#define ARM_SMCCC_1_1 8
#define ARM64_WORKAROUND_AT_SPECULATE 9
+#define ARM_WORKAROUND_BHB_LOOP_8 10
+#define ARM_WORKAROUND_BHB_LOOP_24 11
+#define ARM_WORKAROUND_BHB_LOOP_32 12
+#define ARM_WORKAROUND_BHB_SMCC_3 13
-#define ARM_NCAPS 10
+#define ARM_NCAPS 14
#ifndef __ASSEMBLY__
diff --git a/xen/include/asm-arm/smccc.h b/xen/include/asm-arm/smccc.h
index 126399dd7088..2abbffc3bd8a 100644
--- a/xen/include/asm-arm/smccc.h
+++ b/xen/include/asm-arm/smccc.h
@@ -334,6 +334,12 @@ void __arm_smccc_1_0_smc(register_t a0, register_t a1, register_t a2,
ARM_SMCCC_OWNER_ARCH, \
0x7FFF)
+#define ARM_SMCCC_ARCH_WORKAROUND_3_FID \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
+ ARM_SMCCC_CONV_32, \
+ ARM_SMCCC_OWNER_ARCH, \
+ 0x3FFF)
+
/* SMCCC error codes */
#define ARM_SMCCC_NOT_REQUIRED (-2)
#define ARM_SMCCC_ERR_UNKNOWN_FUNCTION (-1)
["xsa398/xsa398-4.12-5-xen-arm-Allow-to-discover-and-use-SMCCC_ARCH_WORKARO.patch" (application/octet-stream)]
From 21f5a7b22687aa1e384782c8a1c04148f288ad9f Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Thu, 17 Feb 2022 14:52:54 +0000
Subject: xen/arm: Allow to discover and use SMCCC_ARCH_WORKAROUND_3
Allow guest to discover whether or not SMCCC_ARCH_WORKAROUND_3 is
supported and create a fastpath in the code to handle guests request to
do the workaround.
The function SMCCC_ARCH_WORKAROUND_3 will be called by the guest for
flushing the branch history. So we want the handling to be as fast as
possible.
As the mitigation is applied on every guest exit, we can check for the
call before saving all context and return very early.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <julien@xen.org>
(cherry picked from commit c0a56ea0fd92ecb471936b7355ddbecbaea3707c)
diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
index 97bd06217bcd..788d0a1912f0 100644
--- a/xen/arch/arm/arm64/entry.S
+++ b/xen/arch/arm/arm64/entry.S
@@ -343,16 +343,26 @@ guest_sync:
cbnz x1, guest_sync_slowpath /* should be 0 for HVC #0 */
/*
- * Fastest path possible for ARM_SMCCC_ARCH_WORKAROUND_1.
- * The workaround has already been applied on the exception
+ * Fastest path possible for ARM_SMCCC_ARCH_WORKAROUND_1 and
+ * ARM_SMCCC_ARCH_WORKAROUND_3.
+ * The workaround needed has already been applied on the exception
* entry from the guest, so let's quickly get back to the guest.
*
* Note that eor is used because the function identifier cannot
* be encoded as an immediate for cmp.
*/
eor w0, w0, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
- cbnz w0, check_wa2
+ cbz w0, fastpath_out_workaround
+ /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */
+ eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_1_FID ^ ARM_SMCCC_ARCH_WORKAROUND_2_FID)
+ cbz w0, wa2_ssbd
+
+ /* Fastpath out for ARM_SMCCC_ARCH_WORKAROUND_3 */
+ eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_2_FID ^ ARM_SMCCC_ARCH_WORKAROUND_3_FID)
+ cbnz w0, guest_sync_slowpath
+
+fastpath_out_workaround:
/*
* Clobber both x0 and x1 to prevent leakage. Note that thanks
* the eor, x0 = 0.
@@ -361,10 +371,7 @@ guest_sync:
eret
sb
-check_wa2:
- /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */
- eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_1_FID ^ ARM_SMCCC_ARCH_WORKAROUND_2_FID)
- cbnz w0, guest_sync_slowpath
+wa2_ssbd:
#ifdef CONFIG_ARM_SSBD
alternative_cb arm_enable_wa2_handling
b wa2_end
diff --git a/xen/arch/arm/vsmc.c b/xen/arch/arm/vsmc.c
index ecf4faa13da3..643976db6537 100644
--- a/xen/arch/arm/vsmc.c
+++ b/xen/arch/arm/vsmc.c
@@ -123,6 +123,10 @@ static bool handle_arch(struct cpu_user_regs *regs)
break;
}
break;
+ case ARM_SMCCC_ARCH_WORKAROUND_3_FID:
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_SMCC_3) )
+ ret = 0;
+ break;
}
set_user_reg(regs, 0, ret);
@@ -131,6 +135,7 @@ static bool handle_arch(struct cpu_user_regs *regs)
}
case ARM_SMCCC_ARCH_WORKAROUND_1_FID:
+ case ARM_SMCCC_ARCH_WORKAROUND_3_FID:
/* No return value */
return true;
["xsa398/xsa398-4.12-6-x86-spec-ctrl-Cease-using-thunk-lfence-on-AMD.patch" (application/octet-stream)]
From 944afa38d9339a67f0164d07fb7ac8a54e9a4c60 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 7 Mar 2022 16:35:52 +0000
Subject: x86/spec-ctrl: Cease using thunk=lfence on AMD
AMD have updated their Spectre v2 guidance, and lfence/jmp is no longer
considered safe. AMD are recommending using retpoline everywhere.
Update the default heuristics to never select THUNK_LFENCE.
This is part of XSA-398 / CVE-2021-26401.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 8d03080d2a339840d3a59e0932a94f804e45110d)
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index d7919f2ed46c..f9fb1621382a 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1960,9 +1960,9 @@ to use.
If Xen was compiled with INDIRECT_THUNK support, `bti-thunk=` can be used to
select which of the thunks gets patched into the `__x86_indirect_thunk_%reg`
-locations. The default thunk is `retpoline` (generally preferred for Intel
-hardware), with the alternatives being `jmp` (a `jmp *%reg` gadget, minimal
-overhead), and `lfence` (an `lfence; jmp *%reg` gadget, preferred for AMD).
+locations. The default thunk is `retpoline` (generally preferred), with the
+alternatives being `jmp` (a `jmp *%reg` gadget, minimal overhead), and
+`lfence` (an `lfence; jmp *%reg` gadget).
On hardware supporting IBRS (Indirect Branch Restricted Speculation), the
`ibrs=` option can be used to force or prevent Xen using the feature itself.
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index e2fcefc86a60..866b864918fd 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -904,16 +904,10 @@ void __init init_speculation_mitigations(void)
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
{
/*
- * AMD's recommended mitigation is to set lfence as being dispatch
- * serialising, and to use IND_THUNK_LFENCE.
- */
- if ( cpu_has_lfence_dispatch )
- thunk = THUNK_LFENCE;
- /*
- * On Intel hardware, we'd like to use retpoline in preference to
+ * On all hardware, we'd like to use retpoline in preference to
* IBRS, but only if it is safe on this hardware.
*/
- else if ( retpoline_safe(caps) )
+ if ( retpoline_safe(caps) )
thunk = THUNK_RETPOLINE;
else if ( boot_cpu_has(X86_FEATURE_IBRSB) )
ibrs = true;
["xsa398/xsa398-4.13-1-xen-arm-Introduce-new-Arm-processors.patch" (application/octet-stream)]
From 9a8804a92fed77f77afe9fc525c6891bb60f68d3 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Tue, 15 Feb 2022 10:37:51 +0000
Subject: xen/arm: Introduce new Arm processors
Add some new processor identifiers in processor.h and sync Xen
definitions with status of Linux 5.17 (declared in
arch/arm64/include/asm/cputype.h).
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 35d1b85a6b43483f6bd007d48757434e54743e98)
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index 87c8136022df..17cc5cf486f9 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -53,6 +53,7 @@
#define ARM_CPU_PART_CORTEX_A17 0xC0E
#define ARM_CPU_PART_CORTEX_A15 0xC0F
#define ARM_CPU_PART_CORTEX_A53 0xD03
+#define ARM_CPU_PART_CORTEX_A35 0xD04
#define ARM_CPU_PART_CORTEX_A55 0xD05
#define ARM_CPU_PART_CORTEX_A57 0xD07
#define ARM_CPU_PART_CORTEX_A72 0xD08
@@ -60,11 +61,20 @@
#define ARM_CPU_PART_CORTEX_A75 0xD0A
#define ARM_CPU_PART_CORTEX_A76 0xD0B
#define ARM_CPU_PART_NEOVERSE_N1 0xD0C
+#define ARM_CPU_PART_CORTEX_A77 0xD0D
+#define ARM_CPU_PART_NEOVERSE_V1 0xD40
+#define ARM_CPU_PART_CORTEX_A78 0xD41
+#define ARM_CPU_PART_CORTEX_X1 0xD44
+#define ARM_CPU_PART_CORTEX_A710 0xD47
+#define ARM_CPU_PART_CORTEX_X2 0xD48
+#define ARM_CPU_PART_NEOVERSE_N2 0xD49
+#define ARM_CPU_PART_CORTEX_A78C 0xD4B
#define MIDR_CORTEX_A12 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A12)
#define MIDR_CORTEX_A17 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A17)
#define MIDR_CORTEX_A15 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A15)
#define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
+#define MIDR_CORTEX_A35 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A35)
#define MIDR_CORTEX_A55 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A55)
#define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57)
#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72)
@@ -72,6 +82,14 @@
#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75)
#define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A76)
#define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N1)
+#define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A77)
+#define MIDR_NEOVERSE_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V1)
+#define MIDR_CORTEX_A78 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78)
+#define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1)
+#define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710)
+#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2)
+#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)
+#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
/* MPIDR Multiprocessor Affinity Register */
#define _MPIDR_UP (30)
["xsa398/xsa398-4.13-2-xen-arm-move-errata-CSV2-check-earlier.patch" (application/octet-stream)]
From 03db21387b8653d663e8da89c964d611ba509130 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Tue, 15 Feb 2022 10:39:47 +0000
Subject: xen/arm: move errata CSV2 check earlier
CSV2 availability check is done after printing to the user that
workaround 1 will be used. Move the check before to prevent saying to the
user that workaround 1 is used when it is not because it is not needed.
This will also allow to reuse install_bp_hardening_vec function for
other use cases.
Code previously returning "true", now returns "0" to conform to
enable_smccc_arch_workaround_1 returning an int and surrounding code
doing a "return 0" if workaround is not needed.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <julien@xen.org>
(cherry picked from commit 599616d70eb886b9ad0ef9d6b51693ce790504ba)
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index f94bcf74ccec..79620889b49e 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -102,13 +102,6 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry,
printk(XENLOG_INFO "CPU%u will %s on exception entry\n",
smp_processor_id(), desc);
- /*
- * No need to install hardened vector when the processor has
- * ID_AA64PRF0_EL1.CSV2 set.
- */
- if ( cpu_data[smp_processor_id()].pfr64.csv2 )
- return true;
-
spin_lock(&bp_lock);
/*
@@ -167,6 +160,13 @@ static int enable_smccc_arch_workaround_1(void *data)
if ( !entry->matches(entry) )
return 0;
+ /*
+ * No need to install hardened vector when the processor has
+ * ID_AA64PRF0_EL1.CSV2 set.
+ */
+ if ( cpu_data[smp_processor_id()].pfr64.csv2 )
+ return 0;
+
if ( smccc_ver < SMCCC_VERSION(1, 1) )
goto warn;
["xsa398/xsa398-4.13-3-xen-arm-Add-ECBHB-and-CLEARBHB-ID-fields.patch" (application/octet-stream)]
From d99df7d50d366c7a8dc71f5bdc3454f469b00a00 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Wed, 23 Feb 2022 09:42:18 +0000
Subject: xen/arm: Add ECBHB and CLEARBHB ID fields
Introduce ID coprocessor register ID_AA64ISAR2_EL1.
Add definitions in cpufeature and sysregs of ECBHB field in mmfr1 and
CLEARBHB in isar2 ID coprocessor registers.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 4b68d12d98b8790d8002fcc2c25a9d713374a4d7)
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 44126dbf0723..13dac7ccaf94 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -117,6 +117,7 @@ void identify_cpu(struct cpuinfo_arm *c)
c->isa64.bits[0] = READ_SYSREG64(ID_AA64ISAR0_EL1);
c->isa64.bits[1] = READ_SYSREG64(ID_AA64ISAR1_EL1);
+ c->isa64.bits[2] = READ_SYSREG64(ID_AA64ISAR2_EL1);
#endif
c->pfr32.bits[0] = READ_SYSREG32(ID_PFR0_EL1);
diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
index c60029d38f5b..cfd2e1d48699 100644
--- a/xen/include/asm-arm/arm64/sysregs.h
+++ b/xen/include/asm-arm/arm64/sysregs.h
@@ -57,6 +57,10 @@
#define ICH_AP1R2_EL2 __AP1Rx_EL2(2)
#define ICH_AP1R3_EL2 __AP1Rx_EL2(3)
+#ifndef ID_AA64ISAR2_EL1
+#define ID_AA64ISAR2_EL1 S3_0_C0_C6_2
+#endif
+
/* Access to system registers */
#define READ_SYSREG32(name) ((uint32_t)READ_SYSREG64(name))
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 29753fee7831..8519d2987bde 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -183,12 +183,26 @@ struct cpuinfo_arm {
unsigned long lo:4;
unsigned long pan:4;
unsigned long __res1:8;
- unsigned long __res2:32;
+ unsigned long __res2:28;
+ unsigned long ecbhb:4;
};
} mm64;
- struct {
- uint64_t bits[2];
+ union {
+ uint64_t bits[3];
+ struct {
+ /* ISAR0 */
+ unsigned long __res0:64;
+
+ /* ISAR1 */
+ unsigned long __res1:64;
+
+ /* ISAR2 */
+ unsigned long __res3:28;
+ unsigned long clearbhb:4;
+
+ unsigned long __res4:32;
+ };
} isa64;
#endif
["xsa398/xsa398-4.13-4-xen-arm-Add-Spectre-BHB-handling.patch" (application/octet-stream)]
From 47125f5fb2073abb9d5d3f65824cd066e7ec62f1 Mon Sep 17 00:00:00 2001
From: Rahul Singh <rahul.singh@arm.com>
Date: Mon, 14 Feb 2022 18:47:32 +0000
Subject: xen/arm: Add Spectre BHB handling
This commit is adding Spectre BHB handling to Xen on Arm.
The commit is introducing new alternative code to be executed during
exception entry:
- SMCC workaround 3 call
- loop workaround (with 8, 24 or 32 iterations)
- use of new clearbhb instruction
Cpuerrata is modified by this patch to apply the required workaround for
CPU affected by Spectre BHB when CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR is
enabled.
To do this the system previously used to apply smcc workaround 1 is
reused and new alternative code to be copied in the exception handler is
introduced.
To define the type of workaround required by a processor, 4 new cpu
capabilities are introduced (for each number of loop and for smcc
workaround 3).
When a processor is affected, enable_spectre_bhb_workaround is called
and if the processor does not have CSV2 set to 3 or ECBHB feature (which
would mean that the processor is doing what is required in hardware),
the proper code is enabled at exception entry.
In the case where workaround 3 is not supported by the firmware, we
enable workaround 1 when possible as it will also mitigate Spectre BHB
on systems without CSV2.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 62c91eb66a2904eefb1d1d9642e3697a1e3c3a3c)
diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
index d8743d955c4a..4e6382522048 100644
--- a/xen/arch/arm/arm64/bpi.S
+++ b/xen/arch/arm/arm64/bpi.S
@@ -58,16 +58,42 @@ ENTRY(__bp_harden_hyp_vecs_start)
.endr
ENTRY(__bp_harden_hyp_vecs_end)
-ENTRY(__smccc_workaround_1_smc_start)
+.macro mitigate_spectre_bhb_loop count
+ENTRY(__mitigate_spectre_bhb_loop_start_\count)
+ stp x0, x1, [sp, #-16]!
+ mov x0, \count
+.Lspectre_bhb_loop\@:
+ b . + 4
+ subs x0, x0, #1
+ b.ne .Lspectre_bhb_loop\@
+ sb
+ ldp x0, x1, [sp], #16
+ENTRY(__mitigate_spectre_bhb_loop_end_\count)
+.endm
+
+.macro smccc_workaround num smcc_id
+ENTRY(__smccc_workaround_smc_start_\num)
sub sp, sp, #(8 * 4)
stp x0, x1, [sp, #(8 * 2)]
stp x2, x3, [sp, #(8 * 0)]
- mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
+ mov w0, \smcc_id
smc #0
ldp x2, x3, [sp, #(8 * 0)]
ldp x0, x1, [sp, #(8 * 2)]
add sp, sp, #(8 * 4)
-ENTRY(__smccc_workaround_1_smc_end)
+ENTRY(__smccc_workaround_smc_end_\num)
+.endm
+
+ENTRY(__mitigate_spectre_bhb_clear_insn_start)
+ clearbhb
+ isb
+ENTRY(__mitigate_spectre_bhb_clear_insn_end)
+
+mitigate_spectre_bhb_loop 8
+mitigate_spectre_bhb_loop 24
+mitigate_spectre_bhb_loop 32
+smccc_workaround 1, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
+smccc_workaround 3, #ARM_SMCCC_ARCH_WORKAROUND_3_FID
/*
* Local variables:
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index 79620889b49e..8d9e977b7737 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -144,7 +144,16 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry,
return ret;
}
-extern char __smccc_workaround_1_smc_start[], __smccc_workaround_1_smc_end[];
+extern char __smccc_workaround_smc_start_1[], __smccc_workaround_smc_end_1[];
+extern char __smccc_workaround_smc_start_3[], __smccc_workaround_smc_end_3[];
+extern char __mitigate_spectre_bhb_clear_insn_start[],
+ __mitigate_spectre_bhb_clear_insn_end[];
+extern char __mitigate_spectre_bhb_loop_start_8[],
+ __mitigate_spectre_bhb_loop_end_8[];
+extern char __mitigate_spectre_bhb_loop_start_24[],
+ __mitigate_spectre_bhb_loop_end_24[];
+extern char __mitigate_spectre_bhb_loop_start_32[],
+ __mitigate_spectre_bhb_loop_end_32[];
static int enable_smccc_arch_workaround_1(void *data)
{
@@ -176,8 +185,8 @@ static int enable_smccc_arch_workaround_1(void *data)
if ( (int)res.a0 < 0 )
goto warn;
- return !install_bp_hardening_vec(entry,__smccc_workaround_1_smc_start,
- __smccc_workaround_1_smc_end,
+ return !install_bp_hardening_vec(entry,__smccc_workaround_smc_start_1,
+ __smccc_workaround_smc_end_1,
"call ARM_SMCCC_ARCH_WORKAROUND_1");
warn:
@@ -192,6 +201,93 @@ static int enable_smccc_arch_workaround_1(void *data)
return 0;
}
+/*
+ * Spectre BHB Mitigation
+ *
+ * CPU is either:
+ * - Having CVS2.3 so it is not affected.
+ * - Having ECBHB and is clearing the branch history buffer when an exception
+ * to a different exception level is happening so no mitigation is needed.
+ * - Mitigating using a loop on exception entry (number of loop depending on
+ * the CPU).
+ * - Mitigating using the firmware.
+ */
+static int enable_spectre_bhb_workaround(void *data)
+{
+ const struct arm_cpu_capabilities *entry = data;
+
+ /*
+ * Enable callbacks are called on every CPU based on the capabilities, so
+ * double-check whether the CPU matches the entry.
+ */
+ if ( !entry->matches(entry) )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].pfr64.csv2 == 3 )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].mm64.ecbhb )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].isa64.clearbhb )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_clear_insn_start,
+ __mitigate_spectre_bhb_clear_insn_end,
+ "use clearBHB instruction");
+
+ /* Apply solution depending on hwcaps set on arm_errata */
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_8) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_8,
+ __mitigate_spectre_bhb_loop_end_8,
+ "use 8 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_24) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_24,
+ __mitigate_spectre_bhb_loop_end_24,
+ "use 24 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_32) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_32,
+ __mitigate_spectre_bhb_loop_end_32,
+ "use 32 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_SMCC_3) )
+ {
+ struct arm_smccc_res res;
+
+ if ( smccc_ver < SMCCC_VERSION(1, 1) )
+ goto warn;
+
+ arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FID,
+ ARM_SMCCC_ARCH_WORKAROUND_3_FID, &res);
+ /* The return value is in the lower 32-bits. */
+ if ( (int)res.a0 < 0 )
+ {
+ /*
+ * On processor affected with CSV2=0, workaround 1 will mitigate
+ * both Spectre v2 and BHB so use it when available
+ */
+ if ( enable_smccc_arch_workaround_1(data) )
+ return 1;
+
+ goto warn;
+ }
+
+ return !install_bp_hardening_vec(entry,__smccc_workaround_smc_start_3,
+ __smccc_workaround_smc_end_3,
+ "call ARM_SMCCC_ARCH_WORKAROUND_3");
+ }
+
+warn:
+ printk_once("**** No support for any spectre BHB workaround. ****\n"
+ "**** Please update your firmware. ****\n");
+
+ return 0;
+}
+
#endif /* CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR */
/* Hardening Branch predictor code for Arm32 */
@@ -437,19 +533,77 @@ static const struct arm_cpu_capabilities arm_errata[] = {
},
{
.capability = ARM_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ MIDR_RANGE(MIDR_CORTEX_A72, 0, 1 << MIDR_VARIANT_SHIFT),
.enable = enable_smccc_arch_workaround_1,
},
{
- .capability = ARM_HARDEN_BRANCH_PREDICTOR,
+ .capability = ARM_WORKAROUND_BHB_SMCC_3,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .enable = enable_smccc_arch_workaround_1,
+ .enable = enable_spectre_bhb_workaround,
},
{
- .capability = ARM_HARDEN_BRANCH_PREDICTOR,
+ .capability = ARM_WORKAROUND_BHB_SMCC_3,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- .enable = enable_smccc_arch_workaround_1,
+ .enable = enable_spectre_bhb_workaround,
+ },
+ /* spectre BHB */
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_8,
+ MIDR_RANGE(MIDR_CORTEX_A72, 1 << MIDR_VARIANT_SHIFT,
+ (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A76),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A77),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_X1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_X2),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
+ .enable = enable_spectre_bhb_workaround,
},
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+
#endif
#ifdef CONFIG_ARM32_HARDEN_BRANCH_PREDICTOR
{
diff --git a/xen/include/asm-arm/arm64/macros.h b/xen/include/asm-arm/arm64/macros.h
index f981b4f43e84..5100aed6e3ec 100644
--- a/xen/include/asm-arm/arm64/macros.h
+++ b/xen/include/asm-arm/arm64/macros.h
@@ -21,6 +21,11 @@
ldr \dst, [\dst, \tmp]
.endm
+ /* clearbhb instruction clearing the branch history */
+ .macro clearbhb
+ hint #22
+ .endm
+
/*
* Register aliases.
*/
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 8519d2987bde..a1fa3bc1cf58 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -46,8 +46,12 @@
#define ARM_SMCCC_1_1 8
#define ARM64_WORKAROUND_AT_SPECULATE 9
#define ARM_WORKAROUND_858921 10
+#define ARM_WORKAROUND_BHB_LOOP_8 11
+#define ARM_WORKAROUND_BHB_LOOP_24 12
+#define ARM_WORKAROUND_BHB_LOOP_32 13
+#define ARM_WORKAROUND_BHB_SMCC_3 14
-#define ARM_NCAPS 11
+#define ARM_NCAPS 15
#ifndef __ASSEMBLY__
diff --git a/xen/include/asm-arm/smccc.h b/xen/include/asm-arm/smccc.h
index 126399dd7088..2abbffc3bd8a 100644
--- a/xen/include/asm-arm/smccc.h
+++ b/xen/include/asm-arm/smccc.h
@@ -334,6 +334,12 @@ void __arm_smccc_1_0_smc(register_t a0, register_t a1, register_t a2,
ARM_SMCCC_OWNER_ARCH, \
0x7FFF)
+#define ARM_SMCCC_ARCH_WORKAROUND_3_FID \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
+ ARM_SMCCC_CONV_32, \
+ ARM_SMCCC_OWNER_ARCH, \
+ 0x3FFF)
+
/* SMCCC error codes */
#define ARM_SMCCC_NOT_REQUIRED (-2)
#define ARM_SMCCC_ERR_UNKNOWN_FUNCTION (-1)
["xsa398/xsa398-4.13-5-xen-arm-Allow-to-discover-and-use-SMCCC_ARCH_WORKARO.patch" (application/octet-stream)]
From fbabb62dd9e57180400f145a8756624c82de888f Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Thu, 17 Feb 2022 14:52:54 +0000
Subject: xen/arm: Allow to discover and use SMCCC_ARCH_WORKAROUND_3
Allow guest to discover whether or not SMCCC_ARCH_WORKAROUND_3 is
supported and create a fastpath in the code to handle guests request to
do the workaround.
The function SMCCC_ARCH_WORKAROUND_3 will be called by the guest for
flushing the branch history. So we want the handling to be as fast as
possible.
As the mitigation is applied on every guest exit, we can check for the
call before saving all context and return very early.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <julien@xen.org>
(cherry picked from commit c0a56ea0fd92ecb471936b7355ddbecbaea3707c)
diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
index 175ea2981e72..a8c214506786 100644
--- a/xen/arch/arm/arm64/entry.S
+++ b/xen/arch/arm/arm64/entry.S
@@ -338,16 +338,26 @@ guest_sync:
cbnz x1, guest_sync_slowpath /* should be 0 for HVC #0 */
/*
- * Fastest path possible for ARM_SMCCC_ARCH_WORKAROUND_1.
- * The workaround has already been applied on the exception
+ * Fastest path possible for ARM_SMCCC_ARCH_WORKAROUND_1 and
+ * ARM_SMCCC_ARCH_WORKAROUND_3.
+ * The workaround needed has already been applied on the exception
* entry from the guest, so let's quickly get back to the guest.
*
* Note that eor is used because the function identifier cannot
* be encoded as an immediate for cmp.
*/
eor w0, w0, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
- cbnz w0, check_wa2
+ cbz w0, fastpath_out_workaround
+ /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */
+ eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_1_FID ^ ARM_SMCCC_ARCH_WORKAROUND_2_FID)
+ cbz w0, wa2_ssbd
+
+ /* Fastpath out for ARM_SMCCC_ARCH_WORKAROUND_3 */
+ eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_2_FID ^ ARM_SMCCC_ARCH_WORKAROUND_3_FID)
+ cbnz w0, guest_sync_slowpath
+
+fastpath_out_workaround:
/*
* Clobber both x0 and x1 to prevent leakage. Note that thanks
* the eor, x0 = 0.
@@ -356,10 +366,7 @@ guest_sync:
eret
sb
-check_wa2:
- /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */
- eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_1_FID ^ ARM_SMCCC_ARCH_WORKAROUND_2_FID)
- cbnz w0, guest_sync_slowpath
+wa2_ssbd:
#ifdef CONFIG_ARM_SSBD
alternative_cb arm_enable_wa2_handling
b wa2_end
diff --git a/xen/arch/arm/vsmc.c b/xen/arch/arm/vsmc.c
index a36db15fffc0..b633ff2fe897 100644
--- a/xen/arch/arm/vsmc.c
+++ b/xen/arch/arm/vsmc.c
@@ -124,6 +124,10 @@ static bool handle_arch(struct cpu_user_regs *regs)
break;
}
break;
+ case ARM_SMCCC_ARCH_WORKAROUND_3_FID:
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_SMCC_3) )
+ ret = 0;
+ break;
}
set_user_reg(regs, 0, ret);
@@ -132,6 +136,7 @@ static bool handle_arch(struct cpu_user_regs *regs)
}
case ARM_SMCCC_ARCH_WORKAROUND_1_FID:
+ case ARM_SMCCC_ARCH_WORKAROUND_3_FID:
/* No return value */
return true;
["xsa398/xsa398-4.13-6-x86-spec-ctrl-Cease-using-thunk-lfence-on-AMD.patch" (application/octet-stream)]
From 7b9814b250a5a28277bd0866d341a5cfc0f4c1ac Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 7 Mar 2022 16:35:52 +0000
Subject: x86/spec-ctrl: Cease using thunk=lfence on AMD
AMD have updated their Spectre v2 guidance, and lfence/jmp is no longer
considered safe. AMD are recommending using retpoline everywhere.
Update the default heuristics to never select THUNK_LFENCE.
This is part of XSA-398 / CVE-2021-26401.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 8d03080d2a339840d3a59e0932a94f804e45110d)
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index cf9dea62dbfd..eead69ada2c2 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2077,9 +2077,9 @@ to use.
If Xen was compiled with INDIRECT_THUNK support, `bti-thunk=` can be used to
select which of the thunks gets patched into the `__x86_indirect_thunk_%reg`
-locations. The default thunk is `retpoline` (generally preferred for Intel
-hardware), with the alternatives being `jmp` (a `jmp *%reg` gadget, minimal
-overhead), and `lfence` (an `lfence; jmp *%reg` gadget, preferred for AMD).
+locations. The default thunk is `retpoline` (generally preferred), with the
+alternatives being `jmp` (a `jmp *%reg` gadget, minimal overhead), and
+`lfence` (an `lfence; jmp *%reg` gadget).
On hardware supporting IBRS (Indirect Branch Restricted Speculation), the
`ibrs=` option can be used to force or prevent Xen using the feature itself.
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 1cfd02d7d7cf..7447d4a8e5b5 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -908,16 +908,10 @@ void __init init_speculation_mitigations(void)
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
{
/*
- * AMD's recommended mitigation is to set lfence as being dispatch
- * serialising, and to use IND_THUNK_LFENCE.
- */
- if ( cpu_has_lfence_dispatch )
- thunk = THUNK_LFENCE;
- /*
- * On Intel hardware, we'd like to use retpoline in preference to
+ * On all hardware, we'd like to use retpoline in preference to
* IBRS, but only if it is safe on this hardware.
*/
- else if ( retpoline_safe(caps) )
+ if ( retpoline_safe(caps) )
thunk = THUNK_RETPOLINE;
else if ( boot_cpu_has(X86_FEATURE_IBRSB) )
ibrs = true;
["xsa398/xsa398-4.14-1-xen-arm-Introduce-new-Arm-processors.patch" (application/octet-stream)]
From 021466aa73caaa0c5983f02203678e649dd4d22c Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Tue, 15 Feb 2022 10:37:51 +0000
Subject: xen/arm: Introduce new Arm processors
Add some new processor identifiers in processor.h and sync Xen
definitions with status of Linux 5.17 (declared in
arch/arm64/include/asm/cputype.h).
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 35d1b85a6b43483f6bd007d48757434e54743e98)
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index 87c8136022df..17cc5cf486f9 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -53,6 +53,7 @@
#define ARM_CPU_PART_CORTEX_A17 0xC0E
#define ARM_CPU_PART_CORTEX_A15 0xC0F
#define ARM_CPU_PART_CORTEX_A53 0xD03
+#define ARM_CPU_PART_CORTEX_A35 0xD04
#define ARM_CPU_PART_CORTEX_A55 0xD05
#define ARM_CPU_PART_CORTEX_A57 0xD07
#define ARM_CPU_PART_CORTEX_A72 0xD08
@@ -60,11 +61,20 @@
#define ARM_CPU_PART_CORTEX_A75 0xD0A
#define ARM_CPU_PART_CORTEX_A76 0xD0B
#define ARM_CPU_PART_NEOVERSE_N1 0xD0C
+#define ARM_CPU_PART_CORTEX_A77 0xD0D
+#define ARM_CPU_PART_NEOVERSE_V1 0xD40
+#define ARM_CPU_PART_CORTEX_A78 0xD41
+#define ARM_CPU_PART_CORTEX_X1 0xD44
+#define ARM_CPU_PART_CORTEX_A710 0xD47
+#define ARM_CPU_PART_CORTEX_X2 0xD48
+#define ARM_CPU_PART_NEOVERSE_N2 0xD49
+#define ARM_CPU_PART_CORTEX_A78C 0xD4B
#define MIDR_CORTEX_A12 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A12)
#define MIDR_CORTEX_A17 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A17)
#define MIDR_CORTEX_A15 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A15)
#define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
+#define MIDR_CORTEX_A35 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A35)
#define MIDR_CORTEX_A55 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A55)
#define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57)
#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72)
@@ -72,6 +82,14 @@
#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75)
#define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A76)
#define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N1)
+#define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A77)
+#define MIDR_NEOVERSE_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V1)
+#define MIDR_CORTEX_A78 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78)
+#define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1)
+#define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710)
+#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2)
+#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)
+#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
/* MPIDR Multiprocessor Affinity Register */
#define _MPIDR_UP (30)
["xsa398/xsa398-4.14-2-xen-arm-move-errata-CSV2-check-earlier.patch" (application/octet-stream)]
From 6da7a845fb476ef7395185ec08a58c76ebd8c442 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Tue, 15 Feb 2022 10:39:47 +0000
Subject: xen/arm: move errata CSV2 check earlier
CSV2 availability check is done after printing to the user that
workaround 1 will be used. Move the check before to prevent saying to the
user that workaround 1 is used when it is not because it is not needed.
This will also allow to reuse install_bp_hardening_vec function for
other use cases.
Code previously returning "true", now returns "0" to conform to
enable_smccc_arch_workaround_1 returning an int and surrounding code
doing a "return 0" if workaround is not needed.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <julien@xen.org>
(cherry picked from commit 599616d70eb886b9ad0ef9d6b51693ce790504ba)
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index 66d9a1e45cf8..9d79e3bad7e8 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -103,13 +103,6 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry,
printk(XENLOG_INFO "CPU%u will %s on exception entry\n",
smp_processor_id(), desc);
- /*
- * No need to install hardened vector when the processor has
- * ID_AA64PRF0_EL1.CSV2 set.
- */
- if ( cpu_data[smp_processor_id()].pfr64.csv2 )
- return true;
-
spin_lock(&bp_lock);
/*
@@ -168,6 +161,13 @@ static int enable_smccc_arch_workaround_1(void *data)
if ( !entry->matches(entry) )
return 0;
+ /*
+ * No need to install hardened vector when the processor has
+ * ID_AA64PRF0_EL1.CSV2 set.
+ */
+ if ( cpu_data[smp_processor_id()].pfr64.csv2 )
+ return 0;
+
if ( smccc_ver < SMCCC_VERSION(1, 1) )
goto warn;
["xsa398/xsa398-4.14-3-xen-arm-Add-ECBHB-and-CLEARBHB-ID-fields.patch" (application/octet-stream)]
From ee4b53ae1b95966fd9a491668f0eca73028925e1 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Wed, 23 Feb 2022 09:42:18 +0000
Subject: xen/arm: Add ECBHB and CLEARBHB ID fields
Introduce ID coprocessor register ID_AA64ISAR2_EL1.
Add definitions in cpufeature and sysregs of ECBHB field in mmfr1 and
CLEARBHB in isar2 ID coprocessor registers.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 4b68d12d98b8790d8002fcc2c25a9d713374a4d7)
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 44126dbf0723..13dac7ccaf94 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -117,6 +117,7 @@ void identify_cpu(struct cpuinfo_arm *c)
c->isa64.bits[0] = READ_SYSREG64(ID_AA64ISAR0_EL1);
c->isa64.bits[1] = READ_SYSREG64(ID_AA64ISAR1_EL1);
+ c->isa64.bits[2] = READ_SYSREG64(ID_AA64ISAR2_EL1);
#endif
c->pfr32.bits[0] = READ_SYSREG32(ID_PFR0_EL1);
diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
index c60029d38f5b..cfd2e1d48699 100644
--- a/xen/include/asm-arm/arm64/sysregs.h
+++ b/xen/include/asm-arm/arm64/sysregs.h
@@ -57,6 +57,10 @@
#define ICH_AP1R2_EL2 __AP1Rx_EL2(2)
#define ICH_AP1R3_EL2 __AP1Rx_EL2(3)
+#ifndef ID_AA64ISAR2_EL1
+#define ID_AA64ISAR2_EL1 S3_0_C0_C6_2
+#endif
+
/* Access to system registers */
#define READ_SYSREG32(name) ((uint32_t)READ_SYSREG64(name))
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 016a9fe2039a..7be4ee8cf821 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -188,12 +188,26 @@ struct cpuinfo_arm {
unsigned long lo:4;
unsigned long pan:4;
unsigned long __res1:8;
- unsigned long __res2:32;
+ unsigned long __res2:28;
+ unsigned long ecbhb:4;
};
} mm64;
- struct {
- uint64_t bits[2];
+ union {
+ uint64_t bits[3];
+ struct {
+ /* ISAR0 */
+ unsigned long __res0:64;
+
+ /* ISAR1 */
+ unsigned long __res1:64;
+
+ /* ISAR2 */
+ unsigned long __res3:28;
+ unsigned long clearbhb:4;
+
+ unsigned long __res4:32;
+ };
} isa64;
#endif
["xsa398/xsa398-4.14-4-xen-arm-Add-Spectre-BHB-handling.patch" (application/octet-stream)]
From fc56dd212e4574c5fd77f830d077036b330dc1b5 Mon Sep 17 00:00:00 2001
From: Rahul Singh <rahul.singh@arm.com>
Date: Mon, 14 Feb 2022 18:47:32 +0000
Subject: xen/arm: Add Spectre BHB handling
This commit is adding Spectre BHB handling to Xen on Arm.
The commit is introducing new alternative code to be executed during
exception entry:
- SMCC workaround 3 call
- loop workaround (with 8, 24 or 32 iterations)
- use of new clearbhb instruction
Cpuerrata is modified by this patch to apply the required workaround for
CPU affected by Spectre BHB when CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR is
enabled.
To do this the system previously used to apply smcc workaround 1 is
reused and new alternative code to be copied in the exception handler is
introduced.
To define the type of workaround required by a processor, 4 new cpu
capabilities are introduced (for each number of loop and for smcc
workaround 3).
When a processor is affected, enable_spectre_bhb_workaround is called
and if the processor does not have CSV2 set to 3 or ECBHB feature (which
would mean that the processor is doing what is required in hardware),
the proper code is enabled at exception entry.
In the case where workaround 3 is not supported by the firmware, we
enable workaround 1 when possible as it will also mitigate Spectre BHB
on systems without CSV2.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 62c91eb66a2904eefb1d1d9642e3697a1e3c3a3c)
diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
index d8743d955c4a..4e6382522048 100644
--- a/xen/arch/arm/arm64/bpi.S
+++ b/xen/arch/arm/arm64/bpi.S
@@ -58,16 +58,42 @@ ENTRY(__bp_harden_hyp_vecs_start)
.endr
ENTRY(__bp_harden_hyp_vecs_end)
-ENTRY(__smccc_workaround_1_smc_start)
+.macro mitigate_spectre_bhb_loop count
+ENTRY(__mitigate_spectre_bhb_loop_start_\count)
+ stp x0, x1, [sp, #-16]!
+ mov x0, \count
+.Lspectre_bhb_loop\@:
+ b . + 4
+ subs x0, x0, #1
+ b.ne .Lspectre_bhb_loop\@
+ sb
+ ldp x0, x1, [sp], #16
+ENTRY(__mitigate_spectre_bhb_loop_end_\count)
+.endm
+
+.macro smccc_workaround num smcc_id
+ENTRY(__smccc_workaround_smc_start_\num)
sub sp, sp, #(8 * 4)
stp x0, x1, [sp, #(8 * 2)]
stp x2, x3, [sp, #(8 * 0)]
- mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
+ mov w0, \smcc_id
smc #0
ldp x2, x3, [sp, #(8 * 0)]
ldp x0, x1, [sp, #(8 * 2)]
add sp, sp, #(8 * 4)
-ENTRY(__smccc_workaround_1_smc_end)
+ENTRY(__smccc_workaround_smc_end_\num)
+.endm
+
+ENTRY(__mitigate_spectre_bhb_clear_insn_start)
+ clearbhb
+ isb
+ENTRY(__mitigate_spectre_bhb_clear_insn_end)
+
+mitigate_spectre_bhb_loop 8
+mitigate_spectre_bhb_loop 24
+mitigate_spectre_bhb_loop 32
+smccc_workaround 1, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
+smccc_workaround 3, #ARM_SMCCC_ARCH_WORKAROUND_3_FID
/*
* Local variables:
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index 9d79e3bad7e8..1c1149b2c795 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -145,7 +145,16 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry,
return ret;
}
-extern char __smccc_workaround_1_smc_start[], __smccc_workaround_1_smc_end[];
+extern char __smccc_workaround_smc_start_1[], __smccc_workaround_smc_end_1[];
+extern char __smccc_workaround_smc_start_3[], __smccc_workaround_smc_end_3[];
+extern char __mitigate_spectre_bhb_clear_insn_start[],
+ __mitigate_spectre_bhb_clear_insn_end[];
+extern char __mitigate_spectre_bhb_loop_start_8[],
+ __mitigate_spectre_bhb_loop_end_8[];
+extern char __mitigate_spectre_bhb_loop_start_24[],
+ __mitigate_spectre_bhb_loop_end_24[];
+extern char __mitigate_spectre_bhb_loop_start_32[],
+ __mitigate_spectre_bhb_loop_end_32[];
static int enable_smccc_arch_workaround_1(void *data)
{
@@ -177,8 +186,8 @@ static int enable_smccc_arch_workaround_1(void *data)
if ( (int)res.a0 < 0 )
goto warn;
- return !install_bp_hardening_vec(entry,__smccc_workaround_1_smc_start,
- __smccc_workaround_1_smc_end,
+ return !install_bp_hardening_vec(entry,__smccc_workaround_smc_start_1,
+ __smccc_workaround_smc_end_1,
"call ARM_SMCCC_ARCH_WORKAROUND_1");
warn:
@@ -193,6 +202,93 @@ static int enable_smccc_arch_workaround_1(void *data)
return 0;
}
+/*
+ * Spectre BHB Mitigation
+ *
+ * CPU is either:
+ * - Having CVS2.3 so it is not affected.
+ * - Having ECBHB and is clearing the branch history buffer when an exception
+ * to a different exception level is happening so no mitigation is needed.
+ * - Mitigating using a loop on exception entry (number of loop depending on
+ * the CPU).
+ * - Mitigating using the firmware.
+ */
+static int enable_spectre_bhb_workaround(void *data)
+{
+ const struct arm_cpu_capabilities *entry = data;
+
+ /*
+ * Enable callbacks are called on every CPU based on the capabilities, so
+ * double-check whether the CPU matches the entry.
+ */
+ if ( !entry->matches(entry) )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].pfr64.csv2 == 3 )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].mm64.ecbhb )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].isa64.clearbhb )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_clear_insn_start,
+ __mitigate_spectre_bhb_clear_insn_end,
+ "use clearBHB instruction");
+
+ /* Apply solution depending on hwcaps set on arm_errata */
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_8) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_8,
+ __mitigate_spectre_bhb_loop_end_8,
+ "use 8 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_24) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_24,
+ __mitigate_spectre_bhb_loop_end_24,
+ "use 24 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_32) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_32,
+ __mitigate_spectre_bhb_loop_end_32,
+ "use 32 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_SMCC_3) )
+ {
+ struct arm_smccc_res res;
+
+ if ( smccc_ver < SMCCC_VERSION(1, 1) )
+ goto warn;
+
+ arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FID,
+ ARM_SMCCC_ARCH_WORKAROUND_3_FID, &res);
+ /* The return value is in the lower 32-bits. */
+ if ( (int)res.a0 < 0 )
+ {
+ /*
+ * On processor affected with CSV2=0, workaround 1 will mitigate
+ * both Spectre v2 and BHB so use it when available
+ */
+ if ( enable_smccc_arch_workaround_1(data) )
+ return 1;
+
+ goto warn;
+ }
+
+ return !install_bp_hardening_vec(entry,__smccc_workaround_smc_start_3,
+ __smccc_workaround_smc_end_3,
+ "call ARM_SMCCC_ARCH_WORKAROUND_3");
+ }
+
+warn:
+ printk_once("**** No support for any spectre BHB workaround. ****\n"
+ "**** Please update your firmware. ****\n");
+
+ return 0;
+}
+
#endif /* CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR */
/* Hardening Branch predictor code for Arm32 */
@@ -438,19 +534,77 @@ static const struct arm_cpu_capabilities arm_errata[] = {
},
{
.capability = ARM_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ MIDR_RANGE(MIDR_CORTEX_A72, 0, 1 << MIDR_VARIANT_SHIFT),
.enable = enable_smccc_arch_workaround_1,
},
{
- .capability = ARM_HARDEN_BRANCH_PREDICTOR,
+ .capability = ARM_WORKAROUND_BHB_SMCC_3,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .enable = enable_smccc_arch_workaround_1,
+ .enable = enable_spectre_bhb_workaround,
},
{
- .capability = ARM_HARDEN_BRANCH_PREDICTOR,
+ .capability = ARM_WORKAROUND_BHB_SMCC_3,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- .enable = enable_smccc_arch_workaround_1,
+ .enable = enable_spectre_bhb_workaround,
+ },
+ /* spectre BHB */
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_8,
+ MIDR_RANGE(MIDR_CORTEX_A72, 1 << MIDR_VARIANT_SHIFT,
+ (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A76),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A77),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_X1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_X2),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
+ .enable = enable_spectre_bhb_workaround,
},
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+
#endif
#ifdef CONFIG_ARM32_HARDEN_BRANCH_PREDICTOR
{
diff --git a/xen/include/asm-arm/arm64/macros.h b/xen/include/asm-arm/arm64/macros.h
index f981b4f43e84..5100aed6e3ec 100644
--- a/xen/include/asm-arm/arm64/macros.h
+++ b/xen/include/asm-arm/arm64/macros.h
@@ -21,6 +21,11 @@
ldr \dst, [\dst, \tmp]
.endm
+ /* clearbhb instruction clearing the branch history */
+ .macro clearbhb
+ hint #22
+ .endm
+
/*
* Register aliases.
*/
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 7be4ee8cf821..14c7f7d218e2 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -46,8 +46,12 @@
#define ARM_SMCCC_1_1 8
#define ARM64_WORKAROUND_AT_SPECULATE 9
#define ARM_WORKAROUND_858921 10
+#define ARM_WORKAROUND_BHB_LOOP_8 11
+#define ARM_WORKAROUND_BHB_LOOP_24 12
+#define ARM_WORKAROUND_BHB_LOOP_32 13
+#define ARM_WORKAROUND_BHB_SMCC_3 14
-#define ARM_NCAPS 11
+#define ARM_NCAPS 15
#ifndef __ASSEMBLY__
diff --git a/xen/include/asm-arm/smccc.h b/xen/include/asm-arm/smccc.h
index 9d94beb3df2d..b3dbeecc90ad 100644
--- a/xen/include/asm-arm/smccc.h
+++ b/xen/include/asm-arm/smccc.h
@@ -334,6 +334,12 @@ void __arm_smccc_1_0_smc(register_t a0, register_t a1, register_t a2,
ARM_SMCCC_OWNER_ARCH, \
0x7FFF)
+#define ARM_SMCCC_ARCH_WORKAROUND_3_FID \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
+ ARM_SMCCC_CONV_32, \
+ ARM_SMCCC_OWNER_ARCH, \
+ 0x3FFF)
+
/* SMCCC error codes */
#define ARM_SMCCC_NOT_REQUIRED (-2)
#define ARM_SMCCC_ERR_UNKNOWN_FUNCTION (-1)
["xsa398/xsa398-4.14-5-xen-arm-Allow-to-discover-and-use-SMCCC_ARCH_WORKARO.patch" (application/octet-stream)]
From 7cebd77c80ce87f84c63a6043a5ad7115ccab9d5 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Thu, 17 Feb 2022 14:52:54 +0000
Subject: xen/arm: Allow to discover and use SMCCC_ARCH_WORKAROUND_3
Allow guest to discover whether or not SMCCC_ARCH_WORKAROUND_3 is
supported and create a fastpath in the code to handle guests request to
do the workaround.
The function SMCCC_ARCH_WORKAROUND_3 will be called by the guest for
flushing the branch history. So we want the handling to be as fast as
possible.
As the mitigation is applied on every guest exit, we can check for the
call before saving all context and return very early.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <julien@xen.org>
(cherry picked from commit c0a56ea0fd92ecb471936b7355ddbecbaea3707c)
diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
index 175ea2981e72..a8c214506786 100644
--- a/xen/arch/arm/arm64/entry.S
+++ b/xen/arch/arm/arm64/entry.S
@@ -338,16 +338,26 @@ guest_sync:
cbnz x1, guest_sync_slowpath /* should be 0 for HVC #0 */
/*
- * Fastest path possible for ARM_SMCCC_ARCH_WORKAROUND_1.
- * The workaround has already been applied on the exception
+ * Fastest path possible for ARM_SMCCC_ARCH_WORKAROUND_1 and
+ * ARM_SMCCC_ARCH_WORKAROUND_3.
+ * The workaround needed has already been applied on the exception
* entry from the guest, so let's quickly get back to the guest.
*
* Note that eor is used because the function identifier cannot
* be encoded as an immediate for cmp.
*/
eor w0, w0, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
- cbnz w0, check_wa2
+ cbz w0, fastpath_out_workaround
+ /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */
+ eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_1_FID ^ ARM_SMCCC_ARCH_WORKAROUND_2_FID)
+ cbz w0, wa2_ssbd
+
+ /* Fastpath out for ARM_SMCCC_ARCH_WORKAROUND_3 */
+ eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_2_FID ^ ARM_SMCCC_ARCH_WORKAROUND_3_FID)
+ cbnz w0, guest_sync_slowpath
+
+fastpath_out_workaround:
/*
* Clobber both x0 and x1 to prevent leakage. Note that thanks
* the eor, x0 = 0.
@@ -356,10 +366,7 @@ guest_sync:
eret
sb
-check_wa2:
- /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */
- eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_1_FID ^ ARM_SMCCC_ARCH_WORKAROUND_2_FID)
- cbnz w0, guest_sync_slowpath
+wa2_ssbd:
#ifdef CONFIG_ARM_SSBD
alternative_cb arm_enable_wa2_handling
b wa2_end
diff --git a/xen/arch/arm/vsmc.c b/xen/arch/arm/vsmc.c
index a36db15fffc0..b633ff2fe897 100644
--- a/xen/arch/arm/vsmc.c
+++ b/xen/arch/arm/vsmc.c
@@ -124,6 +124,10 @@ static bool handle_arch(struct cpu_user_regs *regs)
break;
}
break;
+ case ARM_SMCCC_ARCH_WORKAROUND_3_FID:
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_SMCC_3) )
+ ret = 0;
+ break;
}
set_user_reg(regs, 0, ret);
@@ -132,6 +136,7 @@ static bool handle_arch(struct cpu_user_regs *regs)
}
case ARM_SMCCC_ARCH_WORKAROUND_1_FID:
+ case ARM_SMCCC_ARCH_WORKAROUND_3_FID:
/* No return value */
return true;
["xsa398/xsa398-4.14-6-x86-spec-ctrl-Cease-using-thunk-lfence-on-AMD.patch" (application/octet-stream)]
From ca304edd3ba8c19211107fd2e898249987557ce5 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 7 Mar 2022 16:35:52 +0000
Subject: x86/spec-ctrl: Cease using thunk=lfence on AMD
AMD have updated their Spectre v2 guidance, and lfence/jmp is no longer
considered safe. AMD are recommending using retpoline everywhere.
Retpoline is incompatible with CET. All CET-capable hardware has efficient
IBRS (specifically, not something retrofitted in microcode), so use IBRS (and
STIBP for consistency sake).
This is a logical change on AMD, but not on Intel as the default calculations
would end up with these settings anyway. Leave behind a message if IBRS is
found to be missing.
Also update the default heuristics to never select THUNK_LFENCE. This causes
AMD CPUs to change their default to retpoline.
Also update the printed message to include the AMD MSR_SPEC_CTRL settings, and
STIBP now that we set it for consistency sake.
This is part of XSA-398 / CVE-2021-26401.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 8d03080d2a339840d3a59e0932a94f804e45110d)
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index fd8f82549152..c0bfbb7a5c27 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2140,9 +2140,9 @@ to use.
If Xen was compiled with INDIRECT_THUNK support, `bti-thunk=` can be used to
select which of the thunks gets patched into the `__x86_indirect_thunk_%reg`
-locations. The default thunk is `retpoline` (generally preferred for Intel
-hardware), with the alternatives being `jmp` (a `jmp *%reg` gadget, minimal
-overhead), and `lfence` (an `lfence; jmp *%reg` gadget, preferred for AMD).
+locations. The default thunk is `retpoline` (generally preferred), with the
+alternatives being `jmp` (a `jmp *%reg` gadget, minimal overhead), and
+`lfence` (an `lfence; jmp *%reg` gadget).
On hardware supporting IBRS (Indirect Branch Restricted Speculation), the
`ibrs=` option can be used to force or prevent Xen using the feature itself.
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 9301d95bd705..7ded6ecba197 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -367,14 +367,19 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
"\n");
/* Settings for Xen's protection, irrespective of guests. */
- printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s, Other:%s%s%s%s%s\n",
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s%s, Other:%s%s%s%s%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- !boot_cpu_has(X86_FEATURE_IBRSB) ? "No" :
+ (!boot_cpu_has(X86_FEATURE_IBRSB) &&
+ !boot_cpu_has(X86_FEATURE_IBRS)) ? "No" :
(default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
- !boot_cpu_has(X86_FEATURE_SSBD) ? "" :
+ (!boot_cpu_has(X86_FEATURE_STIBP) &&
+ !boot_cpu_has(X86_FEATURE_AMD_STIBP)) ? "" :
+ (default_xen_spec_ctrl & SPEC_CTRL_STIBP) ? " STIBP+" : " STIBP-",
+ (!boot_cpu_has(X86_FEATURE_SSBD) &&
+ !boot_cpu_has(X86_FEATURE_AMD_SSBD)) ? "" :
(default_xen_spec_ctrl & SPEC_CTRL_SSBD) ? " SSBD+" : " SSBD-",
!(caps & ARCH_CAPS_TSX_CTRL) ? "" :
(opt_tsx & 1) ? " TSX+" : " TSX-",
@@ -916,10 +921,23 @@ void __init init_speculation_mitigations(void)
/*
* First, disable the use of retpolines if Xen is using shadow stacks, as
* they are incompatible.
+ *
+ * In the absence of retpolines, IBRS needs to be used for speculative
+ * safety. All CET-capable hardware has efficient IBRS.
*/
- if ( cpu_has_xen_shstk &&
- (opt_thunk == THUNK_DEFAULT || opt_thunk == THUNK_RETPOLINE) )
- thunk = THUNK_JMP;
+ if ( cpu_has_xen_shstk )
+ {
+ if ( !has_spec_ctrl )
+ printk(XENLOG_WARNING "?!? CET active, but no MSR_SPEC_CTRL?\n");
+ else if ( opt_ibrs == -1 )
+ {
+ opt_ibrs = ibrs = true;
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS | SPEC_CTRL_STIBP;
+ }
+
+ if ( opt_thunk == THUNK_DEFAULT || opt_thunk == THUNK_RETPOLINE )
+ thunk = THUNK_JMP;
+ }
/*
* Has the user specified any custom BTI mitigations? If so, follow their
@@ -939,16 +957,10 @@ void __init init_speculation_mitigations(void)
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
{
/*
- * AMD's recommended mitigation is to set lfence as being dispatch
- * serialising, and to use IND_THUNK_LFENCE.
- */
- if ( cpu_has_lfence_dispatch )
- thunk = THUNK_LFENCE;
- /*
- * On Intel hardware, we'd like to use retpoline in preference to
+ * On all hardware, we'd like to use retpoline in preference to
* IBRS, but only if it is safe on this hardware.
*/
- else if ( retpoline_safe(caps) )
+ if ( retpoline_safe(caps) )
thunk = THUNK_RETPOLINE;
else if ( has_spec_ctrl )
ibrs = true;
["xsa398/xsa398-4.15-1-xen-arm-Introduce-new-Arm-processors.patch" (application/octet-stream)]
From 5f097af94ec78e170d599a05c6e528cda259891f Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Tue, 15 Feb 2022 10:37:51 +0000
Subject: xen/arm: Introduce new Arm processors
Add some new processor identifiers in processor.h and sync Xen
definitions with status of Linux 5.17 (declared in
arch/arm64/include/asm/cputype.h).
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 35d1b85a6b43483f6bd007d48757434e54743e98)
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index 5c1768cdec25..2e1f5da7853a 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -53,6 +53,7 @@
#define ARM_CPU_PART_CORTEX_A17 0xC0E
#define ARM_CPU_PART_CORTEX_A15 0xC0F
#define ARM_CPU_PART_CORTEX_A53 0xD03
+#define ARM_CPU_PART_CORTEX_A35 0xD04
#define ARM_CPU_PART_CORTEX_A55 0xD05
#define ARM_CPU_PART_CORTEX_A57 0xD07
#define ARM_CPU_PART_CORTEX_A72 0xD08
@@ -60,11 +61,20 @@
#define ARM_CPU_PART_CORTEX_A75 0xD0A
#define ARM_CPU_PART_CORTEX_A76 0xD0B
#define ARM_CPU_PART_NEOVERSE_N1 0xD0C
+#define ARM_CPU_PART_CORTEX_A77 0xD0D
+#define ARM_CPU_PART_NEOVERSE_V1 0xD40
+#define ARM_CPU_PART_CORTEX_A78 0xD41
+#define ARM_CPU_PART_CORTEX_X1 0xD44
+#define ARM_CPU_PART_CORTEX_A710 0xD47
+#define ARM_CPU_PART_CORTEX_X2 0xD48
+#define ARM_CPU_PART_NEOVERSE_N2 0xD49
+#define ARM_CPU_PART_CORTEX_A78C 0xD4B
#define MIDR_CORTEX_A12 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A12)
#define MIDR_CORTEX_A17 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A17)
#define MIDR_CORTEX_A15 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A15)
#define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
+#define MIDR_CORTEX_A35 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A35)
#define MIDR_CORTEX_A55 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A55)
#define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57)
#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72)
@@ -72,6 +82,14 @@
#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75)
#define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A76)
#define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N1)
+#define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A77)
+#define MIDR_NEOVERSE_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V1)
+#define MIDR_CORTEX_A78 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78)
+#define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1)
+#define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710)
+#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2)
+#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)
+#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
/* MPIDR Multiprocessor Affinity Register */
#define _MPIDR_UP (30)
["xsa398/xsa398-4.15-2-xen-arm-move-errata-CSV2-check-earlier.patch" (application/octet-stream)]
From 9a4a4abb595e6600ab3b652676a998f5bbc580d5 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Tue, 15 Feb 2022 10:39:47 +0000
Subject: xen/arm: move errata CSV2 check earlier
CSV2 availability check is done after printing to the user that
workaround 1 will be used. Move the check before to prevent saying to the
user that workaround 1 is used when it is not because it is not needed.
This will also allow to reuse install_bp_hardening_vec function for
other use cases.
Code previously returning "true", now returns "0" to conform to
enable_smccc_arch_workaround_1 returning an int and surrounding code
doing a "return 0" if workaround is not needed.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <julien@xen.org>
(cherry picked from commit 599616d70eb886b9ad0ef9d6b51693ce790504ba)
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index b398d480f113..00f9ebe9cee0 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -103,13 +103,6 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry,
printk(XENLOG_INFO "CPU%u will %s on exception entry\n",
smp_processor_id(), desc);
- /*
- * No need to install hardened vector when the processor has
- * ID_AA64PRF0_EL1.CSV2 set.
- */
- if ( cpu_data[smp_processor_id()].pfr64.csv2 )
- return true;
-
spin_lock(&bp_lock);
/*
@@ -167,6 +160,13 @@ static int enable_smccc_arch_workaround_1(void *data)
if ( !entry->matches(entry) )
return 0;
+ /*
+ * No need to install hardened vector when the processor has
+ * ID_AA64PRF0_EL1.CSV2 set.
+ */
+ if ( cpu_data[smp_processor_id()].pfr64.csv2 )
+ return 0;
+
if ( smccc_ver < SMCCC_VERSION(1, 1) )
goto warn;
["xsa398/xsa398-4.15-3-xen-arm-Add-ECBHB-and-CLEARBHB-ID-fields.patch" (application/octet-stream)]
From 7259e87984a00c6a481b89e1b3da8392750bcc36 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Wed, 23 Feb 2022 09:42:18 +0000
Subject: xen/arm: Add ECBHB and CLEARBHB ID fields
Introduce ID coprocessor register ID_AA64ISAR2_EL1.
Add definitions in cpufeature and sysregs of ECBHB field in mmfr1 and
CLEARBHB in isar2 ID coprocessor registers.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 4b68d12d98b8790d8002fcc2c25a9d713374a4d7)
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 1d88783809e2..6b7b510d26a9 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -122,6 +122,7 @@ void identify_cpu(struct cpuinfo_arm *c)
c->isa64.bits[0] = READ_SYSREG(ID_AA64ISAR0_EL1);
c->isa64.bits[1] = READ_SYSREG(ID_AA64ISAR1_EL1);
+ c->isa64.bits[2] = READ_SYSREG(ID_AA64ISAR2_EL1);
c->zfr64.bits[0] = READ_SYSREG(ID_AA64ZFR0_EL1);
diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
index 077fd95fb768..3115c819705d 100644
--- a/xen/include/asm-arm/arm64/sysregs.h
+++ b/xen/include/asm-arm/arm64/sysregs.h
@@ -84,6 +84,9 @@
#ifndef ID_DFR1_EL1
#define ID_DFR1_EL1 S3_0_C0_C3_5
#endif
+#ifndef ID_AA64ISAR2_EL1
+#define ID_AA64ISAR2_EL1 S3_0_C0_C6_2
+#endif
/* Access to system registers */
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 9ea3970c7003..538145f260bf 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -206,14 +206,15 @@ struct cpuinfo_arm {
unsigned long lo:4;
unsigned long pan:4;
unsigned long __res1:8;
- unsigned long __res2:32;
+ unsigned long __res2:28;
+ unsigned long ecbhb:4;
unsigned long __res3:64;
};
} mm64;
union {
- uint64_t bits[2];
+ uint64_t bits[3];
struct {
/* ISAR0 */
unsigned long __res0:4;
@@ -249,6 +250,12 @@ struct cpuinfo_arm {
unsigned long dgh:4;
unsigned long i8mm:4;
unsigned long __res2:8;
+
+ /* ISAR2 */
+ unsigned long __res3:28;
+ unsigned long clearbhb:4;
+
+ unsigned long __res4:32;
};
} isa64;
["xsa398/xsa398-4.15-4-xen-arm-Add-Spectre-BHB-handling.patch" (application/octet-stream)]
From eed4a84a83f828611210499fd0e995f6c8e73a10 Mon Sep 17 00:00:00 2001
From: Rahul Singh <rahul.singh@arm.com>
Date: Mon, 14 Feb 2022 18:47:32 +0000
Subject: xen/arm: Add Spectre BHB handling
This commit is adding Spectre BHB handling to Xen on Arm.
The commit is introducing new alternative code to be executed during
exception entry:
- SMCC workaround 3 call
- loop workaround (with 8, 24 or 32 iterations)
- use of new clearbhb instruction
Cpuerrata is modified by this patch to apply the required workaround for
CPU affected by Spectre BHB when CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR is
enabled.
To do this the system previously used to apply smcc workaround 1 is
reused and new alternative code to be copied in the exception handler is
introduced.
To define the type of workaround required by a processor, 4 new cpu
capabilities are introduced (for each number of loop and for smcc
workaround 3).
When a processor is affected, enable_spectre_bhb_workaround is called
and if the processor does not have CSV2 set to 3 or ECBHB feature (which
would mean that the processor is doing what is required in hardware),
the proper code is enabled at exception entry.
In the case where workaround 3 is not supported by the firmware, we
enable workaround 1 when possible as it will also mitigate Spectre BHB
on systems without CSV2.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 62c91eb66a2904eefb1d1d9642e3697a1e3c3a3c)
diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
index d8743d955c4a..4e6382522048 100644
--- a/xen/arch/arm/arm64/bpi.S
+++ b/xen/arch/arm/arm64/bpi.S
@@ -58,16 +58,42 @@ ENTRY(__bp_harden_hyp_vecs_start)
.endr
ENTRY(__bp_harden_hyp_vecs_end)
-ENTRY(__smccc_workaround_1_smc_start)
+.macro mitigate_spectre_bhb_loop count
+ENTRY(__mitigate_spectre_bhb_loop_start_\count)
+ stp x0, x1, [sp, #-16]!
+ mov x0, \count
+.Lspectre_bhb_loop\@:
+ b . + 4
+ subs x0, x0, #1
+ b.ne .Lspectre_bhb_loop\@
+ sb
+ ldp x0, x1, [sp], #16
+ENTRY(__mitigate_spectre_bhb_loop_end_\count)
+.endm
+
+.macro smccc_workaround num smcc_id
+ENTRY(__smccc_workaround_smc_start_\num)
sub sp, sp, #(8 * 4)
stp x0, x1, [sp, #(8 * 2)]
stp x2, x3, [sp, #(8 * 0)]
- mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
+ mov w0, \smcc_id
smc #0
ldp x2, x3, [sp, #(8 * 0)]
ldp x0, x1, [sp, #(8 * 2)]
add sp, sp, #(8 * 4)
-ENTRY(__smccc_workaround_1_smc_end)
+ENTRY(__smccc_workaround_smc_end_\num)
+.endm
+
+ENTRY(__mitigate_spectre_bhb_clear_insn_start)
+ clearbhb
+ isb
+ENTRY(__mitigate_spectre_bhb_clear_insn_end)
+
+mitigate_spectre_bhb_loop 8
+mitigate_spectre_bhb_loop 24
+mitigate_spectre_bhb_loop 32
+smccc_workaround 1, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
+smccc_workaround 3, #ARM_SMCCC_ARCH_WORKAROUND_3_FID
/*
* Local variables:
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index 00f9ebe9cee0..ae649d16ef02 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -145,7 +145,16 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry,
return ret;
}
-extern char __smccc_workaround_1_smc_start[], __smccc_workaround_1_smc_end[];
+extern char __smccc_workaround_smc_start_1[], __smccc_workaround_smc_end_1[];
+extern char __smccc_workaround_smc_start_3[], __smccc_workaround_smc_end_3[];
+extern char __mitigate_spectre_bhb_clear_insn_start[],
+ __mitigate_spectre_bhb_clear_insn_end[];
+extern char __mitigate_spectre_bhb_loop_start_8[],
+ __mitigate_spectre_bhb_loop_end_8[];
+extern char __mitigate_spectre_bhb_loop_start_24[],
+ __mitigate_spectre_bhb_loop_end_24[];
+extern char __mitigate_spectre_bhb_loop_start_32[],
+ __mitigate_spectre_bhb_loop_end_32[];
static int enable_smccc_arch_workaround_1(void *data)
{
@@ -176,8 +185,8 @@ static int enable_smccc_arch_workaround_1(void *data)
if ( (int)res.a0 < 0 )
goto warn;
- return !install_bp_hardening_vec(entry,__smccc_workaround_1_smc_start,
- __smccc_workaround_1_smc_end,
+ return !install_bp_hardening_vec(entry,__smccc_workaround_smc_start_1,
+ __smccc_workaround_smc_end_1,
"call ARM_SMCCC_ARCH_WORKAROUND_1");
warn:
@@ -187,6 +196,93 @@ static int enable_smccc_arch_workaround_1(void *data)
return 0;
}
+/*
+ * Spectre BHB Mitigation
+ *
+ * CPU is either:
+ * - Having CVS2.3 so it is not affected.
+ * - Having ECBHB and is clearing the branch history buffer when an exception
+ * to a different exception level is happening so no mitigation is needed.
+ * - Mitigating using a loop on exception entry (number of loop depending on
+ * the CPU).
+ * - Mitigating using the firmware.
+ */
+static int enable_spectre_bhb_workaround(void *data)
+{
+ const struct arm_cpu_capabilities *entry = data;
+
+ /*
+ * Enable callbacks are called on every CPU based on the capabilities, so
+ * double-check whether the CPU matches the entry.
+ */
+ if ( !entry->matches(entry) )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].pfr64.csv2 == 3 )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].mm64.ecbhb )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].isa64.clearbhb )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_clear_insn_start,
+ __mitigate_spectre_bhb_clear_insn_end,
+ "use clearBHB instruction");
+
+ /* Apply solution depending on hwcaps set on arm_errata */
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_8) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_8,
+ __mitigate_spectre_bhb_loop_end_8,
+ "use 8 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_24) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_24,
+ __mitigate_spectre_bhb_loop_end_24,
+ "use 24 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_32) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_32,
+ __mitigate_spectre_bhb_loop_end_32,
+ "use 32 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_SMCC_3) )
+ {
+ struct arm_smccc_res res;
+
+ if ( smccc_ver < SMCCC_VERSION(1, 1) )
+ goto warn;
+
+ arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FID,
+ ARM_SMCCC_ARCH_WORKAROUND_3_FID, &res);
+ /* The return value is in the lower 32-bits. */
+ if ( (int)res.a0 < 0 )
+ {
+ /*
+ * On processor affected with CSV2=0, workaround 1 will mitigate
+ * both Spectre v2 and BHB so use it when available
+ */
+ if ( enable_smccc_arch_workaround_1(data) )
+ return 1;
+
+ goto warn;
+ }
+
+ return !install_bp_hardening_vec(entry,__smccc_workaround_smc_start_3,
+ __smccc_workaround_smc_end_3,
+ "call ARM_SMCCC_ARCH_WORKAROUND_3");
+ }
+
+warn:
+ printk_once("**** No support for any spectre BHB workaround. ****\n"
+ "**** Please update your firmware. ****\n");
+
+ return 0;
+}
+
#endif /* CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR */
/* Hardening Branch predictor code for Arm32 */
@@ -446,19 +542,77 @@ static const struct arm_cpu_capabilities arm_errata[] = {
},
{
.capability = ARM_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ MIDR_RANGE(MIDR_CORTEX_A72, 0, 1 << MIDR_VARIANT_SHIFT),
.enable = enable_smccc_arch_workaround_1,
},
{
- .capability = ARM_HARDEN_BRANCH_PREDICTOR,
+ .capability = ARM_WORKAROUND_BHB_SMCC_3,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .enable = enable_smccc_arch_workaround_1,
+ .enable = enable_spectre_bhb_workaround,
},
{
- .capability = ARM_HARDEN_BRANCH_PREDICTOR,
+ .capability = ARM_WORKAROUND_BHB_SMCC_3,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- .enable = enable_smccc_arch_workaround_1,
+ .enable = enable_spectre_bhb_workaround,
+ },
+ /* spectre BHB */
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_8,
+ MIDR_RANGE(MIDR_CORTEX_A72, 1 << MIDR_VARIANT_SHIFT,
+ (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A76),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A77),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_X1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_X2),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
+ .enable = enable_spectre_bhb_workaround,
},
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+
#endif
#ifdef CONFIG_ARM32_HARDEN_BRANCH_PREDICTOR
{
diff --git a/xen/include/asm-arm/arm64/macros.h b/xen/include/asm-arm/arm64/macros.h
index f981b4f43e84..5100aed6e3ec 100644
--- a/xen/include/asm-arm/arm64/macros.h
+++ b/xen/include/asm-arm/arm64/macros.h
@@ -21,6 +21,11 @@
ldr \dst, [\dst, \tmp]
.endm
+ /* clearbhb instruction clearing the branch history */
+ .macro clearbhb
+ hint #22
+ .endm
+
/*
* Register aliases.
*/
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 538145f260bf..d4e7cde675e5 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -49,8 +49,12 @@
#define ARM64_WORKAROUND_AT_SPECULATE 9
#define ARM_WORKAROUND_858921 10
#define ARM64_WORKAROUND_REPEAT_TLBI 11
+#define ARM_WORKAROUND_BHB_LOOP_8 12
+#define ARM_WORKAROUND_BHB_LOOP_24 13
+#define ARM_WORKAROUND_BHB_LOOP_32 14
+#define ARM_WORKAROUND_BHB_SMCC_3 15
-#define ARM_NCAPS 12
+#define ARM_NCAPS 16
#ifndef __ASSEMBLY__
diff --git a/xen/include/asm-arm/smccc.h b/xen/include/asm-arm/smccc.h
index 9d94beb3df2d..b3dbeecc90ad 100644
--- a/xen/include/asm-arm/smccc.h
+++ b/xen/include/asm-arm/smccc.h
@@ -334,6 +334,12 @@ void __arm_smccc_1_0_smc(register_t a0, register_t a1, register_t a2,
ARM_SMCCC_OWNER_ARCH, \
0x7FFF)
+#define ARM_SMCCC_ARCH_WORKAROUND_3_FID \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
+ ARM_SMCCC_CONV_32, \
+ ARM_SMCCC_OWNER_ARCH, \
+ 0x3FFF)
+
/* SMCCC error codes */
#define ARM_SMCCC_NOT_REQUIRED (-2)
#define ARM_SMCCC_ERR_UNKNOWN_FUNCTION (-1)
["xsa398/xsa398-4.15-5-xen-arm-Allow-to-discover-and-use-SMCCC_ARCH_WORKARO.patch" (application/octet-stream)]
From 65110f49535d15b6dddf4a8f34308231eae761da Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Thu, 17 Feb 2022 14:52:54 +0000
Subject: xen/arm: Allow to discover and use SMCCC_ARCH_WORKAROUND_3
Allow guest to discover whether or not SMCCC_ARCH_WORKAROUND_3 is
supported and create a fastpath in the code to handle guests request to
do the workaround.
The function SMCCC_ARCH_WORKAROUND_3 will be called by the guest for
flushing the branch history. So we want the handling to be as fast as
possible.
As the mitigation is applied on every guest exit, we can check for the
call before saving all context and return very early.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <julien@xen.org>
(cherry picked from commit c0a56ea0fd92ecb471936b7355ddbecbaea3707c)
diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
index 175ea2981e72..a8c214506786 100644
--- a/xen/arch/arm/arm64/entry.S
+++ b/xen/arch/arm/arm64/entry.S
@@ -338,16 +338,26 @@ guest_sync:
cbnz x1, guest_sync_slowpath /* should be 0 for HVC #0 */
/*
- * Fastest path possible for ARM_SMCCC_ARCH_WORKAROUND_1.
- * The workaround has already been applied on the exception
+ * Fastest path possible for ARM_SMCCC_ARCH_WORKAROUND_1 and
+ * ARM_SMCCC_ARCH_WORKAROUND_3.
+ * The workaround needed has already been applied on the exception
* entry from the guest, so let's quickly get back to the guest.
*
* Note that eor is used because the function identifier cannot
* be encoded as an immediate for cmp.
*/
eor w0, w0, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
- cbnz w0, check_wa2
+ cbz w0, fastpath_out_workaround
+ /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */
+ eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_1_FID ^ ARM_SMCCC_ARCH_WORKAROUND_2_FID)
+ cbz w0, wa2_ssbd
+
+ /* Fastpath out for ARM_SMCCC_ARCH_WORKAROUND_3 */
+ eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_2_FID ^ ARM_SMCCC_ARCH_WORKAROUND_3_FID)
+ cbnz w0, guest_sync_slowpath
+
+fastpath_out_workaround:
/*
* Clobber both x0 and x1 to prevent leakage. Note that thanks
* the eor, x0 = 0.
@@ -356,10 +366,7 @@ guest_sync:
eret
sb
-check_wa2:
- /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */
- eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_1_FID ^ ARM_SMCCC_ARCH_WORKAROUND_2_FID)
- cbnz w0, guest_sync_slowpath
+wa2_ssbd:
#ifdef CONFIG_ARM_SSBD
alternative_cb arm_enable_wa2_handling
b wa2_end
diff --git a/xen/arch/arm/vsmc.c b/xen/arch/arm/vsmc.c
index a36db15fffc0..b633ff2fe897 100644
--- a/xen/arch/arm/vsmc.c
+++ b/xen/arch/arm/vsmc.c
@@ -124,6 +124,10 @@ static bool handle_arch(struct cpu_user_regs *regs)
break;
}
break;
+ case ARM_SMCCC_ARCH_WORKAROUND_3_FID:
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_SMCC_3) )
+ ret = 0;
+ break;
}
set_user_reg(regs, 0, ret);
@@ -132,6 +136,7 @@ static bool handle_arch(struct cpu_user_regs *regs)
}
case ARM_SMCCC_ARCH_WORKAROUND_1_FID:
+ case ARM_SMCCC_ARCH_WORKAROUND_3_FID:
/* No return value */
return true;
["xsa398/xsa398-4.15-6-x86-spec-ctrl-Cease-using-thunk-lfence-on-AMD.patch" (application/octet-stream)]
From 1b50f41b3bd800eb72064063da0c64b86d629f3a Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 7 Mar 2022 16:35:52 +0000
Subject: x86/spec-ctrl: Cease using thunk=lfence on AMD
AMD have updated their Spectre v2 guidance, and lfence/jmp is no longer
considered safe. AMD are recommending using retpoline everywhere.
Retpoline is incompatible with CET. All CET-capable hardware has efficient
IBRS (specifically, not something retrofitted in microcode), so use IBRS (and
STIBP for consistency sake).
This is a logical change on AMD, but not on Intel as the default calculations
would end up with these settings anyway. Leave behind a message if IBRS is
found to be missing.
Also update the default heuristics to never select THUNK_LFENCE. This causes
AMD CPUs to change their default to retpoline.
Also update the printed message to include the AMD MSR_SPEC_CTRL settings, and
STIBP now that we set it for consistency sake.
This is part of XSA-398 / CVE-2021-26401.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 8d03080d2a339840d3a59e0932a94f804e45110d)
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 443802b3d2e5..2392537954c8 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2205,9 +2205,9 @@ to use.
If Xen was compiled with INDIRECT_THUNK support, `bti-thunk=` can be used to
select which of the thunks gets patched into the `__x86_indirect_thunk_%reg`
-locations. The default thunk is `retpoline` (generally preferred for Intel
-hardware), with the alternatives being `jmp` (a `jmp *%reg` gadget, minimal
-overhead), and `lfence` (an `lfence; jmp *%reg` gadget, preferred for AMD).
+locations. The default thunk is `retpoline` (generally preferred), with the
+alternatives being `jmp` (a `jmp *%reg` gadget, minimal overhead), and
+`lfence` (an `lfence; jmp *%reg` gadget).
On hardware supporting IBRS (Indirect Branch Restricted Speculation), the
`ibrs=` option can be used to force or prevent Xen using the feature itself.
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 9301d95bd705..7ded6ecba197 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -367,14 +367,19 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
"\n");
/* Settings for Xen's protection, irrespective of guests. */
- printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s, Other:%s%s%s%s%s\n",
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s%s, Other:%s%s%s%s%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- !boot_cpu_has(X86_FEATURE_IBRSB) ? "No" :
+ (!boot_cpu_has(X86_FEATURE_IBRSB) &&
+ !boot_cpu_has(X86_FEATURE_IBRS)) ? "No" :
(default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
- !boot_cpu_has(X86_FEATURE_SSBD) ? "" :
+ (!boot_cpu_has(X86_FEATURE_STIBP) &&
+ !boot_cpu_has(X86_FEATURE_AMD_STIBP)) ? "" :
+ (default_xen_spec_ctrl & SPEC_CTRL_STIBP) ? " STIBP+" : " STIBP-",
+ (!boot_cpu_has(X86_FEATURE_SSBD) &&
+ !boot_cpu_has(X86_FEATURE_AMD_SSBD)) ? "" :
(default_xen_spec_ctrl & SPEC_CTRL_SSBD) ? " SSBD+" : " SSBD-",
!(caps & ARCH_CAPS_TSX_CTRL) ? "" :
(opt_tsx & 1) ? " TSX+" : " TSX-",
@@ -916,10 +921,23 @@ void __init init_speculation_mitigations(void)
/*
* First, disable the use of retpolines if Xen is using shadow stacks, as
* they are incompatible.
+ *
+ * In the absence of retpolines, IBRS needs to be used for speculative
+ * safety. All CET-capable hardware has efficient IBRS.
*/
- if ( cpu_has_xen_shstk &&
- (opt_thunk == THUNK_DEFAULT || opt_thunk == THUNK_RETPOLINE) )
- thunk = THUNK_JMP;
+ if ( cpu_has_xen_shstk )
+ {
+ if ( !has_spec_ctrl )
+ printk(XENLOG_WARNING "?!? CET active, but no MSR_SPEC_CTRL?\n");
+ else if ( opt_ibrs == -1 )
+ {
+ opt_ibrs = ibrs = true;
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS | SPEC_CTRL_STIBP;
+ }
+
+ if ( opt_thunk == THUNK_DEFAULT || opt_thunk == THUNK_RETPOLINE )
+ thunk = THUNK_JMP;
+ }
/*
* Has the user specified any custom BTI mitigations? If so, follow their
@@ -939,16 +957,10 @@ void __init init_speculation_mitigations(void)
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
{
/*
- * AMD's recommended mitigation is to set lfence as being dispatch
- * serialising, and to use IND_THUNK_LFENCE.
- */
- if ( cpu_has_lfence_dispatch )
- thunk = THUNK_LFENCE;
- /*
- * On Intel hardware, we'd like to use retpoline in preference to
+ * On all hardware, we'd like to use retpoline in preference to
* IBRS, but only if it is safe on this hardware.
*/
- else if ( retpoline_safe(caps) )
+ if ( retpoline_safe(caps) )
thunk = THUNK_RETPOLINE;
else if ( has_spec_ctrl )
ibrs = true;
["xsa398/xsa398-4.16-1-xen-arm-Introduce-new-Arm-processors.patch" (application/octet-stream)]
From 8d18b03c95850239a1a9ebaeb565936c7c9ae070 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Tue, 15 Feb 2022 10:37:51 +0000
Subject: xen/arm: Introduce new Arm processors
Add some new processor identifiers in processor.h and sync Xen
definitions with status of Linux 5.17 (declared in
arch/arm64/include/asm/cputype.h).
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 35d1b85a6b43483f6bd007d48757434e54743e98)
diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index 8ab2940f688e..852b5f3c24b8 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -65,6 +65,7 @@
#define ARM_CPU_PART_CORTEX_A17 0xC0E
#define ARM_CPU_PART_CORTEX_A15 0xC0F
#define ARM_CPU_PART_CORTEX_A53 0xD03
+#define ARM_CPU_PART_CORTEX_A35 0xD04
#define ARM_CPU_PART_CORTEX_A55 0xD05
#define ARM_CPU_PART_CORTEX_A57 0xD07
#define ARM_CPU_PART_CORTEX_A72 0xD08
@@ -72,11 +73,20 @@
#define ARM_CPU_PART_CORTEX_A75 0xD0A
#define ARM_CPU_PART_CORTEX_A76 0xD0B
#define ARM_CPU_PART_NEOVERSE_N1 0xD0C
+#define ARM_CPU_PART_CORTEX_A77 0xD0D
+#define ARM_CPU_PART_NEOVERSE_V1 0xD40
+#define ARM_CPU_PART_CORTEX_A78 0xD41
+#define ARM_CPU_PART_CORTEX_X1 0xD44
+#define ARM_CPU_PART_CORTEX_A710 0xD47
+#define ARM_CPU_PART_CORTEX_X2 0xD48
+#define ARM_CPU_PART_NEOVERSE_N2 0xD49
+#define ARM_CPU_PART_CORTEX_A78C 0xD4B
#define MIDR_CORTEX_A12 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A12)
#define MIDR_CORTEX_A17 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A17)
#define MIDR_CORTEX_A15 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A15)
#define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
+#define MIDR_CORTEX_A35 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A35)
#define MIDR_CORTEX_A55 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A55)
#define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57)
#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72)
@@ -84,6 +94,14 @@
#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75)
#define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A76)
#define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N1)
+#define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A77)
+#define MIDR_NEOVERSE_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V1)
+#define MIDR_CORTEX_A78 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78)
+#define MIDR_CORTEX_X1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1)
+#define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710)
+#define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2)
+#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)
+#define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C)
/* MPIDR Multiprocessor Affinity Register */
#define _MPIDR_UP (30)
["xsa398/xsa398-4.16-2-xen-arm-move-errata-CSV2-check-earlier.patch" (application/octet-stream)]
From 3d963874461b3001e33f3ff90e285670f04d16c4 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Tue, 15 Feb 2022 10:39:47 +0000
Subject: xen/arm: move errata CSV2 check earlier
CSV2 availability check is done after printing to the user that
workaround 1 will be used. Move the check before to prevent saying to the
user that workaround 1 is used when it is not because it is not needed.
This will also allow to reuse install_bp_hardening_vec function for
other use cases.
Code previously returning "true", now returns "0" to conform to
enable_smccc_arch_workaround_1 returning an int and surrounding code
doing a "return 0" if workaround is not needed.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <julien@xen.org>
(cherry picked from commit 599616d70eb886b9ad0ef9d6b51693ce790504ba)
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index b398d480f113..00f9ebe9cee0 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -103,13 +103,6 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry,
printk(XENLOG_INFO "CPU%u will %s on exception entry\n",
smp_processor_id(), desc);
- /*
- * No need to install hardened vector when the processor has
- * ID_AA64PRF0_EL1.CSV2 set.
- */
- if ( cpu_data[smp_processor_id()].pfr64.csv2 )
- return true;
-
spin_lock(&bp_lock);
/*
@@ -167,6 +160,13 @@ static int enable_smccc_arch_workaround_1(void *data)
if ( !entry->matches(entry) )
return 0;
+ /*
+ * No need to install hardened vector when the processor has
+ * ID_AA64PRF0_EL1.CSV2 set.
+ */
+ if ( cpu_data[smp_processor_id()].pfr64.csv2 )
+ return 0;
+
if ( smccc_ver < SMCCC_VERSION(1, 1) )
goto warn;
["xsa398/xsa398-4.16-3-xen-arm-Add-ECBHB-and-CLEARBHB-ID-fields.patch" (application/octet-stream)]
From 8aa3833db97e8fe1143c5ece110b9321ce1494ea Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Wed, 23 Feb 2022 09:42:18 +0000
Subject: xen/arm: Add ECBHB and CLEARBHB ID fields
Introduce ID coprocessor register ID_AA64ISAR2_EL1.
Add definitions in cpufeature and sysregs of ECBHB field in mmfr1 and
CLEARBHB in isar2 ID coprocessor registers.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 4b68d12d98b8790d8002fcc2c25a9d713374a4d7)
diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c
index 6e51f530a80e..a58965f7b9bf 100644
--- a/xen/arch/arm/cpufeature.c
+++ b/xen/arch/arm/cpufeature.c
@@ -122,6 +122,7 @@ void identify_cpu(struct cpuinfo_arm *c)
c->isa64.bits[0] = READ_SYSREG(ID_AA64ISAR0_EL1);
c->isa64.bits[1] = READ_SYSREG(ID_AA64ISAR1_EL1);
+ c->isa64.bits[2] = READ_SYSREG(ID_AA64ISAR2_EL1);
c->zfr64.bits[0] = READ_SYSREG(ID_AA64ZFR0_EL1);
diff --git a/xen/include/asm-arm/arm64/sysregs.h b/xen/include/asm-arm/arm64/sysregs.h
index d7e4772f217f..eac08ed33f53 100644
--- a/xen/include/asm-arm/arm64/sysregs.h
+++ b/xen/include/asm-arm/arm64/sysregs.h
@@ -84,6 +84,9 @@
#ifndef ID_DFR1_EL1
#define ID_DFR1_EL1 S3_0_C0_C3_5
#endif
+#ifndef ID_AA64ISAR2_EL1
+#define ID_AA64ISAR2_EL1 S3_0_C0_C6_2
+#endif
/* ID registers (imported from arm64/include/asm/sysreg.h in Linux) */
@@ -139,6 +142,9 @@
#define ID_AA64ISAR1_GPI_NI 0x0
#define ID_AA64ISAR1_GPI_IMP_DEF 0x1
+/* id_aa64isar2 */
+#define ID_AA64ISAR2_CLEARBHB_SHIFT 28
+
/* id_aa64pfr0 */
#define ID_AA64PFR0_CSV3_SHIFT 60
#define ID_AA64PFR0_CSV2_SHIFT 56
@@ -232,6 +238,7 @@
#define ID_AA64MMFR0_PARANGE_52 0x6
/* id_aa64mmfr1 */
+#define ID_AA64MMFR1_ECBHB_SHIFT 60
#define ID_AA64MMFR1_ETS_SHIFT 36
#define ID_AA64MMFR1_TWED_SHIFT 32
#define ID_AA64MMFR1_XNX_SHIFT 28
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index 8a5afbaf0baf..db126508f159 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -243,14 +243,15 @@ struct cpuinfo_arm {
unsigned long lo:4;
unsigned long pan:4;
unsigned long __res1:8;
- unsigned long __res2:32;
+ unsigned long __res2:28;
+ unsigned long ecbhb:4;
unsigned long __res3:64;
};
} mm64;
union {
- register_t bits[2];
+ register_t bits[3];
struct {
/* ISAR0 */
unsigned long __res0:4;
@@ -286,6 +287,12 @@ struct cpuinfo_arm {
unsigned long dgh:4;
unsigned long i8mm:4;
unsigned long __res2:8;
+
+ /* ISAR2 */
+ unsigned long __res3:28;
+ unsigned long clearbhb:4;
+
+ unsigned long __res4:32;
};
} isa64;
["xsa398/xsa398-4.16-4-xen-arm-Add-Spectre-BHB-handling.patch" (application/octet-stream)]
From 789523a2aac88e3668f9c4ad892fa47b5f6bf1a7 Mon Sep 17 00:00:00 2001
From: Rahul Singh <rahul.singh@arm.com>
Date: Mon, 14 Feb 2022 18:47:32 +0000
Subject: xen/arm: Add Spectre BHB handling
This commit is adding Spectre BHB handling to Xen on Arm.
The commit is introducing new alternative code to be executed during
exception entry:
- SMCC workaround 3 call
- loop workaround (with 8, 24 or 32 iterations)
- use of new clearbhb instruction
Cpuerrata is modified by this patch to apply the required workaround for
CPU affected by Spectre BHB when CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR is
enabled.
To do this the system previously used to apply smcc workaround 1 is
reused and new alternative code to be copied in the exception handler is
introduced.
To define the type of workaround required by a processor, 4 new cpu
capabilities are introduced (for each number of loop and for smcc
workaround 3).
When a processor is affected, enable_spectre_bhb_workaround is called
and if the processor does not have CSV2 set to 3 or ECBHB feature (which
would mean that the processor is doing what is required in hardware),
the proper code is enabled at exception entry.
In the case where workaround 3 is not supported by the firmware, we
enable workaround 1 when possible as it will also mitigate Spectre BHB
on systems without CSV2.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Julien Grall <julien@xen.org>
(cherry picked from commit 62c91eb66a2904eefb1d1d9642e3697a1e3c3a3c)
diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
index d8743d955c4a..4e6382522048 100644
--- a/xen/arch/arm/arm64/bpi.S
+++ b/xen/arch/arm/arm64/bpi.S
@@ -58,16 +58,42 @@ ENTRY(__bp_harden_hyp_vecs_start)
.endr
ENTRY(__bp_harden_hyp_vecs_end)
-ENTRY(__smccc_workaround_1_smc_start)
+.macro mitigate_spectre_bhb_loop count
+ENTRY(__mitigate_spectre_bhb_loop_start_\count)
+ stp x0, x1, [sp, #-16]!
+ mov x0, \count
+.Lspectre_bhb_loop\@:
+ b . + 4
+ subs x0, x0, #1
+ b.ne .Lspectre_bhb_loop\@
+ sb
+ ldp x0, x1, [sp], #16
+ENTRY(__mitigate_spectre_bhb_loop_end_\count)
+.endm
+
+.macro smccc_workaround num smcc_id
+ENTRY(__smccc_workaround_smc_start_\num)
sub sp, sp, #(8 * 4)
stp x0, x1, [sp, #(8 * 2)]
stp x2, x3, [sp, #(8 * 0)]
- mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
+ mov w0, \smcc_id
smc #0
ldp x2, x3, [sp, #(8 * 0)]
ldp x0, x1, [sp, #(8 * 2)]
add sp, sp, #(8 * 4)
-ENTRY(__smccc_workaround_1_smc_end)
+ENTRY(__smccc_workaround_smc_end_\num)
+.endm
+
+ENTRY(__mitigate_spectre_bhb_clear_insn_start)
+ clearbhb
+ isb
+ENTRY(__mitigate_spectre_bhb_clear_insn_end)
+
+mitigate_spectre_bhb_loop 8
+mitigate_spectre_bhb_loop 24
+mitigate_spectre_bhb_loop 32
+smccc_workaround 1, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
+smccc_workaround 3, #ARM_SMCCC_ARCH_WORKAROUND_3_FID
/*
* Local variables:
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index 00f9ebe9cee0..ae649d16ef02 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -145,7 +145,16 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry,
return ret;
}
-extern char __smccc_workaround_1_smc_start[], __smccc_workaround_1_smc_end[];
+extern char __smccc_workaround_smc_start_1[], __smccc_workaround_smc_end_1[];
+extern char __smccc_workaround_smc_start_3[], __smccc_workaround_smc_end_3[];
+extern char __mitigate_spectre_bhb_clear_insn_start[],
+ __mitigate_spectre_bhb_clear_insn_end[];
+extern char __mitigate_spectre_bhb_loop_start_8[],
+ __mitigate_spectre_bhb_loop_end_8[];
+extern char __mitigate_spectre_bhb_loop_start_24[],
+ __mitigate_spectre_bhb_loop_end_24[];
+extern char __mitigate_spectre_bhb_loop_start_32[],
+ __mitigate_spectre_bhb_loop_end_32[];
static int enable_smccc_arch_workaround_1(void *data)
{
@@ -176,8 +185,8 @@ static int enable_smccc_arch_workaround_1(void *data)
if ( (int)res.a0 < 0 )
goto warn;
- return !install_bp_hardening_vec(entry,__smccc_workaround_1_smc_start,
- __smccc_workaround_1_smc_end,
+ return !install_bp_hardening_vec(entry,__smccc_workaround_smc_start_1,
+ __smccc_workaround_smc_end_1,
"call ARM_SMCCC_ARCH_WORKAROUND_1");
warn:
@@ -187,6 +196,93 @@ static int enable_smccc_arch_workaround_1(void *data)
return 0;
}
+/*
+ * Spectre BHB Mitigation
+ *
+ * CPU is either:
+ * - Having CVS2.3 so it is not affected.
+ * - Having ECBHB and is clearing the branch history buffer when an exception
+ * to a different exception level is happening so no mitigation is needed.
+ * - Mitigating using a loop on exception entry (number of loop depending on
+ * the CPU).
+ * - Mitigating using the firmware.
+ */
+static int enable_spectre_bhb_workaround(void *data)
+{
+ const struct arm_cpu_capabilities *entry = data;
+
+ /*
+ * Enable callbacks are called on every CPU based on the capabilities, so
+ * double-check whether the CPU matches the entry.
+ */
+ if ( !entry->matches(entry) )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].pfr64.csv2 == 3 )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].mm64.ecbhb )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].isa64.clearbhb )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_clear_insn_start,
+ __mitigate_spectre_bhb_clear_insn_end,
+ "use clearBHB instruction");
+
+ /* Apply solution depending on hwcaps set on arm_errata */
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_8) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_8,
+ __mitigate_spectre_bhb_loop_end_8,
+ "use 8 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_24) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_24,
+ __mitigate_spectre_bhb_loop_end_24,
+ "use 24 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_32) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_32,
+ __mitigate_spectre_bhb_loop_end_32,
+ "use 32 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_SMCC_3) )
+ {
+ struct arm_smccc_res res;
+
+ if ( smccc_ver < SMCCC_VERSION(1, 1) )
+ goto warn;
+
+ arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FID,
+ ARM_SMCCC_ARCH_WORKAROUND_3_FID, &res);
+ /* The return value is in the lower 32-bits. */
+ if ( (int)res.a0 < 0 )
+ {
+ /*
+ * On processor affected with CSV2=0, workaround 1 will mitigate
+ * both Spectre v2 and BHB so use it when available
+ */
+ if ( enable_smccc_arch_workaround_1(data) )
+ return 1;
+
+ goto warn;
+ }
+
+ return !install_bp_hardening_vec(entry,__smccc_workaround_smc_start_3,
+ __smccc_workaround_smc_end_3,
+ "call ARM_SMCCC_ARCH_WORKAROUND_3");
+ }
+
+warn:
+ printk_once("**** No support for any spectre BHB workaround. ****\n"
+ "**** Please update your firmware. ****\n");
+
+ return 0;
+}
+
#endif /* CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR */
/* Hardening Branch predictor code for Arm32 */
@@ -446,19 +542,77 @@ static const struct arm_cpu_capabilities arm_errata[] = {
},
{
.capability = ARM_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ MIDR_RANGE(MIDR_CORTEX_A72, 0, 1 << MIDR_VARIANT_SHIFT),
.enable = enable_smccc_arch_workaround_1,
},
{
- .capability = ARM_HARDEN_BRANCH_PREDICTOR,
+ .capability = ARM_WORKAROUND_BHB_SMCC_3,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .enable = enable_smccc_arch_workaround_1,
+ .enable = enable_spectre_bhb_workaround,
},
{
- .capability = ARM_HARDEN_BRANCH_PREDICTOR,
+ .capability = ARM_WORKAROUND_BHB_SMCC_3,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- .enable = enable_smccc_arch_workaround_1,
+ .enable = enable_spectre_bhb_workaround,
+ },
+ /* spectre BHB */
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_8,
+ MIDR_RANGE(MIDR_CORTEX_A72, 1 << MIDR_VARIANT_SHIFT,
+ (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A76),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A77),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_X1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_X2),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
+ .enable = enable_spectre_bhb_workaround,
},
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+
#endif
#ifdef CONFIG_ARM32_HARDEN_BRANCH_PREDICTOR
{
diff --git a/xen/include/asm-arm/arm64/macros.h b/xen/include/asm-arm/arm64/macros.h
index 5ad66efd6ba4..140e223b4c99 100644
--- a/xen/include/asm-arm/arm64/macros.h
+++ b/xen/include/asm-arm/arm64/macros.h
@@ -27,6 +27,11 @@
sb
.endm
+ /* clearbhb instruction clearing the branch history */
+ .macro clearbhb
+ hint #22
+ .endm
+
/*
* Register aliases.
*/
diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h
index db126508f159..f7368766c07c 100644
--- a/xen/include/asm-arm/cpufeature.h
+++ b/xen/include/asm-arm/cpufeature.h
@@ -63,8 +63,12 @@
#define ARM64_WORKAROUND_AT_SPECULATE 9
#define ARM_WORKAROUND_858921 10
#define ARM64_WORKAROUND_REPEAT_TLBI 11
+#define ARM_WORKAROUND_BHB_LOOP_8 12
+#define ARM_WORKAROUND_BHB_LOOP_24 13
+#define ARM_WORKAROUND_BHB_LOOP_32 14
+#define ARM_WORKAROUND_BHB_SMCC_3 15
-#define ARM_NCAPS 12
+#define ARM_NCAPS 16
#ifndef __ASSEMBLY__
diff --git a/xen/include/asm-arm/smccc.h b/xen/include/asm-arm/smccc.h
index 9d94beb3df2d..b3dbeecc90ad 100644
--- a/xen/include/asm-arm/smccc.h
+++ b/xen/include/asm-arm/smccc.h
@@ -334,6 +334,12 @@ void __arm_smccc_1_0_smc(register_t a0, register_t a1, register_t a2,
ARM_SMCCC_OWNER_ARCH, \
0x7FFF)
+#define ARM_SMCCC_ARCH_WORKAROUND_3_FID \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
+ ARM_SMCCC_CONV_32, \
+ ARM_SMCCC_OWNER_ARCH, \
+ 0x3FFF)
+
/* SMCCC error codes */
#define ARM_SMCCC_NOT_REQUIRED (-2)
#define ARM_SMCCC_ERR_UNKNOWN_FUNCTION (-1)
["xsa398/xsa398-4.16-5-xen-arm-Allow-to-discover-and-use-SMCCC_ARCH_WORKARO.patch" (application/octet-stream)]
From dab616cd3d4856a7a4d4f3a429a82dbdbf1aeeb9 Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Thu, 17 Feb 2022 14:52:54 +0000
Subject: xen/arm: Allow to discover and use SMCCC_ARCH_WORKAROUND_3
Allow guest to discover whether or not SMCCC_ARCH_WORKAROUND_3 is
supported and create a fastpath in the code to handle guests request to
do the workaround.
The function SMCCC_ARCH_WORKAROUND_3 will be called by the guest for
flushing the branch history. So we want the handling to be as fast as
possible.
As the mitigation is applied on every guest exit, we can check for the
call before saving all context and return very early.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <julien@xen.org>
(cherry picked from commit c0a56ea0fd92ecb471936b7355ddbecbaea3707c)
diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
index fc3811ad0ad5..cf7b9d826f54 100644
--- a/xen/arch/arm/arm64/entry.S
+++ b/xen/arch/arm/arm64/entry.S
@@ -336,16 +336,26 @@ guest_sync:
cbnz x1, guest_sync_slowpath /* should be 0 for HVC #0 */
/*
- * Fastest path possible for ARM_SMCCC_ARCH_WORKAROUND_1.
- * The workaround has already been applied on the exception
+ * Fastest path possible for ARM_SMCCC_ARCH_WORKAROUND_1 and
+ * ARM_SMCCC_ARCH_WORKAROUND_3.
+ * The workaround needed has already been applied on the exception
* entry from the guest, so let's quickly get back to the guest.
*
* Note that eor is used because the function identifier cannot
* be encoded as an immediate for cmp.
*/
eor w0, w0, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
- cbnz w0, check_wa2
+ cbz w0, fastpath_out_workaround
+ /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */
+ eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_1_FID ^ ARM_SMCCC_ARCH_WORKAROUND_2_FID)
+ cbz w0, wa2_ssbd
+
+ /* Fastpath out for ARM_SMCCC_ARCH_WORKAROUND_3 */
+ eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_2_FID ^ ARM_SMCCC_ARCH_WORKAROUND_3_FID)
+ cbnz w0, guest_sync_slowpath
+
+fastpath_out_workaround:
/*
* Clobber both x0 and x1 to prevent leakage. Note that thanks
* the eor, x0 = 0.
@@ -354,10 +364,7 @@ guest_sync:
eret
sb
-check_wa2:
- /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */
- eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_1_FID ^ ARM_SMCCC_ARCH_WORKAROUND_2_FID)
- cbnz w0, guest_sync_slowpath
+wa2_ssbd:
#ifdef CONFIG_ARM_SSBD
alternative_cb arm_enable_wa2_handling
b wa2_end
diff --git a/xen/arch/arm/vsmc.c b/xen/arch/arm/vsmc.c
index a36db15fffc0..b633ff2fe897 100644
--- a/xen/arch/arm/vsmc.c
+++ b/xen/arch/arm/vsmc.c
@@ -124,6 +124,10 @@ static bool handle_arch(struct cpu_user_regs *regs)
break;
}
break;
+ case ARM_SMCCC_ARCH_WORKAROUND_3_FID:
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_SMCC_3) )
+ ret = 0;
+ break;
}
set_user_reg(regs, 0, ret);
@@ -132,6 +136,7 @@ static bool handle_arch(struct cpu_user_regs *regs)
}
case ARM_SMCCC_ARCH_WORKAROUND_1_FID:
+ case ARM_SMCCC_ARCH_WORKAROUND_3_FID:
/* No return value */
return true;
["xsa398/xsa398-4.16-6-x86-spec-ctrl-Cease-using-thunk-lfence-on-AMD.patch" (application/octet-stream)]
From c374a8c5cc74535e16410b7a0d9e92bf5de54f79 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 7 Mar 2022 16:35:52 +0000
Subject: x86/spec-ctrl: Cease using thunk=lfence on AMD
AMD have updated their Spectre v2 guidance, and lfence/jmp is no longer
considered safe. AMD are recommending using retpoline everywhere.
Retpoline is incompatible with CET. All CET-capable hardware has efficient
IBRS (specifically, not something retrofitted in microcode), so use IBRS (and
STIBP for consistency sake).
This is a logical change on AMD, but not on Intel as the default calculations
would end up with these settings anyway. Leave behind a message if IBRS is
found to be missing.
Also update the default heuristics to never select THUNK_LFENCE. This causes
AMD CPUs to change their default to retpoline.
Also update the printed message to include the AMD MSR_SPEC_CTRL settings, and
STIBP now that we set it for consistency sake.
This is part of XSA-398 / CVE-2021-26401.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 8d03080d2a339840d3a59e0932a94f804e45110d)
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 995197f4b23e..f606dc0e14c1 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2269,9 +2269,9 @@ to use.
If Xen was compiled with INDIRECT_THUNK support, `bti-thunk=` can be used to
select which of the thunks gets patched into the `__x86_indirect_thunk_%reg`
-locations. The default thunk is `retpoline` (generally preferred for Intel
-hardware), with the alternatives being `jmp` (a `jmp *%reg` gadget, minimal
-overhead), and `lfence` (an `lfence; jmp *%reg` gadget, preferred for AMD).
+locations. The default thunk is `retpoline` (generally preferred), with the
+alternatives being `jmp` (a `jmp *%reg` gadget, minimal overhead), and
+`lfence` (an `lfence; jmp *%reg` gadget).
On hardware supporting IBRS (Indirect Branch Restricted Speculation), the
`ibrs=` option can be used to force or prevent Xen using the feature itself.
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index cbeeb199037e..ae076bec3ab0 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -367,14 +367,19 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
"\n");
/* Settings for Xen's protection, irrespective of guests. */
- printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s, Other:%s%s%s%s%s\n",
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s%s, Other:%s%s%s%s%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- !boot_cpu_has(X86_FEATURE_IBRSB) ? "No" :
+ (!boot_cpu_has(X86_FEATURE_IBRSB) &&
+ !boot_cpu_has(X86_FEATURE_IBRS)) ? "No" :
(default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
- !boot_cpu_has(X86_FEATURE_SSBD) ? "" :
+ (!boot_cpu_has(X86_FEATURE_STIBP) &&
+ !boot_cpu_has(X86_FEATURE_AMD_STIBP)) ? "" :
+ (default_xen_spec_ctrl & SPEC_CTRL_STIBP) ? " STIBP+" : " STIBP-",
+ (!boot_cpu_has(X86_FEATURE_SSBD) &&
+ !boot_cpu_has(X86_FEATURE_AMD_SSBD)) ? "" :
(default_xen_spec_ctrl & SPEC_CTRL_SSBD) ? " SSBD+" : " SSBD-",
!(caps & ARCH_CAPS_TSX_CTRL) ? "" :
(opt_tsx & 1) ? " TSX+" : " TSX-",
@@ -945,10 +950,23 @@ void __init init_speculation_mitigations(void)
/*
* First, disable the use of retpolines if Xen is using shadow stacks, as
* they are incompatible.
+ *
+ * In the absence of retpolines, IBRS needs to be used for speculative
+ * safety. All CET-capable hardware has efficient IBRS.
*/
- if ( cpu_has_xen_shstk &&
- (opt_thunk == THUNK_DEFAULT || opt_thunk == THUNK_RETPOLINE) )
- thunk = THUNK_JMP;
+ if ( cpu_has_xen_shstk )
+ {
+ if ( !has_spec_ctrl )
+ printk(XENLOG_WARNING "?!? CET active, but no MSR_SPEC_CTRL?\n");
+ else if ( opt_ibrs == -1 )
+ {
+ opt_ibrs = ibrs = true;
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS | SPEC_CTRL_STIBP;
+ }
+
+ if ( opt_thunk == THUNK_DEFAULT || opt_thunk == THUNK_RETPOLINE )
+ thunk = THUNK_JMP;
+ }
/*
* Has the user specified any custom BTI mitigations? If so, follow their
@@ -968,16 +986,10 @@ void __init init_speculation_mitigations(void)
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
{
/*
- * AMD's recommended mitigation is to set lfence as being dispatch
- * serialising, and to use IND_THUNK_LFENCE.
- */
- if ( cpu_has_lfence_dispatch )
- thunk = THUNK_LFENCE;
- /*
- * On Intel hardware, we'd like to use retpoline in preference to
+ * On all hardware, we'd like to use retpoline in preference to
* IBRS, but only if it is safe on this hardware.
*/
- else if ( retpoline_safe(caps) )
+ if ( retpoline_safe(caps) )
thunk = THUNK_RETPOLINE;
else if ( has_spec_ctrl )
ibrs = true;
["xsa398/xsa398-4-xen-arm-Add-Spectre-BHB-handling.patch" (application/octet-stream)]
From 62c91eb66a2904eefb1d1d9642e3697a1e3c3a3c Mon Sep 17 00:00:00 2001
From: Rahul Singh <rahul.singh@arm.com>
Date: Mon, 14 Feb 2022 18:47:32 +0000
Subject: xen/arm: Add Spectre BHB handling
This commit is adding Spectre BHB handling to Xen on Arm.
The commit is introducing new alternative code to be executed during
exception entry:
- SMCC workaround 3 call
- loop workaround (with 8, 24 or 32 iterations)
- use of new clearbhb instruction
Cpuerrata is modified by this patch to apply the required workaround for
CPU affected by Spectre BHB when CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR is
enabled.
To do this the system previously used to apply smcc workaround 1 is
reused and new alternative code to be copied in the exception handler is
introduced.
To define the type of workaround required by a processor, 4 new cpu
capabilities are introduced (for each number of loop and for smcc
workaround 3).
When a processor is affected, enable_spectre_bhb_workaround is called
and if the processor does not have CSV2 set to 3 or ECBHB feature (which
would mean that the processor is doing what is required in hardware),
the proper code is enabled at exception entry.
In the case where workaround 3 is not supported by the firmware, we
enable workaround 1 when possible as it will also mitigate Spectre BHB
on systems without CSV2.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Julien Grall <julien@xen.org>
diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S
index d8743d955c4a..4e6382522048 100644
--- a/xen/arch/arm/arm64/bpi.S
+++ b/xen/arch/arm/arm64/bpi.S
@@ -58,16 +58,42 @@ ENTRY(__bp_harden_hyp_vecs_start)
.endr
ENTRY(__bp_harden_hyp_vecs_end)
-ENTRY(__smccc_workaround_1_smc_start)
+.macro mitigate_spectre_bhb_loop count
+ENTRY(__mitigate_spectre_bhb_loop_start_\count)
+ stp x0, x1, [sp, #-16]!
+ mov x0, \count
+.Lspectre_bhb_loop\@:
+ b . + 4
+ subs x0, x0, #1
+ b.ne .Lspectre_bhb_loop\@
+ sb
+ ldp x0, x1, [sp], #16
+ENTRY(__mitigate_spectre_bhb_loop_end_\count)
+.endm
+
+.macro smccc_workaround num smcc_id
+ENTRY(__smccc_workaround_smc_start_\num)
sub sp, sp, #(8 * 4)
stp x0, x1, [sp, #(8 * 2)]
stp x2, x3, [sp, #(8 * 0)]
- mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
+ mov w0, \smcc_id
smc #0
ldp x2, x3, [sp, #(8 * 0)]
ldp x0, x1, [sp, #(8 * 2)]
add sp, sp, #(8 * 4)
-ENTRY(__smccc_workaround_1_smc_end)
+ENTRY(__smccc_workaround_smc_end_\num)
+.endm
+
+ENTRY(__mitigate_spectre_bhb_clear_insn_start)
+ clearbhb
+ isb
+ENTRY(__mitigate_spectre_bhb_clear_insn_end)
+
+mitigate_spectre_bhb_loop 8
+mitigate_spectre_bhb_loop 24
+mitigate_spectre_bhb_loop 32
+smccc_workaround 1, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
+smccc_workaround 3, #ARM_SMCCC_ARCH_WORKAROUND_3_FID
/*
* Local variables:
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index 00f9ebe9cee0..ae649d16ef02 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -145,7 +145,16 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry,
return ret;
}
-extern char __smccc_workaround_1_smc_start[], __smccc_workaround_1_smc_end[];
+extern char __smccc_workaround_smc_start_1[], __smccc_workaround_smc_end_1[];
+extern char __smccc_workaround_smc_start_3[], __smccc_workaround_smc_end_3[];
+extern char __mitigate_spectre_bhb_clear_insn_start[],
+ __mitigate_spectre_bhb_clear_insn_end[];
+extern char __mitigate_spectre_bhb_loop_start_8[],
+ __mitigate_spectre_bhb_loop_end_8[];
+extern char __mitigate_spectre_bhb_loop_start_24[],
+ __mitigate_spectre_bhb_loop_end_24[];
+extern char __mitigate_spectre_bhb_loop_start_32[],
+ __mitigate_spectre_bhb_loop_end_32[];
static int enable_smccc_arch_workaround_1(void *data)
{
@@ -176,8 +185,8 @@ static int enable_smccc_arch_workaround_1(void *data)
if ( (int)res.a0 < 0 )
goto warn;
- return !install_bp_hardening_vec(entry,__smccc_workaround_1_smc_start,
- __smccc_workaround_1_smc_end,
+ return !install_bp_hardening_vec(entry,__smccc_workaround_smc_start_1,
+ __smccc_workaround_smc_end_1,
"call ARM_SMCCC_ARCH_WORKAROUND_1");
warn:
@@ -187,6 +196,93 @@ static int enable_smccc_arch_workaround_1(void *data)
return 0;
}
+/*
+ * Spectre BHB Mitigation
+ *
+ * CPU is either:
+ * - Having CVS2.3 so it is not affected.
+ * - Having ECBHB and is clearing the branch history buffer when an exception
+ * to a different exception level is happening so no mitigation is needed.
+ * - Mitigating using a loop on exception entry (number of loop depending on
+ * the CPU).
+ * - Mitigating using the firmware.
+ */
+static int enable_spectre_bhb_workaround(void *data)
+{
+ const struct arm_cpu_capabilities *entry = data;
+
+ /*
+ * Enable callbacks are called on every CPU based on the capabilities, so
+ * double-check whether the CPU matches the entry.
+ */
+ if ( !entry->matches(entry) )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].pfr64.csv2 == 3 )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].mm64.ecbhb )
+ return 0;
+
+ if ( cpu_data[smp_processor_id()].isa64.clearbhb )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_clear_insn_start,
+ __mitigate_spectre_bhb_clear_insn_end,
+ "use clearBHB instruction");
+
+ /* Apply solution depending on hwcaps set on arm_errata */
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_8) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_8,
+ __mitigate_spectre_bhb_loop_end_8,
+ "use 8 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_24) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_24,
+ __mitigate_spectre_bhb_loop_end_24,
+ "use 24 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_LOOP_32) )
+ return !install_bp_hardening_vec(entry,
+ __mitigate_spectre_bhb_loop_start_32,
+ __mitigate_spectre_bhb_loop_end_32,
+ "use 32 loops workaround");
+
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_SMCC_3) )
+ {
+ struct arm_smccc_res res;
+
+ if ( smccc_ver < SMCCC_VERSION(1, 1) )
+ goto warn;
+
+ arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FID,
+ ARM_SMCCC_ARCH_WORKAROUND_3_FID, &res);
+ /* The return value is in the lower 32-bits. */
+ if ( (int)res.a0 < 0 )
+ {
+ /*
+ * On processor affected with CSV2=0, workaround 1 will mitigate
+ * both Spectre v2 and BHB so use it when available
+ */
+ if ( enable_smccc_arch_workaround_1(data) )
+ return 1;
+
+ goto warn;
+ }
+
+ return !install_bp_hardening_vec(entry,__smccc_workaround_smc_start_3,
+ __smccc_workaround_smc_end_3,
+ "call ARM_SMCCC_ARCH_WORKAROUND_3");
+ }
+
+warn:
+ printk_once("**** No support for any spectre BHB workaround. ****\n"
+ "**** Please update your firmware. ****\n");
+
+ return 0;
+}
+
#endif /* CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR */
/* Hardening Branch predictor code for Arm32 */
@@ -446,19 +542,77 @@ static const struct arm_cpu_capabilities arm_errata[] = {
},
{
.capability = ARM_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ MIDR_RANGE(MIDR_CORTEX_A72, 0, 1 << MIDR_VARIANT_SHIFT),
.enable = enable_smccc_arch_workaround_1,
},
{
- .capability = ARM_HARDEN_BRANCH_PREDICTOR,
+ .capability = ARM_WORKAROUND_BHB_SMCC_3,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .enable = enable_smccc_arch_workaround_1,
+ .enable = enable_spectre_bhb_workaround,
},
{
- .capability = ARM_HARDEN_BRANCH_PREDICTOR,
+ .capability = ARM_WORKAROUND_BHB_SMCC_3,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- .enable = enable_smccc_arch_workaround_1,
+ .enable = enable_spectre_bhb_workaround,
+ },
+ /* spectre BHB */
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_8,
+ MIDR_RANGE(MIDR_CORTEX_A72, 1 << MIDR_VARIANT_SHIFT,
+ (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A76),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A77),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_X1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_X2),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A710),
+ .enable = enable_spectre_bhb_workaround,
},
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_24,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2),
+ .enable = enable_spectre_bhb_workaround,
+ },
+ {
+ .capability = ARM_WORKAROUND_BHB_LOOP_32,
+ MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V1),
+ .enable = enable_spectre_bhb_workaround,
+ },
+
#endif
#ifdef CONFIG_ARM32_HARDEN_BRANCH_PREDICTOR
{
diff --git a/xen/arch/arm/include/asm/arm64/macros.h b/xen/arch/arm/include/asm/arm64/macros.h
index 5ad66efd6ba4..140e223b4c99 100644
--- a/xen/arch/arm/include/asm/arm64/macros.h
+++ b/xen/arch/arm/include/asm/arm64/macros.h
@@ -27,6 +27,11 @@
sb
.endm
+ /* clearbhb instruction clearing the branch history */
+ .macro clearbhb
+ hint #22
+ .endm
+
/*
* Register aliases.
*/
diff --git a/xen/arch/arm/include/asm/cpufeature.h b/xen/arch/arm/include/asm/cpufeature.h
index db126508f159..f7368766c07c 100644
--- a/xen/arch/arm/include/asm/cpufeature.h
+++ b/xen/arch/arm/include/asm/cpufeature.h
@@ -63,8 +63,12 @@
#define ARM64_WORKAROUND_AT_SPECULATE 9
#define ARM_WORKAROUND_858921 10
#define ARM64_WORKAROUND_REPEAT_TLBI 11
+#define ARM_WORKAROUND_BHB_LOOP_8 12
+#define ARM_WORKAROUND_BHB_LOOP_24 13
+#define ARM_WORKAROUND_BHB_LOOP_32 14
+#define ARM_WORKAROUND_BHB_SMCC_3 15
-#define ARM_NCAPS 12
+#define ARM_NCAPS 16
#ifndef __ASSEMBLY__
diff --git a/xen/arch/arm/include/asm/smccc.h b/xen/arch/arm/include/asm/smccc.h
index 9d94beb3df2d..b3dbeecc90ad 100644
--- a/xen/arch/arm/include/asm/smccc.h
+++ b/xen/arch/arm/include/asm/smccc.h
@@ -334,6 +334,12 @@ void __arm_smccc_1_0_smc(register_t a0, register_t a1, register_t a2,
ARM_SMCCC_OWNER_ARCH, \
0x7FFF)
+#define ARM_SMCCC_ARCH_WORKAROUND_3_FID \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
+ ARM_SMCCC_CONV_32, \
+ ARM_SMCCC_OWNER_ARCH, \
+ 0x3FFF)
+
/* SMCCC error codes */
#define ARM_SMCCC_NOT_REQUIRED (-2)
#define ARM_SMCCC_ERR_UNKNOWN_FUNCTION (-1)
["xsa398/xsa398-5-xen-arm-Allow-to-discover-and-use-SMCCC_ARCH_WORKARO.patch" (application/octet-stream)]
From c0a56ea0fd92ecb471936b7355ddbecbaea3707c Mon Sep 17 00:00:00 2001
From: Bertrand Marquis <bertrand.marquis@arm.com>
Date: Thu, 17 Feb 2022 14:52:54 +0000
Subject: xen/arm: Allow to discover and use SMCCC_ARCH_WORKAROUND_3
Allow guest to discover whether or not SMCCC_ARCH_WORKAROUND_3 is
supported and create a fastpath in the code to handle guests request to
do the workaround.
The function SMCCC_ARCH_WORKAROUND_3 will be called by the guest for
flushing the branch history. So we want the handling to be as fast as
possible.
As the mitigation is applied on every guest exit, we can check for the
call before saving all context and return very early.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <julien@xen.org>
diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S
index 8c48087256d6..95f1a9268419 100644
--- a/xen/arch/arm/arm64/entry.S
+++ b/xen/arch/arm/arm64/entry.S
@@ -365,16 +365,26 @@ guest_sync:
cbnz x1, guest_sync_slowpath /* should be 0 for HVC #0 */
/*
- * Fastest path possible for ARM_SMCCC_ARCH_WORKAROUND_1.
- * The workaround has already been applied on the exception
+ * Fastest path possible for ARM_SMCCC_ARCH_WORKAROUND_1 and
+ * ARM_SMCCC_ARCH_WORKAROUND_3.
+ * The workaround needed has already been applied on the exception
* entry from the guest, so let's quickly get back to the guest.
*
* Note that eor is used because the function identifier cannot
* be encoded as an immediate for cmp.
*/
eor w0, w0, #ARM_SMCCC_ARCH_WORKAROUND_1_FID
- cbnz w0, check_wa2
+ cbz w0, fastpath_out_workaround
+ /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */
+ eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_1_FID ^ ARM_SMCCC_ARCH_WORKAROUND_2_FID)
+ cbz w0, wa2_ssbd
+
+ /* Fastpath out for ARM_SMCCC_ARCH_WORKAROUND_3 */
+ eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_2_FID ^ ARM_SMCCC_ARCH_WORKAROUND_3_FID)
+ cbnz w0, guest_sync_slowpath
+
+fastpath_out_workaround:
/*
* Clobber both x0 and x1 to prevent leakage. Note that thanks
* the eor, x0 = 0.
@@ -383,10 +393,7 @@ guest_sync:
eret
sb
-check_wa2:
- /* ARM_SMCCC_ARCH_WORKAROUND_2 handling */
- eor w0, w0, #(ARM_SMCCC_ARCH_WORKAROUND_1_FID ^ ARM_SMCCC_ARCH_WORKAROUND_2_FID)
- cbnz w0, guest_sync_slowpath
+wa2_ssbd:
#ifdef CONFIG_ARM_SSBD
alternative_cb arm_enable_wa2_handling
b wa2_end
diff --git a/xen/arch/arm/vsmc.c b/xen/arch/arm/vsmc.c
index a36db15fffc0..b633ff2fe897 100644
--- a/xen/arch/arm/vsmc.c
+++ b/xen/arch/arm/vsmc.c
@@ -124,6 +124,10 @@ static bool handle_arch(struct cpu_user_regs *regs)
break;
}
break;
+ case ARM_SMCCC_ARCH_WORKAROUND_3_FID:
+ if ( cpus_have_cap(ARM_WORKAROUND_BHB_SMCC_3) )
+ ret = 0;
+ break;
}
set_user_reg(regs, 0, ret);
@@ -132,6 +136,7 @@ static bool handle_arch(struct cpu_user_regs *regs)
}
case ARM_SMCCC_ARCH_WORKAROUND_1_FID:
+ case ARM_SMCCC_ARCH_WORKAROUND_3_FID:
/* No return value */
return true;
["xsa398/xsa398-6-x86-spec-ctrl-Cease-using-thunk-lfence-on-AMD.patch" (application/octet-stream)]
From 8d03080d2a339840d3a59e0932a94f804e45110d Mon Sep 17 00:00:00 2001
From: Andrew Cooper <andrew.cooper3@citrix.com>
Date: Mon, 7 Mar 2022 16:35:52 +0000
Subject: x86/spec-ctrl: Cease using thunk=lfence on AMD
AMD have updated their Spectre v2 guidance, and lfence/jmp is no longer
considered safe. AMD are recommending using retpoline everywhere.
Retpoline is incompatible with CET. All CET-capable hardware has efficient
IBRS (specifically, not something retrofitted in microcode), so use IBRS (and
STIBP for consistency sake).
This is a logical change on AMD, but not on Intel as the default calculations
would end up with these settings anyway. Leave behind a message if IBRS is
found to be missing.
Also update the default heuristics to never select THUNK_LFENCE. This causes
AMD CPUs to change their default to retpoline.
Also update the printed message to include the AMD MSR_SPEC_CTRL settings, and
STIBP now that we set it for consistency sake.
This is part of XSA-398 / CVE-2021-26401.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index eda92ce29e08..1dc7e1ca0706 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2297,9 +2297,9 @@ to use.
If Xen was compiled with INDIRECT_THUNK support, `bti-thunk=` can be used to
select which of the thunks gets patched into the `__x86_indirect_thunk_%reg`
-locations. The default thunk is `retpoline` (generally preferred for Intel
-hardware), with the alternatives being `jmp` (a `jmp *%reg` gadget, minimal
-overhead), and `lfence` (an `lfence; jmp *%reg` gadget, preferred for AMD).
+locations. The default thunk is `retpoline` (generally preferred), with the
+alternatives being `jmp` (a `jmp *%reg` gadget, minimal overhead), and
+`lfence` (an `lfence; jmp *%reg` gadget).
On hardware supporting IBRS (Indirect Branch Restricted Speculation), the
`ibrs=` option can be used to force or prevent Xen using the feature itself.
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 12bf1c5722f9..1408e4c7abd0 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -367,14 +367,19 @@ static void __init print_details(enum ind_thunk thunk, uint64_t caps)
"\n");
/* Settings for Xen's protection, irrespective of guests. */
- printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s, Other:%s%s%s%s%s\n",
+ printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s%s, Other:%s%s%s%s%s\n",
thunk == THUNK_NONE ? "N/A" :
thunk == THUNK_RETPOLINE ? "RETPOLINE" :
thunk == THUNK_LFENCE ? "LFENCE" :
thunk == THUNK_JMP ? "JMP" : "?",
- !boot_cpu_has(X86_FEATURE_IBRSB) ? "No" :
+ (!boot_cpu_has(X86_FEATURE_IBRSB) &&
+ !boot_cpu_has(X86_FEATURE_IBRS)) ? "No" :
(default_xen_spec_ctrl & SPEC_CTRL_IBRS) ? "IBRS+" : "IBRS-",
- !boot_cpu_has(X86_FEATURE_SSBD) ? "" :
+ (!boot_cpu_has(X86_FEATURE_STIBP) &&
+ !boot_cpu_has(X86_FEATURE_AMD_STIBP)) ? "" :
+ (default_xen_spec_ctrl & SPEC_CTRL_STIBP) ? " STIBP+" : " STIBP-",
+ (!boot_cpu_has(X86_FEATURE_SSBD) &&
+ !boot_cpu_has(X86_FEATURE_AMD_SSBD)) ? "" :
(default_xen_spec_ctrl & SPEC_CTRL_SSBD) ? " SSBD+" : " SSBD-",
!(caps & ARCH_CAPS_TSX_CTRL) ? "" :
(opt_tsx & 1) ? " TSX+" : " TSX-",
@@ -947,10 +952,23 @@ void __init init_speculation_mitigations(void)
* First, disable the use of retpolines if Xen is using CET. Retpolines
* are a ROP gadget so incompatbile with Shadow Stacks, while IBT depends
* on executing indirect branches for the safety properties to apply.
+ *
+ * In the absence of retpolines, IBRS needs to be used for speculative
+ * safety. All CET-capable hardware has efficient IBRS.
*/
- if ( (read_cr4() & X86_CR4_CET) &&
- (opt_thunk == THUNK_DEFAULT || opt_thunk == THUNK_RETPOLINE) )
- thunk = THUNK_JMP;
+ if ( read_cr4() & X86_CR4_CET )
+ {
+ if ( !has_spec_ctrl )
+ printk(XENLOG_WARNING "?!? CET active, but no MSR_SPEC_CTRL?\n");
+ else if ( opt_ibrs == -1 )
+ {
+ opt_ibrs = ibrs = true;
+ default_xen_spec_ctrl |= SPEC_CTRL_IBRS | SPEC_CTRL_STIBP;
+ }
+
+ if ( opt_thunk == THUNK_DEFAULT || opt_thunk == THUNK_RETPOLINE )
+ thunk = THUNK_JMP;
+ }
/*
* Has the user specified any custom BTI mitigations? If so, follow their
@@ -970,16 +988,10 @@ void __init init_speculation_mitigations(void)
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) )
{
/*
- * AMD's recommended mitigation is to set lfence as being dispatch
- * serialising, and to use IND_THUNK_LFENCE.
- */
- if ( cpu_has_lfence_dispatch )
- thunk = THUNK_LFENCE;
- /*
- * On Intel hardware, we'd like to use retpoline in preference to
+ * On all hardware, we'd like to use retpoline in preference to
* IBRS, but only if it is safe on this hardware.
*/
- else if ( retpoline_safe(caps) )
+ if ( retpoline_safe(caps) )
thunk = THUNK_RETPOLINE;
else if ( has_spec_ctrl )
ibrs = true;
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic