[prev in list] [next in list] [prev in thread] [next in thread]
List: fwts-devel
Subject: Re: [FWTS-Live] fwts MTRR tests fail on AMD platform if MMIO is above 4G
From: Alex Hung <alex.hung () canonical ! com>
Date: 2020-10-24 0:26:14
Message-ID: CAJ=jquYKC3-GObDzHinq_4Po9RjyL+MMh-Xy3-a_j2TPfSBBKw () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/related)]
[Attachment #4 (multipart/alternative)]
On Fri, Oct 23, 2020 at 2:09 AM Tung, Larry <Larry.Tung@amd.com> wrote:
> [AMD Official Use Only - Internal Distribution Only]
>
>
>
> Hi Alex,
>
>
>
> I've submitted a bug below,
>
> https://bugs.launchpad.net/fwts/+bug/1901146
>
Hi Larry,
Thanks for sharing the information. Let's move our discussion to the
launchpad so it is easier to manage.
>
>
> =====================
>
> The effective memory type can be override by PAT mechanism and MTRRs.
>
>
>
> The PAT(MSR0000_0277) register contains 8 page-attribute fields from PA0
> to PA7. PA fields are selected by using three bits from the page-table
> entries {PAT, PCD, PWT}, and each fields have corresponding memory type in
> the PAT register.
>
>
>
> If the memory range is not specified using the fixed-range and
> variable-range MTRRs, the default memory type in MTRRdefType(MSR0000_02FF)
> would be used.
>
>
>
> The combination of MTRR and PAT memtype are described in AMD64
> Architecture Programmer's Manual Volume 2 (#24593)
>
> https://www.amd.com/system/files/TechDocs/24593.pdf
>
>
>
>
>
> Best Regards,
>
> Larry
>
>
>
> *From:* Alex Hung <alex.hung@canonical.com>
> *Sent:* Thursday, October 22, 2020 3:52 PM
> *To:* Tung, Larry <Larry.Tung@amd.com>
> *Cc:* Lien, Eliot <Eliot.Lien@amd.com>
> *Subject:* Re: [FWTS-Live] fwts MTRR tests fail on AMD platform if MMIO
> is above 4G
>
>
>
> [CAUTION: External Email]
>
>
>
>
>
> On Thu, Oct 22, 2020 at 1:33 AM Tung, Larry <Larry.Tung@amd.com> wrote:
>
> [AMD Official Use Only - Internal Distribution Only]
>
>
>
> Hi Alex,
>
>
>
> Thanks for the information.
>
>
>
> After clarifying, the TOM2-WB (Sys_Cfg[22]=1) would be override with PAT
> (MSR0000_0277) memtype. The page-attribute used to access the PCIe video
> controller MMIO should be WC.
>
>
>
> So, I wonder if it's possible to add a "page-attribute" checking into the
> test tool ?
>
>
>
> Certainly. We really appreciate your feedback and technical details. I was
> able to find a public AMD datasheet
> "54945_PPR_Family_17h_Models_00h-0Fh.pdf", but I don't quite understand how
> it overrides TOM2-WB.
>
>
>
> Will you be able to share some details?
>
>
>
> FYI, you can also send emails to our mailing list "
> fwts-devel@lists.ubuntu.com" where more people can discuss if all
> information is publicly available.
>
>
>
>
>
>
>
>
>
> Thanks,
>
>
>
> Best Regards,
>
> Larry
>
>
>
> *From:* Alex Hung <alex.hung@canonical.com>
> *Sent:* Thursday, October 22, 2020 3:06 AM
> *To:* Tung, Larry <Larry.Tung@amd.com>
> *Cc:* Lien, Eliot <Eliot.Lien@amd.com>
> *Subject:* Re: [FWTS-Live] fwts MTRR tests fail on AMD platform if MMIO
> is above 4G
>
>
>
> [CAUTION: External Email]
>
> Hi,
>
>
>
> That MSR is used to check memory types such as below:
>
>
>
> /* Get the default memory type of memory between 4GB and second top of
> * memory (TOM2) - i.e. is it write back (WB)
> */
> if (strstr(fwts_cpuinfo->vendor_id, "AMD") ||
> strstr(fwts_cpuinfo->vendor_id, "Hygon")) {
> if (fwts_cpu_readmsr(fw, 0, AMD_SYS_CFG_MSR, &amd_sys_conf) == FWTS_OK)
> if (amd_sys_conf & 0x200000)
> amd_Tom2ForceMemTypeWB = true;
> }
>
>
>
> ....
>
>
>
>
>
> /* On AMD platforms, Tom2ForceMemTypeWB overwrites other memory types */
> if (amd_Tom2ForceMemTypeWB && start >= 0x100000000) {
> type = WRITE_BACK;
> return type;
> }
>
>
>
>
>
> However, memory-mapped IO (like this one) should not be write-back:
>
>
>
> "Write back is a storage method in which data is written into the cache
> every time a change occurs, but is written into the corresponding location
> in main memory only at specified intervals or under certain conditions.
>
>
>
> PCI memory-mapped I/O registers aren't RAM that stays the same until the
> CPU changes them - and MMIO can change by itself to reflect hardware
> changes unlike memory. As a result, it is unsuitable for the CPU to write
> to MMIO "only at specific intervals or under certain conditions".
>
>
>
> The scenario should be: you write a command to PCI MMIO, expect the PCI
> hardware to react as soon as writes are issued, and want to read an ack
> from it; however the PCI hardware will not do anything until many
> milliseconds later, long after your reads....
>
>
>
> There are similar discussion @
> https://community.intel.com/t5/Intel-ISA-Extensions/TSX-and-PCI-consistent-memory/td-p/1107558
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcommunity.intel. \
> com%2Ft5%2FIntel-ISA-Extensions%2FTSX-and-PCI-consistent-memory%2Ftd-p%2F1107558&dat \
> a=04%7C01%7CLarry.Tung%40amd.com%7C337e705aab604c47282708d8765f7261%7C3dd8961fe4884e \
> 608e11a82d994e183d%7C0%7C0%7C637389499601755643%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w \
> LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MYGzTQbZ9cAwTJ40oIoVDe90oZ1oRIZU73tyz7BiANk%3D&reserved=0>,
> and I quote
>
>
>
> =======================
>
> FYI: The WB type will not work with memory-mapped IO. You can program
> the bits to set up the mapping as WB, but the system will crash as soon as
> it gets a transaction that it does not know how to handle. It is
> theoretically possible to use WP or WT to get cached reads from MMIO, but
> coherence has to be handled in software.
>
> =======================
>
>
>
>
>
> On Wed, Oct 21, 2020 at 3:04 AM Tung, Larry <Larry.Tung@amd.com> wrote:
>
> [AMD Official Use Only - Internal Distribution Only]
>
>
>
> Hi Alex,
>
>
>
> On AMD system, there is a MSR to force the 4GB ~ TOM2 as WB memory type.
> However, we encounter MTRR test failed on PCIe GFx resource with incorrect
> WB memory type, if BAR is assigned to above 4G memory range. I looked into
> /mtrr.c form git hub, code logic seems would check if PCIe BAR is
> prefetchable, then it will return the "mustnot" mask as WB which may cause
> this failure.
>
>
>
> May I have your comment on this please? Thanks
>
>
>
> 1.
>
> /* if it's PCI mmio -> uncached typically except for video */
>
> if (strstr(string, "0000:")) {
>
> bool pref;
>
>
>
> if (check_prefetchable(fw, string, address,
> &pref) != FWTS_OK)
>
> return FWTS_ERROR;
>
>
>
> if (pref) {
>
> *must = 0;
>
> *mustnot = WRITE_BACK;
>
> } else {
>
> *must = UNCACHED;
>
> *mustnot = (~UNCACHED) &
> (~DEFAULT);
>
> }
>
> }
>
> 1.
>
> if ((type & type_mustnot) != 0) {
>
> failed++;
>
> fwts_failed(fw,
> LOG_LEVEL_CRITICAL,
>
>
> "MTRRIncorrectAttr",
>
> "Memory range
> 0x%" PRIx64 " to 0x%" PRIx64 " (%s) "
>
> "has incorrect
> attribute%s.",
>
> start, end,
>
> c2,
> cache_to_string(type & type_mustnot));
>
> if (type_must == UNCACHED)
>
> skiperror = true;
>
> }
>
>
>
> Error log with FWTS Version V20.07.00
>
> mtrr: MTRR tests.
>
>
> --------------------------------------------------------------------------------
>
> MTRR overview
>
> -------------
>
> Reg 0: 0x0000000000000000 - 0x000000007fffffff ( 2048 MB) Write-Back
>
> Reg 1: 0x0000000080000000 - 0x00000000bfffffff ( 1024 MB) Write-Back
>
>
>
> Test 1 of 3: Validate the kernel MTRR IOMEM setup.
>
> FAILED [CRITICAL] MTRRIncorrectAttr: Test 1, Memory range 0xfce0000000 to
>
> 0xfcefffffff (0000:08:00.0) has incorrect attribute Write-Back.
>
> FAILED [CRITICAL] MTRRIncorrectAttr: Test 1, Memory range 0xfcf0000000 to
>
> 0xfcf01fffff (0000:08:00.0) has incorrect attribute Write-Back.
>
>
>
> Test 2 of 3: Validate the MTRR setup across all processors.
>
> PASSED: Test 2, All processors have the a consistent MTRR setup.
>
>
>
> Test 3 of 3: Test for AMD MtrrFixDramModEn being cleared by the BIOS.
>
> PASSED: Test 3, No MtrrFixDramModEn error detected.
>
>
>
>
>
>
>
> Best Regards,
>
> Larry
>
>
>
>
>
> --
>
> Cheers,
> Alex Hung
>
>
>
> --
>
> Cheers,
> Alex Hung
>
--
Cheers,
Alex Hung
[Attachment #7 (text/html)]
<div dir="ltr"><div dir="ltr"><div class="gmail_default" \
style="font-family:verdana,sans-serif"><br></div></div><br><div \
class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Oct 23, 2020 at 2:09 AM \
Tung, Larry <<a href="mailto:Larry.Tung@amd.com">Larry.Tung@amd.com</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="overflow-wrap: break-word;" lang="EN-US">
<div class="gmail-m_-8327252174044908851WordSection1">
<p class="gmail-m_-8327252174044908851msipheader4d0fcdd7" style="margin:0in"><span \
style="font-size:10pt;font-family:"Arial",sans-serif;color:rgb(0,120,215)">[AMD \
Official Use Only - Internal Distribution Only]</span><u></u><u></u></p> <p \
class="MsoNormal"><u></u> <u></u></p> <p class="MsoNormal">Hi \
Alex,<u></u><u></u></p> <p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I've submitted a bug below,<u></u><u></u></p>
<p class="MsoNormal"><a href="https://bugs.launchpad.net/fwts/+bug/1901146" \
target="_blank">https://bugs.launchpad.net/fwts/+bug/1901146</a></p></div></div></blockquote><div><br></div><div><div \
style="font-family:verdana,sans-serif" class="gmail_default"><div \
class="gmail_default" style="font-family:verdana,sans-serif">Hi Larry,</div><div \
class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div \
class="gmail_default" style="font-family:verdana,sans-serif">Thanks for sharing the \
information. Let's move our discussion to the launchpad so it is easier to \
manage.<br></div><br></div><br></div><div> </div><blockquote class="gmail_quote" \
style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: break-word;" \
lang="EN-US"><div class="gmail-m_-8327252174044908851WordSection1"><p \
class="MsoNormal"><u></u><u></u></p> <p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">=====================<u></u><u></u></p>
<p class="MsoNormal">The effective memory type can be override by PAT mechanism and \
MTRRs.<u></u><u></u></p> <p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">The PAT(MSR0000_0277) register contains 8 page-attribute fields \
from PA0 to PA7. PA fields are selected by using three bits from the page-table \
entries {PAT, PCD, PWT}, and each fields have corresponding memory type in the PAT \
register.<u></u><u></u></p> <p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">If the memory range is not specified using the fixed-range and \
variable-range MTRRs, the default memory type in MTRRdefType(MSR0000_02FF) would be \
used.<u></u><u></u></p> <p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">The combination of MTRR and PAT memtype are described in AMD64 \
Architecture Programmer's Manual Volume 2 (#24593)<u></u><u></u></p> <p \
class="MsoNormal"><a href="https://www.amd.com/system/files/TechDocs/24593.pdf" \
target="_blank">https://www.amd.com/system/files/TechDocs/24593.pdf</a><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><img style="width: 6.243in; height: 6.0902in;" \
id="gmail-m_-8327252174044908851Picture_x0020_3" src="cid:17557ca73ed4ce8e91" \
width="599" height="585" border="0"><u></u><u></u></p> <p class="MsoNormal"><u></u> \
<u></u></p> <div>
<p class="MsoNormal">Best Regards,<u></u><u></u></p>
<p class="MsoNormal">Larry<u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div style="border-color:rgb(225,225,225) currentcolor \
currentcolor;border-style:solid none none;border-width:1pt medium medium;padding:3pt \
0in 0in"> <p class="MsoNormal"><b>From:</b> Alex Hung <<a \
href="mailto:alex.hung@canonical.com" target="_blank">alex.hung@canonical.com</a>> \
<br> <b>Sent:</b> Thursday, October 22, 2020 3:52 PM<br>
<b>To:</b> Tung, Larry <<a href="mailto:Larry.Tung@amd.com" \
target="_blank">Larry.Tung@amd.com</a>><br> <b>Cc:</b> Lien, Eliot <<a \
href="mailto:Eliot.Lien@amd.com" target="_blank">Eliot.Lien@amd.com</a>><br> \
<b>Subject:</b> Re: [FWTS-Live] fwts MTRR tests fail on AMD platform if MMIO is above \
4G<u></u><u></u></p> </div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">[CAUTION: External Email] <u></u><u></u></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"><u></u> \
<u></u></span></p> </div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Thu, Oct 22, 2020 at 1:33 AM Tung, Larry <<a \
href="mailto:Larry.Tung@amd.com" target="_blank">Larry.Tung@amd.com</a>> \
wrote:<u></u><u></u></p> </div>
<blockquote style="border-color:currentcolor currentcolor currentcolor \
rgb(204,204,204);border-style:none none none solid;border-width:medium medium medium \
1pt;padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt"> <div>
<div>
<p style="margin:0in"><span \
style="font-size:10pt;font-family:"Arial",sans-serif;color:rgb(0,120,215)">[AMD \
Official Use Only - Internal Distribution Only]</span><u></u><u></u></p> <p \
class="MsoNormal"> <u></u><u></u></p> <p class="MsoNormal">Hi \
Alex,<u></u><u></u></p> <p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Thanks for the information.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">After clarifying, the TOM2-WB (Sys_Cfg[22]=1)
<span style="color:black;background:white none repeat scroll 0% 0%">would be override \
with PAT (MSR0000_0277) memtype. The page-attribute used to access the PCIe video \
controller MMIO should be WC.</span><u></u><u></u></p> <p class="MsoNormal"> \
<u></u><u></u></p> <p class="MsoNormal">So, I wonder if it's possible to add a \
"page-attribute" checking into the test tool ?<u></u><u></u></p> </div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<div>
<p class="MsoNormal"><span \
style="font-family:"Verdana",sans-serif">Certainly. We really appreciate \
your feedback and technical details. I was able to find a public AMD datasheet \
"54945_PPR_Family_17h_Models_00h-0Fh.pdf", but I don't quite understand \
how it overrides TOM2-WB. <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"><u></u> \
<u></u></span></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">Will \
you be able to share some details?<u></u><u></u></span></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"><u></u> \
<u></u></span></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">FYI, \
you can also send emails to our mailing list "<span \
class="gmail-m_-8327252174044908851gmail-gi"><a \
href="mailto:fwts-devel@lists.ubuntu.com" \
target="_blank">fwts-devel@lists.ubuntu.com</a>" where more people can discuss \
if all information is publicly available.</span><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"><u></u> \
<u></u></span></p> </div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-color:currentcolor currentcolor currentcolor \
rgb(204,204,204);border-style:none none none solid;border-width:medium medium medium \
1pt;padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt"> <div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">Best Regards,<u></u><u></u></p>
<p class="MsoNormal">Larry<u></u><u></u></p>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div style="border-style:solid none none;border-width:1pt medium medium;padding:3pt \
0in 0in;border-color:currentcolor"> <p class="MsoNormal"><b>From:</b> Alex Hung \
<<a href="mailto:alex.hung@canonical.com" \
target="_blank">alex.hung@canonical.com</a>> <br>
<b>Sent:</b> Thursday, October 22, 2020 3:06 AM<br>
<b>To:</b> Tung, Larry <<a href="mailto:Larry.Tung@amd.com" \
target="_blank">Larry.Tung@amd.com</a>><br> <b>Cc:</b> Lien, Eliot <<a \
href="mailto:Eliot.Lien@amd.com" target="_blank">Eliot.Lien@amd.com</a>><br> \
<b>Subject:</b> Re: [FWTS-Live] fwts MTRR tests fail on AMD platform if MMIO is above \
4G<u></u><u></u></p> </div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">[CAUTION: External Email]
<u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal"><span \
style="font-family:"Verdana",sans-serif">Hi,</span><u></u><u></u></p> \
</div> <div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">That \
MSR is used to check memory types such as below:</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">/* Get \
the default memory type of memory between 4GB and second top of<br>
* memory (TOM2) - i.e. is it write back (WB)<br>
*/<br>
if (strstr(fwts_cpuinfo->vendor_id, "AMD") || \
strstr(fwts_cpuinfo->vendor_id, "Hygon")) {<br> if (fwts_cpu_readmsr(fw, \
0, AMD_SYS_CFG_MSR, &amd_sys_conf) == FWTS_OK)<br> if (amd_sys_conf & \
0x200000)<br> amd_Tom2ForceMemTypeWB = true;<br>
}</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span \
style="font-family:"Verdana",sans-serif">....</span><u></u><u></u></p> \
</div> <div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">/* On \
AMD platforms, Tom2ForceMemTypeWB overwrites other memory types */<br> if \
(amd_Tom2ForceMemTypeWB && start >= 0x100000000) {<br> type = \
WRITE_BACK;<br> return type;<br>
}</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span \
style="font-family:"Verdana",sans-serif">However, memory-mapped IO (like \
this one) should not be write-back:</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span \
style="font-family:"Verdana",sans-serif">"</span><span \
style="font-size:12pt;font-family:"Arial",sans-serif;color:rgb(32,33,36);background:white \
none repeat scroll 0% 0%">Write back is a storage method in which data is written \
into the cache every time a change occurs, but is written into the corresponding \
location in main memory only at specified intervals or under certain \
conditions.</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">PCI \
memory-mapped I/O registers aren't RAM that stays the same until the CPU changes \
them - and MMIO can change by itself to reflect hardware changes unlike memory. As a \
result, it is unsuitable for the CPU to write to MMIO "only at specific \
intervals or under certain conditions". </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">The \
scenario should be: you write a command to PCI MMIO, expect the PCI hardware to react \
as soon as writes are issued, and want to read an ack from it; however the PCI \
hardware will not do anything until many milliseconds later, long after your \
reads....</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">There \
are similar discussion @ <a \
href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcommunity.inte \
l.com%2Ft5%2FIntel-ISA-Extensions%2FTSX-and-PCI-consistent-memory%2Ftd-p%2F1107558& \
;data=04%7C01%7CLarry.Tung%40amd.com%7C337e705aab604c47282708d8765f7261%7C3dd8961fe488 \
4e608e11a82d994e183d%7C0%7C0%7C637389499601755643%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w \
LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=MYGzTQbZ9cAwTJ40oIoVDe90oZ1oRIZU73tyz7BiANk%3D&reserved=0" \
target="_blank"> https://community.intel.com/t5/Intel-ISA-Extensions/TSX-and-PCI-consistent-memory/td-p/1107558</a>, \
and I quote</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span \
style="font-family:"Verdana",sans-serif">=======================</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif">FYI: \
The WB type will not work with memory-mapped IO. You can program the bits to set \
up the mapping as WB, but the system will crash as soon as it gets a transaction \
that it does not know how to handle. It is theoretically possible to use WP or WT \
to get cached reads from MMIO, but coherence has to be handled in software. \
</span><u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><span \
style="font-family:"Verdana",sans-serif">=======================</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Verdana",sans-serif"> \
</span><u></u><u></u></p> </div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">On Wed, Oct 21, 2020 at 3:04 AM Tung, Larry <<a \
href="mailto:Larry.Tung@amd.com" target="_blank">Larry.Tung@amd.com</a>> \
wrote:<u></u><u></u></p> </div>
<blockquote style="border-style:none none none solid;border-width:medium medium \
medium 1pt;padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt;border-color:currentcolor \
currentcolor currentcolor rgb(204,204,204)"> <div>
<p style="margin:0in"><span \
style="font-size:10pt;font-family:"Arial",sans-serif;color:rgb(0,120,215)">[AMD \
Official Use Only - Internal Distribution Only]</span><u></u><u></u></p> <p \
class="MsoNormal"> <u></u><u></u></p> <div>
<p class="MsoNormal">Hi Alex,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">On AMD system, there is a MSR to force the 4GB ~ TOM2 as WB \
memory type. However, we encounter MTRR test failed on PCIe GFx resource with \
incorrect WB memory type, if BAR is assigned to above 4G memory range. I looked into \
/mtrr.c form git hub, code logic seems would check if PCIe BAR is prefetchable, then \
it will return the "mustnot" mask as WB which may cause this \
failure.<u></u><u></u></p> <p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">May I have your comment on this please? Thanks<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<ol type="1" start="1">
<li class="MsoNormal">
<u></u><u></u></li></ol>
<p class="MsoNormal"> /* if it's PCI mmio -> \
uncached typically except for video */<u></u><u></u></p> <p class="MsoNormal"> \
if (strstr(string, "0000:")) {<u></u><u></u></p> <p class="MsoNormal"> \
bool pref;<u></u><u></u></p> <p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> if \
(check_prefetchable(fw, string, address, &pref) != FWTS_OK)<u></u><u></u></p> <p \
class="MsoNormal"> \
return FWTS_ERROR;<u></u><u></u></p> <p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> if \
(pref) {<u></u><u></u></p> <p class="MsoNormal"> \
*must = 0;<u></u><u></u></p> <p class="MsoNormal"> \
*<span style="color:black;background:yellow none repeat scroll 0% 0%">mustnot = \
WRITE_BACK;</span><u></u><u></u></p> <p class="MsoNormal"> \
} else {<u></u><u></u></p> <p class="MsoNormal"> \
*must = UNCACHED;<u></u><u></u></p> <p class="MsoNormal"> \
*mustnot = (~UNCACHED) & (~DEFAULT);<u></u><u></u></p> <p class="MsoNormal"> \
}<u></u><u></u></p> <p class="MsoNormal"> \
}<u></u><u></u></p> <ol type="1" start="2">
<li class="MsoNormal">
<u></u><u></u></li></ol>
<p class="MsoNormal">
<span style="color:black;background:yellow none repeat scroll 0% 0%">if ((type & \
type_mustnot) != 0) {</span><u></u><u></u></p> <p class="MsoNormal"> \
failed++;<u></u><u></u></p> <p class="MsoNormal"> \
fwts_failed(fw, LOG_LEVEL_CRITICAL,<u></u><u></u></p> <p class="MsoNormal"> \
"MTRRIncorrectAttr",<u></u><u></u></p> <p class="MsoNormal"> \
"Memory range 0x%" PRIx64 " to 0x%" PRIx64 " (%s) \
"<u></u><u></u></p> <p class="MsoNormal"> \
"has incorrect attribute%s.",<u></u><u></u></p> <p class="MsoNormal"> \
start, end,<u></u><u></u></p> <p class="MsoNormal"> \
c2, cache_to_string(type & type_mustnot));<u></u><u></u></p> <p \
class="MsoNormal"> \
if (type_must == UNCACHED)<u></u><u></u></p> <p class="MsoNormal"> \
skiperror = true;<u></u><u></u></p> <p class="MsoNormal"> \
}<u></u><u></u></p> <p class="MsoNormal"><img style="width: 6.125in; height: \
0.5069in;" id="gmail-m_-8327252174044908851m_5453432093231756307gmail-m_-21134759238815528gmail-m_-6842749692505274978gmail-m_1214234099230522012Picture_x0020_1" \
src="cid:17557ca73ee6917eb2" width="588" height="49" border="0"><u></u><u></u></p> <p \
class="MsoNormal"><img style="width: 6.1041in; height: 1.0347in;" \
id="gmail-m_-8327252174044908851m_5453432093231756307gmail-m_-21134759238815528gmail-m_-6842749692505274978gmail-m_1214234099230522012Picture_x0020_3" \
src="cid:17557ca73ef8546ed3" width="586" height="99" border="0"><u></u><u></u></p> <p \
class="MsoNormal"> <u></u><u></u></p> <p class="MsoNormal">Error log with FWTS \
Version V20.07.00<u></u><u></u></p> <table style="border-collapse:collapse" \
cellspacing="0" cellpadding="0" border="0"> <tbody>
<tr>
<td style="width:503.75pt;border:1pt solid windowtext;padding:0in 5.4pt" width="672" \
valign="top"> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier New";color:black">mtrr: MTRR \
tests.</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier \
New";color:black">--------------------------------------------------------------------------------</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10pt;font-family:"Courier \
New";color:black">MTRR overview</span><u></u><u></u></p> <p \
class="MsoNormal"><span style="font-size:10pt;font-family:"Courier \
New";color:black">-------------</span><u></u><u></u></p> <p \
class="MsoNormal"><span style="font-size:10pt;font-family:"Courier \
New";color:black">Reg 0: 0x0000000000000000 - 0x000000007fffffff ( 2048 MB) \
Write-Back</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier New";color:black">Reg 1: \
0x0000000080000000 - 0x00000000bfffffff ( 1024 MB) \
Write-Back</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier New";color:black"> \
</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier New";color:black">Test 1 of 3: \
Validate the kernel MTRR IOMEM setup.</span><u></u><u></u></p> <p \
class="MsoNormal"><span style="font-size:10pt;font-family:"Courier \
New";color:red">FAILED </span><span \
style="font-size:10pt;font-family:"Courier New";color:black">[CRITICAL] \
MTRRIncorrectAttr: Test 1, Memory range 0xfce0000000 to</span><u></u><u></u></p> <p \
class="MsoNormal"><span style="font-size:10pt;font-family:"Courier \
New";color:black">0xfcefffffff (0000:08:00.0) has incorrect attribute \
Write-Back.</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier New";color:red">FAILED \
</span><span style="font-size:10pt;font-family:"Courier \
New";color:black">[CRITICAL] MTRRIncorrectAttr: Test 1, Memory range \
0xfcf0000000 to</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier New";color:black">0xfcf01fffff \
(0000:08:00.0) has incorrect attribute Write-Back.</span><u></u><u></u></p> <p \
class="MsoNormal"><span style="font-size:10pt;font-family:"Courier \
New";color:black"> </span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier New";color:black">Test 2 of 3: \
Validate the MTRR setup across all processors.</span><u></u><u></u></p> <p \
class="MsoNormal"><span style="font-size:10pt;font-family:"Courier \
New";color:black">PASSED: Test 2, All processors have the a consistent MTRR \
setup.</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier New";color:black"> \
</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier New";color:black">Test 3 of 3: \
Test for AMD MtrrFixDramModEn being cleared by the BIOS.</span><u></u><u></u></p> <p \
class="MsoNormal"><span style="font-size:10pt;font-family:"Courier \
New";color:black">PASSED: Test 3, No MtrrFixDramModEn error \
detected.</span><u></u><u></u></p> <p class="MsoNormal"> <u></u><u></u></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span style="color:black"> </span><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Best Regards,<u></u><u></u></p>
<p class="MsoNormal">Larry<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<br>
-- <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Cheers,<br>
Alex Hung<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<br>
-- <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Cheers,<br>
Alex Hung<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" \
class="gmail_signature"><div dir="ltr">Cheers,<br>Alex Hung<br></div></div></div>
["image001.jpg" (image/jpeg)]
["image003.jpg" (image/jpeg)]
["image005.jpg" (image/jpeg)]
[Attachment #11 (text/plain)]
--
fwts-devel mailing list
fwts-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/fwts-devel
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic