From cfe-commits Thu Jul 30 01:39:24 2015 From: Richard Smith Date: Thu, 30 Jul 2015 01:39:24 +0000 To: cfe-commits Subject: Re: r243597 - Avoid failure to canonicalize '..'. Message-Id: X-MARC-Message: https://marc.info/?l=cfe-commits&m=143822070500784 MIME-Version: 1 Content-Type: multipart/mixed; boundary="--===============1823845955485480542==" --===============1823845955485480542== Content-Type: multipart/alternative; boundary=bcaec548a05ff63270051c0dc5c4 --bcaec548a05ff63270051c0dc5c4 Content-Type: text/plain; charset=UTF-8 On Wed, Jul 29, 2015 at 6:27 PM, Sean Silva wrote: > On Wed, Jul 29, 2015 at 6:16 PM, Richard Smith > wrote: > >> On Wed, Jul 29, 2015 at 6:13 PM, Sean Silva >> wrote: >> >>> On Wed, Jul 29, 2015 at 6:11 PM, Richard Smith >>> wrote: >>> >>>> On Wed, Jul 29, 2015 at 5:26 PM, Sean Silva >>>> wrote: >>>> >>>>> Author: silvas >>>>> Date: Wed Jul 29 19:26:34 2015 >>>>> New Revision: 243597 >>>>> >>>>> URL: http://llvm.org/viewvc/llvm-project?rev=243597&view=rev >>>>> Log: >>>>> Avoid failure to canonicalize '..'. >>>>> >>>>> Also fix completely broken and untested code which was hiding the >>>>> primary bug. The !LLVM_ON_UNIX branch of the ifdef was actually a >>>>> no-op. >>>>> >>>>> I ran into this in the wild. It was causing failures in our SDK build. >>>>> >>>>> Ideally we'd have a perfect llvm::sys::fs::canonical, but at least this >>>>> is a step in the right direction, and fixes an obviously broken case. >>>>> In some sense the test case I've added here is an integration test. We >>>>> should have these routines thoroughly unit tested in llvm::sys::fs. >>>>> >>>> >>>> This may be correct for the !LLVM_ON_UNIX case (I'm not sure whether >>>> the path foo/bar/.. is always the same as foo on Windows), but it's wrong >>>> for the LLVM_ON_UNIX case. Please revert (except for the bugfix in >>>> getCanonicalName); the right fix is to fix the places that are introducing >>>> the unwanted .. path components. >>>> >>> >>> I ran into this in the wild where they came from users. >>> >> >> So why do you want to remove them? (Note: I'm OK with removing blah/.. >> components on Windows, if indeed that is a correct thing to do there. But >> I'm not OK with it on Unix-like systems, where it's not correct in >> general.) Which of the three callers of this function did you need this >> change for? >> > > As of r243600 we should only be hitting the '..' part of this in the > !LLVM_ON_UNIX branch. > Thank you, that seems fine to me. > -- Sean Silva > > >> >> >>> -- Sean Silva >>> >>> >>>> >>>> Added: >>>>> cfe/trunk/test/Modules/Inputs/module-map-path-hash/ >>>>> cfe/trunk/test/Modules/Inputs/module-map-path-hash/a.h >>>>> cfe/trunk/test/Modules/Inputs/module-map-path-hash/module.modulemap >>>>> cfe/trunk/test/Modules/module-map-path-hash.cpp >>>>> Modified: >>>>> cfe/trunk/lib/Basic/FileManager.cpp >>>>> >>>>> Modified: cfe/trunk/lib/Basic/FileManager.cpp >>>>> URL: >>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Basic/FileManager.cpp?rev=243597&r1=243596&r2=243597&view=diff >>>>> >>>>> >>>>> ============================================================================== >>>>> --- cfe/trunk/lib/Basic/FileManager.cpp (original) >>>>> +++ cfe/trunk/lib/Basic/FileManager.cpp Wed Jul 29 19:26:34 2015 >>>>> @@ -514,7 +514,7 @@ void FileManager::modifyFileEntry(FileEn >>>>> File->ModTime = ModificationTime; >>>>> } >>>>> >>>>> -/// Remove '.' path components from the given absolute path. >>>>> +/// Remove '.' and '..' path components from the given absolute path. >>>>> /// \return \c true if any changes were made. >>>>> // FIXME: Move this to llvm::sys::path. >>>>> bool FileManager::removeDotPaths(SmallVectorImpl &Path) { >>>>> @@ -525,24 +525,24 @@ bool FileManager::removeDotPaths(SmallVe >>>>> >>>>> // Skip the root path, then look for traversal in the components. >>>>> StringRef Rel = path::relative_path(P); >>>>> - bool AnyDots = false; >>>>> for (StringRef C : llvm::make_range(path::begin(Rel), >>>>> path::end(Rel))) { >>>>> - if (C == ".") { >>>>> - AnyDots = true; >>>>> + if (C == ".") >>>>> + continue; >>>>> + if (C == "..") { >>>>> + if (!ComponentStack.empty()) >>>>> + ComponentStack.pop_back(); >>>>> continue; >>>>> } >>>>> ComponentStack.push_back(C); >>>>> } >>>>> >>>>> - if (!AnyDots) >>>>> - return false; >>>>> - >>>>> SmallString<256> Buffer = path::root_path(P); >>>>> for (StringRef C : ComponentStack) >>>>> path::append(Buffer, C); >>>>> >>>>> + bool Changed = (Path != Buffer); >>>>> Path.swap(Buffer); >>>>> - return true; >>>>> + return Changed; >>>>> } >>>>> >>>>> StringRef FileManager::getCanonicalName(const DirectoryEntry *Dir) { >>>>> @@ -567,6 +567,9 @@ StringRef FileManager::getCanonicalName( >>>>> llvm::sys::fs::make_absolute(CanonicalNameBuf); >>>>> llvm::sys::path::native(CanonicalNameBuf); >>>>> removeDotPaths(CanonicalNameBuf); >>>>> + char *Mem = >>>>> CanonicalNameStorage.Allocate(CanonicalNameBuf.size()); >>>>> + memcpy(Mem, CanonicalNameBuf.data(), CanonicalNameBuf.size()); >>>>> + CanonicalName = StringRef(Mem, CanonicalNameBuf.size()); >>>>> #endif >>>>> >>>>> CanonicalDirNames.insert(std::make_pair(Dir, CanonicalName)); >>>>> >>>>> Added: cfe/trunk/test/Modules/Inputs/module-map-path-hash/a.h >>>>> URL: >>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/module-map-path-hash/a.h?rev=243597&view=auto >>>>> >>>>> ============================================================================== >>>>> --- cfe/trunk/test/Modules/Inputs/module-map-path-hash/a.h (added) >>>>> +++ cfe/trunk/test/Modules/Inputs/module-map-path-hash/a.h Wed Jul 29 >>>>> 19:26:34 2015 >>>>> @@ -0,0 +1,2 @@ >>>>> +#pragma once >>>>> +int a = 42; >>>>> >>>>> Added: >>>>> cfe/trunk/test/Modules/Inputs/module-map-path-hash/module.modulemap >>>>> URL: >>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/Inputs/module-map-path-hash/module.modulemap?rev=243597&view=auto >>>>> >>>>> ============================================================================== >>>>> --- >>>>> cfe/trunk/test/Modules/Inputs/module-map-path-hash/module.modulemap (added) >>>>> +++ >>>>> cfe/trunk/test/Modules/Inputs/module-map-path-hash/module.modulemap Wed Jul >>>>> 29 19:26:34 2015 >>>>> @@ -0,0 +1,3 @@ >>>>> +module a { >>>>> + header "a.h" >>>>> +} >>>>> >>>>> Added: cfe/trunk/test/Modules/module-map-path-hash.cpp >>>>> URL: >>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Modules/module-map-path-hash.cpp?rev=243597&view=auto >>>>> >>>>> ============================================================================== >>>>> --- cfe/trunk/test/Modules/module-map-path-hash.cpp (added) >>>>> +++ cfe/trunk/test/Modules/module-map-path-hash.cpp Wed Jul 29 >>>>> 19:26:34 2015 >>>>> @@ -0,0 +1,9 @@ >>>>> +// rm -rf %t >>>>> +// RUN: %clang_cc1 -fmodules -fimplicit-module-maps -x c++ >>>>> -Rmodule-build -I%S/Inputs/module-map-path-hash -fmodules-cache-path=%t >>>>> -fsyntax-only %s >>>>> +// xUN: %clang_cc1 -fmodules -fimplicit-module-maps -x c++ >>>>> -Rmodule-build -I%S/Inputs//module-map-path-hash -fmodules-cache-path=%t >>>>> -fsyntax-only %s 2>&1 | FileCheck -allow-empty %s >>>>> +// xUN: %clang_cc1 -fmodules -fimplicit-module-maps -x c++ >>>>> -Rmodule-build -I%S/Inputs/./module-map-path-hash -fmodules-cache-path=%t >>>>> -fsyntax-only %s 2>&1 | FileCheck -allow-empty %s >>>>> >>>> >>>> Did you mean to commit with disabled RUN lines here? >>>> >>>> >>>>> +// RUN: %clang_cc1 -fmodules -fimplicit-module-maps -x c++ >>>>> -Rmodule-build -I%S/Inputs/../Inputs/module-map-path-hash >>>>> -fmodules-cache-path=%t -fsyntax-only %s 2>&1 | FileCheck -allow-empty %s >>>>> + >>>>> +#include "a.h" >>>>> + >>>>> +// CHECK-NOT: remark: building module >>>>> >>>>> >>>>> _______________________________________________ >>>>> cfe-commits mailing list >>>>> cfe-commits@cs.uiuc.edu >>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits >>>>> >>>> >>>> >>> >> > --bcaec548a05ff63270051c0dc5c4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: Quoted-printable
On W= ed, Jul 29, 2015 at 6:27 PM, Sean Silva <chisophugis@gmail.com>= wrote:
On Wed, Jul 2= 9, 2015 at 6:16 PM, Richard Smith <richard@metafoo.co.uk> wrote:
On Wed, Jul 29, 2015 at 6:13 P= M, Sean Silva <chisophugis@gmail.com> wrote:
On Wed, Jul 29, 2015 at 6:11 PM, Richard Smith <richard@metafoo.co.uk> wrote:
On Wed, Jul 29, 2015 at 5:26 PM, Sean Silva <<= a href=3D"mailto:chisophugis@gmail.com" target=3D"_blank">chisophugis@gmail= .com> wrote:
A= uthor: silvas
Date: Wed Jul 29 19:26:34 2015
New Revision: 243597

URL: http://llvm.org/viewv= c/llvm-project?rev=3D243597&view=3Drev
Log:
Avoid failure to canonicalize '..'.

Also fix completely broken and untested code which was hiding the
primary bug. The !LLVM_ON_UNIX branch of the ifdef was actually a no-op.

I ran into this in the wild. It was causing failures in our SDK build.

Ideally we'd have a perfect llvm::sys::fs::canonical, but at least this=
is a step in the right direction, and fixes an obviously broken case.
In some sense the test case I've added here is an integration test. We<= br> should have these routines thoroughly unit tested in llvm::sys::fs.

This may be correct for the !LLVM_ON_UN= IX case (I'm not sure whether the path foo/bar/.. is always the same as= foo on Windows), but it's wrong for the LLVM_ON_UNIX case. Please reve= rt (except for the bugfix in getCanonicalName); the right fix is to fix the= places that are introducing the unwanted .. path components.

I ran into this in the wi= ld where they came from users.
So why do you want to remove them? (Note: I'm OK wit= h removing blah/.. components on Windows, if indeed that is a correct thing= to do there. But I'm not OK with it on Unix-like systems, where it'= ;s not correct in general.) Which of the three callers of this function did= you need this change for?

As of r243600 we should only be hitting the '..' par= t of this in the !LLVM_ON_UNIX branch.
=

Thank you, that seems fine to me.
=C2=A0
=
-- Sean Silva
=C2=A0
=C2=A0
-- Sean Silva
=C2=A0

Added:
=C2=A0 =C2=A0 cfe/trunk/test/Modules/Inputs/module-map-path-hash/
=C2=A0 =C2=A0 cfe/trunk/test/Modules/Inputs/module-map-path-hash/a.h
=C2=A0 =C2=A0 cfe/trunk/test/Modules/Inputs/module-map-path-hash/module.mod= ulemap
=C2=A0 =C2=A0 cfe/trunk/test/Modules/module-map-path-hash.cpp
Modified:
=C2=A0 =C2=A0 cfe/trunk/lib/Basic/FileManager.cpp

Modified: cfe/trunk/lib/Basic/FileManager.cpp
URL: http://llvm.org/viewvc/llvm-pro= ject/cfe/trunk/lib/Basic/FileManager.cpp?rev=3D243597&r1=3D243596&r= 2=3D243597&view=3Ddiff

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D
--- cfe/trunk/lib/Basic/FileManager.cpp (original)
+++ cfe/trunk/lib/Basic/FileManager.cpp Wed Jul 29 19:26:34 2015
@@ -514,7 +514,7 @@ void FileManager::modifyFileEntry(FileEn
=C2=A0 =C2=A0File->ModTime =3D ModificationTime;
=C2=A0}

-/// Remove '.' path components from the given absolute path.
+/// Remove '.' and '..' path components from the given abs= olute path.
=C2=A0/// \return \c true if any changes were made.
=C2=A0// FIXME: Move this to llvm::sys::path.
=C2=A0bool FileManager::removeDotPaths(SmallVectorImpl<char> &Pat= h) {
@@ -525,24 +525,24 @@ bool FileManager::removeDotPaths(SmallVe

=C2=A0 =C2=A0// Skip the root path, then look for traversal in the componen= ts.
=C2=A0 =C2=A0StringRef Rel =3D path::relative_path(P);
-=C2=A0 bool AnyDots =3D false;
=C2=A0 =C2=A0for (StringRef C : llvm::make_range(path::begin(Rel), path::en= d(Rel))) {
-=C2=A0 =C2=A0 if (C =3D=3D ".") {
-=C2=A0 =C2=A0 =C2=A0 AnyDots =3D true;
+=C2=A0 =C2=A0 if (C =3D=3D ".")
+=C2=A0 =C2=A0 =C2=A0 continue;
+=C2=A0 =C2=A0 if (C =3D=3D "..") {
+=C2=A0 =C2=A0 =C2=A0 if (!ComponentStack.empty())
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 ComponentStack.pop_back();
=C2=A0 =C2=A0 =C2=A0 =C2=A0continue;
=C2=A0 =C2=A0 =C2=A0}
=C2=A0 =C2=A0 =C2=A0ComponentStack.push_back(C);
=C2=A0 =C2=A0}

-=C2=A0 if (!AnyDots)
-=C2=A0 =C2=A0 return false;
-
=C2=A0 =C2=A0SmallString<256> Buffer =3D path::root_path(P);
=C2=A0 =C2=A0for (StringRef C : ComponentStack)
=C2=A0 =C2=A0 =C2=A0path::append(Buffer, C);

+=C2=A0 bool Changed =3D (Path !=3D Buffer);
=C2=A0 =C2=A0Path.swap(Buffer);
-=C2=A0 return true;
+=C2=A0 return Changed;
=C2=A0}

=C2=A0StringRef FileManager::getCanonicalName(const DirectoryEntry *Dir) {<= br> @@ -567,6 +567,9 @@ StringRef FileManager::getCanonicalName(
=C2=A0 =C2=A0llvm::sys::fs::make_absolute(CanonicalNameBuf);
=C2=A0 =C2=A0llvm::sys::path::native(CanonicalNameBuf);
=C2=A0 =C2=A0removeDotPaths(CanonicalNameBuf);
+=C2=A0 char *Mem =3D CanonicalNameStorage.Allocate<char>(CanonicalNa= meBuf.size());
+=C2=A0 memcpy(Mem, CanonicalNameBuf.data(), CanonicalNameBuf.size());
+=C2=A0 CanonicalName =3D StringRef(Mem, CanonicalNameBuf.size());
=C2=A0#endif

=C2=A0 =C2=A0CanonicalDirNames.insert(std::make_pair(Dir, CanonicalName));<= br>
Added: cfe/trunk/test/Modules/Inputs/module-map-path-hash/a.h
URL: http://llvm.org/viewvc/llvm-proje= ct/cfe/trunk/test/Modules/Inputs/module-map-path-hash/a.h?rev=3D243597&= view=3Dauto
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D
--- cfe/trunk/test/Modules/Inputs/module-map-path-hash/a.h (added)
+++ cfe/trunk/test/Modules/Inputs/module-map-path-hash/a.h Wed Jul 29 19:26= :34 2015
@@ -0,0 +1,2 @@
+#pragma once
+int a =3D 42;

Added: cfe/trunk/test/Modules/Inputs/module-map-path-hash/module.modulemap<= br>
URL: http://llvm.org/viewv= c/llvm-project/cfe/trunk/test/Modules/Inputs/module-map-path-hash/module.mo= dulemap?rev=3D243597&view=3Dauto
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D
--- cfe/trunk/test/Modules/Inputs/module-map-path-hash/module.modulemap (ad= ded)
+++ cfe/trunk/test/Modules/Inputs/module-map-path-hash/module.modulemap Wed= Jul 29 19:26:34 2015
@@ -0,0 +1,3 @@
+module a {
+=C2=A0 header "a.h"
+}

Added: cfe/trunk/test/Modules/module-map-path-hash.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe= /trunk/test/Modules/module-map-path-hash.cpp?rev=3D243597&view=3Dauto
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D
--- cfe/trunk/test/Modules/module-map-path-hash.cpp (added)
+++ cfe/trunk/test/Modules/module-map-path-hash.cpp Wed Jul 29 19:26:34 201= 5
@@ -0,0 +1,9 @@
+// rm -rf %t
+// RUN: %clang_cc1 -fmodules -fimplicit-module-maps -x c++ -Rmodule-build = -I%S/Inputs/module-map-path-hash -fmodules-cache-path=3D%t -fsyntax-only %s=
+// xUN: %clang_cc1 -fmodules -fimplicit-module-maps -x c++ -Rmodule-build = -I%S/Inputs//module-map-path-hash -fmodules-cache-path=3D%t -fsyntax-only %= s 2>&1 | FileCheck -allow-empty %s
+// xUN: %clang_cc1 -fmodules -fimplicit-module-maps -x c++ -Rmodule-build = -I%S/Inputs/./module-map-path-hash -fmodules-cache-path=3D%t -fsyntax-only = %s 2>&1 | FileCheck -allow-empty %s
+// RUN: %clang_cc1 -fmodules -fimplicit-module-maps -x c++ -Rmodule-build = -I%S/Inputs/../Inputs/module-map-path-hash -fmodules-cache-path=3D%t -fsynt= ax-only %s 2>&1 | FileCheck -allow-empty %s
+
+#include "a.h"
+
+// CHECK-NOT: remark: building module


_______________________________________________
cfe-commits mailing list
cfe-com= mits@cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-c= ommits





--bcaec548a05ff63270051c0dc5c4-- --===============1823845955485480542== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ cfe-commits mailing list cfe-commits@cs.uiuc.edu http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits --===============1823845955485480542==--