[prev in list] [next in list] [prev in thread] [next in thread]
List: openembedded-core
Subject: Re: [OE-core] [hardknott][PATCH] gcc: Fix CVE-2021-42574
From: "Khem Raj" <raj.khem () gmail ! com>
Date: 2021-12-31 17:25:09
Message-ID: CAMKF1sqN3kmkQ8JB2gskJhO=HVhXq9rCRD_Qh8A6YUTyOKYDqQ () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
On Thu, Dec 30, 2021 at 4:29 PM Anuj Mittal <anuj.mittal@intel.com> wrote:
> I can't apply this.
>
> git am says:
>
> error: cannot convert from Y to UTF-8
> fatal: could not parse patch
>
Key is When git send-email asks for UTF-8 encoding for the patch then do
not enter "y" but just press
Enter
> Thanks,
>
> Anuj
>
> On Wed, 2021-12-29 at 01:13 -0800, pgowda wrote:
> > Upstream-Status: Backport
> > [https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=004bb936d6d5f177af26ad
> > 4905595e843d5665a5]
> > Upstream-Status: Backport
> > [https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=bd5e882cf6e0def3dd1bc1
> > 06075d59a303fe0d1e]
> > Upstream-Status: Backport
> > [https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=51c500269bf53749b10780
> > 7d84271385fad35628]
> > Upstream-Status: Backport
> > [https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=1a7f2c0774129750fdf73e
> > 9f1b78f0ce983c9ab3]
> > Upstream-Status: Backport
> > [https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=bef32d4a28595e933f24fe
> > f378cf052a30b674a7]
> >
> > Signed-off-by: pgowda <pgowda.cve@gmail.com>
> > ---
> > meta/recipes-devtools/gcc/gcc-10.2.inc | 5 +
> > .../gcc/gcc/0001-CVE-2021-42574.patch | 2906
> > +++++++++++++++++
> > .../gcc/gcc/0002-CVE-2021-42574.patch | 2270 +++++++++++++
> > .../gcc/gcc/0003-CVE-2021-42574.patch | 1724 ++++++++++
> > .../gcc/gcc/0004-CVE-2021-42574.patch | 138 +
> > .../gcc/gcc/0005-CVE-2021-42574.patch | 575 ++++
> > 6 files changed, 7618 insertions(+)
> > create mode 100644 meta/recipes-devtools/gcc/gcc/0001-CVE-2021-
> > 42574.patch
> > create mode 100644 meta/recipes-devtools/gcc/gcc/0002-CVE-2021-
> > 42574.patch
> > create mode 100644 meta/recipes-devtools/gcc/gcc/0003-CVE-2021-
> > 42574.patch
> > create mode 100644 meta/recipes-devtools/gcc/gcc/0004-CVE-2021-
> > 42574.patch
> > create mode 100644 meta/recipes-devtools/gcc/gcc/0005-CVE-2021-
> > 42574.patch
> >
> > diff --git a/meta/recipes-devtools/gcc/gcc-10.2.inc b/meta/recipes-
> > devtools/gcc/gcc-10.2.inc
> > index 5626bf20f0..fd7f16d7bf 100644
> > --- a/meta/recipes-devtools/gcc/gcc-10.2.inc
> > +++ b/meta/recipes-devtools/gcc/gcc-10.2.inc
> > @@ -74,6 +74,11 @@ SRC_URI = "\
> > file://0002-CVE-2021-35465.patch \
> > file://0003-CVE-2021-35465.patch \
> > file://0004-CVE-2021-35465.patch \
> > + file://0001-CVE-2021-42574.patch \
> > + file://0002-CVE-2021-42574.patch \
> > + file://0003-CVE-2021-42574.patch \
> > + file://0004-CVE-2021-42574.patch \
> > + file://0005-CVE-2021-42574.patch \
> > "
> > SRC_URI[sha256sum] =
> > "b8dd4368bb9c7f0b98188317ee0254dd8cc99d1e3a18d0ff146c855fe16c1d8c"
> >
> > diff --git a/meta/recipes-devtools/gcc/gcc/0001-CVE-2021-42574.patch
> > b/meta/recipes-devtools/gcc/gcc/0001-CVE-2021-42574.patch
> > new file mode 100644
> > index 0000000000..e0f4f7d32f
> > --- /dev/null
> > +++ b/meta/recipes-devtools/gcc/gcc/0001-CVE-2021-42574.patch
> > @@ -0,0 +1,2906 @@
> > +From 004bb936d6d5f177af26ad4905595e843d5665a5 Mon Sep 17 00:00:00
> > 2001
> > +From: Lewis Hyatt <lhyatt@gmail.com>
> > +Date: Tue, 14 Jul 2020 12:05:56 -0400
> > +Subject: [PATCH] diagnostics: Support conversion of tabs to spaces
> > [PR49973]
> > + [PR86904]
> > +
> > +Supports conversion of tabs to spaces when outputting diagnostics.
> > Also
> > +adds -fdiagnostics-column-unit and -fdiagnostics-column-origin
> > options to
> > +control how the column number is output, thereby resolving the two
> > PRs.
> > +
> > +gcc/c-family/ChangeLog:
> > +
> > + PR other/86904
> > + * c-indentation.c (should_warn_for_misleading_indentation):
> > Get
> > + global tabstop from the new source.
> > + * c-opts.c (c_common_handle_option): Remove handling of -
> > ftabstop, which
> > + is now a common option.
> > + * c.opt: Likewise.
> > +
> > +gcc/ChangeLog:
> > +
> > + PR preprocessor/49973
> > + PR other/86904
> > + * common.opt: Handle -ftabstop here instead of in c-family
> > + options. Add -fdiagnostics-column-unit= and
> > + -fdiagnostics-column-origin= options.
> > + * opts.c (common_handle_option): Handle the new options.
> > + * diagnostic-format-json.cc (json_from_expanded_location):
> > Add
> > + diagnostic_context argument. Use it to convert column
> > numbers as per
> > + the new options.
> > + (json_from_location_range): Likewise.
> > + (json_from_fixit_hint): Likewise.
> > + (json_end_diagnostic): Pass the new context argument to
> > helper
> > + functions above. Add "column-origin" field to the output.
> > + (test_unknown_location): Add the new context argument to
> > calls to
> > + helper functions.
> > + (test_bad_endpoints): Likewise.
> > + * diagnostic-show-locus.c
> > + (exploc_with_display_col::exploc_with_display_col): Support
> > + tabstop parameter.
> > + (layout_point::layout_point): Make use of class
> > + exploc_with_display_col.
> > + (layout_range::layout_range): Likewise.
> > + (struct line_bounds): Clarify that the units are now always
> > + display columns. Rename members accordingly. Add
> > constructor.
> > + (layout::print_source_line): Add support for tab expansion.
> > + (make_range): Adapt to class layout_range changes.
> > + (layout::maybe_add_location_range): Likewise.
> > + (layout::layout): Adapt to class exploc_with_display_col
> > changes.
> > + (layout::calculate_x_offset_display): Support tabstop
> > parameter.
> > + (layout::print_annotation_line): Adapt to struct line_bounds
> > changes.
> > + (layout::print_line): Likewise.
> > + (line_label::line_label): Add diagnostic_context argument.
> > + (get_affected_range): Likewise.
> > + (get_printed_columns): Likewise.
> > + (layout::print_any_labels): Adapt to struct line_label
> > changes.
> > + (class correction): Add m_tabstop member.
> > + (correction::correction): Add tabstop argument.
> > + (correction::compute_display_cols): Use m_tabstop.
> > + (class line_corrections): Add m_context member.
> > + (line_corrections::line_corrections): Add diagnostic_context
> > argument.
> > + (line_corrections::add_hint): Use m_context to handle
> > tabstops.
> > + (layout::print_trailing_fixits): Adapt to class
> > line_corrections
> > + changes.
> > + (test_layout_x_offset_display_utf8): Support tabstop
> > parameter.
> > + (test_layout_x_offset_display_tab): New selftest.
> > + (test_one_liner_colorized_utf8): Likewise.
> > + (test_tab_expansion): Likewise.
> > + (test_diagnostic_show_locus_one_liner_utf8): Call the new
> > tests.
> > + (diagnostic_show_locus_c_tests): Likewise.
> > + (test_overlapped_fixit_printing): Adapt to helper class and
> > + function changes.
> > + (test_overlapped_fixit_printing_utf8): Likewise.
> > + (test_overlapped_fixit_printing_2): Likewise.
> > + * diagnostic.h (enum diagnostics_column_unit): New enum.
> > + (struct diagnostic_context): Add members for the new options.
> > + (diagnostic_converted_column): Declare.
> > + (json_from_expanded_location): Add new context argument.
> > + * diagnostic.c (diagnostic_initialize): Initialize new
> > members.
> > + (diagnostic_converted_column): New function.
> > + (maybe_line_and_column): Be willing to output a column of 0.
> > + (diagnostic_get_location_text): Convert column number as per
> > the new
> > + options.
> > + (diagnostic_report_current_module): Likewise.
> > + (assert_location_text): Add origin and column_unit arguments
> > for
> > + testing the new functionality.
> > + (test_diagnostic_get_location_text): Test the new
> > functionality.
> > + * doc/invoke.texi: Document the new options and behavior.
> > + * input.h (location_compute_display_column): Add tabstop
> > argument.
> > + * input.c (location_compute_display_column): Likewise.
> > + (test_cpp_utf8): Add selftests for tab expansion.
> > + * tree-diagnostic-path.cc (default_tree_make_json_for_path):
> > Pass the
> > + new context argument to json_from_expanded_location().
> > +
> > +libcpp/ChangeLog:
> > +
> > + PR preprocessor/49973
> > + PR other/86904
> > + * include/cpplib.h (struct cpp_options): Removed support for
> > -ftabstop,
> > + which is now handled by diagnostic_context.
> > + (class cpp_display_width_computation): New class.
> > + (cpp_byte_column_to_display_column): Add optional tabstop
> > argument.
> > + (cpp_display_width): Likewise.
> > + (cpp_display_column_to_byte_column): Likewise.
> > + * charset.c
> > + (cpp_display_width_computation::cpp_display_width_computation
> > ): New
> > + function.
> > + (cpp_display_width_computation::advance_display_cols):
> > Likewise.
> > + (compute_next_display_width): Removed and implemented this
> > + functionality in a new function...
> > + (cpp_display_width_computation::process_next_codepoint):
> > ...here.
> > + (cpp_byte_column_to_display_column): Added tabstop argument.
> > + Reimplemented in terms of class
> > cpp_display_width_computation.
> > + (cpp_display_column_to_byte_column): Likewise.
> > + * init.c (cpp_create_reader): Remove handling of -ftabstop,
> > which is now
> > + handled by diagnostic_context.
> > +
> > +gcc/testsuite/ChangeLog:
> > +
> > + PR preprocessor/49973
> > + PR other/86904
> > + * c-c++-common/Wmisleading-indentation-3.c: Adjust expected
> > output
> > + for new defaults.
> > + * c-c++-common/Wmisleading-indentation.c: Likewise.
> > + * c-c++-common/diagnostic-format-json-1.c: Likewise.
> > + * c-c++-common/diagnostic-format-json-2.c: Likewise.
> > + * c-c++-common/diagnostic-format-json-3.c: Likewise.
> > + * c-c++-common/diagnostic-format-json-4.c: Likewise.
> > + * c-c++-common/diagnostic-format-json-5.c: Likewise.
> > + * c-c++-common/missing-close-symbol.c: Likewise.
> > + * g++.dg/diagnostic/bad-binary-ops.C: Likewise.
> > + * g++.dg/parse/error4.C: Likewise.
> > + * g++.old-deja/g++.brendan/crash11.C: Likewise.
> > + * g++.old-deja/g++.pt/overload2.C: Likewise.
> > + * g++.old-deja/g++.robertl/eb109.C: Likewise.
> > + * gcc.dg/analyzer/malloc-paths-9.c: Likewise.
> > + * gcc.dg/bad-binary-ops.c: Likewise.
> > + * gcc.dg/format/branch-1.c: Likewise.
> > + * gcc.dg/format/pr79210.c: Likewise.
> > + * gcc.dg/plugin/diagnostic-test-expressions-1.c: Likewise.
> > + * gcc.dg/plugin/diagnostic-test-string-literals-1.c:
> > Likewise.
> > + * gcc.dg/redecl-4.c: Likewise.
> > + * gfortran.dg/diagnostic-format-json-1.F90: Likewise.
> > + * gfortran.dg/diagnostic-format-json-2.F90: Likewise.
> > + * gfortran.dg/diagnostic-format-json-3.F90: Likewise.
> > + * go.dg/arrayclear.go: Add a comment explaining why adding a
> > + comment was necessary to work around a dejagnu bug.
> > + * c-c++-common/diagnostic-units-1.c: New test.
> > + * c-c++-common/diagnostic-units-2.c: New test.
> > + * c-c++-common/diagnostic-units-3.c: New test.
> > + * c-c++-common/diagnostic-units-4.c: New test.
> > + * c-c++-common/diagnostic-units-5.c: New test.
> > + * c-c++-common/diagnostic-units-6.c: New test.
> > + * c-c++-common/diagnostic-units-7.c: New test.
> > + * c-c++-common/diagnostic-units-8.c: New test.
> > +
> > +CVE: CVE-2021-42574
> > +Upstream-Status: Backport
> > [https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=004bb936d6d5f177af26ad
> > 4905595e843d5665a5]
> > +Signed-off-by: Pgowda <pgowda.cve@gmail.com>
> > +---
> > + gcc/c-family/c-indentation.c | 5 +-
> > + gcc/c-family/c-opts.c | 6 -
> > + gcc/c-family/c.opt | 4 -
> > + gcc/common.opt | 21 +
> > + gcc/diagnostic-format-json.cc | 55 +-
> > + gcc/diagnostic-show-locus.c | 504 +++++++++++++--
> > ---
> > + gcc/diagnostic.c | 113 +++-
> > + gcc/diagnostic.h | 28 +-
> > + gcc/doc/invoke.texi | 68 ++-
> > + gcc/input.c | 72 ++-
> > + gcc/input.h | 4 +-
> > + gcc/opts.c | 14 +
> > + .../c-c++-common/Wmisleading-indentation-3.c | 12 +-
> > + .../c-c++-common/Wmisleading-indentation.c | 6 +-
> > + .../c-c++-common/diagnostic-format-json-1.c | 5 +
> > + .../c-c++-common/diagnostic-format-json-2.c | 5 +
> > + .../c-c++-common/diagnostic-format-json-3.c | 5 +
> > + .../c-c++-common/diagnostic-format-json-4.c | 9 +
> > + .../c-c++-common/diagnostic-format-json-5.c | 9 +
> > + .../c-c++-common/diagnostic-units-1.c | 28 +
> > + .../c-c++-common/diagnostic-units-2.c | 28 +
> > + .../c-c++-common/diagnostic-units-3.c | 28 +
> > + .../c-c++-common/diagnostic-units-4.c | 28 +
> > + .../c-c++-common/diagnostic-units-5.c | 28 +
> > + .../c-c++-common/diagnostic-units-6.c | 28 +
> > + .../c-c++-common/diagnostic-units-7.c | 28 +
> > + .../c-c++-common/diagnostic-units-8.c | 28 +
> > + .../c-c++-common/missing-close-symbol.c | 6 +-
> > + .../g++.dg/diagnostic/bad-binary-ops.C | 8 +-
> > + gcc/testsuite/g++.dg/parse/error4.C | 2 +-
> > + .../g++.old-deja/g++.brendan/crash11.C | 4 +-
> > + gcc/testsuite/g++.old-deja/g++.pt/overload2.C | 2 +-
> > + .../g++.old-deja/g++.robertl/eb109.C | 4 +-
> > + .../gcc.dg/analyzer/malloc-paths-9.c | 2 +-
> > + gcc/testsuite/gcc.dg/bad-binary-ops.c | 8 +-
> > + gcc/testsuite/gcc.dg/format/branch-1.c | 2 +-
> > + gcc/testsuite/gcc.dg/format/pr79210.c | 2 +-
> > + .../plugin/diagnostic-test-expressions-1.c | 16 +-
> > + .../diagnostic-test-string-literals-1.c | 4 +-
> > + gcc/testsuite/gcc.dg/redecl-4.c | 2 +-
> > + .../gfortran.dg/diagnostic-format-json-1.F90 | 5 +
> > + .../gfortran.dg/diagnostic-format-json-2.F90 | 5 +
> > + .../gfortran.dg/diagnostic-format-json-3.F90 | 5 +
> > + gcc/testsuite/go.dg/arrayclear.go | 3 +
> > + gcc/tree-diagnostic-path.cc | 5 +-
> > + libcpp/charset.c | 98 ++--
> > + libcpp/include/cpplib.h | 40 +-
> > + libcpp/init.c | 1 -
> > + 48 files changed, 1106 insertions(+), 287 deletions(-)
> > + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-1.c
> > + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-2.c
> > + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-3.c
> > + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-4.c
> > + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-5.c
> > + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-6.c
> > + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-7.c
> > + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-8.c
> > +
> > +diff --git a/gcc/c-family/c-indentation.c b/gcc/c-family/c-
> > indentation.c
> > +--- a/gcc/c-family/c-indentation.c 2020-07-22 23:35:17.296384022
> > -0700
> > ++++ b/gcc/c-family/c-indentation.c 2021-12-25 01:20:53.475636694
> > -0800
> > +@@ -24,8 +24,7 @@ along with GCC; see the file COPYING3.
> > + #include "c-common.h"
> > + #include "c-indentation.h"
> > + #include "selftest.h"
> > +-
> > +-extern cpp_options *cpp_opts;
> > ++#include "diagnostic.h"
> > +
> > + /* Round up VIS_COLUMN to nearest tab stop. */
> > +
> > +@@ -294,7 +293,7 @@ should_warn_for_misleading_indentation (
> > + expanded_location next_stmt_exploc = expand_location
> > (next_stmt_loc);
> > + expanded_location guard_exploc = expand_location (guard_loc);
> > +
> > +- const unsigned int tab_width = cpp_opts->tabstop;
> > ++ const unsigned int tab_width = global_dc->tabstop;
> > +
> > + /* They must be in the same file. */
> > + if (next_stmt_exploc.file != body_exploc.file)
> > +diff --git a/gcc/c-family/c.opt b/gcc/c-family/c.opt
> > +--- a/gcc/c-family/c.opt 2021-12-24 20:23:42.816809230 -0800
> > ++++ b/gcc/c-family/c.opt 2021-12-25 01:20:53.475636694 -0800
> > +@@ -1876,10 +1876,6 @@ Enum(strong_eval_order) String(some) Val
> > + EnumValue
> > + Enum(strong_eval_order) String(all) Value(2)
> > +
> > +-ftabstop=
> > +-C ObjC C++ ObjC++ Joined RejectNegative UInteger
> > +--ftabstop=<number> Distance between tab stops for column
> > reporting.
> > +-
> > + ftemplate-backtrace-limit=
> > + C++ ObjC++ Joined RejectNegative UInteger
> > Var(template_backtrace_limit) Init(10)
> > + Set the maximum number of template instantiation notes for a single
> > warning or error.
> > +diff --git a/gcc/c-family/c-opts.c b/gcc/c-family/c-opts.c
> > +--- a/gcc/c-family/c-opts.c 2021-12-24 20:23:44.824774786 -0800
> > ++++ b/gcc/c-family/c-opts.c 2021-12-25 01:20:53.475636694 -0800
> > +@@ -504,12 +504,6 @@ c_common_handle_option (size_t scode, co
> > + cpp_opts->track_macro_expansion = 2;
> > + break;
> > +
> > +- case OPT_ftabstop_:
> > +- /* It is documented that we silently ignore silly values. */
> > +- if (value >= 1 && value <= 100)
> > +- cpp_opts->tabstop = value;
> > +- break;
> > +-
> > + case OPT_fexec_charset_:
> > + cpp_opts->narrow_charset = arg;
> > + break;
> > +diff --git a/gcc/common.opt b/gcc/common.opt
> > +--- a/gcc/common.opt 2021-12-24 20:23:42.480814993 -0800
> > ++++ b/gcc/common.opt 2021-12-25 01:20:53.475636694 -0800
> > +@@ -1325,6 +1325,14 @@ Enum(diagnostic_url_rule) String(always)
> > + EnumValue
> > + Enum(diagnostic_url_rule) String(auto) Value(DIAGNOSTICS_URL_AUTO)
> > +
> > ++fdiagnostics-column-unit=
> > ++Common Joined RejectNegative Enum(diagnostics_column_unit)
> > ++-fdiagnostics-column-unit=[display|byte] Select whether column
> > numbers are output as display columns (default) or raw bytes.
> > ++
> > ++fdiagnostics-column-origin=
> > ++Common Joined RejectNegative UInteger
> > ++-fdiagnostics-column-origin=<number> Set the number of the first
> > column. The default is 1-based as per GNU style, but some utilities
> > may expect 0-based, for example.
> > ++
> > + fdiagnostics-format=
> > + Common Joined RejectNegative Enum(diagnostics_output_format)
> > + -fdiagnostics-format=[text|json] Select output format.
> > +@@ -1334,6 +1342,15 @@ SourceInclude
> > + diagnostic.h
> > +
> > + Enum
> > ++Name(diagnostics_column_unit) Type(int)
> > ++
> > ++EnumValue
> > ++Enum(diagnostics_column_unit) String(display)
> > Value(DIAGNOSTICS_COLUMN_UNIT_DISPLAY)
> > ++
> > ++EnumValue
> > ++Enum(diagnostics_column_unit) String(byte)
> > Value(DIAGNOSTICS_COLUMN_UNIT_BYTE)
> > ++
> > ++Enum
> > + Name(diagnostics_output_format) Type(int)
> > +
> > + EnumValue
> > +@@ -1362,6 +1379,10 @@ fdiagnostics-path-format=
> > + Common Joined RejectNegative Var(flag_diagnostics_path_format)
> > Enum(diagnostic_path_format) Init(DPF_INLINE_EVENTS)
> > + Specify how to print any control-flow path associated with a
> > diagnostic.
> > +
> > ++ftabstop=
> > ++Common Joined RejectNegative UInteger
> > ++-ftabstop=<number> Distance between tab stops for column
> > reporting.
> > ++
> > + Enum
> > + Name(diagnostic_path_format) Type(int)
> > +
> > +diff --git a/gcc/diagnostic.c b/gcc/diagnostic.c
> > +--- a/gcc/diagnostic.c 2020-07-22 23:35:17.556386887 -0700
> > ++++ b/gcc/diagnostic.c 2021-12-25 01:23:41.300841207 -0800
> > +@@ -38,6 +38,7 @@ along with GCC; see the file COPYING3.
> > + #include "selftest.h"
> > + #include "selftest-diagnostic.h"
> > + #include "opts.h"
> > ++#include "cpplib.h"
> > +
> > + #ifdef HAVE_TERMIOS_H
> > + # include <termios.h>
> > +@@ -219,6 +220,9 @@ diagnostic_initialize (diagnostic_contex
> > + context->min_margin_width = 0;
> > + context->show_ruler_p = false;
> > + context->parseable_fixits_p = false;
> > ++ context->column_unit = DIAGNOSTICS_COLUMN_UNIT_DISPLAY;
> > ++ context->column_origin = 1;
> > ++ context->tabstop = 8;
> > + context->edit_context_ptr = NULL;
> > + context->diagnostic_group_nesting_depth = 0;
> > + context->diagnostic_group_emission_count = 0;
> > +@@ -353,8 +357,51 @@ diagnostic_get_color_for_kind (diagnosti
> > + return diagnostic_kind_color[kind];
> > + }
> > +
> > ++/* Given an expanded_location, convert the column (which is in 1-
> > based bytes)
> > ++ to the requested units, without converting the origin.
> > ++ Return -1 if the column is invalid (<= 0). */
> > ++
> > ++static int
> > ++convert_column_unit (enum diagnostics_column_unit column_unit,
> > ++ int tabstop,
> > ++ expanded_location s)
> > ++{
> > ++ if (s.column <= 0)
> > ++ return -1;
> > ++
> > ++ switch (column_unit)
> > ++ {
> > ++ default:
> > ++ gcc_unreachable ();
> > ++
> > ++ case DIAGNOSTICS_COLUMN_UNIT_DISPLAY:
> > ++ {
> > ++ cpp_char_column_policy policy (tabstop, cpp_wcwidth);
> > ++ return location_compute_display_column (s, policy);
> > ++ }
> > ++
> > ++ case DIAGNOSTICS_COLUMN_UNIT_BYTE:
> > ++ return s.column;
> > ++ }
> > ++}
> > ++
> > ++/* Given an expanded_location, convert the column (which is in 1-
> > based bytes)
> > ++ to the requested units and origin. Return -1 if the column is
> > ++ invalid (<= 0). */
> > ++int
> > ++diagnostic_converted_column (diagnostic_context *context,
> > expanded_location s)
> > ++{
> > ++ int one_based_col
> > ++ = convert_column_unit (context->column_unit, context->tabstop,
> > s);
> > ++ if (one_based_col <= 0)
> > ++ return -1;
> > ++ return one_based_col + (context->column_origin - 1);
> > ++}
> > ++
> > + /* Return a formatted line and column ':%line:%column'. Elided if
> > +- zero. The result is a statically allocated buffer. */
> > ++ line == 0 or col < 0. (A column of 0 may be valid due to the
> > ++ -fdiagnostics-column-origin option.)
> > ++ The result is a statically allocated buffer. */
> > +
> > + static const char *
> > + maybe_line_and_column (int line, int col)
> > +@@ -363,8 +410,9 @@ maybe_line_and_column (int line, int col
> > +
> > + if (line)
> > + {
> > +- size_t l = snprintf (result, sizeof (result),
> > +- col ? ":%d:%d" : ":%d", line, col);
> > ++ size_t l
> > ++ = snprintf (result, sizeof (result),
> > ++ col >= 0 ? ":%d:%d" : ":%d", line, col);
> > + gcc_checking_assert (l < sizeof (result));
> > + }
> > + else
> > +@@ -383,8 +431,14 @@ diagnostic_get_location_text (diagnostic
> > + const char *locus_cs = colorize_start (pp_show_color (pp),
> > "locus");
> > + const char *locus_ce = colorize_stop (pp_show_color (pp));
> > + const char *file = s.file ? s.file : progname;
> > +- int line = strcmp (file, N_("<built-in>")) ? s.line : 0;
> > +- int col = context->show_column ? s.column : 0;
> > ++ int line = 0;
> > ++ int col = -1;
> > ++ if (strcmp (file, N_("<built-in>")))
> > ++ {
> > ++ line = s.line;
> > ++ if (context->show_column)
> > ++ col = diagnostic_converted_column (context, s);
> > ++ }
> > +
> > + const char *line_col = maybe_line_and_column (line, col);
> > + return build_message_string ("%s%s%s:%s", locus_cs, file,
> > +@@ -650,14 +704,20 @@ diagnostic_report_current_module (diagno
> > + if (! MAIN_FILE_P (map))
> > + {
> > + bool first = true;
> > ++ expanded_location s = {};
> > + do
> > + {
> > + where = linemap_included_from (map);
> > + map = linemap_included_from_linemap (line_table, map);
> > +- const char *line_col
> > +- = maybe_line_and_column (SOURCE_LINE (map, where),
> > +- first && context-
> > >show_column
> > +- ? SOURCE_COLUMN (map, where)
> > : 0);
> > ++ s.file = LINEMAP_FILE (map);
> > ++ s.line = SOURCE_LINE (map, where);
> > ++ int col = -1;
> > ++ if (first && context->show_column)
> > ++ {
> > ++ s.column = SOURCE_COLUMN (map, where);
> > ++ col = diagnostic_converted_column (context, s);
> > ++ }
> > ++ const char *line_col = maybe_line_and_column (s.line,
> > col);
> > + static const char *const msgs[] =
> > + {
> > + N_("In file included from"),
> > +@@ -666,7 +726,7 @@ diagnostic_report_current_module (diagno
> > + unsigned index = !first;
> > + pp_verbatim (context->printer, "%s%s %r%s%s%R",
> > + first ? "" : ",\n", _(msgs[index]),
> > +- "locus", LINEMAP_FILE (map), line_col);
> > ++ "locus", s.file, line_col);
> > + first = false;
> > + }
> > + while (! MAIN_FILE_P (map));
> > +@@ -2042,10 +2102,15 @@ test_print_parseable_fixits_replace ()
> > + static void
> > + assert_location_text (const char *expected_loc_text,
> > + const char *filename, int line, int column,
> > +- bool show_column)
> > ++ bool show_column,
> > ++ int origin = 1,
> > ++ enum diagnostics_column_unit column_unit
> > ++ = DIAGNOSTICS_COLUMN_UNIT_BYTE)
> > + {
> > + test_diagnostic_context dc;
> > + dc.show_column = show_column;
> > ++ dc.column_unit = column_unit;
> > ++ dc.column_origin = origin;
> > +
> > + expanded_location xloc;
> > + xloc.file = filename;
> > +@@ -2069,7 +2134,10 @@ test_diagnostic_get_location_text ()
> > + assert_location_text ("PROGNAME:", NULL, 0, 0, true);
> > + assert_location_text ("<built-in>:", "<built-in>", 42, 10, true);
> > + assert_location_text ("foo.c:42:10:", "foo.c", 42, 10, true);
> > +- assert_location_text ("foo.c:42:", "foo.c", 42, 0, true);
> > ++ assert_location_text ("foo.c:42:9:", "foo.c", 42, 10, true, 0);
> > ++ assert_location_text ("foo.c:42:1010:", "foo.c", 42, 10, true,
> > 1001);
> > ++ for (int origin = 0; origin != 2; ++origin)
> > ++ assert_location_text ("foo.c:42:", "foo.c", 42, 0, true,
> > origin);
> > + assert_location_text ("foo.c:", "foo.c", 0, 10, true);
> > + assert_location_text ("foo.c:42:", "foo.c", 42, 10, false);
> > + assert_location_text ("foo.c:", "foo.c", 0, 10, false);
> > +@@ -2077,6 +2145,41 @@ test_diagnostic_get_location_text ()
> > + maybe_line_and_column (INT_MAX, INT_MAX);
> > + maybe_line_and_column (INT_MIN, INT_MIN);
> > +
> > ++ {
> > ++ /* In order to test display columns vs byte columns, we need to
> > create a
> > ++ file for location_get_source_line() to read. */
> > ++
> > ++ const char *const content = "smile \xf0\x9f\x98\x82\n";
> > ++ const int line_bytes = strlen (content) - 1;
> > ++ const int def_tabstop = 8;
> > ++ const int display_width = cpp_display_width (content,
> > line_bytes,
> > ++ def_tabstop);
> > ++ ASSERT_EQ (line_bytes - 2, display_width);
> > ++ temp_source_file tmp (SELFTEST_LOCATION, ".c", content);
> > ++ const char *const fname = tmp.get_filename ();
> > ++ const int buf_len = strlen (fname) + 16;
> > ++ char *const expected = XNEWVEC (char, buf_len);
> > ++
> > ++ snprintf (expected, buf_len, "%s:1:%d:", fname, line_bytes);
> > ++ assert_location_text (expected, fname, 1, line_bytes, true,
> > ++ 1, DIAGNOSTICS_COLUMN_UNIT_BYTE);
> > ++
> > ++ snprintf (expected, buf_len, "%s:1:%d:", fname, line_bytes -
> > 1);
> > ++ assert_location_text (expected, fname, 1, line_bytes, true,
> > ++ 0, DIAGNOSTICS_COLUMN_UNIT_BYTE);
> > ++
> > ++ snprintf (expected, buf_len, "%s:1:%d:", fname, display_width);
> > ++ assert_location_text (expected, fname, 1, line_bytes, true,
> > ++ 1, DIAGNOSTICS_COLUMN_UNIT_DISPLAY);
> > ++
> > ++ snprintf (expected, buf_len, "%s:1:%d:", fname, display_width -
> > 1);
> > ++ assert_location_text (expected, fname, 1, line_bytes, true,
> > ++ 0, DIAGNOSTICS_COLUMN_UNIT_DISPLAY);
> > ++
> > ++ XDELETEVEC (expected);
> > ++ }
> > ++
> > ++
> > + progname = old_progname;
> > + }
> > +
> > +diff --git a/gcc/diagnostic-format-json.cc b/gcc/diagnostic-format-
> > json.cc
> > +--- a/gcc/diagnostic-format-json.cc 2020-07-22 23:35:17.556386887
> > -0700
> > ++++ b/gcc/diagnostic-format-json.cc 2021-12-25 01:20:53.475636694
> > -0800
> > +@@ -23,6 +23,7 @@ along with GCC; see the file COPYING3.
> > + #include "system.h"
> > + #include "coretypes.h"
> > + #include "diagnostic.h"
> > ++#include "selftest-diagnostic.h"
> > + #include "diagnostic-metadata.h"
> > + #include "json.h"
> > + #include "selftest.h"
> > +@@ -43,21 +44,43 @@ static json::array *cur_children_array;
> > + /* Generate a JSON object for LOC. */
> > +
> > + json::value *
> > +-json_from_expanded_location (location_t loc)
> > ++json_from_expanded_location (diagnostic_context *context,
> > location_t loc)
> > + {
> > + expanded_location exploc = expand_location (loc);
> > + json::object *result = new json::object ();
> > + if (exploc.file)
> > + result->set ("file", new json::string (exploc.file));
> > + result->set ("line", new json::integer_number (exploc.line));
> > +- result->set ("column", new json::integer_number (exploc.column));
> > ++
> > ++ const enum diagnostics_column_unit orig_unit = context-
> > >column_unit;
> > ++ struct
> > ++ {
> > ++ const char *name;
> > ++ enum diagnostics_column_unit unit;
> > ++ } column_fields[] = {
> > ++ {"display-column", DIAGNOSTICS_COLUMN_UNIT_DISPLAY},
> > ++ {"byte-column", DIAGNOSTICS_COLUMN_UNIT_BYTE}
> > ++ };
> > ++ int the_column = INT_MIN;
> > ++ for (int i = 0; i != sizeof column_fields / sizeof
> > (*column_fields); ++i)
> > ++ {
> > ++ context->column_unit = column_fields[i].unit;
> > ++ const int col = diagnostic_converted_column (context,
> > exploc);
> > ++ result->set (column_fields[i].name, new json::integer_number
> > (col));
> > ++ if (column_fields[i].unit == orig_unit)
> > ++ the_column = col;
> > ++ }
> > ++ gcc_assert (the_column != INT_MIN);
> > ++ result->set ("column", new json::integer_number (the_column));
> > ++ context->column_unit = orig_unit;
> > + return result;
> > + }
> > +
> > + /* Generate a JSON object for LOC_RANGE. */
> > +
> > + static json::object *
> > +-json_from_location_range (const location_range *loc_range, unsigned
> > range_idx)
> > ++json_from_location_range (diagnostic_context *context,
> > ++ const location_range *loc_range, unsigned
> > range_idx)
> > + {
> > + location_t caret_loc = get_pure_location (loc_range->m_loc);
> > +
> > +@@ -68,13 +91,13 @@ json_from_location_range (const location
> > + location_t finish_loc = get_finish (loc_range->m_loc);
> > +
> > + json::object *result = new json::object ();
> > +- result->set ("caret", json_from_expanded_location (caret_loc));
> > ++ result->set ("caret", json_from_expanded_location (context,
> > caret_loc));
> > + if (start_loc != caret_loc
> > + && start_loc != UNKNOWN_LOCATION)
> > +- result->set ("start", json_from_expanded_location (start_loc));
> > ++ result->set ("start", json_from_expanded_location (context,
> > start_loc));
> > + if (finish_loc != caret_loc
> > + && finish_loc != UNKNOWN_LOCATION)
> > +- result->set ("finish", json_from_expanded_location
> > (finish_loc));
> > ++ result->set ("finish", json_from_expanded_location (context,
> > finish_loc));
> > +
> > + if (loc_range->m_label)
> > + {
> > +@@ -91,14 +114,14 @@ json_from_location_range (const location
> > + /* Generate a JSON object for HINT. */
> > +
> > + static json::object *
> > +-json_from_fixit_hint (const fixit_hint *hint)
> > ++json_from_fixit_hint (diagnostic_context *context, const fixit_hint
> > *hint)
> > + {
> > + json::object *fixit_obj = new json::object ();
> > +
> > + location_t start_loc = hint->get_start_loc ();
> > +- fixit_obj->set ("start", json_from_expanded_location
> > (start_loc));
> > ++ fixit_obj->set ("start", json_from_expanded_location (context,
> > start_loc));
> > + location_t next_loc = hint->get_next_loc ();
> > +- fixit_obj->set ("next", json_from_expanded_location (next_loc));
> > ++ fixit_obj->set ("next", json_from_expanded_location (context,
> > next_loc));
> > + fixit_obj->set ("string", new json::string (hint->get_string
> > ()));
> > +
> > + return fixit_obj;
> > +@@ -190,11 +213,13 @@ json_end_diagnostic (diagnostic_context
> > + else
> > + {
> > + /* Otherwise, make diag_obj be the top-level object within
> > the group;
> > +- add a "children" array. */
> > ++ add a "children" array and record the column origin. */
> > + toplevel_array->append (diag_obj);
> > + cur_group = diag_obj;
> > + cur_children_array = new json::array ();
> > + diag_obj->set ("children", cur_children_array);
> > ++ diag_obj->set ("column-origin",
> > ++ new json::integer_number (context-
> > >column_origin));
> > + }
> > +
> > + const rich_location *richloc = diagnostic->richloc;
> > +@@ -205,7 +230,7 @@ json_end_diagnostic (diagnostic_context
> > + for (unsigned int i = 0; i < richloc->get_num_locations (); i++)
> > + {
> > + const location_range *loc_range = richloc->get_range (i);
> > +- json::object *loc_obj = json_from_location_range (loc_range,
> > i);
> > ++ json::object *loc_obj = json_from_location_range (context,
> > loc_range, i);
> > + if (loc_obj)
> > + loc_array->append (loc_obj);
> > + }
> > +@@ -217,7 +242,7 @@ json_end_diagnostic (diagnostic_context
> > + for (unsigned int i = 0; i < richloc->get_num_fixit_hints ();
> > i++)
> > + {
> > + const fixit_hint *hint = richloc->get_fixit_hint (i);
> > +- json::object *fixit_obj = json_from_fixit_hint (hint);
> > ++ json::object *fixit_obj = json_from_fixit_hint (context,
> > hint);
> > + fixit_array->append (fixit_obj);
> > + }
> > + }
> > +@@ -320,7 +345,8 @@ namespace selftest {
> > + static void
> > + test_unknown_location ()
> > + {
> > +- delete json_from_expanded_location (UNKNOWN_LOCATION);
> > ++ test_diagnostic_context dc;
> > ++ delete json_from_expanded_location (&dc, UNKNOWN_LOCATION);
> > + }
> > +
> > + /* Verify that we gracefully handle attempts to serialize bad
> > +@@ -338,7 +364,8 @@ test_bad_endpoints ()
> > + loc_range.m_range_display_kind = SHOW_RANGE_WITH_CARET;
> > + loc_range.m_label = NULL;
> > +
> > +- json::object *obj = json_from_location_range (&loc_range, 0);
> > ++ test_diagnostic_context dc;
> > ++ json::object *obj = json_from_location_range (&dc, &loc_range,
> > 0);
> > + /* We should have a "caret" value, but no "start" or "finish"
> > values. */
> > + ASSERT_TRUE (obj != NULL);
> > + ASSERT_TRUE (obj->get ("caret") != NULL);
> > +diff --git a/gcc/diagnostic.h b/gcc/diagnostic.h
> > +--- a/gcc/diagnostic.h 2020-07-22 23:35:17.556386887 -0700
> > ++++ b/gcc/diagnostic.h 2021-12-25 01:20:53.479636627 -0800
> > +@@ -24,6 +24,20 @@ along with GCC; see the file COPYING3.
> > + #include "pretty-print.h"
> > + #include "diagnostic-core.h"
> > +
> > ++/* An enum for controlling what units to use for the column number
> > ++ when diagnostics are output, used by the -fdiagnostics-column-
> > unit option.
> > ++ Tabs will be expanded or not according to the value of -
> > ftabstop. The origin
> > ++ (default 1) is controlled by -fdiagnostics-column-origin. */
> > ++
> > ++enum diagnostics_column_unit
> > ++{
> > ++ /* The default from GCC 11 onwards: display columns. */
> > ++ DIAGNOSTICS_COLUMN_UNIT_DISPLAY,
> > ++
> > ++ /* The behavior in GCC 10 and earlier: simple bytes. */
> > ++ DIAGNOSTICS_COLUMN_UNIT_BYTE
> > ++};
> > ++
> > + /* Enum for overriding the standard output format. */
> > +
> > + enum diagnostics_output_format
> > +@@ -280,6 +294,15 @@ struct diagnostic_context
> > + rest of the diagnostic. */
> > + bool parseable_fixits_p;
> > +
> > ++ /* What units to use when outputting the column number. */
> > ++ enum diagnostics_column_unit column_unit;
> > ++
> > ++ /* The origin for the column number (1-based or 0-based
> > typically). */
> > ++ int column_origin;
> > ++
> > ++ /* The size of the tabstop for tab expansion. */
> > ++ int tabstop;
> > ++
> > + /* If non-NULL, an edit_context to which fix-it hints should be
> > + applied, for generating patches. */
> > + edit_context *edit_context_ptr;
> > +@@ -458,6 +481,8 @@ diagnostic_same_line (const diagnostic_c
> > + }
> > +
> > + extern const char *diagnostic_get_color_for_kind (diagnostic_t
> > kind);
> > ++extern int diagnostic_converted_column (diagnostic_context
> > *context,
> > ++ expanded_location s);
> > +
> > + /* Pure text formatting support functions. */
> > + extern char *file_name_as_prefix (diagnostic_context *, const char
> > *);
> > +@@ -470,6 +495,7 @@ extern void diagnostic_output_format_ini
> > + /* Compute the number of digits in the decimal representation of an
> > integer. */
> > + extern int num_digits (int);
> > +
> > +-extern json::value *json_from_expanded_location (location_t loc);
> > ++extern json::value *json_from_expanded_location (diagnostic_context
> > *context,
> > ++ location_t loc);
> > +
> > + #endif /* ! GCC_DIAGNOSTIC_H */
> > +diff --git a/gcc/diagnostic-show-locus.c b/gcc/diagnostic-show-
> > locus.c
> > +--- a/gcc/diagnostic-show-locus.c 2020-07-22 23:35:17.556386887
> > -0700
> > ++++ b/gcc/diagnostic-show-locus.c 2021-12-25 01:20:53.479636627
> > -0800
> > +@@ -175,9 +175,10 @@ enum column_unit {
> > + class exploc_with_display_col : public expanded_location
> > + {
> > + public:
> > +- exploc_with_display_col (const expanded_location &exploc)
> > ++ exploc_with_display_col (const expanded_location &exploc, int
> > tabstop)
> > + : expanded_location (exploc),
> > +- m_display_col (location_compute_display_column (exploc)) {}
> > ++ m_display_col (location_compute_display_column (exploc,
> > tabstop))
> > ++ {}
> > +
> > + int m_display_col;
> > + };
> > +@@ -189,11 +190,11 @@ class exploc_with_display_col : public e
> > + class layout_point
> > + {
> > + public:
> > +- layout_point (const expanded_location &exploc)
> > ++ layout_point (const exploc_with_display_col &exploc)
> > + : m_line (exploc.line)
> > + {
> > + m_columns[CU_BYTES] = exploc.column;
> > +- m_columns[CU_DISPLAY_COLS] = location_compute_display_column
> > (exploc);
> > ++ m_columns[CU_DISPLAY_COLS] = exploc.m_display_col;
> > + }
> > +
> > + linenum_type m_line;
> > +@@ -205,10 +206,10 @@ class layout_point
> > + class layout_range
> > + {
> > + public:
> > +- layout_range (const expanded_location *start_exploc,
> > +- const expanded_location *finish_exploc,
> > ++ layout_range (const exploc_with_display_col &start_exploc,
> > ++ const exploc_with_display_col &finish_exploc,
> > + enum range_display_kind range_display_kind,
> > +- const expanded_location *caret_exploc,
> > ++ const exploc_with_display_col &caret_exploc,
> > + unsigned original_idx,
> > + const range_label *label);
> > +
> > +@@ -226,22 +227,18 @@ class layout_range
> > +
> > + /* A struct for use by layout::print_source_line for telling
> > + layout::print_annotation_line the extents of the source line
> > that
> > +- it printed, so that underlines can be clipped appropriately. */
> > ++ it printed, so that underlines can be clipped appropriately.
> > Units
> > ++ are 1-based display columns. */
> > +
> > + struct line_bounds
> > + {
> > +- int m_first_non_ws;
> > +- int m_last_non_ws;
> > ++ int m_first_non_ws_disp_col;
> > ++ int m_last_non_ws_disp_col;
> > +
> > +- void convert_to_display_cols (char_span line)
> > ++ line_bounds ()
> > + {
> > +- m_first_non_ws = cpp_byte_column_to_display_column
> > (line.get_buffer (),
> > +- line.length
> > (),
> > +-
> > m_first_non_ws)
> > ;
> > +-
> > +- m_last_non_ws = cpp_byte_column_to_display_column
> > (line.get_buffer (),
> > +- line.length
> > (),
> > +-
> > m_last_non_ws);
> > ++ m_first_non_ws_disp_col = INT_MAX;
> > ++ m_last_non_ws_disp_col = 0;
> > + }
> > + };
> > +
> > +@@ -351,8 +348,8 @@ class layout
> > + private:
> > + bool will_show_line_p (linenum_type row) const;
> > + void print_leading_fixits (linenum_type row);
> > +- void print_source_line (linenum_type row, const char *line, int
> > line_bytes,
> > +- line_bounds *lbounds_out);
> > ++ line_bounds print_source_line (linenum_type row, const char
> > *line,
> > ++ int line_bytes);
> > + bool should_print_annotation_line_p (linenum_type row) const;
> > + void start_annotation_line (char margin_char = ' ') const;
> > + void print_annotation_line (linenum_type row, const line_bounds
> > lbounds);
> > +@@ -513,16 +510,16 @@ colorizer::get_color_by_name (const char
> > + Initialize various layout_point fields from expanded_location
> > + equivalents; we've already filtered on file. */
> > +
> > +-layout_range::layout_range (const expanded_location *start_exploc,
> > +- const expanded_location *finish_exploc,
> > ++layout_range::layout_range (const exploc_with_display_col
> > &start_exploc,
> > ++ const exploc_with_display_col
> > &finish_exploc,
> > + enum range_display_kind
> > range_display_kind,
> > +- const expanded_location *caret_exploc,
> > ++ const exploc_with_display_col
> > &caret_exploc,
> > + unsigned original_idx,
> > + const range_label *label)
> > +-: m_start (*start_exploc),
> > +- m_finish (*finish_exploc),
> > ++: m_start (start_exploc),
> > ++ m_finish (finish_exploc),
> > + m_range_display_kind (range_display_kind),
> > +- m_caret (*caret_exploc),
> > ++ m_caret (caret_exploc),
> > + m_original_idx (original_idx),
> > + m_label (label)
> > + {
> > +@@ -646,6 +643,9 @@ layout_range::intersects_line_p (linenum
> > +
> > + #if CHECKING_P
> > +
> > ++/* Default for when we don't care what the tab expansion is set
> > to. */
> > ++static const int def_tabstop = 8;
> > ++
> > + /* Create some expanded locations for testing layout_range. The
> > filename
> > + member of the explocs is set to the empty string. This member
> > will only be
> > + inspected by the calls to location_compute_display_column() made
> > from the
> > +@@ -662,8 +662,11 @@ make_range (int start_line, int start_co
> > + = {"", start_line, start_col, NULL, false};
> > + const expanded_location finish_exploc
> > + = {"", end_line, end_col, NULL, false};
> > +- return layout_range (&start_exploc, &finish_exploc,
> > SHOW_RANGE_WITHOUT_CARET,
> > +- &start_exploc, 0, NULL);
> > ++ return layout_range (exploc_with_display_col (start_exploc,
> > def_tabstop),
> > ++ exploc_with_display_col (finish_exploc,
> > def_tabstop),
> > ++ SHOW_RANGE_WITHOUT_CARET,
> > ++ exploc_with_display_col (start_exploc,
> > def_tabstop),
> > ++ 0, NULL);
> > + }
> > +
> > + /* Selftests for layout_range::contains_point and
> > +@@ -964,7 +967,7 @@ layout::layout (diagnostic_context * con
> > + : m_context (context),
> > + m_pp (context->printer),
> > + m_primary_loc (richloc->get_range (0)->m_loc),
> > +- m_exploc (richloc->get_expanded_location (0)),
> > ++ m_exploc (richloc->get_expanded_location (0), context->tabstop),
> > + m_colorizer (context, diagnostic_kind),
> > + m_colorize_source_p (context->colorize_source_p),
> > + m_show_labels_p (context->show_labels_p),
> > +@@ -1060,7 +1063,10 @@ layout::maybe_add_location_range (const
> > +
> > + /* Everything is now known to be in the correct source file,
> > + but it may require further sanitization. */
> > +- layout_range ri (&start, &finish, loc_range-
> > >m_range_display_kind, &caret,
> > ++ layout_range ri (exploc_with_display_col (start, m_context-
> > >tabstop),
> > ++ exploc_with_display_col (finish, m_context-
> > >tabstop),
> > ++ loc_range->m_range_display_kind,
> > ++ exploc_with_display_col (caret, m_context-
> > >tabstop),
> > + original_idx, loc_range->m_label);
> > +
> > + /* If we have a range that finishes before it starts (perhaps
> > +@@ -1394,7 +1400,7 @@ layout::calculate_x_offset_display ()
> > + = get_line_bytes_without_trailing_whitespace (line.get_buffer
> > (),
> > + line.length ());
> > + int eol_display_column
> > +- = cpp_display_width (line.get_buffer (), line_bytes);
> > ++ = cpp_display_width (line.get_buffer (), line_bytes, m_context-
> > >tabstop);
> > + if (caret_display_column > eol_display_column
> > + || !caret_display_column)
> > + {
> > +@@ -1445,16 +1451,13 @@ layout::calculate_x_offset_display ()
> > + }
> > +
> > + /* Print line ROW of source code, potentially colorized at any
> > ranges, and
> > +- populate *LBOUNDS_OUT.
> > +- LINE is the source line (not necessarily 0-terminated) and
> > LINE_BYTES
> > +- is its length in bytes.
> > +- This function deals only with byte offsets, not display columns,
> > so
> > +- m_x_offset_display must be converted from display to byte
> > units. In
> > +- particular, LINE_BYTES and LBOUNDS_OUT are in bytes. */
> > ++ return the line bounds. LINE is the source line (not
> > necessarily
> > ++ 0-terminated) and LINE_BYTES is its length in bytes. In order
> > to handle both
> > ++ colorization and tab expansion, this function tracks the line
> > position in
> > ++ both byte and display column units. */
> > +
> > +-void
> > +-layout::print_source_line (linenum_type row, const char *line, int
> > line_bytes,
> > +- line_bounds *lbounds_out)
> > ++line_bounds
> > ++layout::print_source_line (linenum_type row, const char *line, int
> > line_bytes)
> > + {
> > + m_colorizer.set_normal_text ();
> > +
> > +@@ -1469,30 +1472,29 @@ layout::print_source_line (linenum_type
> > + else
> > + pp_space (m_pp);
> > +
> > +- /* We will stop printing the source line at any trailing
> > whitespace, and start
> > +- printing it as per m_x_offset_display. */
> > ++ /* We will stop printing the source line at any trailing
> > whitespace. */
> > + line_bytes = get_line_bytes_without_trailing_whitespace (line,
> > +
> > line_bytes);
> > +- int x_offset_bytes = 0;
> > +- if (m_x_offset_display)
> > +- {
> > +- x_offset_bytes = cpp_display_column_to_byte_column (line,
> > line_bytes,
> > +-
> > m_x_offset_display);
> > +- /* In case the leading portion of the line that will be
> > skipped over ends
> > +- with a character with wcwidth > 1, then it is possible we
> > skipped too
> > +- much, so account for that by padding with spaces. */
> > +- const int overage
> > +- = cpp_byte_column_to_display_column (line, line_bytes,
> > x_offset_bytes)
> > +- - m_x_offset_display;
> > +- for (int column = 0; column < overage; ++column)
> > +- pp_space (m_pp);
> > +- line += x_offset_bytes;
> > +- }
> > +
> > +- /* Print the line. */
> > +- int first_non_ws = INT_MAX;
> > +- int last_non_ws = 0;
> > +- for (int col_byte = 1 + x_offset_bytes; col_byte <= line_bytes;
> > col_byte++)
> > ++ /* This object helps to keep track of which display column we are
> > at, which is
> > ++ necessary for computing the line bounds in display units, for
> > doing
> > ++ tab expansion, and for implementing m_x_offset_display. */
> > ++ cpp_display_width_computation dw (line, line_bytes, m_context-
> > >tabstop);
> > ++
> > ++ /* Skip the first m_x_offset_display display columns. In case
> > the leading
> > ++ portion that will be skipped ends with a character with
> > wcwidth > 1, then
> > ++ it is possible we skipped too much, so account for that by
> > padding with
> > ++ spaces. Note that this does the right thing too in case a tab
> > was the last
> > ++ character to be skipped over; the tab is effectively replaced
> > by the
> > ++ correct number of trailing spaces needed to offset by the
> > desired number of
> > ++ display columns. */
> > ++ for (int skipped_display_cols = dw.advance_display_cols
> > (m_x_offset_display);
> > ++ skipped_display_cols > m_x_offset_display; --
> > skipped_display_cols)
> > ++ pp_space (m_pp);
> > ++
> > ++ /* Print the line and compute the line_bounds. */
> > ++ line_bounds lbounds;
> > ++ while (!dw.done ())
> > + {
> > + /* Assuming colorization is enabled for the caret and
> > underline
> > + characters, we may also colorize the associated characters
> > +@@ -1510,7 +1512,8 @@ layout::print_source_line (linenum_type
> > + {
> > + bool in_range_p;
> > + point_state state;
> > +- in_range_p = get_state_at_point (row, col_byte,
> > ++ const int start_byte_col = dw.bytes_processed () + 1;
> > ++ in_range_p = get_state_at_point (row, start_byte_col,
> > + 0, INT_MAX,
> > + CU_BYTES,
> > + &state);
> > +@@ -1519,22 +1522,44 @@ layout::print_source_line (linenum_type
> > + else
> > + m_colorizer.set_normal_text ();
> > + }
> > +- char c = *line;
> > +- if (c == '\0' || c == '\t' || c == '\r')
> > +- c = ' ';
> > +- if (c != ' ')
> > ++
> > ++ /* Get the display width of the next character to be output,
> > expanding
> > ++ tabs and replacing some control bytes with spaces as
> > necessary. */
> > ++ const char *c = dw.next_byte ();
> > ++ const int start_disp_col = dw.display_cols_processed () + 1;
> > ++ const int this_display_width = dw.process_next_codepoint ();
> > ++ if (*c == '\t')
> > ++ {
> > ++ /* The returned display width is the number of spaces into
> > which the
> > ++ tab should be expanded. */
> > ++ for (int i = 0; i != this_display_width; ++i)
> > ++ pp_space (m_pp);
> > ++ continue;
> > ++ }
> > ++ if (*c == '\0' || *c == '\r')
> > + {
> > +- last_non_ws = col_byte;
> > +- if (first_non_ws == INT_MAX)
> > +- first_non_ws = col_byte;
> > ++ /* cpp_wcwidth() promises to return 1 for all control
> > bytes, and we
> > ++ want to output these as a single space too, so this case
> > is
> > ++ actually the same as the '\t' case. */
> > ++ gcc_assert (this_display_width == 1);
> > ++ pp_space (m_pp);
> > ++ continue;
> > + }
> > +- pp_character (m_pp, c);
> > +- line++;
> > ++
> > ++ /* We have a (possibly multibyte) character to output; update
> > the line
> > ++ bounds if it is not whitespace. */
> > ++ if (*c != ' ')
> > ++ {
> > ++ lbounds.m_last_non_ws_disp_col = dw.display_cols_processed
> > ();
> > ++ if (lbounds.m_first_non_ws_disp_col == INT_MAX)
> > ++ lbounds.m_first_non_ws_disp_col = start_disp_col;
> > ++ }
> > ++
> > ++ /* Output the character. */
> > ++ while (c != dw.next_byte ()) pp_character (m_pp, *c++);
> > + }
> > + print_newline ();
> > +-
> > +- lbounds_out->m_first_non_ws = first_non_ws;
> > +- lbounds_out->m_last_non_ws = last_non_ws;
> > ++ return lbounds;
> > + }
> > +
> > + /* Determine if we should print an annotation line for ROW.
> > +@@ -1576,14 +1601,13 @@ layout::start_annotation_line (char marg
> > + }
> > +
> > + /* Print a line consisting of the caret/underlines for the given
> > +- source line. This function works with display columns, rather
> > than byte
> > +- counts; in particular, LBOUNDS should be in display column
> > units. */
> > ++ source line. */
> > +
> > + void
> > + layout::print_annotation_line (linenum_type row, const line_bounds
> > lbounds)
> > + {
> > + int x_bound = get_x_bound_for_row (row, m_exploc.m_display_col,
> > +- lbounds.m_last_non_ws);
> > ++ lbounds.m_last_non_ws_disp_col);
> > +
> > + start_annotation_line ();
> > + pp_space (m_pp);
> > +@@ -1593,8 +1617,8 @@ layout::print_annotation_line (linenum_t
> > + bool in_range_p;
> > + point_state state;
> > + in_range_p = get_state_at_point (row, column,
> > +- lbounds.m_first_non_ws,
> > +- lbounds.m_last_non_ws,
> > ++
> > lbounds.m_first_non_ws_disp_col,
> > ++
> > lbounds.m_last_non_ws_disp_col,
> > + CU_DISPLAY_COLS,
> > + &state);
> > + if (in_range_p)
> > +@@ -1631,12 +1655,14 @@ layout::print_annotation_line (linenum_t
> > + class line_label
> > + {
> > + public:
> > +- line_label (int state_idx, int column, label_text text)
> > ++ line_label (diagnostic_context *context, int state_idx, int
> > column,
> > ++ label_text text)
> > + : m_state_idx (state_idx), m_column (column),
> > + m_text (text), m_label_line (0), m_has_vbar (true)
> > + {
> > + const int bytes = strlen (text.m_buffer);
> > +- m_display_width = cpp_display_width (text.m_buffer, bytes);
> > ++ m_display_width
> > ++ = cpp_display_width (text.m_buffer, bytes, context->tabstop);
> > + }
> > +
> > + /* Sorting is primarily by column, then by state index. */
> > +@@ -1696,7 +1722,7 @@ layout::print_any_labels (linenum_type r
> > + if (text.m_buffer == NULL)
> > + continue;
> > +
> > +- labels.safe_push (line_label (i, disp_col, text));
> > ++ labels.safe_push (line_label (m_context, i, disp_col, text));
> > + }
> > + }
> > +
> > +@@ -1976,7 +2002,8 @@ public:
> > +
> > + /* Get the range of bytes or display columns that HINT would
> > affect. */
> > + static column_range
> > +-get_affected_range (const fixit_hint *hint, enum column_unit
> > col_unit)
> > ++get_affected_range (diagnostic_context *context,
> > ++ const fixit_hint *hint, enum column_unit
> > col_unit)
> > + {
> > + expanded_location exploc_start = expand_location (hint-
> > >get_start_loc ());
> > + expanded_location exploc_finish = expand_location (hint-
> > >get_next_loc ());
> > +@@ -1986,11 +2013,13 @@ get_affected_range (const fixit_hint *hi
> > + int finish_column;
> > + if (col_unit == CU_DISPLAY_COLS)
> > + {
> > +- start_column = location_compute_display_column
> > (exploc_start);
> > ++ start_column
> > ++ = location_compute_display_column (exploc_start, context-
> > >tabstop);
> > + if (hint->insertion_p ())
> > + finish_column = start_column - 1;
> > + else
> > +- finish_column = location_compute_display_column
> > (exploc_finish);
> > ++ finish_column
> > ++ = location_compute_display_column (exploc_finish, context-
> > >tabstop);
> > + }
> > + else
> > + {
> > +@@ -2003,12 +2032,12 @@ get_affected_range (const fixit_hint *hi
> > + /* Get the range of display columns that would be printed for
> > HINT. */
> > +
> > + static column_range
> > +-get_printed_columns (const fixit_hint *hint)
> > ++get_printed_columns (diagnostic_context *context, const fixit_hint
> > *hint)
> > + {
> > + expanded_location exploc = expand_location (hint->get_start_loc
> > ());
> > +- int start_column = location_compute_display_column (exploc);
> > +- int hint_width = cpp_display_width (hint->get_string (),
> > +- hint->get_length ());
> > ++ int start_column = location_compute_display_column (exploc,
> > context->tabstop);
> > ++ int hint_width = cpp_display_width (hint->get_string (), hint-
> > >get_length (),
> > ++ context->tabstop);
> > + int final_hint_column = start_column + hint_width - 1;
> > + if (hint->insertion_p ())
> > + {
> > +@@ -2018,7 +2047,8 @@ get_printed_columns (const fixit_hint *h
> > + {
> > + exploc = expand_location (hint->get_next_loc ());
> > + --exploc.column;
> > +- int finish_column = location_compute_display_column (exploc);
> > ++ int finish_column
> > ++ = location_compute_display_column (exploc, context->tabstop);
> > + return column_range (start_column,
> > + MAX (finish_column, final_hint_column));
> > + }
> > +@@ -2035,12 +2065,14 @@ public:
> > + correction (column_range affected_bytes,
> > + column_range affected_columns,
> > + column_range printed_columns,
> > +- const char *new_text, size_t new_text_len)
> > ++ const char *new_text, size_t new_text_len,
> > ++ int tabstop)
> > + : m_affected_bytes (affected_bytes),
> > + m_affected_columns (affected_columns),
> > + m_printed_columns (printed_columns),
> > + m_text (xstrdup (new_text)),
> > + m_byte_length (new_text_len),
> > ++ m_tabstop (tabstop),
> > + m_alloc_sz (new_text_len + 1)
> > + {
> > + compute_display_cols ();
> > +@@ -2058,7 +2090,7 @@ public:
> > +
> > + void compute_display_cols ()
> > + {
> > +- m_display_cols = cpp_display_width (m_text, m_byte_length);
> > ++ m_display_cols = cpp_display_width (m_text, m_byte_length,
> > m_tabstop);
> > + }
> > +
> > + void overwrite (int dst_offset, const char_span &src_span)
> > +@@ -2086,6 +2118,7 @@ public:
> > + char *m_text;
> > + size_t m_byte_length; /* Not including null-terminator. */
> > + int m_display_cols;
> > ++ int m_tabstop;
> > + size_t m_alloc_sz;
> > + };
> > +
> > +@@ -2121,13 +2154,15 @@ correction::ensure_terminated ()
> > + class line_corrections
> > + {
> > + public:
> > +- line_corrections (const char *filename, linenum_type row)
> > +- : m_filename (filename), m_row (row)
> > ++ line_corrections (diagnostic_context *context, const char
> > *filename,
> > ++ linenum_type row)
> > ++ : m_context (context), m_filename (filename), m_row (row)
> > + {}
> > + ~line_corrections ();
> > +
> > + void add_hint (const fixit_hint *hint);
> > +
> > ++ diagnostic_context *m_context;
> > + const char *m_filename;
> > + linenum_type m_row;
> > + auto_vec <correction *> m_corrections;
> > +@@ -2173,9 +2208,10 @@ source_line::source_line (const char *fi
> > + void
> > + line_corrections::add_hint (const fixit_hint *hint)
> > + {
> > +- column_range affected_bytes = get_affected_range (hint,
> > CU_BYTES);
> > +- column_range affected_columns = get_affected_range (hint,
> > CU_DISPLAY_COLS);
> > +- column_range printed_columns = get_printed_columns (hint);
> > ++ column_range affected_bytes = get_affected_range (m_context,
> > hint, CU_BYTES);
> > ++ column_range affected_columns = get_affected_range (m_context,
> > hint,
> > ++
> > CU_DISPLAY_COLS);
> > ++ column_range printed_columns = get_printed_columns (m_context,
> > hint);
> > +
> > + /* Potentially consolidate. */
> > + if (!m_corrections.is_empty ())
> > +@@ -2243,7 +2279,8 @@ line_corrections::add_hint (const fixit_
> > + affected_columns,
> > + printed_columns,
> > + hint->get_string (),
> > +- hint->get_length ()));
> > ++ hint->get_length (),
> > ++ m_context->tabstop));
> > + }
> > +
> > + /* If there are any fixit hints on source line ROW, print them.
> > +@@ -2257,7 +2294,7 @@ layout::print_trailing_fixits (linenum_t
> > + {
> > + /* Build a list of correction instances for the line,
> > + potentially consolidating hints (for the sake of
> > readability). */
> > +- line_corrections corrections (m_exploc.file, row);
> > ++ line_corrections corrections (m_context, m_exploc.file, row);
> > + for (unsigned int i = 0; i < m_fixit_hints.length (); i++)
> > + {
> > + const fixit_hint *hint = m_fixit_hints[i];
> > +@@ -2499,15 +2536,11 @@ layout::print_line (linenum_type row)
> > + if (!line)
> > + return;
> > +
> > +- line_bounds lbounds;
> > + print_leading_fixits (row);
> > +- print_source_line (row, line.get_buffer (), line.length (),
> > &lbounds);
> > ++ const line_bounds lbounds
> > ++ = print_source_line (row, line.get_buffer (), line.length ());
> > + if (should_print_annotation_line_p (row))
> > +- {
> > +- if (lbounds.m_first_non_ws != INT_MAX)
> > +- lbounds.convert_to_display_cols (line);
> > +- print_annotation_line (row, lbounds);
> > +- }
> > ++ print_annotation_line (row, lbounds);
> > + if (m_show_labels_p)
> > + print_any_labels (row);
> > + print_trailing_fixits (row);
> > +@@ -2670,9 +2703,11 @@ test_layout_x_offset_display_utf8 (const
> > +
> > + char_span lspan = location_get_source_line (tmp.get_filename (),
> > 1);
> > + ASSERT_EQ (line_display_cols,
> > +- cpp_display_width (lspan.get_buffer (), lspan.length
> > ()));
> > ++ cpp_display_width (lspan.get_buffer (), lspan.length (),
> > ++ def_tabstop));
> > + ASSERT_EQ (line_display_cols,
> > +- location_compute_display_column (expand_location
> > (line_end)));
> > ++ location_compute_display_column (expand_location
> > (line_end),
> > ++ def_tabstop));
> > + ASSERT_EQ (0, memcmp (lspan.get_buffer () + (emoji_col - 1),
> > + "\xf0\x9f\x98\x82\xf0\x9f\x98\x82", 8));
> > +
> > +@@ -2774,6 +2809,111 @@ test_layout_x_offset_display_utf8 (const
> > +
> > + }
> > +
> > ++static void
> > ++test_layout_x_offset_display_tab (const line_table_case &case_)
> > ++{
> > ++ const char *content
> > ++ = "This line is very long, so that we can use it to test the
> > logic for "
> > ++ "clipping long lines. Also this: `\t' is a tab that occupies
> > 1 byte and "
> > ++ "a variable number of display columns, starting at column
> > #103.\n";
> > ++
> > ++ /* Number of bytes in the line, subtracting one to remove the
> > newline. */
> > ++ const int line_bytes = strlen (content) - 1;
> > ++
> > ++ /* The column where the tab begins. Byte or display is the same
> > as there are
> > ++ no multibyte characters earlier on the line. */
> > ++ const int tab_col = 103;
> > ++
> > ++ /* Effective extra size of the tab beyond what a single space
> > would have taken
> > ++ up, indexed by tabstop. */
> > ++ static const int num_tabstops = 11;
> > ++ int extra_width[num_tabstops];
> > ++ for (int tabstop = 1; tabstop != num_tabstops; ++tabstop)
> > ++ {
> > ++ const int this_tab_size = tabstop - (tab_col - 1) % tabstop;
> > ++ extra_width[tabstop] = this_tab_size - 1;
> > ++ }
> > ++ /* Example of this calculation: if tabstop is 10, the tab
> > starting at column
> > ++ #103 has to expand into 8 spaces, covering columns 103-110, so
> > that the
> > ++ next character is at column #111. So it takes up 7 more
> > columns than
> > ++ a space would have taken up. */
> > ++ ASSERT_EQ (7, extra_width[10]);
> > ++
> > ++ temp_source_file tmp (SELFTEST_LOCATION, ".c", content);
> > ++ line_table_test ltt (case_);
> > ++
> > ++ linemap_add (line_table, LC_ENTER, false, tmp.get_filename (),
> > 1);
> > ++
> > ++ location_t line_end = linemap_position_for_column (line_table,
> > line_bytes);
> > ++
> > ++ /* Don't attempt to run the tests if column data might be
> > unavailable. */
> > ++ if (line_end > LINE_MAP_MAX_LOCATION_WITH_COLS)
> > ++ return;
> > ++
> > ++ /* Check that cpp_display_width handles the tabs as expected. */
> > ++ char_span lspan = location_get_source_line (tmp.get_filename (),
> > 1);
> > ++ ASSERT_EQ ('\t', *(lspan.get_buffer () + (tab_col - 1)));
> > ++ for (int tabstop = 1; tabstop != num_tabstops; ++tabstop)
> > ++ {
> > ++ ASSERT_EQ (line_bytes + extra_width[tabstop],
> > ++ cpp_display_width (lspan.get_buffer (), lspan.length
> > (),
> > ++ tabstop));
> > ++ ASSERT_EQ (line_bytes + extra_width[tabstop],
> > ++ location_compute_display_column (expand_location
> > (line_end),
> > ++ tabstop));
> > ++ }
> > ++
> > ++ /* Check that the tab is expanded to the expected number of
> > spaces. */
> > ++ rich_location richloc (line_table,
> > ++ linemap_position_for_column (line_table,
> > ++ tab_col + 1));
> > ++ for (int tabstop = 1; tabstop != num_tabstops; ++tabstop)
> > ++ {
> > ++ test_diagnostic_context dc;
> > ++ dc.tabstop = tabstop;
> > ++ layout test_layout (&dc, &richloc, DK_ERROR);
> > ++ test_layout.print_line (1);
> > ++ const char *out = pp_formatted_text (dc.printer);
> > ++ ASSERT_EQ (NULL, strchr (out, '\t'));
> > ++ const char *left_quote = strchr (out, '`');
> > ++ const char *right_quote = strchr (out, '\'');
> > ++ ASSERT_NE (NULL, left_quote);
> > ++ ASSERT_NE (NULL, right_quote);
> > ++ ASSERT_EQ (right_quote - left_quote, extra_width[tabstop] +
> > 2);
> > ++ }
> > ++
> > ++ /* Check that the line is offset properly and that the tab is
> > broken up
> > ++ into the expected number of spaces when it is the last
> > character skipped
> > ++ over. */
> > ++ for (int tabstop = 1; tabstop != num_tabstops; ++tabstop)
> > ++ {
> > ++ test_diagnostic_context dc;
> > ++ dc.tabstop = tabstop;
> > ++ static const int small_width = 24;
> > ++ dc.caret_max_width = small_width - 4;
> > ++ dc.min_margin_width = test_left_margin - test_linenum_sep +
> > 1;
> > ++ dc.show_line_numbers_p = true;
> > ++ layout test_layout (&dc, &richloc, DK_ERROR);
> > ++ test_layout.print_line (1);
> > ++
> > ++ /* We have arranged things so that two columns will be
> > printed before
> > ++ the caret. If the tab results in more than one space, this
> > should
> > ++ produce two spaces in the output; otherwise, it will be a
> > single space
> > ++ preceded by the opening quote before the tab character. */
> > ++ const char *output1
> > ++ = " 1 | ' is a tab that occupies 1 byte and a variable
> > number of "
> > ++ "display columns, starting at column #103.\n"
> > ++ " | ^\n\n";
> > ++ const char *output2
> > ++ = " 1 | ` ' is a tab that occupies 1 byte and a variable
> > number of "
> > ++ "display columns, starting at column #103.\n"
> > ++ " | ^\n\n";
> > ++ const char *expected_output = (extra_width[tabstop] ? output1
> > : output2);
> > ++ ASSERT_STREQ (expected_output, pp_formatted_text
> > (dc.printer));
> > ++ }
> > ++}
> > ++
> > ++
> > + /* Verify that diagnostic_show_locus works sanely on
> > UNKNOWN_LOCATION. */
> > +
> > + static void
> > +@@ -3854,6 +3994,27 @@ test_one_liner_labels_utf8 ()
> > + }
> > + }
> > +
> > ++/* Make sure that colorization codes don't interrupt a multibyte
> > ++ sequence, which would corrupt it. */
> > ++static void
> > ++test_one_liner_colorized_utf8 ()
> > ++{
> > ++ test_diagnostic_context dc;
> > ++ dc.colorize_source_p = true;
> > ++ diagnostic_color_init (&dc, DIAGNOSTICS_COLOR_YES);
> > ++ const location_t pi = linemap_position_for_column (line_table,
> > 12);
> > ++ rich_location richloc (line_table, pi);
> > ++ diagnostic_show_locus (&dc, &richloc, DK_ERROR);
> > ++
> > ++ /* In order to avoid having the test depend on exactly how the
> > colorization
> > ++ was effected, just confirm there are two pi characters in the
> > output. */
> > ++ const char *result = pp_formatted_text (dc.printer);
> > ++ const char *null_term = result + strlen (result);
> > ++ const char *first_pi = strstr (result, "\xcf\x80");
> > ++ ASSERT_TRUE (first_pi && first_pi <= null_term - 2);
> > ++ ASSERT_STR_CONTAINS (first_pi + 2, "\xcf\x80");
> > ++}
> > ++
> > + /* Run the various one-liner tests. */
> > +
> > + static void
> > +@@ -3884,8 +4045,10 @@ test_diagnostic_show_locus_one_liner_utf
> > + ASSERT_EQ (31, LOCATION_COLUMN (line_end));
> > +
> > + char_span lspan = location_get_source_line (tmp.get_filename (),
> > 1);
> > +- ASSERT_EQ (25, cpp_display_width (lspan.get_buffer (),
> > lspan.length ()));
> > +- ASSERT_EQ (25, location_compute_display_column (expand_location
> > (line_end)));
> > ++ ASSERT_EQ (25, cpp_display_width (lspan.get_buffer (),
> > lspan.length (),
> > ++ def_tabstop));
> > ++ ASSERT_EQ (25, location_compute_display_column (expand_location
> > (line_end),
> > ++ def_tabstop));
> > +
> > + test_one_liner_simple_caret_utf8 ();
> > + test_one_liner_caret_and_range_utf8 ();
> > +@@ -3900,6 +4063,7 @@ test_diagnostic_show_locus_one_liner_utf
> > + test_one_liner_many_fixits_1_utf8 ();
> > + test_one_liner_many_fixits_2_utf8 ();
> > + test_one_liner_labels_utf8 ();
> > ++ test_one_liner_colorized_utf8 ();
> > + }
> > +
> > + /* Verify that gcc_rich_location::add_location_if_nearby works. */
> > +@@ -4272,25 +4436,28 @@ test_overlapped_fixit_printing (const li
> > + /* Unit-test the line_corrections machinery. */
> > + ASSERT_EQ (3, richloc.get_num_fixit_hints ());
> > + const fixit_hint *hint_0 = richloc.get_fixit_hint (0);
> > +- ASSERT_EQ (column_range (12, 12), get_affected_range (hint_0,
> > CU_BYTES));
> > + ASSERT_EQ (column_range (12, 12),
> > +- get_affected_range (hint_0,
> > CU_DISPLAY_COLS));
> > +- ASSERT_EQ (column_range (12, 22), get_printed_columns
> > (hint_0));
> > ++ get_affected_range (&dc, hint_0, CU_BYTES));
> > ++ ASSERT_EQ (column_range (12, 12),
> > ++ get_affected_range (&dc, hint_0, CU_DISPLAY_COLS));
> > ++ ASSERT_EQ (column_range (12, 22), get_printed_columns (&dc,
> > hint_0));
> > + const fixit_hint *hint_1 = richloc.get_fixit_hint (1);
> > +- ASSERT_EQ (column_range (18, 18), get_affected_range (hint_1,
> > CU_BYTES));
> > + ASSERT_EQ (column_range (18, 18),
> > +- get_affected_range (hint_1,
> > CU_DISPLAY_COLS));
> > +- ASSERT_EQ (column_range (18, 20), get_printed_columns
> > (hint_1));
> > ++ get_affected_range (&dc, hint_1, CU_BYTES));
> > ++ ASSERT_EQ (column_range (18, 18),
> > ++ get_affected_range (&dc, hint_1, CU_DISPLAY_COLS));
> > ++ ASSERT_EQ (column_range (18, 20), get_printed_columns (&dc,
> > hint_1));
> > + const fixit_hint *hint_2 = richloc.get_fixit_hint (2);
> > +- ASSERT_EQ (column_range (29, 28), get_affected_range (hint_2,
> > CU_BYTES));
> > + ASSERT_EQ (column_range (29, 28),
> > +- get_affected_range (hint_2,
> > CU_DISPLAY_COLS));
> > +- ASSERT_EQ (column_range (29, 29), get_printed_columns
> > (hint_2));
> > ++ get_affected_range (&dc, hint_2, CU_BYTES));
> > ++ ASSERT_EQ (column_range (29, 28),
> > ++ get_affected_range (&dc, hint_2, CU_DISPLAY_COLS));
> > ++ ASSERT_EQ (column_range (29, 29), get_printed_columns (&dc,
> > hint_2));
> > +
> > + /* Add each hint in turn to a line_corrections instance,
> > + and verify that they are consolidated into one correction
> > instance
> > + as expected. */
> > +- line_corrections lc (tmp.get_filename (), 1);
> > ++ line_corrections lc (&dc, tmp.get_filename (), 1);
> > +
> > + /* The first replace hint by itself. */
> > + lc.add_hint (hint_0);
> > +@@ -4484,25 +4651,28 @@ test_overlapped_fixit_printing_utf8 (con
> > + /* Unit-test the line_corrections machinery. */
> > + ASSERT_EQ (3, richloc.get_num_fixit_hints ());
> > + const fixit_hint *hint_0 = richloc.get_fixit_hint (0);
> > +- ASSERT_EQ (column_range (14, 14), get_affected_range (hint_0,
> > CU_BYTES));
> > ++ ASSERT_EQ (column_range (14, 14),
> > ++ get_affected_range (&dc, hint_0, CU_BYTES));
> > + ASSERT_EQ (column_range (12, 12),
> > +- get_affected_range (hint_0,
> > CU_DISPLAY_COLS));
> > +- ASSERT_EQ (column_range (12, 22), get_printed_columns
> > (hint_0));
> > ++ get_affected_range (&dc, hint_0, CU_DISPLAY_COLS));
> > ++ ASSERT_EQ (column_range (12, 22), get_printed_columns (&dc,
> > hint_0));
> > + const fixit_hint *hint_1 = richloc.get_fixit_hint (1);
> > +- ASSERT_EQ (column_range (22, 22), get_affected_range (hint_1,
> > CU_BYTES));
> > ++ ASSERT_EQ (column_range (22, 22),
> > ++ get_affected_range (&dc, hint_1, CU_BYTES));
> > + ASSERT_EQ (column_range (18, 18),
> > +- get_affected_range (hint_1,
> > CU_DISPLAY_COLS));
> > +- ASSERT_EQ (column_range (18, 20), get_printed_columns
> > (hint_1));
> > ++ get_affected_range (&dc, hint_1, CU_DISPLAY_COLS));
> > ++ ASSERT_EQ (column_range (18, 20), get_printed_columns (&dc,
> > hint_1));
> > + const fixit_hint *hint_2 = richloc.get_fixit_hint (2);
> > +- ASSERT_EQ (column_range (35, 34), get_affected_range (hint_2,
> > CU_BYTES));
> > ++ ASSERT_EQ (column_range (35, 34),
> > ++ get_affected_range (&dc, hint_2, CU_BYTES));
> > + ASSERT_EQ (column_range (30, 29),
> > +- get_affected_range (hint_2,
> > CU_DISPLAY_COLS));
> > +- ASSERT_EQ (column_range (30, 30), get_printed_columns
> > (hint_2));
> > ++ get_affected_range (&dc, hint_2, CU_DISPLAY_COLS));
> > ++ ASSERT_EQ (column_range (30, 30), get_printed_columns (&dc,
> > hint_2));
> > +
> > + /* Add each hint in turn to a line_corrections instance,
> > + and verify that they are consolidated into one correction
> > instance
> > + as expected. */
> > +- line_corrections lc (tmp.get_filename (), 1);
> > ++ line_corrections lc (&dc, tmp.get_filename (), 1);
> > +
> > + /* The first replace hint by itself. */
> > + lc.add_hint (hint_0);
> > +@@ -4689,6 +4859,8 @@ test_overlapped_fixit_printing_2 (const
> > +
> > + /* Two insertions, in the wrong order. */
> > + {
> > ++ test_diagnostic_context dc;
> > ++
> > + rich_location richloc (line_table, col_20);
> > + richloc.add_fixit_insert_before (col_23, "{");
> > + richloc.add_fixit_insert_before (col_21, "}");
> > +@@ -4696,14 +4868,15 @@ test_overlapped_fixit_printing_2 (const
> > + /* These fixits should be accepted; they can't be
> > consolidated. */
> > + ASSERT_EQ (2, richloc.get_num_fixit_hints ());
> > + const fixit_hint *hint_0 = richloc.get_fixit_hint (0);
> > +- ASSERT_EQ (column_range (23, 22), get_affected_range (hint_0,
> > CU_BYTES));
> > +- ASSERT_EQ (column_range (23, 23), get_printed_columns
> > (hint_0));
> > ++ ASSERT_EQ (column_range (23, 22),
> > ++ get_affected_range (&dc, hint_0, CU_BYTES));
> > ++ ASSERT_EQ (column_range (23, 23), get_printed_columns (&dc,
> > hint_0));
> > + const fixit_hint *hint_1 = richloc.get_fixit_hint (1);
> > +- ASSERT_EQ (column_range (21, 20), get_affected_range (hint_1,
> > CU_BYTES));
> > +- ASSERT_EQ (column_range (21, 21), get_printed_columns
> > (hint_1));
> > ++ ASSERT_EQ (column_range (21, 20),
> > ++ get_affected_range (&dc, hint_1, CU_BYTES));
> > ++ ASSERT_EQ (column_range (21, 21), get_printed_columns (&dc,
> > hint_1));
> > +
> > + /* Verify that they're printed correctly. */
> > +- test_diagnostic_context dc;
> > + diagnostic_show_locus (&dc, &richloc, DK_ERROR);
> > + ASSERT_STREQ (" int a5[][0][0] = { 1, 2 };\n"
> > + " ^\n"
> > +@@ -4955,6 +5128,65 @@ test_fixit_deletion_affecting_newline (c
> > + pp_formatted_text (dc.printer));
> > + }
> > +
> > ++static void
> > ++test_tab_expansion (const line_table_case &case_)
> > ++{
> > ++ /* Create a tempfile and write some text to it. This example
> > uses a tabstop
> > ++ of 8, as the column numbers attempt to indicate:
> > ++
> > ++ .....................000.01111111111.22222333333 display
> > ++ .....................123.90123456789.56789012345 columns */
> > ++ const char *content = " \t This: `\t' is a tab.\n";
> > ++ /* ....................000 00000011111 11111222222 byte
> > ++ ....................123 45678901234 56789012345 columns */
> > ++
> > ++ const int tabstop = 8;
> > ++ const int first_non_ws_byte_col = 7;
> > ++ const int right_quote_byte_col = 15;
> > ++ const int last_byte_col = 25;
> > ++ ASSERT_EQ (35, cpp_display_width (content, last_byte_col,
> > tabstop));
> > ++
> > ++ temp_source_file tmp (SELFTEST_LOCATION, ".c", content);
> > ++ line_table_test ltt (case_);
> > ++ linemap_add (line_table, LC_ENTER, false, tmp.get_filename (),
> > 1);
> > ++
> > ++ /* Don't attempt to run the tests if column data might be
> > unavailable. */
> > ++ location_t line_end = linemap_position_for_column (line_table,
> > last_byte_col);
> > ++ if (line_end > LINE_MAP_MAX_LOCATION_WITH_COLS)
> > ++ return;
> > ++
> > ++ /* Check that the leading whitespace with mixed tabs and spaces
> > is expanded
> > ++ into 11 spaces. Recall that print_line() also puts one space
> > before
> > ++ everything too. */
> > ++ {
> > ++ test_diagnostic_context dc;
> > ++ dc.tabstop = tabstop;
> > ++ rich_location richloc (line_table,
> > ++ linemap_position_for_column (line_table,
> > ++ first_non_ws_
> > byte_col));
> > ++ layout test_layout (&dc, &richloc, DK_ERROR);
> > ++ test_layout.print_line (1);
> > ++ ASSERT_STREQ (" This: ` ' is a tab.\n"
> > ++ " ^\n",
> > ++ pp_formatted_text (dc.printer));
> > ++ }
> > ++
> > ++ /* Confirm the display width was tracked correctly across the
> > internal tab
> > ++ as well. */
> > ++ {
> > ++ test_diagnostic_context dc;
> > ++ dc.tabstop = tabstop;
> > ++ rich_location richloc (line_table,
> > ++ linemap_position_for_column (line_table,
> > ++ right_quote_b
> > yte_col));
> > ++ layout test_layout (&dc, &richloc, DK_ERROR);
> > ++ test_layout.print_line (1);
> > ++ ASSERT_STREQ (" This: ` ' is a tab.\n"
> > ++ " ^\n",
> > ++ pp_formatted_text (dc.printer));
> > ++ }
> > ++}
> > ++
> > + /* Verify that line numbers are correctly printed for the case of
> > + a multiline range in which the width of the line numbers changes
> > + (e.g. from "9" to "10"). */
> > +@@ -5012,6 +5244,7 @@ diagnostic_show_locus_c_tests ()
> > + test_layout_range_for_multiple_lines ();
> > +
> > + for_each_line_table_case (test_layout_x_offset_display_utf8);
> > ++ for_each_line_table_case (test_layout_x_offset_display_tab);
> > +
> > + test_get_line_bytes_without_trailing_whitespace ();
> > +
> > +@@ -5029,6 +5262,7 @@ diagnostic_show_locus_c_tests ()
> > + for_each_line_table_case
> > (test_fixit_insert_containing_newline_2);
> > + for_each_line_table_case (test_fixit_replace_containing_newline);
> > + for_each_line_table_case (test_fixit_deletion_affecting_newline);
> > ++ for_each_line_table_case (test_tab_expansion);
> > +
> > + test_line_numbers_multiline_range ();
> > + }
> > +diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
> > +--- a/gcc/doc/invoke.texi 2021-12-24 20:23:46.876739587 -0800
> > ++++ b/gcc/doc/invoke.texi 2021-12-25 01:20:53.487636494 -0800
> > +@@ -293,7 +293,9 @@ Objective-C and Objective-C++ Dialects}.
> > + -fdiagnostics-show-template-tree -fno-elide-type @gol
> > +
> > -fdiagnostics-path-format=@r{[}none@r{|}separate-events@r
> {|}inline-events@r{
> > ]} @gol
> > + -fdiagnostics-show-path-depths @gol
> > +--fno-show-column}
> > ++-fno-show-column @gol
> > ++-fdiagnostics-column-unit=@r{[}display@r{|}byte@r{]} @gol
> > ++-fdiagnostics-column-origin=@var{origin}}
> > +
> > + @item Warning Options
> > + @xref{Warning Options,,Options to Request or Suppress Warnings}.
> > +@@ -4424,6 +4426,31 @@ Do not print column numbers in diagnosti
> > + diagnostics are being scanned by a program that does not understand
> > the
> > + column numbers, such as @command{dejagnu}.
> > +
> > ++@item -fdiagnostics-column-unit=@var{UNIT}
> > ++@opindex fdiagnostics-column-unit
> > ++Select the units for the column number. This affects traditional
> > diagnostics
> > ++(in the absence of @option{-fno-show-column}), as well as JSON
> > format
> > ++diagnostics if requested.
> > ++
> > ++The default @var{UNIT}, @samp{display}, considers the number of
> > display
> > ++columns occupied by each character. This may be larger than the
> > number
> > ++of bytes required to encode the character, in the case of tab
> > ++characters, or it may be smaller, in the case of multibyte
> > characters.
> > ++For example, the character ``GREEK SMALL LETTER PI (U+03C0)''
> > occupies one
> > ++display column, and its UTF-8 encoding requires two bytes; the
> > character
> > ++``SLIGHTLY SMILING FACE (U+1F642)'' occupies two display columns,
> > and
> > ++its UTF-8 encoding requires four bytes.
> > ++
> > ++Setting @var{UNIT} to @samp{byte} changes the column number to the
> > raw byte
> > ++count in all cases, as was traditionally output by GCC prior to
> > version 11.1.0.
> > ++
> > ++@item -fdiagnostics-column-origin=@var{ORIGIN}
> > ++@opindex fdiagnostics-column-origin
> > ++Select the origin for column numbers, i.e. the column number
> > assigned to the
> > ++first column. The default value of 1 corresponds to traditional
> > GCC
> > ++behavior and to the GNU style guide. Some utilities may perform
> > better with an
> > ++origin of 0; any non-negative value may be specified.
> > ++
> > + @item -fdiagnostics-format=@var{FORMAT}
> > + @opindex fdiagnostics-format
> > + Select a different format for printing diagnostics.
> > +@@ -4459,11 +4486,15 @@ might be printed in JSON form (after for
> > + "locations": [
> > + @{
> > + "caret": @{
> > ++ "display-column": 3,
> > ++ "byte-column": 3,
> > + "column": 3,
> > + "file": "misleading-indentation.c",
> > + "line": 15
> > + @},
> > + "finish": @{
> > ++ "display-column": 4,
> > ++ "byte-column": 4,
> > + "column": 4,
> > + "file": "misleading-indentation.c",
> > + "line": 15
> > +@@ -4479,6 +4510,8 @@ might be printed in JSON form (after for
> > + "locations": [
> > + @{
> > + "caret": @{
> > ++ "display-column": 5,
> > ++ "byte-column": 5,
> > + "column": 5,
> > + "file": "misleading-indentation.c",
> > + "line": 17
> > +@@ -4488,6 +4521,7 @@ might be printed in JSON form (after for
> > + "message": "...this statement, but the latter is
> > @dots{}"
> > + @}
> > + ]
> > ++ "column-origin": 1,
> > + @},
> > + @dots{}
> > + ]
> > +@@ -4500,10 +4534,34 @@ A diagnostic has a @code{kind}. If this
> > + an @code{option} key describing the command-line option controlling
> > the
> > + warning.
> > +
> > +-A diagnostic can contain zero or more locations. Each location has
> > up
> > +-to three positions within it: a @code{caret} position and optional
> > +-@code{start} and @code{finish} positions. A location can also have
> > +-an optional @code{label} string. For example, this error:
> > ++A diagnostic can contain zero or more locations. Each location has
> >
[Attachment #5 (text/html)]
<div><br></div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On \
Thu, Dec 30, 2021 at 4:29 PM Anuj Mittal <<a \
href="mailto:anuj.mittal@intel.com">anuj.mittal@intel.com</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 \
.8ex;border-left:1px #ccc solid;padding-left:1ex">I can't apply this. <br> <br>
git am says:<br>
<br>
error: cannot convert from Y to UTF-8<br>
fatal: could not parse patch<br>
</blockquote><div dir="auto"><br></div><div dir="auto">Key is When git send-email \
asks for UTF-8 encoding for the patch then do not enter "y" but just press</div><div \
dir="auto">Enter </div><div dir="auto"><br></div><blockquote class="gmail_quote" \
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br> \
Thanks,<br> <br>
Anuj<br>
<br>
On Wed, 2021-12-29 at 01:13 -0800, pgowda wrote:<br>
> Upstream-Status: Backport<br>
> [<a href="https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=004bb936d6d5f177af26ad" \
rel="noreferrer" target="_blank">https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=004bb936d6d5f177af26ad</a><br>
> 4905595e843d5665a5]<br>
> Upstream-Status: Backport<br>
> [<a href="https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=bd5e882cf6e0def3dd1bc1" \
rel="noreferrer" target="_blank">https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=bd5e882cf6e0def3dd1bc1</a><br>
> 06075d59a303fe0d1e]<br>
> Upstream-Status: Backport<br>
> [<a href="https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=51c500269bf53749b10780" \
rel="noreferrer" target="_blank">https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=51c500269bf53749b10780</a><br>
> 7d84271385fad35628]<br>
> Upstream-Status: Backport<br>
> [<a href="https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=1a7f2c0774129750fdf73e" \
rel="noreferrer" target="_blank">https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=1a7f2c0774129750fdf73e</a><br>
> 9f1b78f0ce983c9ab3]<br>
> Upstream-Status: Backport<br>
> [<a href="https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=bef32d4a28595e933f24fe" \
rel="noreferrer" target="_blank">https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=bef32d4a28595e933f24fe</a><br>
> f378cf052a30b674a7]<br>
> <br>
> Signed-off-by: pgowda <<a href="mailto:pgowda.cve@gmail.com" \
target="_blank">pgowda.cve@gmail.com</a>><br> > ---<br>
> meta/recipes-devtools/gcc/gcc-10.2.inc | 5 +<br>
> .../gcc/gcc/0001-CVE-2021-42574.patch | 2906<br>
> +++++++++++++++++<br>
> .../gcc/gcc/0002-CVE-2021-42574.patch | 2270 +++++++++++++<br>
> .../gcc/gcc/0003-CVE-2021-42574.patch | 1724 ++++++++++<br>
> .../gcc/gcc/0004-CVE-2021-42574.patch | 138 +<br>
> .../gcc/gcc/0005-CVE-2021-42574.patch | 575 ++++<br>
> 6 files changed, 7618 insertions(+)<br>
> create mode 100644 meta/recipes-devtools/gcc/gcc/0001-CVE-2021-<br>
> 42574.patch<br>
> create mode 100644 meta/recipes-devtools/gcc/gcc/0002-CVE-2021-<br>
> 42574.patch<br>
> create mode 100644 meta/recipes-devtools/gcc/gcc/0003-CVE-2021-<br>
> 42574.patch<br>
> create mode 100644 meta/recipes-devtools/gcc/gcc/0004-CVE-2021-<br>
> 42574.patch<br>
> create mode 100644 meta/recipes-devtools/gcc/gcc/0005-CVE-2021-<br>
> 42574.patch<br>
> <br>
> diff --git a/meta/recipes-devtools/gcc/gcc-10.2.inc b/meta/recipes-<br>
> devtools/gcc/gcc-10.2.inc<br>
> index 5626bf20f0..fd7f16d7bf 100644<br>
> --- a/meta/recipes-devtools/gcc/gcc-10.2.inc<br>
> +++ b/meta/recipes-devtools/gcc/gcc-10.2.inc<br>
> @@ -74,6 +74,11 @@ SRC_URI = "\<br>
> file://0002-CVE-2021-35465.patch \<br>
> file://0003-CVE-2021-35465.patch \<br>
> file://0004-CVE-2021-35465.patch \<br>
> + file://0001-CVE-2021-42574.patch \<br>
> + file://0002-CVE-2021-42574.patch \<br>
> + file://0003-CVE-2021-42574.patch \<br>
> + file://0004-CVE-2021-42574.patch \<br>
> + file://0005-CVE-2021-42574.patch \<br>
> "<br>
> SRC_URI[sha256sum] =<br>
> "b8dd4368bb9c7f0b98188317ee0254dd8cc99d1e3a18d0ff146c855fe16c1d8c"<br>
> <br>
> diff --git a/meta/recipes-devtools/gcc/gcc/0001-CVE-2021-42574.patch<br>
> b/meta/recipes-devtools/gcc/gcc/0001-CVE-2021-42574.patch<br>
> new file mode 100644<br>
> index 0000000000..e0f4f7d32f<br>
> --- /dev/null<br>
> +++ b/meta/recipes-devtools/gcc/gcc/0001-CVE-2021-42574.patch<br>
> @@ -0,0 +1,2906 @@<br>
> +From 004bb936d6d5f177af26ad4905595e843d5665a5 Mon Sep 17 00:00:00<br>
> 2001<br>
> +From: Lewis Hyatt <<a href="mailto:lhyatt@gmail.com" \
target="_blank">lhyatt@gmail.com</a>><br> > +Date: Tue, 14 Jul 2020 12:05:56 \
-0400<br> > +Subject: [PATCH] diagnostics: Support conversion of tabs to \
spaces<br> > [PR49973]<br>
> + [PR86904]<br>
> +<br>
> +Supports conversion of tabs to spaces when outputting diagnostics.<br>
> Also<br>
> +adds -fdiagnostics-column-unit and -fdiagnostics-column-origin<br>
> options to<br>
> +control how the column number is output, thereby resolving the two<br>
> PRs.<br>
> +<br>
> +gcc/c-family/ChangeLog:<br>
> +<br>
> + PR other/86904<br>
> + * c-indentation.c (should_warn_for_misleading_indentation):<br>
> Get<br>
> + global tabstop from the new source.<br>
> + * c-opts.c (c_common_handle_option): Remove handling of -<br>
> ftabstop, which<br>
> + is now a common option.<br>
> + * c.opt: Likewise.<br>
> +<br>
> +gcc/ChangeLog:<br>
> +<br>
> + PR preprocessor/49973<br>
> + PR other/86904<br>
> + * common.opt: Handle -ftabstop here instead of in c-family<br>
> + options. Add -fdiagnostics-column-unit= and<br>
> + -fdiagnostics-column-origin= options.<br>
> + * opts.c (common_handle_option): Handle the new options.<br>
> + * diagnostic-format-json.cc (json_from_expanded_location):<br>
> Add<br>
> + diagnostic_context argument. Use it to convert column<br>
> numbers as per<br>
> + the new options.<br>
> + (json_from_location_range): Likewise.<br>
> + (json_from_fixit_hint): Likewise.<br>
> + (json_end_diagnostic): Pass the new context argument to<br>
> helper<br>
> + functions above. Add "column-origin" field to the \
output.<br> > + (test_unknown_location): Add the new context argument \
to<br> > calls to<br>
> + helper functions.<br>
> + (test_bad_endpoints): Likewise.<br>
> + * diagnostic-show-locus.c<br>
> + (exploc_with_display_col::exploc_with_display_col): Support<br>
> + tabstop parameter.<br>
> + (layout_point::layout_point): Make use of class<br>
> + exploc_with_display_col.<br>
> + (layout_range::layout_range): Likewise.<br>
> + (struct line_bounds): Clarify that the units are now always<br>
> + display columns. Rename members accordingly. Add<br>
> constructor.<br>
> + (layout::print_source_line): Add support for tab expansion.<br>
> + (make_range): Adapt to class layout_range changes.<br>
> + (layout::maybe_add_location_range): Likewise.<br>
> + (layout::layout): Adapt to class exploc_with_display_col<br>
> changes.<br>
> + (layout::calculate_x_offset_display): Support tabstop<br>
> parameter.<br>
> + (layout::print_annotation_line): Adapt to struct line_bounds<br>
> changes.<br>
> + (layout::print_line): Likewise.<br>
> + (line_label::line_label): Add diagnostic_context argument.<br>
> + (get_affected_range): Likewise.<br>
> + (get_printed_columns): Likewise.<br>
> + (layout::print_any_labels): Adapt to struct line_label<br>
> changes.<br>
> + (class correction): Add m_tabstop member.<br>
> + (correction::correction): Add tabstop argument.<br>
> + (correction::compute_display_cols): Use m_tabstop.<br>
> + (class line_corrections): Add m_context member.<br>
> + (line_corrections::line_corrections): Add diagnostic_context<br>
> argument.<br>
> + (line_corrections::add_hint): Use m_context to handle<br>
> tabstops.<br>
> + (layout::print_trailing_fixits): Adapt to class<br>
> line_corrections<br>
> + changes.<br>
> + (test_layout_x_offset_display_utf8): Support tabstop<br>
> parameter.<br>
> + (test_layout_x_offset_display_tab): New selftest.<br>
> + (test_one_liner_colorized_utf8): Likewise.<br>
> + (test_tab_expansion): Likewise.<br>
> + (test_diagnostic_show_locus_one_liner_utf8): Call the new<br>
> tests.<br>
> + (diagnostic_show_locus_c_tests): Likewise.<br>
> + (test_overlapped_fixit_printing): Adapt to helper class and<br>
> + function changes.<br>
> + (test_overlapped_fixit_printing_utf8): Likewise.<br>
> + (test_overlapped_fixit_printing_2): Likewise.<br>
> + * diagnostic.h (enum diagnostics_column_unit): New enum.<br>
> + (struct diagnostic_context): Add members for the new options.<br>
> + (diagnostic_converted_column): Declare.<br>
> + (json_from_expanded_location): Add new context argument.<br>
> + * diagnostic.c (diagnostic_initialize): Initialize new<br>
> members.<br>
> + (diagnostic_converted_column): New function.<br>
> + (maybe_line_and_column): Be willing to output a column of 0.<br>
> + (diagnostic_get_location_text): Convert column number as per<br>
> the new<br>
> + options.<br>
> + (diagnostic_report_current_module): Likewise.<br>
> + (assert_location_text): Add origin and column_unit arguments<br>
> for<br>
> + testing the new functionality.<br>
> + (test_diagnostic_get_location_text): Test the new<br>
> functionality.<br>
> + * doc/invoke.texi: Document the new options and behavior.<br>
> + * input.h (location_compute_display_column): Add tabstop<br>
> argument.<br>
> + * input.c (location_compute_display_column): Likewise.<br>
> + (test_cpp_utf8): Add selftests for tab expansion.<br>
> + * tree-diagnostic-path.cc (default_tree_make_json_for_path):<br>
> Pass the<br>
> + new context argument to json_from_expanded_location().<br>
> +<br>
> +libcpp/ChangeLog:<br>
> +<br>
> + PR preprocessor/49973<br>
> + PR other/86904<br>
> + * include/cpplib.h (struct cpp_options): Removed support \
for<br> > -ftabstop,<br>
> + which is now handled by diagnostic_context.<br>
> + (class cpp_display_width_computation): New class.<br>
> + (cpp_byte_column_to_display_column): Add optional tabstop<br>
> argument.<br>
> + (cpp_display_width): Likewise.<br>
> + (cpp_display_column_to_byte_column): Likewise.<br>
> + * charset.c<br>
> + (cpp_display_width_computation::cpp_display_width_computation<br>
> ): New<br>
> + function.<br>
> + (cpp_display_width_computation::advance_display_cols):<br>
> Likewise.<br>
> + (compute_next_display_width): Removed and implemented this<br>
> + functionality in a new function...<br>
> + (cpp_display_width_computation::process_next_codepoint):<br>
> ...here.<br>
> + (cpp_byte_column_to_display_column): Added tabstop argument.<br>
> + Reimplemented in terms of class<br>
> cpp_display_width_computation.<br>
> + (cpp_display_column_to_byte_column): Likewise.<br>
> + * init.c (cpp_create_reader): Remove handling of -ftabstop,<br>
> which is now<br>
> + handled by diagnostic_context.<br>
> +<br>
> +gcc/testsuite/ChangeLog:<br>
> +<br>
> + PR preprocessor/49973<br>
> + PR other/86904<br>
> + * c-c++-common/Wmisleading-indentation-3.c: Adjust expected<br>
> output<br>
> + for new defaults.<br>
> + * c-c++-common/Wmisleading-indentation.c: Likewise.<br>
> + * c-c++-common/diagnostic-format-json-1.c: Likewise.<br>
> + * c-c++-common/diagnostic-format-json-2.c: Likewise.<br>
> + * c-c++-common/diagnostic-format-json-3.c: Likewise.<br>
> + * c-c++-common/diagnostic-format-json-4.c: Likewise.<br>
> + * c-c++-common/diagnostic-format-json-5.c: Likewise.<br>
> + * c-c++-common/missing-close-symbol.c: Likewise.<br>
> + * g++.dg/diagnostic/bad-binary-ops.C: Likewise.<br>
> + * g++.dg/parse/error4.C: Likewise.<br>
> + * g++.old-deja/g++.brendan/crash11.C: Likewise.<br>
> + * g++.old-deja/g++.pt/overload2.C: Likewise.<br>
> + * g++.old-deja/g++.robertl/eb109.C: Likewise.<br>
> + * gcc.dg/analyzer/malloc-paths-9.c: Likewise.<br>
> + * gcc.dg/bad-binary-ops.c: Likewise.<br>
> + * gcc.dg/format/branch-1.c: Likewise.<br>
> + * gcc.dg/format/pr79210.c: Likewise.<br>
> + * gcc.dg/plugin/diagnostic-test-expressions-1.c: Likewise.<br>
> + * gcc.dg/plugin/diagnostic-test-string-literals-1.c:<br>
> Likewise.<br>
> + * gcc.dg/redecl-4.c: Likewise.<br>
> + * gfortran.dg/diagnostic-format-json-1.F90: Likewise.<br>
> + * gfortran.dg/diagnostic-format-json-2.F90: Likewise.<br>
> + * gfortran.dg/diagnostic-format-json-3.F90: Likewise.<br>
> + * go.dg/arrayclear.go: Add a comment explaining why adding a<br>
> + comment was necessary to work around a dejagnu bug.<br>
> + * c-c++-common/diagnostic-units-1.c: New test.<br>
> + * c-c++-common/diagnostic-units-2.c: New test.<br>
> + * c-c++-common/diagnostic-units-3.c: New test.<br>
> + * c-c++-common/diagnostic-units-4.c: New test.<br>
> + * c-c++-common/diagnostic-units-5.c: New test.<br>
> + * c-c++-common/diagnostic-units-6.c: New test.<br>
> + * c-c++-common/diagnostic-units-7.c: New test.<br>
> + * c-c++-common/diagnostic-units-8.c: New test.<br>
> +<br>
> +CVE: CVE-2021-42574<br>
> +Upstream-Status: Backport<br>
> [<a href="https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=004bb936d6d5f177af26ad" \
rel="noreferrer" target="_blank">https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=004bb936d6d5f177af26ad</a><br>
> 4905595e843d5665a5]<br>
> +Signed-off-by: Pgowda <<a href="mailto:pgowda.cve@gmail.com" \
target="_blank">pgowda.cve@gmail.com</a>><br> > +---<br>
> + gcc/c-family/c-indentation.c | 5 +-<br>
> + gcc/c-family/c-opts.c | 6 \
-<br> > + gcc/c-family/c.opt \
| 4 -<br> > + gcc/common.opt \
| 21 +<br> > + gcc/diagnostic-format-json.cc | \
55 +-<br> > + gcc/diagnostic-show-locus.c | \
504 +++++++++++++--<br> > ---<br>
> + gcc/diagnostic.c | \
113 +++-<br> > + gcc/diagnostic.h \
| 28 +-<br> > + gcc/doc/invoke.texi \
| 68 ++-<br> > + gcc/input.c \
| 72 ++-<br> > + gcc/input.h \
| 4 +-<br> > + gcc/opts.c \
| 14 +<br> > + .../c-c++-common/Wmisleading-indentation-3.c | 12 +-<br>
> + .../c-c++-common/Wmisleading-indentation.c | 6 +-<br>
> + .../c-c++-common/diagnostic-format-json-1.c | 5 +<br>
> + .../c-c++-common/diagnostic-format-json-2.c | 5 +<br>
> + .../c-c++-common/diagnostic-format-json-3.c | 5 +<br>
> + .../c-c++-common/diagnostic-format-json-4.c | 9 +<br>
> + .../c-c++-common/diagnostic-format-json-5.c | 9 +<br>
> + .../c-c++-common/diagnostic-units-1.c | 28 +<br>
> + .../c-c++-common/diagnostic-units-2.c | 28 +<br>
> + .../c-c++-common/diagnostic-units-3.c | 28 +<br>
> + .../c-c++-common/diagnostic-units-4.c | 28 +<br>
> + .../c-c++-common/diagnostic-units-5.c | 28 +<br>
> + .../c-c++-common/diagnostic-units-6.c | 28 +<br>
> + .../c-c++-common/diagnostic-units-7.c | 28 +<br>
> + .../c-c++-common/diagnostic-units-8.c | 28 +<br>
> + .../c-c++-common/missing-close-symbol.c | 6 +-<br>
> + .../g++.dg/diagnostic/bad-binary-ops.C | 8 +-<br>
> + gcc/testsuite/g++.dg/parse/error4.C | 2 +-<br>
> + .../g++.old-deja/g++.brendan/crash11.C | 4 +-<br>
> + gcc/testsuite/g++.old-deja/g++.pt/overload2.C | 2 +-<br>
> + .../g++.old-deja/g++.robertl/eb109.C | 4 +-<br>
> + .../gcc.dg/analyzer/malloc-paths-9.c | 2 +-<br>
> + gcc/testsuite/gcc.dg/bad-binary-ops.c | 8 +-<br>
> + gcc/testsuite/gcc.dg/format/branch-1.c | 2 +-<br>
> + gcc/testsuite/gcc.dg/format/pr79210.c | 2 +-<br>
> + .../plugin/diagnostic-test-expressions-1.c | 16 +-<br>
> + .../diagnostic-test-string-literals-1.c | 4 +-<br>
> + gcc/testsuite/gcc.dg/redecl-4.c | 2 +-<br>
> + .../gfortran.dg/diagnostic-format-json-1.F90 | 5 +<br>
> + .../gfortran.dg/diagnostic-format-json-2.F90 | 5 +<br>
> + .../gfortran.dg/diagnostic-format-json-3.F90 | 5 +<br>
> + gcc/testsuite/go.dg/arrayclear.go | 3 +<br>
> + gcc/tree-diagnostic-path.cc | 5 +-<br>
> + libcpp/charset.c | \
98 ++--<br> > + libcpp/include/cpplib.h \
| 40 +-<br> > + libcpp/init.c \
| 1 -<br> > + 48 files changed, 1106 insertions(+), 287 deletions(-)<br>
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-1.c<br>
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-2.c<br>
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-3.c<br>
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-4.c<br>
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-5.c<br>
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-6.c<br>
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-7.c<br>
> + create mode 100644 gcc/testsuite/c-c++-common/diagnostic-units-8.c<br>
> +<br>
> +diff --git a/gcc/c-family/c-indentation.c b/gcc/c-family/c-<br>
> indentation.c<br>
> +--- a/gcc/c-family/c-indentation.c 2020-07-22 23:35:17.296384022<br>
> -0700<br>
> ++++ b/gcc/c-family/c-indentation.c 2021-12-25 01:20:53.475636694<br>
> -0800<br>
> +@@ -24,8 +24,7 @@ along with GCC; see the file COPYING3.<br>
> + #include "c-common.h"<br>
> + #include "c-indentation.h"<br>
> + #include "selftest.h"<br>
> +-<br>
> +-extern cpp_options *cpp_opts;<br>
> ++#include "diagnostic.h"<br>
> + <br>
> + /* Round up VIS_COLUMN to nearest tab stop. */<br>
> + <br>
> +@@ -294,7 +293,7 @@ should_warn_for_misleading_indentation (<br>
> + expanded_location next_stmt_exploc = expand_location<br>
> (next_stmt_loc);<br>
> + expanded_location guard_exploc = expand_location (guard_loc);<br>
> + <br>
> +- const unsigned int tab_width = cpp_opts->tabstop;<br>
> ++ const unsigned int tab_width = global_dc->tabstop;<br>
> + <br>
> + /* They must be in the same file. */<br>
> + if (next_stmt_exploc.file != body_exploc.file)<br>
> +diff --git a/gcc/c-family/c.opt b/gcc/c-family/c.opt<br>
> +--- a/gcc/c-family/c.opt 2021-12-24 20:23:42.816809230 -0800<br>
> ++++ b/gcc/c-family/c.opt 2021-12-25 01:20:53.475636694 -0800<br>
> +@@ -1876,10 +1876,6 @@ Enum(strong_eval_order) String(some) Val<br>
> + EnumValue<br>
> + Enum(strong_eval_order) String(all) Value(2)<br>
> + <br>
> +-ftabstop=<br>
> +-C ObjC C++ ObjC++ Joined RejectNegative UInteger<br>
> +--ftabstop=<number> Distance between tab stops for column<br>
> reporting.<br>
> +-<br>
> + ftemplate-backtrace-limit=<br>
> + C++ ObjC++ Joined RejectNegative UInteger<br>
> Var(template_backtrace_limit) Init(10)<br>
> + Set the maximum number of template instantiation notes for a single<br>
> warning or error.<br>
> +diff --git a/gcc/c-family/c-opts.c b/gcc/c-family/c-opts.c<br>
> +--- a/gcc/c-family/c-opts.c 2021-12-24 20:23:44.824774786 -0800<br>
> ++++ b/gcc/c-family/c-opts.c 2021-12-25 01:20:53.475636694 -0800<br>
> +@@ -504,12 +504,6 @@ c_common_handle_option (size_t scode, co<br>
> + cpp_opts->track_macro_expansion = 2;<br>
> + break;<br>
> + <br>
> +- case OPT_ftabstop_:<br>
> +- /* It is documented that we silently ignore silly values. */<br>
> +- if (value >= 1 && value <= 100)<br>
> +- cpp_opts->tabstop = value;<br>
> +- break;<br>
> +-<br>
> + case OPT_fexec_charset_:<br>
> + cpp_opts->narrow_charset = arg;<br>
> + break;<br>
> +diff --git a/gcc/common.opt b/gcc/common.opt<br>
> +--- a/gcc/common.opt 2021-12-24 20:23:42.480814993 -0800<br>
> ++++ b/gcc/common.opt 2021-12-25 01:20:53.475636694 -0800<br>
> +@@ -1325,6 +1325,14 @@ Enum(diagnostic_url_rule) String(always)<br>
> + EnumValue<br>
> + Enum(diagnostic_url_rule) String(auto) Value(DIAGNOSTICS_URL_AUTO)<br>
> + <br>
> ++fdiagnostics-column-unit=<br>
> ++Common Joined RejectNegative Enum(diagnostics_column_unit)<br>
> ++-fdiagnostics-column-unit=[display|byte] Select whether column<br>
> numbers are output as display columns (default) or raw bytes.<br>
> ++<br>
> ++fdiagnostics-column-origin=<br>
> ++Common Joined RejectNegative UInteger<br>
> ++-fdiagnostics-column-origin=<number> Set the number of the first<br>
> column. The default is 1-based as per GNU style, but some utilities<br>
> may expect 0-based, for example.<br>
> ++<br>
> + fdiagnostics-format=<br>
> + Common Joined RejectNegative Enum(diagnostics_output_format)<br>
> + -fdiagnostics-format=[text|json] Select output format.<br>
> +@@ -1334,6 +1342,15 @@ SourceInclude<br>
> + diagnostic.h<br>
> + <br>
> + Enum<br>
> ++Name(diagnostics_column_unit) Type(int)<br>
> ++<br>
> ++EnumValue<br>
> ++Enum(diagnostics_column_unit) String(display)<br>
> Value(DIAGNOSTICS_COLUMN_UNIT_DISPLAY)<br>
> ++<br>
> ++EnumValue<br>
> ++Enum(diagnostics_column_unit) String(byte)<br>
> Value(DIAGNOSTICS_COLUMN_UNIT_BYTE)<br>
> ++<br>
> ++Enum<br>
> + Name(diagnostics_output_format) Type(int)<br>
> + <br>
> + EnumValue<br>
> +@@ -1362,6 +1379,10 @@ fdiagnostics-path-format=<br>
> + Common Joined RejectNegative Var(flag_diagnostics_path_format)<br>
> Enum(diagnostic_path_format) Init(DPF_INLINE_EVENTS)<br>
> + Specify how to print any control-flow path associated with a<br>
> diagnostic.<br>
> + <br>
> ++ftabstop=<br>
> ++Common Joined RejectNegative UInteger<br>
> ++-ftabstop=<number> Distance between tab stops for column<br>
> reporting.<br>
> ++<br>
> + Enum<br>
> + Name(diagnostic_path_format) Type(int)<br>
> + <br>
> +diff --git a/gcc/diagnostic.c b/gcc/diagnostic.c<br>
> +--- a/gcc/diagnostic.c 2020-07-22 23:35:17.556386887 -0700<br>
> ++++ b/gcc/diagnostic.c 2021-12-25 01:23:41.300841207 -0800<br>
> +@@ -38,6 +38,7 @@ along with GCC; see the file COPYING3.<br>
> + #include "selftest.h"<br>
> + #include "selftest-diagnostic.h"<br>
> + #include "opts.h"<br>
> ++#include "cpplib.h"<br>
> + <br>
> + #ifdef HAVE_TERMIOS_H<br>
> + # include <termios.h><br>
> +@@ -219,6 +220,9 @@ diagnostic_initialize (diagnostic_contex<br>
> + context->min_margin_width = 0;<br>
> + context->show_ruler_p = false;<br>
> + context->parseable_fixits_p = false;<br>
> ++ context->column_unit = DIAGNOSTICS_COLUMN_UNIT_DISPLAY;<br>
> ++ context->column_origin = 1;<br>
> ++ context->tabstop = 8;<br>
> + context->edit_context_ptr = NULL;<br>
> + context->diagnostic_group_nesting_depth = 0;<br>
> + context->diagnostic_group_emission_count = 0;<br>
> +@@ -353,8 +357,51 @@ diagnostic_get_color_for_kind (diagnosti<br>
> + return diagnostic_kind_color[kind];<br>
> + }<br>
> + <br>
> ++/* Given an expanded_location, convert the column (which is in 1-<br>
> based bytes)<br>
> ++ to the requested units, without converting the origin.<br>
> ++ Return -1 if the column is invalid (<= 0). */<br>
> ++<br>
> ++static int<br>
> ++convert_column_unit (enum diagnostics_column_unit column_unit,<br>
> ++ int tabstop,<br>
> ++ expanded_location s)<br>
> ++{<br>
> ++ if (s.column <= 0)<br>
> ++ return -1;<br>
> ++<br>
> ++ switch (column_unit)<br>
> ++ {<br>
> ++ default:<br>
> ++ gcc_unreachable ();<br>
> ++<br>
> ++ case DIAGNOSTICS_COLUMN_UNIT_DISPLAY:<br>
> ++ {<br>
> ++ cpp_char_column_policy policy (tabstop, cpp_wcwidth);<br>
> ++ return location_compute_display_column (s, policy);<br>
> ++ }<br>
> ++<br>
> ++ case DIAGNOSTICS_COLUMN_UNIT_BYTE:<br>
> ++ return s.column;<br>
> ++ }<br>
> ++}<br>
> ++<br>
> ++/* Given an expanded_location, convert the column (which is in 1-<br>
> based bytes)<br>
> ++ to the requested units and origin. Return -1 if the column is<br>
> ++ invalid (<= 0). */<br>
> ++int<br>
> ++diagnostic_converted_column (diagnostic_context *context,<br>
> expanded_location s)<br>
> ++{<br>
> ++ int one_based_col<br>
> ++ = convert_column_unit (context->column_unit, \
context->tabstop,<br> > s);<br>
> ++ if (one_based_col <= 0)<br>
> ++ return -1;<br>
> ++ return one_based_col + (context->column_origin - 1);<br>
> ++}<br>
> ++<br>
> + /* Return a formatted line and column ':%line:%column'. Elided \
if<br> > +- zero. The result is a statically allocated buffer. */<br>
> ++ line == 0 or col < 0. (A column of 0 may be valid due to the<br>
> ++ -fdiagnostics-column-origin option.)<br>
> ++ The result is a statically allocated buffer. */<br>
> + <br>
> + static const char *<br>
> + maybe_line_and_column (int line, int col)<br>
> +@@ -363,8 +410,9 @@ maybe_line_and_column (int line, int col<br>
> + <br>
> + if (line)<br>
> + {<br>
> +- size_t l = snprintf (result, sizeof (result),<br>
> +- col ? ":%d:%d" : \
":%d", line, col);<br> > ++ size_t l<br>
> ++ = snprintf (result, sizeof (result),<br>
> ++ col >= 0 ? ":%d:%d" : \
":%d", line, col);<br> > + gcc_checking_assert (l < \
sizeof (result));<br> > + }<br>
> + else<br>
> +@@ -383,8 +431,14 @@ diagnostic_get_location_text (diagnostic<br>
> + const char *locus_cs = colorize_start (pp_show_color (pp),<br>
> "locus");<br>
> + const char *locus_ce = colorize_stop (pp_show_color (pp));<br>
> + const char *file = s.file ? s.file : progname;<br>
> +- int line = strcmp (file, N_("<built-in>")) ? s.line : \
0;<br> > +- int col = context->show_column ? s.column : 0;<br>
> ++ int line = 0;<br>
> ++ int col = -1;<br>
> ++ if (strcmp (file, N_("<built-in>")))<br>
> ++ {<br>
> ++ line = s.line;<br>
> ++ if (context->show_column)<br>
> ++ col = diagnostic_converted_column (context, s);<br>
> ++ }<br>
> + <br>
> + const char *line_col = maybe_line_and_column (line, col);<br>
> + return build_message_string ("%s%s%s:%s", locus_cs, file,<br>
> +@@ -650,14 +704,20 @@ diagnostic_report_current_module (diagno<br>
> + if (! MAIN_FILE_P (map))<br>
> + {<br>
> + bool first = true;<br>
> ++ expanded_location s = {};<br>
> + do<br>
> + {<br>
> + where = linemap_included_from (map);<br>
> + map = linemap_included_from_linemap (line_table, \
map);<br> > +- const char *line_col<br>
> +- = maybe_line_and_column (SOURCE_LINE (map, \
where),<br> > +- \
first && context-<br> > >show_column<br>
> +- ? \
SOURCE_COLUMN (map, where)<br> > : 0);<br>
> ++ s.file = LINEMAP_FILE (map);<br>
> ++ s.line = SOURCE_LINE (map, where);<br>
> ++ int col = -1;<br>
> ++ if (first && context->show_column)<br>
> ++ {<br>
> ++ s.column = SOURCE_COLUMN (map, where);<br>
> ++ col = diagnostic_converted_column (context, \
s);<br> > ++ }<br>
> ++ const char *line_col = maybe_line_and_column \
(s.line,<br> > col);<br>
> + static const char *const msgs[] =<br>
> + {<br>
> + N_("In file included from"),<br>
> +@@ -666,7 +726,7 @@ diagnostic_report_current_module (diagno<br>
> + unsigned index = !first;<br>
> + pp_verbatim (context->printer, "%s%s \
%r%s%s%R",<br> > + first ? \
"" : ",\n", _(msgs[index]),<br> > +- \
"locus", LINEMAP_FILE (map), line_col);<br> > ++ \
"locus", s.file, line_col);<br> > + first = \
false;<br> > + }<br>
> + while (! MAIN_FILE_P (map));<br>
> +@@ -2042,10 +2102,15 @@ test_print_parseable_fixits_replace ()<br>
> + static void<br>
> + assert_location_text (const char *expected_loc_text,<br>
> + const char *filename, int line, int \
column,<br> > +- bool show_column)<br>
> ++ bool show_column,<br>
> ++ int origin = 1,<br>
> ++ enum diagnostics_column_unit \
column_unit<br> > ++ = \
DIAGNOSTICS_COLUMN_UNIT_BYTE)<br> > + {<br>
> + test_diagnostic_context dc;<br>
> + dc.show_column = show_column;<br>
> ++ dc.column_unit = column_unit;<br>
> ++ dc.column_origin = origin;<br>
> + <br>
> + expanded_location xloc;<br>
> + xloc.file = filename;<br>
> +@@ -2069,7 +2134,10 @@ test_diagnostic_get_location_text ()<br>
> + assert_location_text ("PROGNAME:", NULL, 0, 0, true);<br>
> + assert_location_text ("<built-in>:", \
"<built-in>", 42, 10, true);<br> > + assert_location_text \
("foo.c:42:10:", "foo.c", 42, 10, true);<br> > +- \
assert_location_text ("foo.c:42:", "foo.c", 42, 0, true);<br> \
> ++ assert_location_text ("foo.c:42:9:", "foo.c", 42, 10, \
true, 0);<br> > ++ assert_location_text ("foo.c:42:1010:", \
"foo.c", 42, 10, true,<br> > 1001);<br>
> ++ for (int origin = 0; origin != 2; ++origin)<br>
> ++ assert_location_text ("foo.c:42:", "foo.c", 42, 0, \
true,<br> > origin);<br>
> + assert_location_text ("foo.c:", "foo.c", 0, 10, \
true);<br> > + assert_location_text ("foo.c:42:", "foo.c", \
42, 10, false);<br> > + assert_location_text ("foo.c:", \
"foo.c", 0, 10, false);<br> > +@@ -2077,6 +2145,41 @@ \
test_diagnostic_get_location_text ()<br> > + maybe_line_and_column (INT_MAX, \
INT_MAX);<br> > + maybe_line_and_column (INT_MIN, INT_MIN);<br>
> + <br>
> ++ {<br>
> ++ /* In order to test display columns vs byte columns, we need to<br>
> create a<br>
> ++ file for location_get_source_line() to read. */<br>
> ++<br>
> ++ const char *const content = "smile \xf0\x9f\x98\x82\n";<br>
> ++ const int line_bytes = strlen (content) - 1;<br>
> ++ const int def_tabstop = 8;<br>
> ++ const int display_width = cpp_display_width (content,<br>
> line_bytes,<br>
> ++ \
def_tabstop);<br> > ++ ASSERT_EQ (line_bytes - 2, display_width);<br>
> ++ temp_source_file tmp (SELFTEST_LOCATION, ".c", content);<br>
> ++ const char *const fname = tmp.get_filename ();<br>
> ++ const int buf_len = strlen (fname) + 16;<br>
> ++ char *const expected = XNEWVEC (char, buf_len);<br>
> ++<br>
> ++ snprintf (expected, buf_len, "%s:1:%d:", fname, \
line_bytes);<br> > ++ assert_location_text (expected, fname, 1, line_bytes, \
true,<br> > ++ 1, \
DIAGNOSTICS_COLUMN_UNIT_BYTE);<br> > ++<br>
> ++ snprintf (expected, buf_len, "%s:1:%d:", fname, line_bytes \
-<br> > 1);<br>
> ++ assert_location_text (expected, fname, 1, line_bytes, true,<br>
> ++ 0, \
DIAGNOSTICS_COLUMN_UNIT_BYTE);<br> > ++<br>
> ++ snprintf (expected, buf_len, "%s:1:%d:", fname, \
display_width);<br> > ++ assert_location_text (expected, fname, 1, \
line_bytes, true,<br> > ++ 1, \
DIAGNOSTICS_COLUMN_UNIT_DISPLAY);<br> > ++<br>
> ++ snprintf (expected, buf_len, "%s:1:%d:", fname, display_width \
-<br> > 1);<br>
> ++ assert_location_text (expected, fname, 1, line_bytes, true,<br>
> ++ 0, \
DIAGNOSTICS_COLUMN_UNIT_DISPLAY);<br> > ++<br>
> ++ XDELETEVEC (expected);<br>
> ++ }<br>
> ++<br>
> ++<br>
> + progname = old_progname;<br>
> + }<br>
> + <br>
> +diff --git a/gcc/diagnostic-format-json.cc b/gcc/diagnostic-format-<br>
> json.cc<br>
> +--- a/gcc/diagnostic-format-json.cc 2020-07-22 23:35:17.556386887<br>
> -0700<br>
> ++++ b/gcc/diagnostic-format-json.cc 2021-12-25 01:20:53.475636694<br>
> -0800<br>
> +@@ -23,6 +23,7 @@ along with GCC; see the file COPYING3.<br>
> + #include "system.h"<br>
> + #include "coretypes.h"<br>
> + #include "diagnostic.h"<br>
> ++#include "selftest-diagnostic.h"<br>
> + #include "diagnostic-metadata.h"<br>
> + #include "json.h"<br>
> + #include "selftest.h"<br>
> +@@ -43,21 +44,43 @@ static json::array *cur_children_array;<br>
> + /* Generate a JSON object for LOC. */<br>
> + <br>
> + json::value *<br>
> +-json_from_expanded_location (location_t loc)<br>
> ++json_from_expanded_location (diagnostic_context *context,<br>
> location_t loc)<br>
> + {<br>
> + expanded_location exploc = expand_location (loc);<br>
> + json::object *result = new json::object ();<br>
> + if (exploc.file)<br>
> + result->set ("file", new json::string (exploc.file));<br>
> + result->set ("line", new json::integer_number \
(exploc.line));<br> > +- result->set ("column", new \
json::integer_number (exploc.column));<br> > ++<br>
> ++ const enum diagnostics_column_unit orig_unit = context-<br>
> >column_unit;<br>
> ++ struct<br>
> ++ {<br>
> ++ const char *name;<br>
> ++ enum diagnostics_column_unit unit;<br>
> ++ } column_fields[] = {<br>
> ++ {"display-column", DIAGNOSTICS_COLUMN_UNIT_DISPLAY},<br>
> ++ {"byte-column", DIAGNOSTICS_COLUMN_UNIT_BYTE}<br>
> ++ };<br>
> ++ int the_column = INT_MIN;<br>
> ++ for (int i = 0; i != sizeof column_fields / sizeof<br>
> (*column_fields); ++i)<br>
> ++ {<br>
> ++ context->column_unit = column_fields[i].unit;<br>
> ++ const int col = diagnostic_converted_column (context,<br>
> exploc);<br>
> ++ result->set (column_fields[i].name, new json::integer_number<br>
> (col));<br>
> ++ if (column_fields[i].unit == orig_unit)<br>
> ++ the_column = col;<br>
> ++ }<br>
> ++ gcc_assert (the_column != INT_MIN);<br>
> ++ result->set ("column", new json::integer_number \
(the_column));<br> > ++ context->column_unit = orig_unit;<br>
> + return result;<br>
> + }<br>
> + <br>
> + /* Generate a JSON object for LOC_RANGE. */<br>
> + <br>
> + static json::object *<br>
> +-json_from_location_range (const location_range *loc_range, unsigned<br>
> range_idx)<br>
> ++json_from_location_range (diagnostic_context *context,<br>
> ++ const location_range \
*loc_range, unsigned<br> > range_idx)<br>
> + {<br>
> + location_t caret_loc = get_pure_location (loc_range->m_loc);<br>
> + <br>
> +@@ -68,13 +91,13 @@ json_from_location_range (const location<br>
> + location_t finish_loc = get_finish (loc_range->m_loc);<br>
> + <br>
> + json::object *result = new json::object ();<br>
> +- result->set ("caret", json_from_expanded_location \
(caret_loc));<br> > ++ result->set ("caret", \
json_from_expanded_location (context,<br> > caret_loc));<br>
> + if (start_loc != caret_loc<br>
> + && start_loc != UNKNOWN_LOCATION)<br>
> +- result->set ("start", json_from_expanded_location \
(start_loc));<br> > ++ result->set ("start", \
json_from_expanded_location (context,<br> > start_loc));<br>
> + if (finish_loc != caret_loc<br>
> + && finish_loc != UNKNOWN_LOCATION)<br>
> +- result->set ("finish", json_from_expanded_location<br>
> (finish_loc));<br>
> ++ result->set ("finish", json_from_expanded_location \
(context,<br> > finish_loc));<br>
> + <br>
> + if (loc_range->m_label)<br>
> + {<br>
> +@@ -91,14 +114,14 @@ json_from_location_range (const location<br>
> + /* Generate a JSON object for HINT. */<br>
> + <br>
> + static json::object *<br>
> +-json_from_fixit_hint (const fixit_hint *hint)<br>
> ++json_from_fixit_hint (diagnostic_context *context, const fixit_hint<br>
> *hint)<br>
> + {<br>
> + json::object *fixit_obj = new json::object ();<br>
> + <br>
> + location_t start_loc = hint->get_start_loc ();<br>
> +- fixit_obj->set ("start", json_from_expanded_location<br>
> (start_loc));<br>
> ++ fixit_obj->set ("start", json_from_expanded_location \
(context,<br> > start_loc));<br>
> + location_t next_loc = hint->get_next_loc ();<br>
> +- fixit_obj->set ("next", json_from_expanded_location \
(next_loc));<br> > ++ fixit_obj->set ("next", \
json_from_expanded_location (context,<br> > next_loc));<br>
> + fixit_obj->set ("string", new json::string \
(hint->get_string<br> > ()));<br>
> + <br>
> + return fixit_obj;<br>
> +@@ -190,11 +213,13 @@ json_end_diagnostic (diagnostic_context<br>
> + else<br>
> + {<br>
> + /* Otherwise, make diag_obj be the top-level object within<br>
> the group;<br>
> +- add a "children" array. */<br>
> ++ add a "children" array and record the column origin. \
*/<br> > + toplevel_array->append (diag_obj);<br>
> + cur_group = diag_obj;<br>
> + cur_children_array = new json::array ();<br>
> + diag_obj->set ("children", cur_children_array);<br>
> ++ diag_obj->set ("column-origin",<br>
> ++ new json::integer_number (context-<br>
> >column_origin));<br>
> + }<br>
> + <br>
> + const rich_location *richloc = diagnostic->richloc;<br>
> +@@ -205,7 +230,7 @@ json_end_diagnostic (diagnostic_context<br>
> + for (unsigned int i = 0; i < richloc->get_num_locations (); i++)<br>
> + {<br>
> + const location_range *loc_range = richloc->get_range (i);<br>
> +- json::object *loc_obj = json_from_location_range (loc_range,<br>
> i);<br>
> ++ json::object *loc_obj = json_from_location_range (context,<br>
> loc_range, i);<br>
> + if (loc_obj)<br>
> + loc_array->append (loc_obj);<br>
> + }<br>
> +@@ -217,7 +242,7 @@ json_end_diagnostic (diagnostic_context<br>
> + for (unsigned int i = 0; i < richloc->get_num_fixit_hints \
();<br> > i++)<br>
> + {<br>
> + const fixit_hint *hint = richloc->get_fixit_hint (i);<br>
> +- json::object *fixit_obj = json_from_fixit_hint (hint);<br>
> ++ json::object *fixit_obj = json_from_fixit_hint (context,<br>
> hint);<br>
> + fixit_array->append (fixit_obj);<br>
> + }<br>
> + }<br>
> +@@ -320,7 +345,8 @@ namespace selftest {<br>
> + static void<br>
> + test_unknown_location ()<br>
> + {<br>
> +- delete json_from_expanded_location (UNKNOWN_LOCATION);<br>
> ++ test_diagnostic_context dc;<br>
> ++ delete json_from_expanded_location (&dc, UNKNOWN_LOCATION);<br>
> + }<br>
> + <br>
> + /* Verify that we gracefully handle attempts to serialize bad<br>
> +@@ -338,7 +364,8 @@ test_bad_endpoints ()<br>
> + loc_range.m_range_display_kind = SHOW_RANGE_WITH_CARET;<br>
> + loc_range.m_label = NULL;<br>
> + <br>
> +- json::object *obj = json_from_location_range (&loc_range, 0);<br>
> ++ test_diagnostic_context dc;<br>
> ++ json::object *obj = json_from_location_range (&dc, &loc_range,<br>
> 0);<br>
> + /* We should have a "caret" value, but no "start" or \
"finish"<br> > values. */<br>
> + ASSERT_TRUE (obj != NULL);<br>
> + ASSERT_TRUE (obj->get ("caret") != NULL);<br>
> +diff --git a/gcc/diagnostic.h b/gcc/diagnostic.h<br>
> +--- a/gcc/diagnostic.h 2020-07-22 23:35:17.556386887 -0700<br>
> ++++ b/gcc/diagnostic.h 2021-12-25 01:20:53.479636627 -0800<br>
> +@@ -24,6 +24,20 @@ along with GCC; see the file COPYING3.<br>
> + #include "pretty-print.h"<br>
> + #include "diagnostic-core.h"<br>
> + <br>
> ++/* An enum for controlling what units to use for the column number<br>
> ++ when diagnostics are output, used by the -fdiagnostics-column-<br>
> unit option.<br>
> ++ Tabs will be expanded or not according to the value of -<br>
> ftabstop. The origin<br>
> ++ (default 1) is controlled by -fdiagnostics-column-origin. */<br>
> ++<br>
> ++enum diagnostics_column_unit<br>
> ++{<br>
> ++ /* The default from GCC 11 onwards: display columns. */<br>
> ++ DIAGNOSTICS_COLUMN_UNIT_DISPLAY,<br>
> ++<br>
> ++ /* The behavior in GCC 10 and earlier: simple bytes. */<br>
> ++ DIAGNOSTICS_COLUMN_UNIT_BYTE<br>
> ++};<br>
> ++<br>
> + /* Enum for overriding the standard output format. */<br>
> + <br>
> + enum diagnostics_output_format<br>
> +@@ -280,6 +294,15 @@ struct diagnostic_context<br>
> + rest of the diagnostic. */<br>
> + bool parseable_fixits_p;<br>
> + <br>
> ++ /* What units to use when outputting the column number. */<br>
> ++ enum diagnostics_column_unit column_unit;<br>
> ++<br>
> ++ /* The origin for the column number (1-based or 0-based<br>
> typically). */<br>
> ++ int column_origin;<br>
> ++<br>
> ++ /* The size of the tabstop for tab expansion. */<br>
> ++ int tabstop;<br>
> ++<br>
> + /* If non-NULL, an edit_context to which fix-it hints should be<br>
> + applied, for generating patches. */<br>
> + edit_context *edit_context_ptr;<br>
> +@@ -458,6 +481,8 @@ diagnostic_same_line (const diagnostic_c<br>
> + }<br>
> + <br>
> + extern const char *diagnostic_get_color_for_kind (diagnostic_t<br>
> kind);<br>
> ++extern int diagnostic_converted_column (diagnostic_context<br>
> *context,<br>
> ++ \
expanded_location s);<br> > + <br>
> + /* Pure text formatting support functions. */<br>
> + extern char *file_name_as_prefix (diagnostic_context *, const char<br>
> *);<br>
> +@@ -470,6 +495,7 @@ extern void diagnostic_output_format_ini<br>
> + /* Compute the number of digits in the decimal representation of an<br>
> integer. */<br>
> + extern int num_digits (int);<br>
> + <br>
> +-extern json::value *json_from_expanded_location (location_t loc);<br>
> ++extern json::value *json_from_expanded_location (diagnostic_context<br>
> *context,<br>
> ++ \
location_t loc);<br> > + <br>
> + #endif /* ! GCC_DIAGNOSTIC_H */<br>
> +diff --git a/gcc/diagnostic-show-locus.c b/gcc/diagnostic-show-<br>
> locus.c<br>
> +--- a/gcc/diagnostic-show-locus.c 2020-07-22 23:35:17.556386887<br>
> -0700<br>
> ++++ b/gcc/diagnostic-show-locus.c 2021-12-25 01:20:53.479636627<br>
> -0800<br>
> +@@ -175,9 +175,10 @@ enum column_unit {<br>
> + class exploc_with_display_col : public expanded_location<br>
> + {<br>
> + public:<br>
> +- exploc_with_display_col (const expanded_location &exploc)<br>
> ++ exploc_with_display_col (const expanded_location &exploc, int<br>
> tabstop)<br>
> + : expanded_location (exploc),<br>
> +- m_display_col (location_compute_display_column (exploc)) {}<br>
> ++ m_display_col (location_compute_display_column (exploc,<br>
> tabstop))<br>
> ++ {}<br>
> + <br>
> + int m_display_col;<br>
> + };<br>
> +@@ -189,11 +190,11 @@ class exploc_with_display_col : public e<br>
> + class layout_point<br>
> + {<br>
> + public:<br>
> +- layout_point (const expanded_location &exploc)<br>
> ++ layout_point (const exploc_with_display_col &exploc)<br>
> + : m_line (exploc.line)<br>
> + {<br>
> + m_columns[CU_BYTES] = exploc.column;<br>
> +- m_columns[CU_DISPLAY_COLS] = location_compute_display_column<br>
> (exploc);<br>
> ++ m_columns[CU_DISPLAY_COLS] = exploc.m_display_col;<br>
> + }<br>
> + <br>
> + linenum_type m_line;<br>
> +@@ -205,10 +206,10 @@ class layout_point<br>
> + class layout_range<br>
> + {<br>
> + public:<br>
> +- layout_range (const expanded_location *start_exploc,<br>
> +- const expanded_location *finish_exploc,<br>
> ++ layout_range (const exploc_with_display_col &start_exploc,<br>
> ++ const exploc_with_display_col \
&finish_exploc,<br> > + enum range_display_kind \
range_display_kind,<br> > +- const expanded_location \
*caret_exploc,<br> > ++ const exploc_with_display_col \
&caret_exploc,<br> > + unsigned original_idx,<br>
> + const range_label *label);<br>
> + <br>
> +@@ -226,22 +227,18 @@ class layout_range<br>
> + <br>
> + /* A struct for use by layout::print_source_line for telling<br>
> + layout::print_annotation_line the extents of the source line<br>
> that<br>
> +- it printed, so that underlines can be clipped appropriately. */<br>
> ++ it printed, so that underlines can be clipped appropriately. <br>
> Units<br>
> ++ are 1-based display columns. */<br>
> + <br>
> + struct line_bounds<br>
> + {<br>
> +- int m_first_non_ws;<br>
> +- int m_last_non_ws;<br>
> ++ int m_first_non_ws_disp_col;<br>
> ++ int m_last_non_ws_disp_col;<br>
> + <br>
> +- void convert_to_display_cols (char_span line)<br>
> ++ line_bounds ()<br>
> + {<br>
> +- m_first_non_ws = cpp_byte_column_to_display_column<br>
> (line.get_buffer (),<br>
> +- \
line.length<br> > (),<br>
> +-<br>
> \
m_first_non_ws)<br> > ;<br>
> +-<br>
> +- m_last_non_ws = cpp_byte_column_to_display_column<br>
> (line.get_buffer (),<br>
> +- \
line.length<br> > (),<br>
> +- \
<br> > m_last_non_ws);<br>
> ++ m_first_non_ws_disp_col = INT_MAX;<br>
> ++ m_last_non_ws_disp_col = 0;<br>
> + }<br>
> + };<br>
> + <br>
> +@@ -351,8 +348,8 @@ class layout<br>
> + private:<br>
> + bool will_show_line_p (linenum_type row) const;<br>
> + void print_leading_fixits (linenum_type row);<br>
> +- void print_source_line (linenum_type row, const char *line, int<br>
> line_bytes,<br>
> +- line_bounds *lbounds_out);<br>
> ++ line_bounds print_source_line (linenum_type row, const char<br>
> *line,<br>
> ++ int \
line_bytes);<br> > + bool should_print_annotation_line_p (linenum_type row) \
const;<br> > + void start_annotation_line (char margin_char = ' ') \
const;<br> > + void print_annotation_line (linenum_type row, const \
line_bounds<br> > lbounds);<br>
> +@@ -513,16 +510,16 @@ colorizer::get_color_by_name (const char<br>
> + Initialize various layout_point fields from expanded_location<br>
> + equivalents; we've already filtered on file. */<br>
> + <br>
> +-layout_range::layout_range (const expanded_location *start_exploc,<br>
> +- const expanded_location \
*finish_exploc,<br> > ++layout_range::layout_range (const \
exploc_with_display_col<br> > &start_exploc,<br>
> ++ const \
exploc_with_display_col<br> > &finish_exploc,<br>
> + enum \
range_display_kind<br> > range_display_kind,<br>
> +- const expanded_location \
*caret_exploc,<br> > ++ const \
exploc_with_display_col<br> > &caret_exploc,<br>
> + unsigned original_idx,<br>
> + const range_label \
*label)<br> > +-: m_start (*start_exploc),<br>
> +- m_finish (*finish_exploc),<br>
> ++: m_start (start_exploc),<br>
> ++ m_finish (finish_exploc),<br>
> + m_range_display_kind (range_display_kind),<br>
> +- m_caret (*caret_exploc),<br>
> ++ m_caret (caret_exploc),<br>
> + m_original_idx (original_idx),<br>
> + m_label (label)<br>
> + {<br>
> +@@ -646,6 +643,9 @@ layout_range::intersects_line_p (linenum<br>
> + <br>
> + #if CHECKING_P<br>
> + <br>
> ++/* Default for when we don't care what the tab expansion is set<br>
> to. */<br>
> ++static const int def_tabstop = 8;<br>
> ++<br>
> + /* Create some expanded locations for testing layout_range. The<br>
> filename<br>
> + member of the explocs is set to the empty string. This member<br>
> will only be<br>
> + inspected by the calls to location_compute_display_column() made<br>
> from the<br>
> +@@ -662,8 +662,11 @@ make_range (int start_line, int start_co<br>
> + = {"", start_line, start_col, NULL, false};<br>
> + const expanded_location finish_exploc<br>
> + = {"", end_line, end_col, NULL, false};<br>
> +- return layout_range (&start_exploc, &finish_exploc,<br>
> SHOW_RANGE_WITHOUT_CARET,<br>
> +- &start_exploc, 0, NULL);<br>
> ++ return layout_range (exploc_with_display_col (start_exploc,<br>
> def_tabstop),<br>
> ++ exploc_with_display_col \
(finish_exploc,<br> > def_tabstop),<br>
> ++ SHOW_RANGE_WITHOUT_CARET,<br>
> ++ exploc_with_display_col \
(start_exploc,<br> > def_tabstop),<br>
> ++ 0, NULL);<br>
> + }<br>
> + <br>
> + /* Selftests for layout_range::contains_point and<br>
> +@@ -964,7 +967,7 @@ layout::layout (diagnostic_context * con<br>
> + : m_context (context),<br>
> + m_pp (context->printer),<br>
> + m_primary_loc (richloc->get_range (0)->m_loc),<br>
> +- m_exploc (richloc->get_expanded_location (0)),<br>
> ++ m_exploc (richloc->get_expanded_location (0), context->tabstop),<br>
> + m_colorizer (context, diagnostic_kind),<br>
> + m_colorize_source_p (context->colorize_source_p),<br>
> + m_show_labels_p (context->show_labels_p),<br>
> +@@ -1060,7 +1063,10 @@ layout::maybe_add_location_range (const<br>
> + <br>
> + /* Everything is now known to be in the correct source file,<br>
> + but it may require further sanitization. */<br>
> +- layout_range ri (&start, &finish, loc_range-<br>
> >m_range_display_kind, &caret,<br>
> ++ layout_range ri (exploc_with_display_col (start, m_context-<br>
> >tabstop),<br>
> ++ exploc_with_display_col (finish, \
m_context-<br> > >tabstop),<br>
> ++ loc_range->m_range_display_kind,<br>
> ++ exploc_with_display_col (caret, \
m_context-<br> > >tabstop),<br>
> + original_idx, loc_range->m_label);<br>
> + <br>
> + /* If we have a range that finishes before it starts (perhaps<br>
> +@@ -1394,7 +1400,7 @@ layout::calculate_x_offset_display ()<br>
> + = get_line_bytes_without_trailing_whitespace (line.get_buffer<br>
> (),<br>
> + \
line.length ());<br> > + int eol_display_column<br>
> +- = cpp_display_width (line.get_buffer (), line_bytes);<br>
> ++ = cpp_display_width (line.get_buffer (), line_bytes, m_context-<br>
> >tabstop);<br>
> + if (caret_display_column > eol_display_column<br>
> + || !caret_display_column)<br>
> + {<br>
> +@@ -1445,16 +1451,13 @@ layout::calculate_x_offset_display ()<br>
> + }<br>
> + <br>
> + /* Print line ROW of source code, potentially colorized at any<br>
> ranges, and<br>
> +- populate *LBOUNDS_OUT.<br>
> +- LINE is the source line (not necessarily 0-terminated) and<br>
> LINE_BYTES<br>
> +- is its length in bytes.<br>
> +- This function deals only with byte offsets, not display columns,<br>
> so<br>
> +- m_x_offset_display must be converted from display to byte<br>
> units. In<br>
> +- particular, LINE_BYTES and LBOUNDS_OUT are in bytes. */<br>
> ++ return the line bounds. LINE is the source line (not<br>
> necessarily<br>
> ++ 0-terminated) and LINE_BYTES is its length in bytes. In order<br>
> to handle both<br>
> ++ colorization and tab expansion, this function tracks the line<br>
> position in<br>
> ++ both byte and display column units. */<br>
> + <br>
> +-void<br>
> +-layout::print_source_line (linenum_type row, const char *line, int<br>
> line_bytes,<br>
> +- line_bounds *lbounds_out)<br>
> ++line_bounds<br>
> ++layout::print_source_line (linenum_type row, const char *line, int<br>
> line_bytes)<br>
> + {<br>
> + m_colorizer.set_normal_text ();<br>
> + <br>
> +@@ -1469,30 +1472,29 @@ layout::print_source_line (linenum_type<br>
> + else<br>
> + pp_space (m_pp);<br>
> + <br>
> +- /* We will stop printing the source line at any trailing<br>
> whitespace, and start<br>
> +- printing it as per m_x_offset_display. */<br>
> ++ /* We will stop printing the source line at any trailing<br>
> whitespace. */<br>
> + line_bytes = get_line_bytes_without_trailing_whitespace (line,<br>
> + \
<br> > line_bytes);<br>
> +- int x_offset_bytes = 0;<br>
> +- if (m_x_offset_display)<br>
> +- {<br>
> +- x_offset_bytes = cpp_display_column_to_byte_column (line,<br>
> line_bytes,<br>
> +- \
<br> > m_x_offset_display);<br>
> +- /* In case the leading portion of the line that will be<br>
> skipped over ends<br>
> +- with a character with wcwidth > 1, then it is possible we<br>
> skipped too<br>
> +- much, so account for that by padding with spaces. */<br>
> +- const int overage<br>
> +- = cpp_byte_column_to_display_column (line, line_bytes,<br>
> x_offset_bytes)<br>
> +- - m_x_offset_display;<br>
> +- for (int column = 0; column < overage; ++column)<br>
> +- pp_space (m_pp);<br>
> +- line += x_offset_bytes;<br>
> +- }<br>
> + <br>
> +- /* Print the line. */<br>
> +- int first_non_ws = INT_MAX;<br>
> +- int last_non_ws = 0;<br>
> +- for (int col_byte = 1 + x_offset_bytes; col_byte <= line_bytes;<br>
> col_byte++)<br>
> ++ /* This object helps to keep track of which display column we are<br>
> at, which is<br>
> ++ necessary for computing the line bounds in display units, for<br>
> doing<br>
> ++ tab expansion, and for implementing m_x_offset_display. */<br>
> ++ cpp_display_width_computation dw (line, line_bytes, m_context-<br>
> >tabstop);<br>
> ++<br>
> ++ /* Skip the first m_x_offset_display display columns. In case<br>
> the leading<br>
> ++ portion that will be skipped ends with a character with<br>
> wcwidth > 1, then<br>
> ++ it is possible we skipped too much, so account for that by<br>
> padding with<br>
> ++ spaces. Note that this does the right thing too in case a tab<br>
> was the last<br>
> ++ character to be skipped over; the tab is effectively replaced<br>
> by the<br>
> ++ correct number of trailing spaces needed to offset by the<br>
> desired number of<br>
> ++ display columns. */<br>
> ++ for (int skipped_display_cols = dw.advance_display_cols<br>
> (m_x_offset_display);<br>
> ++ skipped_display_cols > m_x_offset_display; --<br>
> skipped_display_cols)<br>
> ++ pp_space (m_pp);<br>
> ++<br>
> ++ /* Print the line and compute the line_bounds. */<br>
> ++ line_bounds lbounds;<br>
> ++ while (!dw.done ())<br>
> + {<br>
> + /* Assuming colorization is enabled for the caret and<br>
> underline<br>
> + characters, we may also colorize the associated characters<br>
> +@@ -1510,7 +1512,8 @@ layout::print_source_line (linenum_type<br>
> + {<br>
> + bool in_range_p;<br>
> + point_state state;<br>
> +- in_range_p = get_state_at_point (row, col_byte,<br>
> ++ const int start_byte_col = dw.bytes_processed () + 1;<br>
> ++ in_range_p = get_state_at_point (row, start_byte_col,<br>
> + \
0, INT_MAX,<br> > + \
CU_BYTES,<br> > + \
&state);<br> > +@@ -1519,22 +1522,44 @@ layout::print_source_line \
(linenum_type<br> > + else<br>
> + m_colorizer.set_normal_text ();<br>
> + }<br>
> +- char c = *line;<br>
> +- if (c == '\0' || c == '\t' || c == \
'\r')<br> > +- c = ' ';<br>
> +- if (c != ' ')<br>
> ++<br>
> ++ /* Get the display width of the next character to be output,<br>
> expanding<br>
> ++ tabs and replacing some control bytes with spaces as<br>
> necessary. */<br>
> ++ const char *c = dw.next_byte ();<br>
> ++ const int start_disp_col = dw.display_cols_processed () + 1;<br>
> ++ const int this_display_width = dw.process_next_codepoint ();<br>
> ++ if (*c == '\t')<br>
> ++ {<br>
> ++ /* The returned display width is the number of spaces into<br>
> which the<br>
> ++ tab should be expanded. */<br>
> ++ for (int i = 0; i != this_display_width; ++i)<br>
> ++ pp_space (m_pp);<br>
> ++ continue;<br>
> ++ }<br>
> ++ if (*c == '\0' || *c == '\r')<br>
> + {<br>
> +- last_non_ws = col_byte;<br>
> +- if (first_non_ws == INT_MAX)<br>
> +- first_non_ws = col_byte;<br>
> ++ /* cpp_wcwidth() promises to return 1 for all control<br>
> bytes, and we<br>
> ++ want to output these as a single space too, so this \
case<br> > is<br>
> ++ actually the same as the '\t' case. */<br>
> ++ gcc_assert (this_display_width == 1);<br>
> ++ pp_space (m_pp);<br>
> ++ continue;<br>
> + }<br>
> +- pp_character (m_pp, c);<br>
> +- line++;<br>
> ++<br>
> ++ /* We have a (possibly multibyte) character to output; update<br>
> the line<br>
> ++ bounds if it is not whitespace. */<br>
> ++ if (*c != ' ')<br>
> ++ {<br>
> ++ lbounds.m_last_non_ws_disp_col = dw.display_cols_processed<br>
> ();<br>
> ++ if (lbounds.m_first_non_ws_disp_col == INT_MAX)<br>
> ++ lbounds.m_first_non_ws_disp_col = start_disp_col;<br>
> ++ }<br>
> ++<br>
> ++ /* Output the character. */<br>
> ++ while (c != dw.next_byte ()) pp_character (m_pp, *c++);<br>
> + }<br>
> + print_newline ();<br>
> +-<br>
> +- lbounds_out->m_first_non_ws = first_non_ws;<br>
> +- lbounds_out->m_last_non_ws = last_non_ws;<br>
> ++ return lbounds;<br>
> + }<br>
> + <br>
> + /* Determine if we should print an annotation line for ROW.<br>
> +@@ -1576,14 +1601,13 @@ layout::start_annotation_line (char marg<br>
> + }<br>
> + <br>
> + /* Print a line consisting of the caret/underlines for the given<br>
> +- source line. This function works with display columns, rather<br>
> than byte<br>
> +- counts; in particular, LBOUNDS should be in display column<br>
> units. */<br>
> ++ source line. */<br>
> + <br>
> + void<br>
> + layout::print_annotation_line (linenum_type row, const line_bounds<br>
> lbounds)<br>
> + {<br>
> + int x_bound = get_x_bound_for_row (row, m_exploc.m_display_col,<br>
> +- \
lbounds.m_last_non_ws);<br> > ++ \
lbounds.m_last_non_ws_disp_col);<br> > + <br>
> + start_annotation_line ();<br>
> + pp_space (m_pp);<br>
> +@@ -1593,8 +1617,8 @@ layout::print_annotation_line (linenum_t<br>
> + bool in_range_p;<br>
> + point_state state;<br>
> + in_range_p = get_state_at_point (row, column,<br>
> +- \
lbounds.m_first_non_ws,<br> > +- \
lbounds.m_last_non_ws,<br> > ++ \
<br> > lbounds.m_first_non_ws_disp_col,<br>
> ++ <br>
> lbounds.m_last_non_ws_disp_col,<br>
> + \
CU_DISPLAY_COLS,<br> > + \
&state);<br> > + if (in_range_p)<br>
> +@@ -1631,12 +1655,14 @@ layout::print_annotation_line (linenum_t<br>
> + class line_label<br>
> + {<br>
> + public:<br>
> +- line_label (int state_idx, int column, label_text text)<br>
> ++ line_label (diagnostic_context *context, int state_idx, int<br>
> column,<br>
> ++ label_text text)<br>
> + : m_state_idx (state_idx), m_column (column),<br>
> + m_text (text), m_label_line (0), m_has_vbar (true)<br>
> + {<br>
> + const int bytes = strlen (text.m_buffer);<br>
> +- m_display_width = cpp_display_width (text.m_buffer, bytes);<br>
> ++ m_display_width<br>
> ++ = cpp_display_width (text.m_buffer, bytes, \
context->tabstop);<br> > + }<br>
> + <br>
> + /* Sorting is primarily by column, then by state index. */<br>
> +@@ -1696,7 +1722,7 @@ layout::print_any_labels (linenum_type r<br>
> + if (text.m_buffer == NULL)<br>
> + continue;<br>
> + <br>
> +- labels.safe_push (line_label (i, disp_col, text));<br>
> ++ labels.safe_push (line_label (m_context, i, disp_col, text));<br>
> + }<br>
> + }<br>
> + <br>
> +@@ -1976,7 +2002,8 @@ public:<br>
> + <br>
> + /* Get the range of bytes or display columns that HINT would<br>
> affect. */<br>
> + static column_range<br>
> +-get_affected_range (const fixit_hint *hint, enum column_unit<br>
> col_unit)<br>
> ++get_affected_range (diagnostic_context *context,<br>
> ++ const fixit_hint *hint, enum \
column_unit<br> > col_unit)<br>
> + {<br>
> + expanded_location exploc_start = expand_location (hint-<br>
> >get_start_loc ());<br>
> + expanded_location exploc_finish = expand_location (hint-<br>
> >get_next_loc ());<br>
> +@@ -1986,11 +2013,13 @@ get_affected_range (const fixit_hint *hi<br>
> + int finish_column;<br>
> + if (col_unit == CU_DISPLAY_COLS)<br>
> + {<br>
> +- start_column = location_compute_display_column<br>
> (exploc_start);<br>
> ++ start_column<br>
> ++ = location_compute_display_column (exploc_start, context-<br>
> >tabstop);<br>
> + if (hint->insertion_p ())<br>
> + finish_column = start_column - 1;<br>
> + else<br>
> +- finish_column = location_compute_display_column<br>
> (exploc_finish);<br>
> ++ finish_column<br>
> ++ = location_compute_display_column (exploc_finish, context-<br>
> >tabstop);<br>
> + }<br>
> + else<br>
> + {<br>
> +@@ -2003,12 +2032,12 @@ get_affected_range (const fixit_hint *hi<br>
> + /* Get the range of display columns that would be printed for<br>
> HINT. */<br>
> + <br>
> + static column_range<br>
> +-get_printed_columns (const fixit_hint *hint)<br>
> ++get_printed_columns (diagnostic_context *context, const fixit_hint<br>
> *hint)<br>
> + {<br>
> + expanded_location exploc = expand_location (hint->get_start_loc<br>
> ());<br>
> +- int start_column = location_compute_display_column (exploc);<br>
> +- int hint_width = cpp_display_width (hint->get_string (),<br>
> +- \
hint->get_length ());<br> > ++ int start_column = \
location_compute_display_column (exploc,<br> > context->tabstop);<br>
> ++ int hint_width = cpp_display_width (hint->get_string (), hint-<br>
> >get_length (),<br>
> ++ \
context->tabstop);<br> > + int final_hint_column = start_column + \
hint_width - 1;<br> > + if (hint->insertion_p ())<br>
> + {<br>
> +@@ -2018,7 +2047,8 @@ get_printed_columns (const fixit_hint *h<br>
> + {<br>
> + exploc = expand_location (hint->get_next_loc ());<br>
> + --exploc.column;<br>
> +- int finish_column = location_compute_display_column (exploc);<br>
> ++ int finish_column<br>
> ++ = location_compute_display_column (exploc, \
context->tabstop);<br> > + return column_range (start_column,<br>
> + MAX (finish_column, \
final_hint_column));<br> > + }<br>
> +@@ -2035,12 +2065,14 @@ public:<br>
> + correction (column_range affected_bytes,<br>
> + column_range affected_columns,<br>
> + column_range printed_columns,<br>
> +- const char *new_text, size_t new_text_len)<br>
> ++ const char *new_text, size_t new_text_len,<br>
> ++ int tabstop)<br>
> + : m_affected_bytes (affected_bytes),<br>
> + m_affected_columns (affected_columns),<br>
> + m_printed_columns (printed_columns),<br>
> + m_text (xstrdup (new_text)),<br>
> + m_byte_length (new_text_len),<br>
> ++ m_tabstop (tabstop),<br>
> + m_alloc_sz (new_text_len + 1)<br>
> + {<br>
> + compute_display_cols ();<br>
> +@@ -2058,7 +2090,7 @@ public:<br>
> + <br>
> + void compute_display_cols ()<br>
> + {<br>
> +- m_display_cols = cpp_display_width (m_text, m_byte_length);<br>
> ++ m_display_cols = cpp_display_width (m_text, m_byte_length,<br>
> m_tabstop);<br>
> + }<br>
> + <br>
> + void overwrite (int dst_offset, const char_span &src_span)<br>
> +@@ -2086,6 +2118,7 @@ public:<br>
> + char *m_text;<br>
> + size_t m_byte_length; /* Not including null-terminator. */<br>
> + int m_display_cols;<br>
> ++ int m_tabstop;<br>
> + size_t m_alloc_sz;<br>
> + };<br>
> + <br>
> +@@ -2121,13 +2154,15 @@ correction::ensure_terminated ()<br>
> + class line_corrections<br>
> + {<br>
> + public:<br>
> +- line_corrections (const char *filename, linenum_type row)<br>
> +- : m_filename (filename), m_row (row)<br>
> ++ line_corrections (diagnostic_context *context, const char<br>
> *filename,<br>
> ++ linenum_type row)<br>
> ++ : m_context (context), m_filename (filename), m_row (row)<br>
> + {}<br>
> + ~line_corrections ();<br>
> + <br>
> + void add_hint (const fixit_hint *hint);<br>
> + <br>
> ++ diagnostic_context *m_context;<br>
> + const char *m_filename;<br>
> + linenum_type m_row;<br>
> + auto_vec <correction *> m_corrections;<br>
> +@@ -2173,9 +2208,10 @@ source_line::source_line (const char *fi<br>
> + void<br>
> + line_corrections::add_hint (const fixit_hint *hint)<br>
> + {<br>
> +- column_range affected_bytes = get_affected_range (hint,<br>
> CU_BYTES);<br>
> +- column_range affected_columns = get_affected_range (hint,<br>
> CU_DISPLAY_COLS);<br>
> +- column_range printed_columns = get_printed_columns (hint);<br>
> ++ column_range affected_bytes = get_affected_range (m_context,<br>
> hint, CU_BYTES);<br>
> ++ column_range affected_columns = get_affected_range (m_context,<br>
> hint,<br>
> ++ \
<br> > CU_DISPLAY_COLS);<br>
> ++ column_range printed_columns = get_printed_columns (m_context,<br>
> hint);<br>
> + <br>
> + /* Potentially consolidate. */<br>
> + if (!m_corrections.is_empty ())<br>
> +@@ -2243,7 +2279,8 @@ line_corrections::add_hint (const fixit_<br>
> + \
affected_columns,<br> > + \
printed_columns,<br> > + \
hint->get_string (),<br> > +- \
hint->get_length ()));<br> > ++ \
hint->get_length (),<br> > ++ \
m_context->tabstop));<br> > + }<br>
> + <br>
> + /* If there are any fixit hints on source line ROW, print them.<br>
> +@@ -2257,7 +2294,7 @@ layout::print_trailing_fixits (linenum_t<br>
> + {<br>
> + /* Build a list of correction instances for the line,<br>
> + potentially consolidating hints (for the sake of<br>
> readability). */<br>
> +- line_corrections corrections (m_exploc.file, row);<br>
> ++ line_corrections corrections (m_context, m_exploc.file, row);<br>
> + for (unsigned int i = 0; i < m_fixit_hints.length (); i++)<br>
> + {<br>
> + const fixit_hint *hint = m_fixit_hints[i];<br>
> +@@ -2499,15 +2536,11 @@ layout::print_line (linenum_type row)<br>
> + if (!line)<br>
> + return;<br>
> + <br>
> +- line_bounds lbounds;<br>
> + print_leading_fixits (row);<br>
> +- print_source_line (row, line.get_buffer (), line.length (),<br>
> &lbounds);<br>
> ++ const line_bounds lbounds<br>
> ++ = print_source_line (row, line.get_buffer (), line.length ());<br>
> + if (should_print_annotation_line_p (row))<br>
> +- {<br>
> +- if (lbounds.m_first_non_ws != INT_MAX)<br>
> +- lbounds.convert_to_display_cols (line);<br>
> +- print_annotation_line (row, lbounds);<br>
> +- }<br>
> ++ print_annotation_line (row, lbounds);<br>
> + if (m_show_labels_p)<br>
> + print_any_labels (row);<br>
> + print_trailing_fixits (row);<br>
> +@@ -2670,9 +2703,11 @@ test_layout_x_offset_display_utf8 (const<br>
> + <br>
> + char_span lspan = location_get_source_line (tmp.get_filename (),<br>
> 1);<br>
> + ASSERT_EQ (line_display_cols,<br>
> +- cpp_display_width (lspan.get_buffer (), lspan.length<br>
> ()));<br>
> ++ cpp_display_width (lspan.get_buffer (), lspan.length \
(),<br> > ++ \
def_tabstop));<br> > + ASSERT_EQ (line_display_cols,<br>
> +- location_compute_display_column (expand_location<br>
> (line_end)));<br>
> ++ location_compute_display_column (expand_location<br>
> (line_end),<br>
> ++ \
def_tabstop));<br> > + ASSERT_EQ (0, memcmp (lspan.get_buffer () + (emoji_col \
- 1),<br> > + \
"\xf0\x9f\x98\x82\xf0\x9f\x98\x82", 8));<br> > + <br>
> +@@ -2774,6 +2809,111 @@ test_layout_x_offset_display_utf8 (const<br>
> + <br>
> + }<br>
> + <br>
> ++static void<br>
> ++test_layout_x_offset_display_tab (const line_table_case &case_)<br>
> ++{<br>
> ++ const char *content<br>
> ++ = "This line is very long, so that we can use it to test the<br>
> logic for "<br>
> ++ "clipping long lines. Also this: `\t' is a tab that \
occupies<br> > 1 byte and "<br>
> ++ "a variable number of display columns, starting at column<br>
> #103.\n";<br>
> ++<br>
> ++ /* Number of bytes in the line, subtracting one to remove the<br>
> newline. */<br>
> ++ const int line_bytes = strlen (content) - 1;<br>
> ++<br>
> ++ /* The column where the tab begins. Byte or display is the same<br>
> as there are<br>
> ++ no multibyte characters earlier on the line. */<br>
> ++ const int tab_col = 103;<br>
> ++<br>
> ++ /* Effective extra size of the tab beyond what a single space<br>
> would have taken<br>
> ++ up, indexed by tabstop. */<br>
> ++ static const int num_tabstops = 11;<br>
> ++ int extra_width[num_tabstops];<br>
> ++ for (int tabstop = 1; tabstop != num_tabstops; ++tabstop)<br>
> ++ {<br>
> ++ const int this_tab_size = tabstop - (tab_col - 1) % tabstop;<br>
> ++ extra_width[tabstop] = this_tab_size - 1;<br>
> ++ }<br>
> ++ /* Example of this calculation: if tabstop is 10, the tab<br>
> starting at column<br>
> ++ #103 has to expand into 8 spaces, covering columns 103-110, so<br>
> that the<br>
> ++ next character is at column #111. So it takes up 7 more<br>
> columns than<br>
> ++ a space would have taken up. */<br>
> ++ ASSERT_EQ (7, extra_width[10]);<br>
> ++<br>
> ++ temp_source_file tmp (SELFTEST_LOCATION, ".c", content);<br>
> ++ line_table_test ltt (case_);<br>
> ++<br>
> ++ linemap_add (line_table, LC_ENTER, false, tmp.get_filename (),<br>
> 1);<br>
> ++<br>
> ++ location_t line_end = linemap_position_for_column (line_table,<br>
> line_bytes);<br>
> ++<br>
> ++ /* Don't attempt to run the tests if column data might be<br>
> unavailable. */<br>
> ++ if (line_end > LINE_MAP_MAX_LOCATION_WITH_COLS)<br>
> ++ return;<br>
> ++<br>
> ++ /* Check that cpp_display_width handles the tabs as expected. */<br>
> ++ char_span lspan = location_get_source_line (tmp.get_filename (),<br>
> 1);<br>
> ++ ASSERT_EQ ('\t', *(lspan.get_buffer () + (tab_col - 1)));<br>
> ++ for (int tabstop = 1; tabstop != num_tabstops; ++tabstop)<br>
> ++ {<br>
> ++ ASSERT_EQ (line_bytes + extra_width[tabstop],<br>
> ++ cpp_display_width (lspan.get_buffer (), \
lspan.length<br> > (),<br>
> ++ \
tabstop));<br> > ++ ASSERT_EQ (line_bytes + extra_width[tabstop],<br>
> ++ location_compute_display_column \
(expand_location<br> > (line_end),<br>
> ++ \
tabstop));<br> > ++ }<br>
> ++<br>
> ++ /* Check that the tab is expanded to the expected number of<br>
> spaces. */<br>
> ++ rich_location richloc (line_table,<br>
> ++ linemap_position_for_column \
(line_table,<br> > ++ \
tab_col + 1));<br> > ++ for (int tabstop = 1; tabstop != num_tabstops; \
++tabstop)<br> > ++ {<br>
> ++ test_diagnostic_context dc;<br>
> ++ dc.tabstop = tabstop;<br>
> ++ layout test_layout (&dc, &richloc, DK_ERROR);<br>
> ++ test_layout.print_line (1);<br>
> ++ const char *out = pp_formatted_text (dc.printer);<br>
> ++ ASSERT_EQ (NULL, strchr (out, '\t'));<br>
> ++ const char *left_quote = strchr (out, '`');<br>
> ++ const char *right_quote = strchr (out, '\'');<br>
> ++ ASSERT_NE (NULL, left_quote);<br>
> ++ ASSERT_NE (NULL, right_quote);<br>
> ++ ASSERT_EQ (right_quote - left_quote, extra_width[tabstop] +<br>
> 2);<br>
> ++ }<br>
> ++<br>
> ++ /* Check that the line is offset properly and that the tab is<br>
> broken up<br>
> ++ into the expected number of spaces when it is the last<br>
> character skipped<br>
> ++ over. */<br>
> ++ for (int tabstop = 1; tabstop != num_tabstops; ++tabstop)<br>
> ++ {<br>
> ++ test_diagnostic_context dc;<br>
> ++ dc.tabstop = tabstop;<br>
> ++ static const int small_width = 24;<br>
> ++ dc.caret_max_width = small_width - 4;<br>
> ++ dc.min_margin_width = test_left_margin - test_linenum_sep +<br>
> 1;<br>
> ++ dc.show_line_numbers_p = true;<br>
> ++ layout test_layout (&dc, &richloc, DK_ERROR);<br>
> ++ test_layout.print_line (1);<br>
> ++<br>
> ++ /* We have arranged things so that two columns will be<br>
> printed before<br>
> ++ the caret. If the tab results in more than one space, this<br>
> should<br>
> ++ produce two spaces in the output; otherwise, it will be a<br>
> single space<br>
> ++ preceded by the opening quote before the tab character. */<br>
> ++ const char *output1<br>
> ++ = " 1 | ' is a tab that occupies 1 byte and a \
variable<br> > number of "<br>
> ++ "display columns, starting at column #103.\n"<br>
> ++ " | ^\n\n";<br>
> ++ const char *output2<br>
> ++ = " 1 | ` ' is a tab that occupies 1 byte and a \
variable<br> > number of "<br>
> ++ "display columns, starting at column #103.\n"<br>
> ++ " | ^\n\n";<br>
> ++ const char *expected_output = (extra_width[tabstop] ? output1<br>
> : output2);<br>
> ++ ASSERT_STREQ (expected_output, pp_formatted_text<br>
> (dc.printer));<br>
> ++ }<br>
> ++}<br>
> ++<br>
> ++<br>
> + /* Verify that diagnostic_show_locus works sanely on<br>
> UNKNOWN_LOCATION. */<br>
> + <br>
> + static void<br>
> +@@ -3854,6 +3994,27 @@ test_one_liner_labels_utf8 ()<br>
> + }<br>
> + }<br>
> + <br>
> ++/* Make sure that colorization codes don't interrupt a multibyte<br>
> ++ sequence, which would corrupt it. */<br>
> ++static void<br>
> ++test_one_liner_colorized_utf8 ()<br>
> ++{<br>
> ++ test_diagnostic_context dc;<br>
> ++ dc.colorize_source_p = true;<br>
> ++ diagnostic_color_init (&dc, DIAGNOSTICS_COLOR_YES);<br>
> ++ const location_t pi = linemap_position_for_column (line_table,<br>
> 12);<br>
> ++ rich_location richloc (line_table, pi);<br>
> ++ diagnostic_show_locus (&dc, &richloc, DK_ERROR);<br>
> ++<br>
> ++ /* In order to avoid having the test depend on exactly how the<br>
> colorization<br>
> ++ was effected, just confirm there are two pi characters in the<br>
> output. */<br>
> ++ const char *result = pp_formatted_text (dc.printer);<br>
> ++ const char *null_term = result + strlen (result);<br>
> ++ const char *first_pi = strstr (result, "\xcf\x80");<br>
> ++ ASSERT_TRUE (first_pi && first_pi <= null_term - 2);<br>
> ++ ASSERT_STR_CONTAINS (first_pi + 2, "\xcf\x80");<br>
> ++}<br>
> ++<br>
> + /* Run the various one-liner tests. */<br>
> + <br>
> + static void<br>
> +@@ -3884,8 +4045,10 @@ test_diagnostic_show_locus_one_liner_utf<br>
> + ASSERT_EQ (31, LOCATION_COLUMN (line_end));<br>
> + <br>
> + char_span lspan = location_get_source_line (tmp.get_filename (),<br>
> 1);<br>
> +- ASSERT_EQ (25, cpp_display_width (lspan.get_buffer (),<br>
> lspan.length ()));<br>
> +- ASSERT_EQ (25, location_compute_display_column (expand_location<br>
> (line_end)));<br>
> ++ ASSERT_EQ (25, cpp_display_width (lspan.get_buffer (),<br>
> lspan.length (),<br>
> ++ \
def_tabstop));<br> > ++ ASSERT_EQ (25, location_compute_display_column \
(expand_location<br> > (line_end),<br>
> ++ \
def_tabstop));<br> > + <br>
> + test_one_liner_simple_caret_utf8 ();<br>
> + test_one_liner_caret_and_range_utf8 ();<br>
> +@@ -3900,6 +4063,7 @@ test_diagnostic_show_locus_one_liner_utf<br>
> + test_one_liner_many_fixits_1_utf8 ();<br>
> + test_one_liner_many_fixits_2_utf8 ();<br>
> + test_one_liner_labels_utf8 ();<br>
> ++ test_one_liner_colorized_utf8 ();<br>
> + }<br>
> + <br>
> + /* Verify that gcc_rich_location::add_location_if_nearby works. */<br>
> +@@ -4272,25 +4436,28 @@ test_overlapped_fixit_printing (const li<br>
> + /* Unit-test the line_corrections machinery. */<br>
> + ASSERT_EQ (3, richloc.get_num_fixit_hints ());<br>
> + const fixit_hint *hint_0 = richloc.get_fixit_hint (0);<br>
> +- ASSERT_EQ (column_range (12, 12), get_affected_range (hint_0,<br>
> CU_BYTES));<br>
> + ASSERT_EQ (column_range (12, 12),<br>
> +- get_affected_range \
(hint_0,<br> > CU_DISPLAY_COLS));<br>
> +- ASSERT_EQ (column_range (12, 22), get_printed_columns<br>
> (hint_0));<br>
> ++ get_affected_range (&dc, hint_0, CU_BYTES));<br>
> ++ ASSERT_EQ (column_range (12, 12),<br>
> ++ get_affected_range (&dc, hint_0, \
CU_DISPLAY_COLS));<br> > ++ ASSERT_EQ (column_range (12, 22), \
get_printed_columns (&dc,<br> > hint_0));<br>
> + const fixit_hint *hint_1 = richloc.get_fixit_hint (1);<br>
> +- ASSERT_EQ (column_range (18, 18), get_affected_range (hint_1,<br>
> CU_BYTES));<br>
> + ASSERT_EQ (column_range (18, 18),<br>
> +- get_affected_range \
(hint_1,<br> > CU_DISPLAY_COLS));<br>
> +- ASSERT_EQ (column_range (18, 20), get_printed_columns<br>
> (hint_1));<br>
> ++ get_affected_range (&dc, hint_1, CU_BYTES));<br>
> ++ ASSERT_EQ (column_range (18, 18),<br>
> ++ get_affected_range (&dc, hint_1, \
CU_DISPLAY_COLS));<br> > ++ ASSERT_EQ (column_range (18, 20), \
get_printed_columns (&dc,<br> > hint_1));<br>
> + const fixit_hint *hint_2 = richloc.get_fixit_hint (2);<br>
> +- ASSERT_EQ (column_range (29, 28), get_affected_range (hint_2,<br>
> CU_BYTES));<br>
> + ASSERT_EQ (column_range (29, 28),<br>
> +- get_affected_range \
(hint_2,<br> > CU_DISPLAY_COLS));<br>
> +- ASSERT_EQ (column_range (29, 29), get_printed_columns<br>
> (hint_2));<br>
> ++ get_affected_range (&dc, hint_2, CU_BYTES));<br>
> ++ ASSERT_EQ (column_range (29, 28),<br>
> ++ get_affected_range (&dc, hint_2, \
CU_DISPLAY_COLS));<br> > ++ ASSERT_EQ (column_range (29, 29), \
get_printed_columns (&dc,<br> > hint_2));<br>
> + <br>
> + /* Add each hint in turn to a line_corrections instance,<br>
> + and verify that they are consolidated into one correction<br>
> instance<br>
> + as expected. */<br>
> +- line_corrections lc (tmp.get_filename (), 1);<br>
> ++ line_corrections lc (&dc, tmp.get_filename (), 1);<br>
> + <br>
> + /* The first replace hint by itself. */<br>
> + lc.add_hint (hint_0);<br>
> +@@ -4484,25 +4651,28 @@ test_overlapped_fixit_printing_utf8 (con<br>
> + /* Unit-test the line_corrections machinery. */<br>
> + ASSERT_EQ (3, richloc.get_num_fixit_hints ());<br>
> + const fixit_hint *hint_0 = richloc.get_fixit_hint (0);<br>
> +- ASSERT_EQ (column_range (14, 14), get_affected_range (hint_0,<br>
> CU_BYTES));<br>
> ++ ASSERT_EQ (column_range (14, 14),<br>
> ++ get_affected_range (&dc, hint_0, CU_BYTES));<br>
> + ASSERT_EQ (column_range (12, 12),<br>
> +- get_affected_range \
(hint_0,<br> > CU_DISPLAY_COLS));<br>
> +- ASSERT_EQ (column_range (12, 22), get_printed_columns<br>
> (hint_0));<br>
> ++ get_affected_range (&dc, hint_0, \
CU_DISPLAY_COLS));<br> > ++ ASSERT_EQ (column_range (12, 22), \
get_printed_columns (&dc,<br> > hint_0));<br>
> + const fixit_hint *hint_1 = richloc.get_fixit_hint (1);<br>
> +- ASSERT_EQ (column_range (22, 22), get_affected_range (hint_1,<br>
> CU_BYTES));<br>
> ++ ASSERT_EQ (column_range (22, 22),<br>
> ++ get_affected_range (&dc, hint_1, CU_BYTES));<br>
> + ASSERT_EQ (column_range (18, 18),<br>
> +- get_affected_range \
(hint_1,<br> > CU_DISPLAY_COLS));<br>
> +- ASSERT_EQ (column_range (18, 20), get_printed_columns<br>
> (hint_1));<br>
> ++ get_affected_range (&dc, hint_1, \
CU_DISPLAY_COLS));<br> > ++ ASSERT_EQ (column_range (18, 20), \
get_printed_columns (&dc,<br> > hint_1));<br>
> + const fixit_hint *hint_2 = richloc.get_fixit_hint (2);<br>
> +- ASSERT_EQ (column_range (35, 34), get_affected_range (hint_2,<br>
> CU_BYTES));<br>
> ++ ASSERT_EQ (column_range (35, 34),<br>
> ++ get_affected_range (&dc, hint_2, CU_BYTES));<br>
> + ASSERT_EQ (column_range (30, 29),<br>
> +- get_affected_range \
(hint_2,<br> > CU_DISPLAY_COLS));<br>
> +- ASSERT_EQ (column_range (30, 30), get_printed_columns<br>
> (hint_2));<br>
> ++ get_affected_range (&dc, hint_2, \
CU_DISPLAY_COLS));<br> > ++ ASSERT_EQ (column_range (30, 30), \
get_printed_columns (&dc,<br> > hint_2));<br>
> + <br>
> + /* Add each hint in turn to a line_corrections instance,<br>
> + and verify that they are consolidated into one correction<br>
> instance<br>
> + as expected. */<br>
> +- line_corrections lc (tmp.get_filename (), 1);<br>
> ++ line_corrections lc (&dc, tmp.get_filename (), 1);<br>
> + <br>
> + /* The first replace hint by itself. */<br>
> + lc.add_hint (hint_0);<br>
> +@@ -4689,6 +4859,8 @@ test_overlapped_fixit_printing_2 (const<br>
> + <br>
> + /* Two insertions, in the wrong order. */<br>
> + {<br>
> ++ test_diagnostic_context dc;<br>
> ++<br>
> + rich_location richloc (line_table, col_20);<br>
> + richloc.add_fixit_insert_before (col_23, "{");<br>
> + richloc.add_fixit_insert_before (col_21, "}");<br>
> +@@ -4696,14 +4868,15 @@ test_overlapped_fixit_printing_2 (const<br>
> + /* These fixits should be accepted; they can't be<br>
> consolidated. */<br>
> + ASSERT_EQ (2, richloc.get_num_fixit_hints ());<br>
> + const fixit_hint *hint_0 = richloc.get_fixit_hint (0);<br>
> +- ASSERT_EQ (column_range (23, 22), get_affected_range (hint_0,<br>
> CU_BYTES));<br>
> +- ASSERT_EQ (column_range (23, 23), get_printed_columns<br>
> (hint_0));<br>
> ++ ASSERT_EQ (column_range (23, 22),<br>
> ++ get_affected_range (&dc, hint_0, CU_BYTES));<br>
> ++ ASSERT_EQ (column_range (23, 23), get_printed_columns (&dc,<br>
> hint_0));<br>
> + const fixit_hint *hint_1 = richloc.get_fixit_hint (1);<br>
> +- ASSERT_EQ (column_range (21, 20), get_affected_range (hint_1,<br>
> CU_BYTES));<br>
> +- ASSERT_EQ (column_range (21, 21), get_printed_columns<br>
> (hint_1));<br>
> ++ ASSERT_EQ (column_range (21, 20),<br>
> ++ get_affected_range (&dc, hint_1, CU_BYTES));<br>
> ++ ASSERT_EQ (column_range (21, 21), get_printed_columns (&dc,<br>
> hint_1));<br>
> + <br>
> + /* Verify that they're printed correctly. */<br>
> +- test_diagnostic_context dc;<br>
> + diagnostic_show_locus (&dc, &richloc, DK_ERROR);<br>
> + ASSERT_STREQ (" int a5[][0][0] = { 1, 2 };\n"<br>
> + " \
^\n"<br> > +@@ -4955,6 +5128,65 @@ test_fixit_deletion_affecting_newline \
(c<br> > + pp_formatted_text (dc.printer));<br>
> + }<br>
> + <br>
> ++static void<br>
> ++test_tab_expansion (const line_table_case &case_)<br>
> ++{<br>
> ++ /* Create a tempfile and write some text to it. This example<br>
> uses a tabstop<br>
> ++ of 8, as the column numbers attempt to indicate:<br>
> ++<br>
> ++ .....................000.01111111111.22222333333 display<br>
> ++ .....................123.90123456789.56789012345 columns */<br>
> ++ const char *content = " \t This: `\t' is a tab.\n";<br>
> ++ /* ....................000 00000011111 11111222222 byte<br>
> ++ ....................123 45678901234 56789012345 columns */<br>
> ++<br>
> ++ const int tabstop = 8;<br>
> ++ const int first_non_ws_byte_col = 7;<br>
> ++ const int right_quote_byte_col = 15;<br>
> ++ const int last_byte_col = 25;<br>
> ++ ASSERT_EQ (35, cpp_display_width (content, last_byte_col,<br>
> tabstop));<br>
> ++<br>
> ++ temp_source_file tmp (SELFTEST_LOCATION, ".c", content);<br>
> ++ line_table_test ltt (case_);<br>
> ++ linemap_add (line_table, LC_ENTER, false, tmp.get_filename (),<br>
> 1);<br>
> ++<br>
> ++ /* Don't attempt to run the tests if column data might be<br>
> unavailable. */<br>
> ++ location_t line_end = linemap_position_for_column (line_table,<br>
> last_byte_col);<br>
> ++ if (line_end > LINE_MAP_MAX_LOCATION_WITH_COLS)<br>
> ++ return;<br>
> ++<br>
> ++ /* Check that the leading whitespace with mixed tabs and spaces<br>
> is expanded<br>
> ++ into 11 spaces. Recall that print_line() also puts one space<br>
> before<br>
> ++ everything too. */<br>
> ++ {<br>
> ++ test_diagnostic_context dc;<br>
> ++ dc.tabstop = tabstop;<br>
> ++ rich_location richloc (line_table,<br>
> ++ linemap_position_for_column \
(line_table,<br> > ++ \
first_non_ws_<br> > byte_col));<br>
> ++ layout test_layout (&dc, &richloc, DK_ERROR);<br>
> ++ test_layout.print_line (1);<br>
> ++ ASSERT_STREQ (" This: ` ' is a \
tab.\n"<br> > ++ " \
^\n",<br> > ++ pp_formatted_text \
(dc.printer));<br> > ++ }<br>
> ++<br>
> ++ /* Confirm the display width was tracked correctly across the<br>
> internal tab<br>
> ++ as well. */<br>
> ++ {<br>
> ++ test_diagnostic_context dc;<br>
> ++ dc.tabstop = tabstop;<br>
> ++ rich_location richloc (line_table,<br>
> ++ linemap_position_for_column \
(line_table,<br> > ++ \
right_quote_b<br> > yte_col));<br>
> ++ layout test_layout (&dc, &richloc, DK_ERROR);<br>
> ++ test_layout.print_line (1);<br>
> ++ ASSERT_STREQ (" This: ` ' is a \
tab.\n"<br> > ++ " \
^\n",<br> > ++ pp_formatted_text \
(dc.printer));<br> > ++ }<br>
> ++}<br>
> ++<br>
> + /* Verify that line numbers are correctly printed for the case of<br>
> + a multiline range in which the width of the line numbers changes<br>
> + (e.g. from "9" to "10"). */<br>
> +@@ -5012,6 +5244,7 @@ diagnostic_show_locus_c_tests ()<br>
> + test_layout_range_for_multiple_lines ();<br>
> + <br>
> + for_each_line_table_case (test_layout_x_offset_display_utf8);<br>
> ++ for_each_line_table_case (test_layout_x_offset_display_tab);<br>
> + <br>
> + test_get_line_bytes_without_trailing_whitespace ();<br>
> + <br>
> +@@ -5029,6 +5262,7 @@ diagnostic_show_locus_c_tests ()<br>
> + for_each_line_table_case<br>
> (test_fixit_insert_containing_newline_2);<br>
> + for_each_line_table_case (test_fixit_replace_containing_newline);<br>
> + for_each_line_table_case (test_fixit_deletion_affecting_newline);<br>
> ++ for_each_line_table_case (test_tab_expansion);<br>
> + <br>
> + test_line_numbers_multiline_range ();<br>
> + }<br>
> +diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi<br>
> +--- a/gcc/doc/invoke.texi 2021-12-24 20:23:46.876739587 -0800<br>
> ++++ b/gcc/doc/invoke.texi 2021-12-25 01:20:53.487636494 -0800<br>
> +@@ -293,7 +293,9 @@ Objective-C and Objective-C++ Dialects}.<br>
> + -fdiagnostics-show-template-tree -fno-elide-type @gol<br>
> +<br>
> -fdiagnostics-path-format=@r{[}none@r{|}separate-events@r{|}inline-events@r{<br>
> ]} @gol<br>
> + -fdiagnostics-show-path-depths @gol<br>
> +--fno-show-column}<br>
> ++-fno-show-column @gol<br>
> ++-fdiagnostics-column-unit=@r{[}display@r{|}byte@r{]} @gol<br>
> ++-fdiagnostics-column-origin=@var{origin}}<br>
> + <br>
> + @item Warning Options<br>
> + @xref{Warning Options,,Options to Request or Suppress Warnings}.<br>
> +@@ -4424,6 +4426,31 @@ Do not print column numbers in diagnosti<br>
> + diagnostics are being scanned by a program that does not understand<br>
> the<br>
> + column numbers, such as @command{dejagnu}.<br>
> + <br>
> ++@item -fdiagnostics-column-unit=@var{UNIT}<br>
> ++@opindex fdiagnostics-column-unit<br>
> ++Select the units for the column number. This affects traditional<br>
> diagnostics<br>
> ++(in the absence of @option{-fno-show-column}), as well as JSON<br>
> format<br>
> ++diagnostics if requested.<br>
> ++<br>
> ++The default @var{UNIT}, @samp{display}, considers the number of<br>
> display<br>
> ++columns occupied by each character. This may be larger than the<br>
> number<br>
> ++of bytes required to encode the character, in the case of tab<br>
> ++characters, or it may be smaller, in the case of multibyte<br>
> characters.<br>
> ++For example, the character ``GREEK SMALL LETTER PI (U+03C0)''<br>
> occupies one<br>
> ++display column, and its UTF-8 encoding requires two bytes; the<br>
> character<br>
> ++``SLIGHTLY SMILING FACE (U+1F642)'' occupies two display columns,<br>
> and<br>
> ++its UTF-8 encoding requires four bytes.<br>
> ++<br>
> ++Setting @var{UNIT} to @samp{byte} changes the column number to the<br>
> raw byte<br>
> ++count in all cases, as was traditionally output by GCC prior to<br>
> version 11.1.0.<br>
> ++<br>
> ++@item -fdiagnostics-column-origin=@var{ORIGIN}<br>
> ++@opindex fdiagnostics-column-origin<br>
> ++Select the origin for column numbers, i.e. the column number<br>
> assigned to the<br>
> ++first column. The default value of 1 corresponds to traditional<br>
> GCC<br>
> ++behavior and to the GNU style guide. Some utilities may perform<br>
> better with an<br>
> ++origin of 0; any non-negative value may be specified.<br>
> ++<br>
> + @item -fdiagnostics-format=@var{FORMAT}<br>
> + @opindex fdiagnostics-format<br>
> + Select a different format for printing diagnostics.<br>
> +@@ -4459,11 +4486,15 @@ might be printed in JSON form (after for<br>
> + "locations": [<br>
> + @{<br>
> + "caret": @{<br>
> ++ "display-column": 3,<br>
> ++ "byte-column": 3,<br>
> + "column": 3,<br>
> + "file": \
"misleading-indentation.c",<br> > + \
"line": 15<br> > + @},<br>
> + "finish": @{<br>
> ++ "display-column": 4,<br>
> ++ "byte-column": 4,<br>
> + "column": 4,<br>
> + "file": \
"misleading-indentation.c",<br> > + \
"line": 15<br> > +@@ -4479,6 +4510,8 @@ might be printed in JSON form \
(after for<br> > + "locations": [<br>
> + @{<br>
> + "caret": @{<br>
> ++ "display-column": \
5,<br> > ++ \
"byte-column": 5,<br> > + \
"column": 5,<br> > + \
"file": "misleading-indentation.c",<br> > + \
"line": 17<br> > +@@ -4488,6 +4521,7 @@ might be printed in JSON form \
(after for<br> > + "message": \
"...this statement, but the latter is<br> > @dots{}"<br>
> + @}<br>
> + ]<br>
> ++ "column-origin": 1,<br>
> + @},<br>
> + @dots{}<br>
> + ]<br>
> +@@ -4500,10 +4534,34 @@ A diagnostic has a @code{kind}. If this<br>
> + an @code{option} key describing the command-line option controlling<br>
> the<br>
> + warning.<br>
> + <br>
> +-A diagnostic can contain zero or more locations. Each location has<br>
> up<br>
> +-to three positions within it: a @code{caret} position and optional<br>
> +-@code{start} and @code{finish} positions. A location can also have<br>
> +-an optional @code{label} string. For example, this error:<br>
> ++A diagnostic can contain zero or more locations. Each location has<br>
> </blockquote></div></div>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160090): https://lists.openembedded.org/g/openembedded-core/message/160090
Mute This Topic: https://lists.openembedded.org/mt/88011740/4454766
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [openembedded-core@marc.info]
-=-=-=-=-=-=-=-=-=-=-=-
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic