[prev in list] [next in list] [prev in thread] [next in thread] 

List:       postgresql-admin
Subject:    RE: Vacuum Tuning Question
From:       Murthy Nunna <mnunna () fnal ! gov>
Date:       2023-05-20 15:49:22
Message-ID: DM8PR09MB66772B0EC9D8D79174A543E2B87D9 () DM8PR09MB6677 ! namprd09 ! prod ! outlook ! com
[Download RAW message or body]

[Attachment #2 (text/plain)]

Jeff,

Your following statement made me understand this parameter better… Thank you!
"If you want autovacuum_freeze_max_age to be lower just make it lower.  You are \
basically saying you want it to behave as if it were lower, but without actually \
making it lower. You don't want the ratio to be much more than 100%, but expecting it \
to never even get close to 100% doesn't make any sense.  On a busy system, it will \
likely approach the value you told it to, that is what the setting is for."

At this point I am trying to comprehend \
https://www.postgresql.org/docs/14/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND

I understand default for autovacuum_freeze_max_age is 200 million. I currently have \
this setting at 1.5 billion. I am planning on lowering this to 1 billion in the next \
server restart and expect the ratio to get close to 100%. With 1 billion setting, if \
I base my ratio with max allowed 2 billion, the ratio will be 50%. That makes me feel \
better because the autovacuum will attempt to prevent wraparound when it gets close \
to 1 billion setting. If my understanding is wrong and if I start seeing dreaded \
warning such as "HINT:  To avoid a database shutdown, execute a database-wide VACUUM \
in that database." then I will have another billion leg room of transactions to set \
autovacuum_freeze_max_age to while I work on manual vacuuming. But only disadvantage \
is I have to restart.

This is that I currently have. Please comment when you are able to. Thanks again!


select name, setting, unit, context, source::char(8) from pg_settings where name like \
'%vacuum%';



                 name                  |  setting   | unit |  context   |  source

---------------------------------------+------------+------+------------+----------

autovacuum                            | on         |      | sighup     | configur

autovacuum_analyze_scale_factor       | 0.05       |      | sighup     | configur

autovacuum_analyze_threshold          | 0          |      | sighup     | configur

autovacuum_freeze_max_age             | 1500000000 |      | postmaster | configur

autovacuum_max_workers                | 5          |      | postmaster | configur

autovacuum_multixact_freeze_max_age   | 400000000  |      | postmaster | default

autovacuum_naptime                    | 60         | s    | sighup     | configur

autovacuum_vacuum_cost_delay          | -1         | ms   | sighup     | configur

autovacuum_vacuum_cost_limit          | -1         |      | sighup     | configur

autovacuum_vacuum_insert_scale_factor | 0.1        |      | sighup     | configur

autovacuum_vacuum_insert_threshold    | 0          |      | sighup     | configur

autovacuum_vacuum_scale_factor        | 0.1        |      | sighup     | configur

autovacuum_vacuum_threshold           | 0          |      | sighup     | configur

autovacuum_work_mem                   | -1         | kB   | sighup     | default

log_autovacuum_min_duration           | 0          | ms   | sighup     | configur

vacuum_cost_delay                     | 2          | ms   | user       | configur

vacuum_cost_limit                     | 200        |      | user       | configur

vacuum_cost_page_dirty                | 20         |      | user       | default

vacuum_cost_page_hit                  | 1          |      | user       | default

vacuum_cost_page_miss                 | 2          |      | user       | default

vacuum_defer_cleanup_age              | 0          |      | sighup     | default

vacuum_failsafe_age                   | 1600000000 |      | user       | default

vacuum_freeze_min_age                 | 50000000   |      | user       | default

vacuum_freeze_table_age               | 150000000  |      | user       | default

vacuum_multixact_failsafe_age         | 1600000000 |      | user       | default

vacuum_multixact_freeze_min_age       | 5000000    |      | user       | default

vacuum_multixact_freeze_table_age     | 150000000  |      | user       | default

(27 rows)














From: Jeff Janes <jeff.janes@gmail.com>
Sent: Friday, May 19, 2023 7:51 PM
To: Murthy Nunna <mnunna@fnal.gov>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: Vacuum Tuning Question

On Fri, May 19, 2023 at 11:36 AM Murthy Nunna \
<mnunna@fnal.gov<mailto:mnunna@fnal.gov>> wrote: Hi,

I have a fairly large database (several TBs) where auto vacuum is enabled. My focus \
is to keep the datfrozenxid reasonably low and avoid manual vacuum.

Why?  The age of datfrozenxid needs to be kept below 2 billion, but you should \
already be staying way below that, without needing to do anything.  What benefit do \
you think you will accrue from keeping it even more lower?

 > ...So, eventually I am afraid I have to vacuum the tables manually

Autovacuum to prevent wrap around will automatically kick in at \
autovacuum_freeze_max_age (if nothing makes it happen sooner).  There is no need to \
do a manual vacuum to accomplish that, so nothing needs to be done to avoid it.

which has its own problems like creating massive WALs in a short time etc. I would \
like to avoid manual vacuuming for this reason.

While manual vacuums are unthrottled by default, you can change vacuum_cost_delay to \
be the same as autovacuum_vacuum_cost_delay so they will be throttled in the same way \
as autovac is.  So if you were to have a good reason to do regular manual vacuums \
(which I don't think you do), this would not be much of a counterargument.  (There \
are other counterarguments which are better, like autocancelling upon lock conflicts, \
or just not wanting to write your own scheduling code when autovacuum already \
exists.)

SELECT freez, txns, ROUND(100*(txns/freez::float)) AS perc, datname
FROM (SELECT foo.freez::int, age(datfrozenxid) AS txns, datname
FROM pg_database d JOIN (SELECT setting AS freez FROM pg_settings WHERE name = \
'autovacuum_freeze_max_age') AS foo ON (true) WHERE d.datallowconn) AS foo2 ORDER BY \
3 DESC, 4 ASC;

...

Following are the settings I have. I am wondering if there is a way autovacuum  can \
keep the above "datfrozenxid" low and not keep increasing. Thank you ahead time for \
reading my post.

If you want autovacuum_freeze_max_age to be lower just make it lower.  You are \
basically saying you want it to behave as if it were lower, but without actually \
making it lower. You don't want the ratio to be much more than 100%, but expecting it \
to never even get close to 100% doesn't make any sense.  On a busy system, it will \
likely approach the value you told it to, that is what the setting is for.

Cheers,

Jeff


[Attachment #3 (text/html)]

<html xmlns:v="urn:schemas-microsoft-com:vml" \
xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns="http://www.w3.org/TR/REC-html40"> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:r_ansi;
	panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	font-size:11.0pt;
	font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
p.MsoNoSpacing, li.MsoNoSpacing, div.MsoNoSpacing
	{mso-style-priority:1;
	margin:0in;
	font-size:11.0pt;
	font-family:"Calibri",sans-serif;}
span.EmailStyle18
	{mso-style-type:personal-reply;
	font-family:"Calibri",sans-serif;
	color:windowtext;
	font-weight:normal;
	font-style:normal;}
.MsoChpDefault
	{mso-style-type:export-only;}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:12.0pt">Jeff,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p>&nbsp;</o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">Your following statement made me \
understand this parameter better… Thank you!<o:p></o:p></span></p> <p \
class="MsoNormal"><span style="font-size:12.0pt">"</span>If you want \
autovacuum_freeze_max_age to be lower just make it lower.&nbsp; You are basically \
saying you want it to behave as if it were lower, but without actually making it \
lower. You don't want the ratio  to be much more than 100%, but expecting it to never \
even get close to 100% doesn't make any sense.&nbsp; On a busy system, it will likely \
approach the value you told it to, that is what the setting is for."<o:p></o:p></p> \
<p class="MsoNormal"><o:p>&nbsp;</o:p></p> <p class="MsoNormal">At this point I am \
trying to comprehend <a \
href="https://www.postgresql.org/docs/14/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND">
 https://www.postgresql.org/docs/14/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND</a><o:p></o:p></p>
 <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal">I understand default for autovacuum_freeze_max_age is 200 \
million. I currently have this setting at 1.5 billion. I am planning on lowering this \
to 1 billion in the next server restart and expect the ratio to get close to 100%. \
With 1 billion  setting, if I base my ratio with max allowed 2 billion, the ratio \
will be 50%. That makes me feel better because the autovacuum will attempt to prevent \
wraparound when it gets close to 1 billion setting. If my understanding is wrong and \
if I start seeing dreaded  warning such as "<span style="font-family:&quot;Courier \
New&quot;;color:black">HINT:&nbsp; To avoid a database shutdown, execute a \
database-wide VACUUM in that database." </span><span style="color:black">then I will \
have another billion leg room of transactions to set </span>autovacuum_freeze_max_age \
to while I work on manual vacuuming. But only disadvantage is I have to \
restart.<o:p></o:p></p> <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal">This is that I currently have. Please comment when you are able \
to. Thanks again!<o:p></o:p></p> <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNoSpacing"><span style="font-size:10.0pt;font-family:&quot;Courier \
New&quot;">select name, setting, unit, context, source::char(8) from pg_settings \
where name like '%vacuum%';<o:p></o:p></span></p> <p class="MsoNoSpacing"><span \
style="font-size:10.0pt;font-family:&quot;Courier \
New&quot;"><o:p>&nbsp;</o:p></span></p> <p class="MsoNoSpacing"><span \
style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
name&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
|&nbsp; setting&nbsp;&nbsp; | unit |&nbsp; context&nbsp;&nbsp; |&nbsp; \
source<o:p></o:p></span></p> <p class="MsoNoSpacing"><span \
style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">---------------------------------------+------------+------+------------+----------<o:p></o:p></span></p>
 <p class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp \
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| on&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| sighup&nbsp;&nbsp;&nbsp;&nbsp; | configur<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_analyze_scale_factor&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
0.05&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
sighup&nbsp;&nbsp;&nbsp;&nbsp; | configur<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_analyze_threshold&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | sighup&nbsp;&nbsp;&nbsp;&nbsp; | \
configur<o:p></o:p></span></p> <p class="MsoNoSpacing"><span \
style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_freeze_max_age&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 1500000000 |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | postmaster | \
configur<o:p></o:p></span></p> <p class="MsoNoSpacing"><span \
style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_max_workers&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 5&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | postmaster | configur<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_multixact_freeze_max_age&nbsp;&nbsp; | 400000000&nbsp; \
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | postmaster | default<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_naptime&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 60&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | s&nbsp;&nbsp;&nbsp; | \
sighup&nbsp;&nbsp;&nbsp;&nbsp; | configur<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_vacuum_cost_delay&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| -1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | ms&nbsp;&nbsp; | \
sighup&nbsp;&nbsp;&nbsp;&nbsp; | configur<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_vacuum_cost_limit&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| -1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| sighup&nbsp;&nbsp;&nbsp;&nbsp; | configur<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_vacuum_insert_scale_factor | \
0.1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
sighup&nbsp;&nbsp;&nbsp;&nbsp; | configur<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_vacuum_insert_threshold&nbsp;&nbsp;&nbsp; | \
0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | sighup&nbsp;&nbsp;&nbsp;&nbsp; | \
configur<o:p></o:p></span></p> <p class="MsoNoSpacing"><span \
style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_vacuum_scale_factor&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
0.1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
sighup&nbsp;&nbsp;&nbsp;&nbsp; | configur<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_vacuum_threshold&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | sighup&nbsp;&nbsp;&nbsp;&nbsp; | \
configur<o:p></o:p></span></p> <p class="MsoNoSpacing"><span \
style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">autovacuum_work_mem&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| -1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | kB&nbsp;&nbsp; | \
sighup&nbsp;&nbsp;&nbsp;&nbsp; | default<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">log_autovacuum_min_duration&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | ms&nbsp;&nbsp; | \
sighup&nbsp;&nbsp;&nbsp;&nbsp; | configur<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">vacuum_cost_delay&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | ms&nbsp;&nbsp; | \
user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | configur<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">vacuum_cost_limit&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 200&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | configur<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">vacuum_cost_page_dirty&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 20&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | default<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">vacuum_cost_page_hit&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
default<o:p></o:p></span></p> <p class="MsoNoSpacing"><span \
style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">vacuum_cost_page_miss&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
default<o:p></o:p></span></p> <p class="MsoNoSpacing"><span \
style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">vacuum_defer_cleanup_age&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | sighup&nbsp;&nbsp;&nbsp;&nbsp; | \
default<o:p></o:p></span></p> <p class="MsoNoSpacing"><span \
style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">vacuum_failsafe_age&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 1600000000 |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | default<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">vacuum_freeze_min_age&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 50000000&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | default<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">vacuum_freeze_table_age&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 150000000&nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | default<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">vacuum_multixact_failsafe_age&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; \
| 1600000000 |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | default<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">vacuum_multixact_freeze_min_age&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
5000000&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | default<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:9.0pt;font-family:&quot;Courier \
New&quot;">vacuum_multixact_freeze_table_age&nbsp;&nbsp;&nbsp;&nbsp; | \
150000000&nbsp; |&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | \
user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | default<o:p></o:p></span></p> <p \
class="MsoNoSpacing"><span style="font-size:10.0pt;font-family:&quot;Courier \
New&quot;">(27 rows)<o:p></o:p></span></p> <p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p>&nbsp;</o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p>&nbsp;</o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p>&nbsp;</o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p>&nbsp;</o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p>&nbsp;</o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Jeff Janes &lt;jeff.janes@gmail.com&gt; <br>
<b>Sent:</b> Friday, May 19, 2023 7:51 PM<br>
<b>To:</b> Murthy Nunna &lt;mnunna@fnal.gov&gt;<br>
<b>Cc:</b> pgsql-admin@lists.postgresql.org<br>
<b>Subject:</b> Re: Vacuum Tuning Question<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<div>
<div>
<p class="MsoNormal">On Fri, May 19, 2023 at 11:36 AM Murthy Nunna &lt;<a \
href="mailto:mnunna@fnal.gov">mnunna@fnal.gov</a>&gt; wrote:<o:p></o:p></p> </div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-size:12.0pt">Hi,</span><o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-size:12.0pt">&nbsp;</span><o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-size:12.0pt">I have a fairly large database (several TBs) where auto \
vacuum is enabled. My focus is to keep the datfrozenxid reasonably low and avoid \
manual vacuum. </span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
<div>
<p class="MsoNormal">Why?&nbsp; The age of&nbsp;datfrozenxid needs to be kept below 2 \
billion, but you should already be staying way&nbsp;below that, without needing to do \
anything.&nbsp; What benefit do you think you&nbsp;will accrue from keeping it even \
more lower?<o:p></o:p></p> </div>
<div>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
<div>
<p class="MsoNormal">&nbsp;&gt; ...<span style="font-size:12.0pt">So, eventually I am \
afraid I have to vacuum the tables manually</span><o:p></o:p></p> </div>
<div>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
<div>
<p class="MsoNormal">Autovacuum to prevent wrap around will automatically kick in at \
autovacuum_freeze_max_age (if nothing makes&nbsp;it happen sooner).&nbsp; There is no \
need to do a manual vacuum to accomplish that, so nothing needs to be done to avoid \
it.<o:p></o:p></p> </div>
<div>
<p class="MsoNormal">&nbsp;<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-size:12.0pt">which has its own problems like creating massive WALs in a \
short time etc. I would like to avoid manual vacuuming for this \
reason.</span><o:p></o:p></p> </div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
<div>
<p class="MsoNormal">While manual vacuums are unthrottled by default, you can change \
vacuum_cost_delay to be the same as&nbsp;autovacuum_vacuum_cost_delay so they will be \
throttled in the same way as autovac is.&nbsp; So if you were to have a good reason \
to do regular  manual vacuums (which I don't think you do), this would not be much of \
a counterargument.&nbsp; (There are other counterarguments which are better, like \
autocancelling upon lock conflicts, or just not wanting to write your own scheduling \
code when autovacuum already  exists.)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">&nbsp;<span \
style="font-size:12.0pt">&nbsp;</span><o:p></o:p></p> </div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-size:10.0pt;font-family:r_ansi">SELECT freez, txns, \
ROUND(100*(txns/freez::float)) AS perc, datname</span><o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-size:10.0pt;font-family:r_ansi">FROM (SELECT foo.freez::int, \
age(datfrozenxid) AS txns, datname</span><o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-size:10.0pt;font-family:r_ansi">FROM pg_database d JOIN (SELECT setting \
AS freez FROM pg_settings WHERE name = 'autovacuum_freeze_max_age') AS \
foo</span><o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-size:10.0pt;font-family:r_ansi">ON (true) WHERE d.datallowconn) AS foo2 \
ORDER BY 3 DESC, 4 ASC;</span><o:p></o:p></p> </div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<div>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">...<o:p></o:p></p> </div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal">&nbsp;<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:&quot;Arial&quot;,sans-serif">Following are the settings I have. I \
am wondering if there is a way autovacuum&nbsp; can keep the above "datfrozenxid" low \
and not keep increasing.  Thank you ahead time for reading my \
post.</span><o:p></o:p></p> </div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
<div>
<p class="MsoNormal">If you want autovacuum_freeze_max_age to be lower just make it \
lower.&nbsp; You are basically saying you want it to behave as if it were lower, but \
without actually making it lower. You don't want the ratio to be much more than 100%, \
but expecting  it to never even get close to 100% doesn't make any sense.&nbsp; On a \
busy system, it will likely approach the value you told it to, that is what the \
setting is for.<o:p></o:p></p> </div>
<div>
<p class="MsoNormal">&nbsp;<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Cheers,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
<div>
<p class="MsoNormal">Jeff<o:p></o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>



[prev in list] [next in list] [prev in thread] [next in thread] 

Configure | About | News | Add a list | Sponsored by KoreLogic