[prev in list] [next in list] [prev in thread] [next in thread]
List: forgerock-openidm
Subject: Re: [OpenIDM] [EXTERNAL] Re: Reconciliation of 160 OpenDJ entries to OpenIDM 3.1 managed user reposi
From: "MacAlpine, Tim" <tlmacal () sandia ! gov>
Date: 2016-01-14 23:07:42
Message-ID: 69bcd5c52f29496db48f3d4516795e23 () ES06AMSNLNT ! srn ! sandia ! gov
[Download RAW message or body]
[Attachment #2 (text/plain)]
OK, I found the problem. Our repojdbc.json file had a LIMIT 4 clause in the \
query-all-ids. I'm not sure why but when I removed it things started working.
Thanks for the breadcrumbs! I learned some new things!
Tim
From: openidm-bounces@forgerock.org [mailto:openidm-bounces@forgerock.org] On Behalf \
Of MacAlpine, Tim
Sent: Thursday, January 14, 2016 3:23 PM
To: openidm@forgerock.org
Subject: Re: [OpenIDM] [EXTERNAL] Re: Reconciliation of 160 OpenDJ entries to OpenIDM \
3.1 managed user repository results in only 4 confirmed and 156 missing
Hi again – an update,
I queried the links table via the repo/link endpoint. For my 8 count load of managed \
users I get eight links returned. They all show a entryUUID as the firstId and the \
managed user _id value (that we have mapped from the LDAP entry) as the secondId. All \
eight look similar in structure to me, but still, if I perform the REST call \
"…openidm/managed/user/?_queryId=query-all-ids" via PostMan, I only get four of \
these back. They are, by the way, the 4 lowest OpenIDM ID values. Maybe coincidental, \
but…
I'm not sure I understand your suggestion to "follow the link from the link table \
manually". Do you mean to build queries from the information or is there some way to \
follow each link in OpenIDM?
Cheers!
Tim
From: openidm-bounces@forgerock.org<mailto:openidm-bounces@forgerock.org> \
[mailto:openidm-bounces@forgerock.org] On Behalf Of MacAlpine, Tim
Sent: Thursday, January 14, 2016 2:15 PM
To: openidm@forgerock.org<mailto:openidm@forgerock.org>
Subject: Re: [OpenIDM] [EXTERNAL] Re: Reconciliation of 160 OpenDJ entries to OpenIDM \
3.1 managed user repository results in only 4 confirmed and 156 missing
Hello, Matthias!
You are of course correct. They are in the repository but are not found by the sync \
process. They can be listed with the built-in UI under the Users tab. They can be \
listed individually by ID or other attributes through the REST API. But if I perform \
a postman all ids query \
(http://openidmhost.my.org/openidm/managed/user/?_queryId=query-all-ids) then I only \
get the four back that are confirmed by the sync. It seems strange to me that it is \
always four entries exactly, and not always the same four if I delete all entries and \
reload from scratch.
I will try to look into the link table – I have to figure out how to list that – \
and see what I can find.
Thanks for giving me something to go on! I'll report back on what I find.
Tim
From: openidm-bounces@forgerock.org<mailto:openidm-bounces@forgerock.org> \
[mailto:openidm-bounces@forgerock.org] On Behalf Of Matthias Tristl
Sent: Thursday, January 14, 2016 12:26 AM
To: openidm@forgerock.org<mailto:openidm@forgerock.org>
Subject: [EXTERNAL] Re: [OpenIDM] Reconciliation of 160 OpenDJ entries to OpenIDM 3.1 \
managed user repository results in only 4 confirmed and 156 missing
Hello Tim,
What you don't tell us is, whether the missing accounts are there in the repo and not \
found by the sync process, or whether they have vanished. 1) if they vanished: check \
why 2) If they are still there, then try to follow the link from the link table \
manually. What is the attribute that is used for the links? Could that somehow be \
changed after the objects where created? Check both, on the source as well as on the \
target side. A good candidate for a change that is not obvious is capitalisation. Are \
you using the DN or the entryUUID for the link (check your uidAttribute property in \
the connector configuration).
Just to remind you: "MISSING" means that the sync process found a link, but following \
the link it can't find a target object.
Matthias
Matthias Tristl : ForgeRock INC
e: matthias.tristl@forgerock.com<mailto:matthias.tristl@forgerock.com>
t: +47 47707662
w: forgerock.com<http://forgerock.com>
On Wed, Jan 13, 2016 at 5:47 PM, MacAlpine, Tim \
<tlmacal@sandia.gov<mailto:tlmacal@sandia.gov>> wrote: I am working with OpenIDM 3.1 \
and OpenDJ. I've setup a connector to access a class of objects in OpenDJ and \
configured several mappings to try to reconcile the entries to managed users. For \
some reason it will only ever confirm 4 entries and the rest are seen as missing. \
When I check the recon audit log it says that all 160 get created on the first pass \
(when repo is empty) but every run after that shows 4 of the entries as confirmed and \
the rest as missing. I've changed the connector to use a different base dn that \
filters the results down to 8 entries, emptied the repo by setting the recon rules to \
delete everything, re-run it with default settings and it creates all 8 entries the \
first time but still only confirms 4 of them after that. It's a different 4 than when \
I used the full 160 objects from OpenDJ.
I'm using a validSource script (source.AccountType == 'G') and a correlation query \
script ("var query = {'_queryId' : 'for-userName', 'uid' : source.uid};query;") in \
all my attempts.
Any clues would be appreciated. Oh, and where did the access to the email list \
archives go? I can't seem to find it anymore since the site reorganization.
Cheers!
Tim MacAlpine
Sandia National Laboratories Dept 09514<tel:09514>
(505) 844-5949
tlmacal@sandia.gov<mailto:tlmacal@sandia.gov>
_______________________________________________
OpenIDM mailing list
OpenIDM@forgerock.org<mailto:OpenIDM@forgerock.org>
https://lists.forgerock.org/mailman/listinfo/openidm
[Attachment #3 (text/html)]
<html xmlns:v="urn:schemas-microsoft-com:vml" \
xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns="http://www.w3.org/TR/REC-html40"> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">OK, \
I found the problem. Our repojdbc.json file had a LIMIT 4 clause in the \
query-all-ids. I'm not sure why but when I removed it things started working. \
<o:p></o:p></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks \
for the breadcrumbs! I learned some new things!<o:p></o:p></span></p> <p \
class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Tim<o:p></o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> \
openidm-bounces@forgerock.org [mailto:openidm-bounces@forgerock.org] <b>On Behalf Of \
</b>MacAlpine, Tim<br> <b>Sent:</b> Thursday, January 14, 2016 3:23 PM<br>
<b>To:</b> openidm@forgerock.org<br>
<b>Subject:</b> Re: [OpenIDM] [EXTERNAL] Re: Reconciliation of 160 OpenDJ entries to \
OpenIDM 3.1 managed user repository results in only 4 confirmed and 156 \
missing<o:p></o:p></span></p> </div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi \
again – an update,<o:p></o:p></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I \
queried the links table via the repo/link endpoint. For my 8 count load of managed \
users I get eight links returned. They all show a entryUUID as the firstId and the \
managed user _id value (that we have mapped from the LDAP entry) as the secondId. All \
eight look similar in structure to me, but still, if I perform the REST call \
"…openidm/managed/user/?_queryId=query-all-ids" via PostMan, I only get four of \
these back. They are, by the way, the 4 lowest OpenIDM ID values. Maybe \
coincidental, but…<o:p></o:p></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I'm \
not sure I understand your suggestion to "</span>follow the link from the link table \
manually<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">".
Do you mean to build queries from the information or is there some way to \
follow each link in OpenIDM?<o:p></o:p></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Cheers!<o:p></o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Tim<o:p></o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a \
href="mailto:openidm-bounces@forgerock.org">openidm-bounces@forgerock.org</a> [<a \
href="mailto:openidm-bounces@forgerock.org">mailto:openidm-bounces@forgerock.org</a>] \
<b>On Behalf Of </b>MacAlpine, Tim<br> <b>Sent:</b> Thursday, January 14, 2016 2:15 \
PM<br> <b>To:</b> <a \
href="mailto:openidm@forgerock.org">openidm@forgerock.org</a><br> <b>Subject:</b> Re: \
[OpenIDM] [EXTERNAL] Re: Reconciliation of 160 OpenDJ entries to OpenIDM 3.1 managed \
user repository results in only 4 confirmed and 156 missing<o:p></o:p></span></p> \
</div> </div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hello, \
Matthias!<o:p></o:p></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">You \
are of course correct. They are in the repository but are not found by the sync \
process. They can be listed with the built-in UI under the Users tab. They can be \
listed individually by ID or other attributes through the REST API. But if I \
perform a postman all ids query (<a \
href="http://openidmhost.my.org/openidm/managed/user/?_queryId=query-all-ids">http://openidmhost.my.org/openidm/managed/user/?_queryId=query-all-ids</a>)
then I only get the four back that are confirmed by the sync. It seems strange to \
me that it is always four entries exactly, and not always the same four if I delete \
all entries and reload from scratch. <o:p></o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I \
will try to look into the link table – I have to figure out how to list that – \
and see what I can find.<o:p></o:p></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks \
for giving me something to go on! I'll report back on what I \
find.<o:p></o:p></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Tim
<o:p></o:p></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a \
href="mailto:openidm-bounces@forgerock.org">openidm-bounces@forgerock.org</a> [<a \
href="mailto:openidm-bounces@forgerock.org">mailto:openidm-bounces@forgerock.org</a>] \
<b>On Behalf Of </b>Matthias Tristl<br> <b>Sent:</b> Thursday, January 14, 2016 12:26 \
AM<br> <b>To:</b> <a \
href="mailto:openidm@forgerock.org">openidm@forgerock.org</a><br> <b>Subject:</b> \
[EXTERNAL] Re: [OpenIDM] Reconciliation of 160 OpenDJ entries to OpenIDM 3.1 managed \
user repository results in only 4 confirmed and 156 missing<o:p></o:p></span></p> <p \
class="MsoNormal"><o:p> </o:p></p> <div>
<p class="MsoNormal">Hello Tim,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">What you don't tell us is, whether the missing accounts are \
there in the repo and not found by the sync process, or whether they have \
vanished. <o:p></o:p></p> </div>
<div>
<p class="MsoNormal">1) if they vanished: check why<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">2) If they are still there, then try to follow the link from the \
link table manually. What is the attribute that is used for the links? Could that \
somehow be changed after the objects where created? Check both, on the source as well \
as on the target side. A good candidate for a change that is not obvious is \
capitalisation. Are you using the DN or the entryUUID for the link (check your \
uidAttribute property in the connector configuration).<o:p></o:p></p> </div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Just to remind you: "MISSING" means that the sync \
process found a link, but following the link it can't find a target \
object. <o:p></o:p></p> </div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Matthias<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Matthias Tristl : ForgeRock INC<br>
e: <a href="mailto:matthias.tristl@forgerock.com">matthias.tristl@forgerock.com</a><br>
t: +47 47707662<br>
w: <a href="http://forgerock.com" target="_blank">forgerock.com</a><o:p></o:p></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Wed, Jan 13, 2016 at 5:47 PM, MacAlpine, Tim <<a \
href="mailto:tlmacal@sandia.gov" target="_blank">tlmacal@sandia.gov</a>> \
wrote:<o:p></o:p></p> <div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I am \
working with OpenIDM 3.1 and OpenDJ. I've setup a connector to access a class of \
objects in OpenDJ and configured several mappings to try to reconcile the entries to \
managed users. For some reason it will only ever confirm 4 entries and the rest are \
seen as missing. When I check the recon audit log it says that all 160 get created on \
the first pass (when repo is empty) but every run after that shows 4 of the entries \
as confirmed and the rest as missing. I've changed the connector to use a different \
base dn that filters the results down to 8 entries, emptied the repo by setting the \
recon rules to delete everything, re-run it with default settings and it creates all \
8 entries the first time but still only confirms 4 of them after that. It's a \
different 4 than when I used the full 160 objects from OpenDJ. <o:p></o:p></p>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I'm \
using a validSource script (source.AccountType == 'G') and a correlation query script \
("var query = {'_queryId' : 'for-userName', 'uid' : source.uid};query;") in all my \
attempts.<o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Any \
clues would be appreciated. Oh, and where did the access to the email list archives \
go? I can't seem to find it anymore since the site reorganization.<o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;margin-bottom:12.0pt">Cheers!<o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="color:#1F497D">Tim MacAlpine</span><o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="color:#1F497D">Sandia National Laboratories Dept <a href="tel:09514" \
target="_blank">09514</a></span><o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="color:#1F497D">(505) 844-5949</span><o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="color:#1F497D"><a href="mailto:tlmacal@sandia.gov" \
target="_blank">tlmacal@sandia.gov</a></span><o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> \
</div> </div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
OpenIDM mailing list<br>
<a href="mailto:OpenIDM@forgerock.org">OpenIDM@forgerock.org</a><br>
<a href="https://lists.forgerock.org/mailman/listinfo/openidm" \
target="_blank">https://lists.forgerock.org/mailman/listinfo/openidm</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>
_______________________________________________
OpenIDM mailing list
OpenIDM@forgerock.org
https://lists.forgerock.org/mailman/listinfo/openidm
--===============3279411318897573135==--
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic