'Re: [OpenIDM] Powershell-Connector: accessing Exchange 2013 efficiently'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       forgerock-openidm
Subject:    Re: [OpenIDM] Powershell-Connector: accessing Exchange 2013 efficiently
From:       Gael Allioux <gael.allioux () forgerock ! com>
Date:       2015-11-18 15:53:09
Message-ID: 564C9EE5.90808 () forgerock ! com
[Download RAW message or body]

[Attachment #2 (text/html)]

<html>
  <head>
    <meta content="text/html; charset=windows-1252"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <br>
    <br>
    <div class="moz-cite-prefix">On 11/18/2015 04:38 PM, Patrick von der
      Hagen wrote:<br>
    </div>
    <blockquote cite="mid:564C9B80.1030904@kit.edu" type="cite">
      <pre wrap="">On 18.11.2015 15:00, Gael Allioux wrote:
</pre>
      <blockquote type="cite">
        <pre wrap="">Patrick,
</pre>
      </blockquote>
      <pre wrap="">Hi Gael,

</pre>
      <blockquote type="cite">
        <pre wrap="">one thing you could also try is to cache your session once \
created. PowerShell connector has a shared "PropertyBag" between instances

See: <a class="moz-txt-link-freetext" \
href="https://bugster.forgerock.org/jira/browse/OPENICF-348">https://bugster.forgerock.org/jira/browse/OPENICF-348</a>

Since these powershell sessions is nothing else than an HTTP session,
I assume it could be shared between connector instances...
</pre>
      </blockquote>
      <pre wrap="">that's extremely interesting. :-)
</pre>
      <blockquote type="cite">
        <pre wrap="">if ( $Connector.Configuration.PropertyBag[session] == null) {
   $Connector.Configuration.PropertyBag[session] = New-PSSession .....
} else {
$session = $Connector.Configuration.PropertyBag[session]
}
</pre>
      </blockquote>
      <pre wrap="">in Powershell more like
if ( $Connector.Configuration.PropertyBag["session"] -eq $null) {
	$Connector.Configuration.PropertyBag["session"] =  New-PSSession ....
Import-PSSession ....
}
$Session = $Connector.Configuration.PropertyBag["session"]

(I added Import-PSSession to the cache, since the call that's really
taking ages).

</pre>
      <blockquote type="cite">
        <pre wrap="">THis feature has been tested with upcoming 1.4.2.0 version of \
the connector but the cache feature was already implemented in 1.4.1.0 ...
</pre>
      </blockquote>
      <pre wrap="">I'm giving it a try in my mailboxstatistics "provisioner", which \
I'm currently working at and which is read-only anyway, so there is little
risk. Really nice, certainly faster.

The code can be found at
<a class="moz-txt-link-freetext" \
href="https://git.scc.kit.edu/patrick.hagen/openidm-powershell/commits/master">https://git.scc.kit.edu/patrick.hagen/openidm-powershell/commits/master</a>
 (only at mailboxstatistics, not used for Exchange2013 yet).

But I wonder....

just to clarify: it is shared between instances and a running OpenICF
daemon is considered to be one instance, even if there are different
provisioner-configurations using different powershell-scripts running on
the same OpenICF daemon.</pre>
    </blockquote>
    1 provisioner = 1 configuration instance = 1 shared cache<br>
    The cache is actually a :<br>
    <br>
    ConcurrentDictionary&lt;string, object&gt; PropertyBag<br>
    <br>
    so if you have, say, 3 different .Net powershell connectors running
    on one .Net remote connector server,<br>
    each instance (each provisioner) will have its own cache.<br>
    <br>
    <blockquote cite="mid:564C9B80.1030904@kit.edu" type="cite">
      <pre wrap="">

I'm a little bit concerned about the setup being thread save? There is
pooling in the OpenICF daemon, so if the instance is the daemon and not
an object in the pool, some threads might use the same session in
parallel? I'm not sure Microsoft hat such a setup in mind when designing
the interface and would feel more comfortable, if each connection in the
pool had such a cache, thus avoiding any threading concerns...
</pre>
    </blockquote>
    PowerShell connector is a "Poolable" Connector. It means the OpenICF<br>
    framework maintains a pool of connector instances.<br>
    <br>
    This is what you configure in the provisioner file with the section<br>
    <br>
        "poolConfigOption" : {<br>
            "maxObjects" : 10,<br>
            "maxIdle" : 10,<br>
            "maxWait" : 150000,<br>
            "minEvictableIdleTimeMillis" : 120000,<br>
            "minIdle" : 1<br>
        },<br>
    <br>
    so in your case, if you create a session and cache it, it can be
    used simultaneously by<br>
    10 instances of the connector.<br>
    Is it safe? I don't know...<br>
    What I know is the session is an http session so it should be
    shareable...<br>
    <br>
    let us know your findings...  <span class="moz-smiley-s1"><span>
        :-) </span></span><br>
    <br>
    <blockquote cite="mid:564C9B80.1030904@kit.edu" type="cite">
      <pre wrap="">

</pre>
      <blockquote type="cite">
        <pre wrap="">

On 04/14/2015 01:09 PM, Patrick von der Hagen wrote:
</pre>
        <blockquote type="cite">
          <pre wrap="">Hi all,

short update on PowerShell, Exchange 2013 and efficency:

The usual way for accessing Exchange-Commandlets is
</pre>
          <blockquote type="cite">
            <pre wrap="">$Session = New-PSSession -ConfigurationName \
                Microsoft.Exchange
-ConnectionUrihttps://SERVER/powershell  -Authentication kerberos
Import-PSSession $Session
</pre>
          </blockquote>
          <pre wrap="">("usual" like "this is how every tutorial starts...")

"New-PSSession" is very fast, but "Import-PSSession" will load more than
700 commandlets and is extremely slow, especially since you can't share
a session (e.g. between read and a subsequent update).

However, we found two options to speed it up.

First, it should be possible to define a new user who has only access to
get-mailbox, enable-mailbox, disable-mailbox,.... instead of of more
than 700 default commandlets. Import-PSSession should run much faster,
but I chose another solution

Instead of using "Import-PSSession" at all, it is possible to invoke a
specific command using an existing session.

$s = New-PSSession -ConfigurationName Microsoft.Exchange
 -ConnectionUrihttps://SERVER/powershell  -Authentication kerberos
$res = Invoke-Command -ScriptBlock {get-mailbox -Identity ab1234}
-Session $s

The speed-up by not importing a session in the first place is amazing,
so this is currently my preferred solution for fast access to Exchange 2013.

On 31.03.2015 19:46, Gael Allioux wrote:
</pre>
          <blockquote type="cite">
            <pre wrap="">Hi,

inline:

On 03/31/2015 07:24 PM, Patrick von der Hagen wrote:
</pre>
            <blockquote type="cite">
              <pre wrap="">Hi all,

one of my top-requirements is managing mailboxes in MS Exchange 2013. In
theory, it should be not to hard to accomplish this using the
powershell-connector.

But there is a "however" just around the corner....

Looks like I can either invoke "add-pssnapin
Microsoft.Exchange.Management.PowerShell.E2010" at the start of all
scripts, which isn't officially supported or get a remote connection to
Exchange by starting/importing a remote session.

$Session = New-PSSession -ConfigurationName Microsoft.Exchange
-ConnectionUrihttps://SERVER/powershell  -Authentication kerberos
Import-PSSession $Session

This works fine, but it is incredibly slow. :-( Currently, it is ok for
testing but would be an issue for production.
</pre>
            </blockquote>
            <pre wrap="">remote session is the way to go. I know it is slow right now \
but next release of the connector will improve that a lot with the ability to cache \
the  connection
and keep them alive for next invocation

<a class="moz-txt-link-freetext" \
href="https://bugster.forgerock.org/jira/browse/OPENICF-363">https://bugster.forgerock.org/jira/browse/OPENICF-363</a>
 <a class="moz-txt-link-freetext" \
href="https://bugster.forgerock.org/jira/browse/OPENICF-349">https://bugster.forgerock.org/jira/browse/OPENICF-349</a>
 <a class="moz-txt-link-freetext" \
href="https://bugster.forgerock.org/jira/browse/OPENICF-348">https://bugster.forgerock.org/jira/browse/OPENICF-348</a>
 <a class="moz-txt-link-freetext" \
href="https://bugster.forgerock.org/jira/browse/OPENICF-347">https://bugster.forgerock.org/jira/browse/OPENICF-347</a>

Keep in mind, Powershell connector is still pretty new. It will
improve and keep sending precious feedback by filing JIRA issues.

</pre>
            <blockquote type="cite">
              <pre wrap="">Are there any ideas how to speed that up? Any experiences \
regarding managing mailboxes in Exchange 2013? I was hoping to use
PsModulesToImport at the connector-configuration, but I got a SnapIn,
not a module. :-(

Second issue: I'm still not sure how to structure my connector... I can
either use "__ACCOUNT__" and add exchange-data to each access or define
"__MAILBOX__" and create two mappings form managed/user to my
powershell-connector, one to AD__ACCOUNT and one to AD__MAILBOX__. I
guess I'll go for the second one, hoping for more speed and easier
handling when not every user gets a mailbox. But I'm not really
confident that's the right choice, so any input is appreciated. I'm
mostly worried about the sequence, like "only create the __MAILBOX__
after __ACCOUNT__ has been created". I guess I could do that inside a
workflow (always nice to get a sequence) or try to have lookups inside
of "validTarget".
</pre>
            </blockquote>
            <pre wrap="">depending on your exchange/ad topology you may have \
significant replication latency so that, yes, create AD account first and then wait \
to create the mailbox.

hope that helps a bit...

</pre>
            <blockquote type="cite">
              <pre wrap="">

_______________________________________________
OpenIDM mailing list
<a class="moz-txt-link-abbreviated" \
href="mailto:OpenIDM@forgerock.org">OpenIDM@forgerock.org</a> <a \
class="moz-txt-link-freetext" \
href="https://lists.forgerock.org/mailman/listinfo/openidm">https://lists.forgerock.org/mailman/listinfo/openidm</a>
 </pre>
            </blockquote>
          </blockquote>
          <pre wrap="">

_______________________________________________
OpenIDM mailing list
<a class="moz-txt-link-abbreviated" \
href="mailto:OpenIDM@forgerock.org">OpenIDM@forgerock.org</a> <a \
class="moz-txt-link-freetext" \
href="https://lists.forgerock.org/mailman/listinfo/openidm">https://lists.forgerock.org/mailman/listinfo/openidm</a>
 </pre>
        </blockquote>
        <pre wrap="">

_______________________________________________
OpenIDM mailing list
<a class="moz-txt-link-abbreviated" \
href="mailto:OpenIDM@forgerock.org">OpenIDM@forgerock.org</a> <a \
class="moz-txt-link-freetext" \
href="https://lists.forgerock.org/mailman/listinfo/openidm">https://lists.forgerock.org/mailman/listinfo/openidm</a>

</pre>
      </blockquote>
      <pre wrap="">
</pre>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
OpenIDM mailing list
<a class="moz-txt-link-abbreviated" \
href="mailto:OpenIDM@forgerock.org">OpenIDM@forgerock.org</a> <a \
class="moz-txt-link-freetext" \
href="https://lists.forgerock.org/mailman/listinfo/openidm">https://lists.forgerock.org/mailman/listinfo/openidm</a>
 </pre>
    </blockquote>
    <br>
  </body>
</html>

_______________________________________________
OpenIDM mailing list
OpenIDM@forgerock.org
https://lists.forgerock.org/mailman/listinfo/openidm

[prev in list] [next in list] [prev in thread] [next in thread] 