[prev in list] [next in list] [prev in thread] [next in thread]
List: calendarserver-changes
Subject: [CalendarServer-changes] [7377]
From: source_changes () macosforge ! org
Date: 2011-04-27 21:09:24
Message-ID: 20110427210925.1F48B1971D1E () lists ! macosforge ! org
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Revision: 7377
http://trac.macosforge.org/projects/calendarserver/changeset/7377
Author: cdaboo@apple.com
Date: 2011-04-27 14:09:24 -0700 (Wed, 27 Apr 2011)
Log Message:
-----------
Merged from trunk and fixed some tests.
Modified Paths:
--------------
CalendarServer/branches/users/cdaboo/pods/bin/caldavd
CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/opendirectory.py
CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/test/test_opendirectory.py
CalendarServer/branches/users/cdaboo/pods/calendarserver/tap/caldav.py
CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_gateway.py
CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_principals.py
CalendarServer/branches/users/cdaboo/pods/conf/auth/accounts-test.xml
CalendarServer/branches/users/cdaboo/pods/conf/caldavd-apple.plist
CalendarServer/branches/users/cdaboo/pods/conf/caldavd-test.plist
CalendarServer/branches/users/cdaboo/pods/conf/caldavd.plist
CalendarServer/branches/users/cdaboo/pods/contrib/migration/calendarmigrator.py
CalendarServer/branches/users/cdaboo/pods/contrib/migration/test/test_migrator.py
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/config.plist
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/ical.py
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/population.py
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/profiles.py
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/sim.py
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/test_profiles.py
CalendarServer/branches/users/cdaboo/pods/doc/calendarserver_export.8
CalendarServer/branches/users/cdaboo/pods/python
CalendarServer/branches/users/cdaboo/pods/support/Makefile.Apple
CalendarServer/branches/users/cdaboo/pods/support/build.sh
CalendarServer/branches/users/cdaboo/pods/support/py.sh
CalendarServer/branches/users/cdaboo/pods/support/shell.sh
CalendarServer/branches/users/cdaboo/pods/testserver
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/adbapi2.py
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/model.py
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/syntax.py
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/test/test_sqlsyntax.py
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/test/test_adbapi2.py
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/util.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/config.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/appleopendirectory.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/augment.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/cachingdirectory.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_augment.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_opendirectory.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/resource.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/scheduling/ischedule.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/servers.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/stdconfig.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_config.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_upgrade.py
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/upgrade.py
CalendarServer/branches/users/cdaboo/pods/txdav/base/datastore/dbapiclient.py
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/calendar_store/ho/me/home1/calendar_1/1.ics
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/common.py
CalendarServer/branches/users/cdaboo/pods/txdav/common/datastore/sql_tables.py
Added Paths:
-----------
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/logger.py
Removed Paths:
-------------
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py
Property Changed:
----------------
CalendarServer/branches/users/cdaboo/pods/
CalendarServer/branches/users/cdaboo/pods/contrib/performance/sim
CalendarServer/branches/users/cdaboo/pods/support/build.sh
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/index_file.py
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/test_index_file.py
CalendarServer/branches/users/cdaboo/pods/txdav/carddav/datastore/index_file.py
CalendarServer/branches/users/cdaboo/pods/txdav/carddav/datastore/test/test_index_file.py
Property changes on: CalendarServer/branches/users/cdaboo/pods
___________________________________________________________________
Modified: svn:mergeinfo
- /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/users/cdaboo/batchupload-6699:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/pycalendar:7085-7206
/CalendarServer/branches/users/cdaboo/pycard:7227-7237
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/glyph/conn-limit:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/dalify:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect:6824-6876
/CalendarServer/branches/users/glyph/dont-start-postgres:6592-6614
/CalendarServer/branches/users/glyph/linux-tests:6893-6900
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7:6369-6445
/CalendarServer/branches/users/glyph/oracle:7106-7155
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/sharedpool:6490-6550
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/subtransactions:7248-7258
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
+ /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/users/cdaboo/batchupload-6699:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/pycalendar:7085-7206
/CalendarServer/branches/users/cdaboo/pycard:7227-7237
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/glyph/conn-limit:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/dalify:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect:6824-6876
/CalendarServer/branches/users/glyph/dont-start-postgres:6592-6614
/CalendarServer/branches/users/glyph/linux-tests:6893-6900
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7:6369-6445
/CalendarServer/branches/users/glyph/oracle:7106-7155
/CalendarServer/branches/users/glyph/oracle-nulls:7340-7351
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/sharedpool:6490-6550
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/subtransactions:7248-7258
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
/CalendarServer/trunk:7297-7364
Modified: CalendarServer/branches/users/cdaboo/pods/bin/caldavd
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/bin/caldavd 2011-04-27 20:10:49 UTC \
(rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/bin/caldavd 2011-04-27 21:09:24 UTC \
(rev 7377) @@ -51,7 +51,7 @@
return 0;
}
-for v in "" "2.6" "2.5"; do
+for v in "2.7" "2.6" "2.5" ""; do
for p in \
"${PYTHON:=}" \
"python${v}" \
Modified: CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/opendirectory.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/opendirectory.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/opendirectory.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -23,7 +23,23 @@
import dsattributes
import base64
from twext.python.log import Logger
+import Foundation
+
+def autoPooled(f):
+ """
+ A decorator which creates an autorelease pool and deletes it, causing it
+ to drain
+ """
+ def autoPooledFunction(*args, **kwds):
+ pool = Foundation.NSAutoreleasePool.alloc().init()
+ try:
+ return f(*args, **kwds)
+ finally:
+ del pool
+ return autoPooledFunction
+
+
log = Logger()
NUM_TRIES = 3
@@ -136,6 +152,7 @@
return names, encodings
+@autoPooled
def odInit(nodeName):
"""
Create an Open Directory object to operate on the specified directory service \
node name. @@ -167,6 +184,7 @@
+@autoPooled
def getNodeAttributes(directory, nodeName, attributes):
"""
Return key attributes for the specified directory node. The attributes
@@ -198,6 +216,7 @@
raise ODError(error)
+@autoPooled
def listAllRecordsWithAttributes_list(directory, recordType, attributes, count=0):
"""
List records in Open Directory, and return key attributes for each one.
@@ -212,7 +231,6 @@
for each record found, or C{None} otherwise.
"""
results = []
-
attributeNames, encodings = attributeNamesFromList(attributes)
tries = NUM_TRIES
@@ -246,6 +264,8 @@
log.error(error)
raise ODError(error)
+
+@autoPooled
def queryRecordsWithAttribute_list(directory, attr, value, matchType, casei, \
recordType, attributes, count=0): """
List records in Open Directory matching specified attribute/value, and return \
key attributes for each one. @@ -263,9 +283,7 @@
@return: C{list} containing a C{list} of C{str} (record name) and C{dict} \
attributes for each record found, or C{None} otherwise.
"""
-
results = []
-
attributeNames, encodings = attributeNamesFromList(attributes)
tries = NUM_TRIES
@@ -301,6 +319,7 @@
raise ODError(error)
+@autoPooled
def queryRecordsWithAttributes_list(directory, compound, casei, recordType, \
attributes, count=0): """
List records in Open Directory matching specified criteria, and return key \
attributes for each one. @@ -385,6 +404,7 @@
raise ODError(error)
+@autoPooled
def authenticateUserBasic(directory, nodeName, user, password):
"""
Authenticate a user with a password to Open Directory.
@@ -428,6 +448,7 @@
raise ODError(error)
+@autoPooled
def authenticateUserDigest(directory, nodeName, user, challenge, response, method):
"""
Authenticate using HTTP Digest credentials to Open Directory.
Modified: CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/test/test_opendirectory.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/test/test_opendirectory.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/test/test_opendirectory.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -898,3 +898,13 @@
(["a", "b"], {"b":"base64"}),
opendirectory.attributeNamesFromList(["a", ("b", "base64")])
)
+
+ def test_autoPooled(self):
+ """
+ Make sure no exception is raised by an autoPooled method
+ """
+ @opendirectory.autoPooled
+ def method(x):
+ return x + 1
+
+ self.assertEquals(2, method(1))
Modified: CalendarServer/branches/users/cdaboo/pods/calendarserver/tap/caldav.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/calendarserver/tap/caldav.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/calendarserver/tap/caldav.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -120,16 +120,35 @@
return (uid, gid)
-PARENT_ENVIRONMENT = {
- "PATH": os.environ.get("PATH", ""),
- "PYTHONPATH": os.environ.get("PYTHONPATH", ""),
- "LD_LIBRARY_PATH": os.environ.get("LD_LIBRARY_PATH", ""),
- "DYLD_LIBRARY_PATH": os.environ.get("DYLD_LIBRARY_PATH", ""),
-}
-if "KRB5_KTNAME" in os.environ:
- PARENT_ENVIRONMENT["KRB5_KTNAME"] = os.environ["KRB5_KTNAME"]
+def _computeEnvVars(parent):
+ """
+ Compute environment variables to be propagated to child processes.
+ """
+ result = {}
+ requiredVars = [
+ "PATH",
+ "PYTHONPATH",
+ "LD_LIBRARY_PATH",
+ "DYLD_LIBRARY_PATH",
+ ]
+ optionalVars = [
+ "KRB5_KTNAME",
+ "ORACLE_HOME",
+ ]
+
+ for varname in requiredVars:
+ result[varname] = parent.get(varname, "")
+ for varname in optionalVars:
+ if varname in parent:
+ result[varname] = parent[varname]
+ return result
+
+PARENT_ENVIRONMENT = _computeEnvVars(os.environ)
+
+
+
class CalDAVStatisticsProtocol (Protocol):
def connectionMade(self):
Modified: CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_gateway.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_gateway.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_gateway.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -15,6 +15,7 @@
##
import os
+import sys
from twext.python.plistlib import readPlistFromString
import xml
@@ -81,7 +82,7 @@
command = command.encode("utf-8")
sourceRoot = \
os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(__file__))))
- python = os.path.join(sourceRoot, "python")
+ python = sys.executable
gateway = os.path.join(sourceRoot, "bin", "calendarserver_command_gateway")
args = [python, gateway, "-f", self.configFileName]
Modified: CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_principals.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_principals.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_principals.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -15,6 +15,7 @@
##
import os
+import sys
from twext.python.filepath import CachingFilePath as FilePath
from twisted.internet import reactor
@@ -83,7 +84,7 @@
Run calendarserver_manage_principals, passing additional as args.
"""
sourceRoot = \
os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(__file__))))
- python = os.path.join(sourceRoot, "python")
+ python = sys.executable
script = os.path.join(sourceRoot, "bin", "calendarserver_manage_principals")
args = [python, script, "-f", self.configFileName]
Modified: CalendarServer/branches/users/cdaboo/pods/conf/auth/accounts-test.xml
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/conf/auth/accounts-test.xml 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/conf/auth/accounts-test.xml 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -38,7 +38,7 @@
<user>
<uid>wsanchez</uid>
<guid>wsanchez</guid>
- <email-address>wsanchez@apple.com</email-address>
+ <email-address>wsanchez@example.com</email-address>
<password>test</password>
<name>Wilfredo Sanchez Vega</name>
<first-name>Wilfredo</first-name>
@@ -47,7 +47,7 @@
<user>
<uid>cdaboo</uid>
<guid>cdaboo</guid>
- <email-address>cdaboo@apple.com</email-address>
+ <email-address>cdaboo@example.com</email-address>
<password>test</password>
<name>Cyrus Daboo</name>
<first-name>Cyrus</first-name>
@@ -56,7 +56,7 @@
<user>
<uid>sagen</uid>
<guid>sagen</guid>
- <email-address>sagen@apple.com</email-address>
+ <email-address>sagen@example.com</email-address>
<password>test</password>
<name>Morgen Sagen</name>
<first-name>Morgen</first-name>
@@ -65,7 +65,7 @@
<user>
<uid>dre</uid>
<guid>andre</guid>
- <email-address>dre@apple.com</email-address>
+ <email-address>dre@example.com</email-address>
<password>test</password>
<name>Andre LaBranche</name>
<first-name>Andre</first-name>
@@ -74,12 +74,21 @@
<user>
<uid>glyph</uid>
<guid>glyph</guid>
- <email-address>glyph@apple.com</email-address>
+ <email-address>glyph@example.com</email-address>
<password>test</password>
<name>Glyph Lefkowitz</name>
<first-name>Glyph</first-name>
<last-name>Lefkowitz</last-name>
</user>
+ <user>
+ <uid>i18nuser</uid>
+ <guid>i18nuser</guid>
+ <email-address>i18nuser@example.com</email-address>
+ <password>i18nuser</password>
+ <name>ま </name>
+ <first-name>ま</first-name>
+ <last-name> </last-name>
+ </user>
<user repeat="99">
<uid>user%02d</uid>
<uid>User %02d</uid>
Modified: CalendarServer/branches/users/cdaboo/pods/conf/caldavd-apple.plist
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/conf/caldavd-apple.plist 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/conf/caldavd-apple.plist 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -171,21 +171,6 @@
A variety of directory services are available for use.
-->
- <!-- XML File Directory Service -->
- <!--
- <key>DirectoryService</key>
- <dict>
- <key>type</key>
- <string>twistedcaldav.directory.xmlfile.XMLDirectoryService</string>
-
- <key>params</key>
- <dict>
- <key>xmlFile</key>
- <string>accounts.xml</string>
- </dict>
- </dict>
- -->
-
<!-- Open Directory Service (Mac OS X) -->
<key>DirectoryService</key>
<dict>
Modified: CalendarServer/branches/users/cdaboo/pods/conf/caldavd-test.plist
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/conf/caldavd-test.plist 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/conf/caldavd-test.plist 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -94,6 +94,14 @@
<key>ServerRoot</key>
<string>./data</string>
+ <!-- Database connection -->
+ <!--
+ <key>DBType</key>
+ <string>postgres</string>
+ <key>DSN</key>
+ <string>:caldav:caldav:::</string>
+ -->
+
<!-- Data root -->
<key>DataRoot</key>
<string>Data</string>
Modified: CalendarServer/branches/users/cdaboo/pods/conf/caldavd.plist
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/conf/caldavd.plist 2011-04-27 20:10:49 \
UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/conf/caldavd.plist 2011-04-27 21:09:24 \
UTC (rev 7377) @@ -80,6 +80,14 @@
<key>ServerRoot</key>
<string>/var/db/caldavd</string>
+ <!-- Database connection -->
+ <!--
+ <key>DBType</key>
+ <string>postgres</string>
+ <key>DSN</key>
+ <string>:caldav:caldav:::</string>
+ -->
+
<!-- Data root -->
<key>DataRoot</key>
<string>Data</string>
Deleted: CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py
===================================================================
--- CalendarServer/trunk/contrib/certupdate/__init__.py 2011-04-27 18:04:16 UTC (rev \
7364)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -1,15 +0,0 @@
-##
-# Copyright (c) 2011 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
Copied: CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py \
(from rev 7364, CalendarServer/trunk/contrib/certupdate/__init__.py) \
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py \
(rev 0)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -0,0 +1,15 @@
+##
+# Copyright (c) 2011 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
Deleted: CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py
===================================================================
--- CalendarServer/trunk/contrib/certupdate/calendarcertupdate.py 2011-04-27 18:04:16 \
UTC (rev 7364)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -1,154 +0,0 @@
-#!/usr/bin/env python
-#
-# CertUpdate script for calendar / addresbook service.
-#
-# This script will be called with the path to the cert file in
-# /etc/certificates and also the keychain persistent reference if
-# we have one available. For the remove command, the handler
-# returns 0 = don't care, 1 = please keep, 2 = an error occurred.
-# For the replace command the handler returns
-# 0 = don't care/ cert replaced, 2 = an error occurred.
-#
-# Copyright (c) 2011 Apple Inc. All Rights Reserved.
-#
-# IMPORTANT NOTE: This file is licensed only for use on Apple-labeled
-# computers and is subject to the terms and conditions of the Apple
-# Software License Agreement accompanying the package this file is a
-# part of. You may not port this file to another platform without
-# Apple's written consent.
-
-import datetime
-import os
-import subprocess
-import sys
-
-from plistlib import readPlist, readPlistFromString, writePlist
-
-LOG = "/var/log/caldavd/certupdate.log"
-SERVICE_NAME = "calendar"
-CALDAVD_PLIST = "/etc/caldavd/caldavd.plist"
-SERVER_ADMIN = "/usr/sbin/serveradmin"
-
-def main():
-
- log(sys.argv)
- numArgs = len(sys.argv) - 1
- if numArgs == 3:
- if sys.argv[1] != "remove":
- die("Bad command line; 'remove' expected", 2)
- if isThisMyCert(CALDAVD_PLIST, sys.argv[2]):
- die("%s is in use by calendar" % (sys.argv[2],), 1)
- else:
- die("%s is not in use by calendar" % (sys.argv[2],), 0)
-
- elif numArgs == 5:
- if sys.argv[1] != "replace":
- die("Bad command line; 'replace' expected", 2)
- if isThisMyCert(CALDAVD_PLIST, sys.argv[2]):
- try:
- replaceCert(CALDAVD_PLIST, sys.argv[4])
- restartService(CALDAVD_PLIST)
- die("Replaced calendar cert with %s" % (sys.argv[4],), 0)
- except Exception, e:
- die("Error replacing calendar cert with %s: %s" % (sys.argv[4], e), \
2)
-
- else:
- die("%s is not in use by calendar" % (sys.argv[2],), 0)
-
- else:
- # Wrong number of args
- die("Bad command line; incorrect number of arguments", 2)
-
-
-def getMyCert(plistPath):
- """
- Return SSLCertificate from the plist at plistPath
- """
- plist = readPlist(plistPath)
- return plist.get("SSLCertificate", None)
-
-
-def isThisMyCert(plistPath, otherCert):
- """
- Compare otherCert against SSLCertificate from the plist at plistPath
- """
- myCert = getMyCert(plistPath)
- return otherCert == myCert
-
-
-def replaceCert(plistPath, otherCert):
- """
- Replace SSL settings in plist at plistPath based on otherCert path
- """
- log("Reading plist %s" % (plistPath,))
- plist = readPlist(plistPath)
- log("Read in plist %s" % (plistPath,))
-
- basePath = otherCert[:-len("cert.pem")]
- log("Base path is %s" % (basePath,))
-
- log("Setting SSLCertificate to %s" % (otherCert,))
- plist["SSLCertificate"] = otherCert
-
- otherChain = basePath + "chain.pem"
- log("Setting SSLAuthorityChain to %s" % (otherChain,))
- plist["SSLAuthorityChain"] = otherChain
-
- otherKey = basePath + "key.pem"
- log("Setting SSLPrivateKey to %s" % (otherKey,))
- plist["SSLPrivateKey"] = otherKey
-
- log("Writing plist %s" % (plistPath,))
- writePlist(plist, plistPath)
-
-
-def restartService(plistPath):
- """
- Use serveradmin to restart the service.
- """
-
- plist = readPlist(plistPath)
-
- if not plist.get("EnableSSL", False):
- log("SSL is not enabled, so no need to restart")
- return
-
- if plist.get("EnableCardDAV", False):
- log("Stopping addressbook service via serveradmin")
- ret = subprocess.call([SERVER_ADMIN, "stop", "addressbook"])
- log("serveradmin exited with %d" % (ret,))
- log("Starting addressbook service via serveradmin")
- ret = subprocess.call([SERVER_ADMIN, "start", "addressbook"])
- log("serveradmin exited with %d" % (ret,))
- elif plist.get("EnableCalDAV", False):
- log("Stopping calendar service via serveradmin")
- ret = subprocess.call([SERVER_ADMIN, "stop", "calendar"])
- log("serveradmin exited with %d" % (ret,))
- log("Starting calendar service via serveradmin")
- ret = subprocess.call([SERVER_ADMIN, "start", "calendar"])
- log("serveradmin exited with %d" % (ret,))
- else:
- log("Neither calendar nor addressbook services were running")
-
-
-def log(msg):
- try:
- timestamp = datetime.datetime.now().strftime("%b %d %H:%M:%S")
- msg = "calendarcertupdate: %s %s" % (timestamp, msg)
- with open(LOG, 'a') as output:
- output.write("%s\n" % (msg,)) # so it appears in our log
- except IOError:
- # Could not write to log
- pass
-
-
-def die(msg, exitCode):
- """
- Log msg and exit with exitCode
- """
- log(msg)
- sys.exit(exitCode)
-
-
-if __name__ == '__main__':
- main()
Copied: CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py \
(from rev 7364, CalendarServer/trunk/contrib/certupdate/calendarcertupdate.py) \
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py \
(rev 0)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -0,0 +1,154 @@
+#!/usr/bin/env python
+#
+# CertUpdate script for calendar / addresbook service.
+#
+# This script will be called with the path to the cert file in
+# /etc/certificates and also the keychain persistent reference if
+# we have one available. For the remove command, the handler
+# returns 0 = don't care, 1 = please keep, 2 = an error occurred.
+# For the replace command the handler returns
+# 0 = don't care/ cert replaced, 2 = an error occurred.
+#
+# Copyright (c) 2011 Apple Inc. All Rights Reserved.
+#
+# IMPORTANT NOTE: This file is licensed only for use on Apple-labeled
+# computers and is subject to the terms and conditions of the Apple
+# Software License Agreement accompanying the package this file is a
+# part of. You may not port this file to another platform without
+# Apple's written consent.
+
+import datetime
+import os
+import subprocess
+import sys
+
+from plistlib import readPlist, readPlistFromString, writePlist
+
+LOG = "/var/log/caldavd/certupdate.log"
+SERVICE_NAME = "calendar"
+CALDAVD_PLIST = "/etc/caldavd/caldavd.plist"
+SERVER_ADMIN = "/usr/sbin/serveradmin"
+
+def main():
+
+ log(sys.argv)
+ numArgs = len(sys.argv) - 1
+ if numArgs == 3:
+ if sys.argv[1] != "remove":
+ die("Bad command line; 'remove' expected", 2)
+ if isThisMyCert(CALDAVD_PLIST, sys.argv[2]):
+ die("%s is in use by calendar" % (sys.argv[2],), 1)
+ else:
+ die("%s is not in use by calendar" % (sys.argv[2],), 0)
+
+ elif numArgs == 5:
+ if sys.argv[1] != "replace":
+ die("Bad command line; 'replace' expected", 2)
+ if isThisMyCert(CALDAVD_PLIST, sys.argv[2]):
+ try:
+ replaceCert(CALDAVD_PLIST, sys.argv[4])
+ restartService(CALDAVD_PLIST)
+ die("Replaced calendar cert with %s" % (sys.argv[4],), 0)
+ except Exception, e:
+ die("Error replacing calendar cert with %s: %s" % (sys.argv[4], e), \
2) +
+ else:
+ die("%s is not in use by calendar" % (sys.argv[2],), 0)
+
+ else:
+ # Wrong number of args
+ die("Bad command line; incorrect number of arguments", 2)
+
+
+def getMyCert(plistPath):
+ """
+ Return SSLCertificate from the plist at plistPath
+ """
+ plist = readPlist(plistPath)
+ return plist.get("SSLCertificate", None)
+
+
+def isThisMyCert(plistPath, otherCert):
+ """
+ Compare otherCert against SSLCertificate from the plist at plistPath
+ """
+ myCert = getMyCert(plistPath)
+ return otherCert == myCert
+
+
+def replaceCert(plistPath, otherCert):
+ """
+ Replace SSL settings in plist at plistPath based on otherCert path
+ """
+ log("Reading plist %s" % (plistPath,))
+ plist = readPlist(plistPath)
+ log("Read in plist %s" % (plistPath,))
+
+ basePath = otherCert[:-len("cert.pem")]
+ log("Base path is %s" % (basePath,))
+
+ log("Setting SSLCertificate to %s" % (otherCert,))
+ plist["SSLCertificate"] = otherCert
+
+ otherChain = basePath + "chain.pem"
+ log("Setting SSLAuthorityChain to %s" % (otherChain,))
+ plist["SSLAuthorityChain"] = otherChain
+
+ otherKey = basePath + "key.pem"
+ log("Setting SSLPrivateKey to %s" % (otherKey,))
+ plist["SSLPrivateKey"] = otherKey
+
+ log("Writing plist %s" % (plistPath,))
+ writePlist(plist, plistPath)
+
+
+def restartService(plistPath):
+ """
+ Use serveradmin to restart the service.
+ """
+
+ plist = readPlist(plistPath)
+
+ if not plist.get("EnableSSL", False):
+ log("SSL is not enabled, so no need to restart")
+ return
+
+ if plist.get("EnableCardDAV", False):
+ log("Stopping addressbook service via serveradmin")
+ ret = subprocess.call([SERVER_ADMIN, "stop", "addressbook"])
+ log("serveradmin exited with %d" % (ret,))
+ log("Starting addressbook service via serveradmin")
+ ret = subprocess.call([SERVER_ADMIN, "start", "addressbook"])
+ log("serveradmin exited with %d" % (ret,))
+ elif plist.get("EnableCalDAV", False):
+ log("Stopping calendar service via serveradmin")
+ ret = subprocess.call([SERVER_ADMIN, "stop", "calendar"])
+ log("serveradmin exited with %d" % (ret,))
+ log("Starting calendar service via serveradmin")
+ ret = subprocess.call([SERVER_ADMIN, "start", "calendar"])
+ log("serveradmin exited with %d" % (ret,))
+ else:
+ log("Neither calendar nor addressbook services were running")
+
+
+def log(msg):
+ try:
+ timestamp = datetime.datetime.now().strftime("%b %d %H:%M:%S")
+ msg = "calendarcertupdate: %s %s" % (timestamp, msg)
+ with open(LOG, 'a') as output:
+ output.write("%s\n" % (msg,)) # so it appears in our log
+ except IOError:
+ # Could not write to log
+ pass
+
+
+def die(msg, exitCode):
+ """
+ Log msg and exit with exitCode
+ """
+ log(msg)
+ sys.exit(exitCode)
+
+
+if __name__ == '__main__':
+ main()
Deleted: CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py
===================================================================
--- CalendarServer/trunk/contrib/certupdate/test/__init__.py 2011-04-27 18:04:16 UTC \
(rev 7364)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -1,15 +0,0 @@
-##
-# Copyright (c) 2011 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
Copied: CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py \
(from rev 7364, CalendarServer/trunk/contrib/certupdate/test/__init__.py) \
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py \
(rev 0)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -0,0 +1,15 @@
+##
+# Copyright (c) 2011 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
Deleted: CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py
===================================================================
--- CalendarServer/trunk/contrib/certupdate/test/test_certupdate.py 2011-04-27 \
18:04:16 UTC (rev 7364)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -1,65 +0,0 @@
-##
-# Copyright (c) 2011 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
-
-from tempfile import mkstemp
-import os
-import twistedcaldav.test.util
-from plistlib import readPlist
-from contrib.certupdate.calendarcertupdate import (
- getMyCert, isThisMyCert, replaceCert
-)
-
-samplePlist = """<?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" \
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0">
-<dict>
- <key>SSLAuthorityChain</key>
- <string>/etc/certificates/original.chain.pem</string>
- <key>SSLCertificate</key>
- <string>/etc/certificates/original.cert.pem</string>
- <key>SSLPrivateKey</key>
- <string>/etc/certificates/original.key.pem</string>
-</dict>
-</plist>
-"""
-
-class CertUpdateTests(twistedcaldav.test.util.TestCase):
- """
- Calendar Server Certificate Update Tests
- """
-
- def setUp(self):
- self.fd, self.path = mkstemp(suffix=".plist")
- out = os.fdopen(self.fd, "w")
- out.write(samplePlist)
- out.close()
-
- def tearDown(self):
- os.remove(self.path)
-
- def test_getMyCert(self):
- self.assertEquals("/etc/certificates/original.cert.pem", \
getMyCert(self.path))
-
- def test_isThisMyCert(self):
- self.assertTrue(isThisMyCert(self.path, \
"/etc/certificates/original.cert.pem"))
- self.assertFalse(isThisMyCert(self.path, "/etc/certificates/not.cert.pem"))
-
- def test_replaceCert(self):
- replaceCert(self.path, "/etc/certificates/new.cert.pem")
- plist = readPlist(self.path)
- self.assertEquals(plist["SSLAuthorityChain"], \
"/etc/certificates/new.chain.pem")
- self.assertEquals(plist["SSLCertificate"], "/etc/certificates/new.cert.pem")
- self.assertEquals(plist["SSLPrivateKey"], "/etc/certificates/new.key.pem")
Copied: CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py \
(from rev 7364, CalendarServer/trunk/contrib/certupdate/test/test_certupdate.py) \
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py \
(rev 0)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -0,0 +1,65 @@
+##
+# Copyright (c) 2011 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from tempfile import mkstemp
+import os
+import twistedcaldav.test.util
+from plistlib import readPlist
+from contrib.certupdate.calendarcertupdate import (
+ getMyCert, isThisMyCert, replaceCert
+)
+
+samplePlist = """<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" \
"http://www.apple.com/DTDs/PropertyList-1.0.dtd"> +<plist version="1.0">
+<dict>
+ <key>SSLAuthorityChain</key>
+ <string>/etc/certificates/original.chain.pem</string>
+ <key>SSLCertificate</key>
+ <string>/etc/certificates/original.cert.pem</string>
+ <key>SSLPrivateKey</key>
+ <string>/etc/certificates/original.key.pem</string>
+</dict>
+</plist>
+"""
+
+class CertUpdateTests(twistedcaldav.test.util.TestCase):
+ """
+ Calendar Server Certificate Update Tests
+ """
+
+ def setUp(self):
+ self.fd, self.path = mkstemp(suffix=".plist")
+ out = os.fdopen(self.fd, "w")
+ out.write(samplePlist)
+ out.close()
+
+ def tearDown(self):
+ os.remove(self.path)
+
+ def test_getMyCert(self):
+ self.assertEquals("/etc/certificates/original.cert.pem", \
getMyCert(self.path)) +
+ def test_isThisMyCert(self):
+ self.assertTrue(isThisMyCert(self.path, \
"/etc/certificates/original.cert.pem")) + \
self.assertFalse(isThisMyCert(self.path, "/etc/certificates/not.cert.pem")) +
+ def test_replaceCert(self):
+ replaceCert(self.path, "/etc/certificates/new.cert.pem")
+ plist = readPlist(self.path)
+ self.assertEquals(plist["SSLAuthorityChain"], \
"/etc/certificates/new.chain.pem") + \
self.assertEquals(plist["SSLCertificate"], "/etc/certificates/new.cert.pem") + \
self.assertEquals(plist["SSLPrivateKey"], "/etc/certificates/new.key.pem")
Modified: CalendarServer/branches/users/cdaboo/pods/contrib/migration/calendarmigrator.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/migration/calendarmigrator.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/migration/calendarmigrator.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -29,7 +29,7 @@
import subprocess
import sys
-from plistlib import readPlist, writePlist
+from plistlib import readPlist, readPlistFromString, writePlist
CALDAV_LAUNCHD_KEY = "org.calendarserver.calendarserver"
CARDDAV_LAUNCHD_KEY = "org.addressbookserver.addressbookserver"
@@ -45,6 +45,7 @@
RESOURCE_MIGRATION_TRIGGER = "trigger_resource_migration"
SERVER_ADMIN = "/usr/sbin/serveradmin"
LAUNCHCTL = "/bin/launchctl"
+DITTO = "/usr/bin/ditto"
verbatimKeys = """
@@ -214,13 +215,50 @@
if enableCalDAV:
unloadService(options, CALDAV_LAUNCHD_KEY)
- newServerRootValue = migrateData(options)
- migrateConfiguration(options, newServerRootValue, enableCalDAV,
- enableCardDAV)
+ # Pull values out of previous plists
+ (
+ oldServerRootValue,
+ oldCalDocumentRootValue,
+ oldCalDataRootValue,
+ oldABDocumentRootValue,
+ uid,
+ gid
+ ) = examinePreviousSystem(
+ options.sourceRoot,
+ options.targetRoot
+ )
+ # Copy data as needed
+ (
+ newServerRoot,
+ newServerRootValue,
+ newDocumentRootValue,
+ newDataRootValue
+ ) = relocateData(
+ options.sourceRoot,
+ options.targetRoot,
+ oldServerRootValue,
+ oldCalDocumentRootValue,
+ oldCalDataRootValue,
+ oldABDocumentRootValue,
+ uid,
+ gid
+ )
+
+ # Combine old and new plists
+ migrateConfiguration(
+ options,
+ newServerRootValue,
+ newDocumentRootValue,
+ newDataRootValue,
+ enableCalDAV,
+ enableCardDAV
+ )
+
configureNotifications()
- triggerResourceMigration(newServerRootValue)
+ triggerResourceMigration(newServerRoot)
+
setRunState(options, enableCalDAV, enableCardDAV)
else:
@@ -310,7 +348,8 @@
open(triggerPath, "w").close()
-def migrateConfiguration(options, newServerRootValue, enableCalDAV, enableCardDAV):
+def migrateConfiguration(options, newServerRootValue,
+ newDocumentRootValue, newDataRootValue, enableCalDAV, enableCardDAV):
"""
Copy files/directories/symlinks from previous system's /etc/caldavd
and /etc/carddavd
@@ -325,7 +364,6 @@
log("New configuration directory does not exist: %s" % (newConfigDir,))
return
-
for configDir in (CALDAVD_CONFIG_DIR, CARDDAVD_CONFIG_DIR):
oldConfigDir = os.path.join(options.sourceRoot, configDir)
@@ -387,8 +425,8 @@
mergePlist(oldCalDAVDPlist, oldCardDAVDPlist, newCalDAVDPlist)
newCalDAVDPlist["ServerRoot"] = newServerRootValue
- newCalDAVDPlist["DocumentRoot"] = "Documents"
- newCalDAVDPlist["DataRoot"] = "Data"
+ newCalDAVDPlist["DocumentRoot"] = newDocumentRootValue
+ newCalDAVDPlist["DataRoot"] = newDataRootValue
newCalDAVDPlist["EnableCalDAV"] = enableCalDAV
newCalDAVDPlist["EnableCardDAV"] = enableCardDAV
@@ -516,250 +554,207 @@
def log(msg):
try:
+ timestamp = datetime.datetime.now().strftime("%b %d %H:%M:%S")
+ msg = "calendarmigrator: %s %s" % (timestamp, msg)
+ print msg # so it appears in Setup.log
with open(LOG, 'a') as output:
- timestamp = datetime.datetime.now().strftime("%b %d %H:%M:%S")
- msg = "calendarmigrator: %s %s" % (timestamp, msg)
output.write("%s\n" % (msg,)) # so it appears in our log
- print msg # so it appears in Setup.log
except IOError:
# Could not write to log
pass
-def migrateData(options):
+def examinePreviousSystem(sourceRoot, targetRoot, diskAccessor=None):
"""
Examines the old caldavd.plist and carddavd.plist to see where data
- lives in the previous system. If there is old data, calls relocateData( )
+ lives in the previous system.
"""
- oldCalDocuments = None
- oldCalData = None
- oldABDocuments = None
- calendarDataInDefaultLocation = True
- addressbookDataInDefaultLocation = True
- uid = -1
- gid = -1
- newServerRoot = None # actual path
- newServerRootValue = NEW_SERVER_ROOT # value to put in plist
+ if diskAccessor is None:
+ diskAccessor = DiskAccessor()
- oldCalConfigDir = os.path.join(options.sourceRoot, CALDAVD_CONFIG_DIR)
+ oldServerRootValue = None
+ oldCalDocumentRootValue = None
+ oldCalDataRootValue = None
+ oldABDocumentRootValue = None
+
+ # Get uid and gid from new caldavd.plist
+ newCalConfigDir = os.path.join(targetRoot, CALDAVD_CONFIG_DIR)
+ newCalPlistPath = os.path.join(newCalConfigDir, CALDAVD_PLIST)
+ if diskAccessor.exists(newCalPlistPath):
+ contents = diskAccessor.readFile(newCalPlistPath)
+ newCalPlist = readPlistFromString(contents)
+ uid, gid = getServerIDs(newCalPlist)
+ log("ServerIDs from %s: %d, %d" % (newCalPlistPath, uid, gid))
+ else:
+ uid = gid = -1
+ log("Can't find new calendar plist at %s" % (newCalPlistPath,))
+
+ # Try and read old caldavd.plist
+ oldCalConfigDir = os.path.join(sourceRoot, CALDAVD_CONFIG_DIR)
oldCalPlistPath = os.path.join(oldCalConfigDir, CALDAVD_PLIST)
- if os.path.exists(oldCalPlistPath):
- oldCalPlist = readPlist(oldCalPlistPath)
- uid, gid = getServerIDs(oldCalPlist)
- log("ServerIDs: %d, %d" % (uid, gid))
+ if diskAccessor.exists(oldCalPlistPath):
+ contents = diskAccessor.readFile(oldCalPlistPath)
+ oldCalPlist = readPlistFromString(contents)
+ log("Found previous caldavd plist at %s" % (oldCalPlistPath,))
+
+ oldServerRootValue = oldCalPlist.get("ServerRoot", None)
+ oldCalDocumentRootValue = oldCalPlist.get("DocumentRoot", None)
+ oldCalDataRootValue = oldCalPlist.get("DataRoot", None)
+
else:
log("Can't find previous calendar plist at %s" % (oldCalPlistPath,))
oldCalPlist = None
- newCalConfigDir = os.path.join(options.targetRoot, CALDAVD_CONFIG_DIR)
- newCalPlistPath = os.path.join(newCalConfigDir, CALDAVD_PLIST)
- if os.path.exists(newCalPlistPath):
- newCalPlist = readPlist(newCalPlistPath)
- uid, gid = getServerIDs(newCalPlist)
- log("ServerIDs: %d, %d" % (uid, gid))
+ # Try and read old carddavd.plist
+ oldABConfigDir = os.path.join(sourceRoot, CARDDAVD_CONFIG_DIR)
+ oldABPlistPath = os.path.join(oldABConfigDir, CARDDAVD_PLIST)
+ if diskAccessor.exists(oldABPlistPath):
+ contents = diskAccessor.readFile(oldABPlistPath)
+ oldABPlist = readPlistFromString(contents)
+ log("Found previous carddavd plist at %s" % (oldABPlistPath,))
- oldABConfigDir = os.path.join(options.sourceRoot, CARDDAVD_CONFIG_DIR)
- oldABPlistPath = os.path.join(oldABConfigDir, CARDDAVD_PLIST)
- if os.path.exists(oldABPlistPath):
- oldABPlist = readPlist(oldABPlistPath)
+ oldABDocumentRootValue = oldABPlist.get("DocumentRoot", None)
else:
- log("Can't find previous addressbook plist at %s" % (oldABPlistPath,))
+ log("Can't find previous carddavd plist at %s" % (oldABPlistPath,))
oldABPlist = None
- if oldCalPlist is not None:
- # See if there is actually any calendar data
+ return (
+ oldServerRootValue,
+ oldCalDocumentRootValue,
+ oldCalDataRootValue,
+ oldABDocumentRootValue,
+ uid,
+ gid
+ )
- oldDocumentRoot = oldCalPlist["DocumentRoot"]
- if oldDocumentRoot.rstrip("/") != "/Library/CalendarServer/Documents":
- log("Calendar data in non-standard location: %s" % (oldDocumentRoot,))
- calendarDataInDefaultLocation = False
- else:
- log("Calendar data in standard location: %s" % (oldDocumentRoot,))
- oldDataRoot = oldCalPlist["DataRoot"]
+def relocateData(sourceRoot, targetRoot, oldServerRootValue,
+ oldCalDocumentRootValue, oldCalDataRootValue, oldABDocumentRootValue,
+ uid, gid, diskAccessor=None):
+ """
+ Copy data from sourceRoot to targetRoot, except when data is on another
+ volume in which case we just refer to it there.
+ """
- oldCalendarsPath = os.path.join(oldDocumentRoot, "calendars")
- if os.path.exists(oldCalendarsPath):
- # There is calendar data
- oldCalDocuments = oldDocumentRoot
- oldCalData = oldDataRoot
- log("Calendar data to migrate from %s and %s" %
- (oldCalDocuments, oldCalData))
+ if diskAccessor is None:
+ diskAccessor = DiskAccessor()
- if calendarDataInDefaultLocation:
- newServerRoot = absolutePathWithRoot(options.targetRoot,
- NEW_SERVER_ROOT)
- newServerRootValue = NEW_SERVER_ROOT
- else:
- newServerRoot = absolutePathWithRoot(options.targetRoot,
- oldDocumentRoot)
- newServerRootValue = oldDocumentRoot
- else:
- log("No calendar data to migrate")
+ log("RelocateData: sourceRoot=%s, targetRoot=%s, oldServerRootValue=%s, \
oldCalDocumentRootValue=%s, oldCalDataRootValue=%s, oldABDocumentRootValue=%s, \
uid=%d, gid=%d" % (sourceRoot, targetRoot, oldServerRootValue, \
oldCalDocumentRootValue, oldCalDataRootValue, oldABDocumentRootValue, uid, gid))
- if oldABPlist is not None:
- # See if there is actually any addressbook data
- oldDocumentRoot = oldABPlist["DocumentRoot"]
- if oldDocumentRoot.rstrip("/") != "/Library/AddressBookServer/Documents":
- log("AddressBook data in non-standard location: %s" % \
(oldDocumentRoot,))
- addressbookDataInDefaultLocation = False
+ if oldServerRootValue:
+ newServerRootValue = oldServerRootValue
+ # Source is Lion; see if ServerRoot refers to an external volume
+ # or a directory in sourceRoot
+ if diskAccessor.exists(oldServerRootValue):
+ # refers to an external volume
+ newServerRoot = newServerRootValue
+ elif diskAccessor.exists(os.path.join(sourceRoot, oldServerRootValue)):
+ # refers to a directory on sourceRoot
+ newServerRoot = absolutePathWithRoot(targetRoot, newServerRootValue)
else:
- log("AddressBook data in standard location: %s" % (oldDocumentRoot,))
+ # It doesn't exist, so use default
+ newServerRootValue = NEW_SERVER_ROOT
+ newServerRoot = absolutePathWithRoot(targetRoot, newServerRootValue)
- oldAddressbooksPath = os.path.join(oldDocumentRoot, "addressbooks")
- if os.path.exists(oldAddressbooksPath):
- # There is addressbook data
- oldABDocuments = oldDocumentRoot
- log("AddressBook data to migrate from %s" % (oldABDocuments,))
+ # If there was an old ServerRoot value, process DocumentRoot and
+ # DataRoot because those could be relative to ServerRoot
+ oldCalDocumentRootValueProcessed = os.path.join(oldServerRootValue,
+ oldCalDocumentRootValue)
+ oldCalDataRootValueProcessed = os.path.join(oldServerRootValue,
+ oldCalDataRootValue)
+ else:
+ newServerRootValue = NEW_SERVER_ROOT
+ newServerRoot = absolutePathWithRoot(targetRoot, newServerRootValue)
+ oldCalDocumentRootValueProcessed = oldCalDocumentRootValue
+ oldCalDataRootValueProcessed = oldCalDataRootValue
- if newServerRoot is None:
- # don't override server root computed from calendar
- if addressbookDataInDefaultLocation:
- newServerRoot = absolutePathWithRoot(options.targetRoot,
- NEW_SERVER_ROOT)
- newServerRootValue = NEW_SERVER_ROOT
- else:
- newServerRoot = absolutePathWithRoot(options.targetRoot,
- oldDocumentRoot)
- newServerRootValue = oldDocumentRoot
- else:
- log("No addressbook data to migrate")
+ # Set default values for these, possibly overridden below:
+ newDocumentRootValue = "Documents"
+ newDocumentRoot = absolutePathWithRoot(
+ targetRoot,
+ os.path.join(newServerRootValue, newDocumentRootValue)
+ )
+ newDataRootValue = "Data"
+ newDataRoot = absolutePathWithRoot(
+ targetRoot,
+ os.path.join(newServerRootValue, newDataRootValue)
+ )
- if (oldCalDocuments or oldABDocuments) and newServerRoot:
- relocateData(oldCalDocuments, oldCalData, oldABDocuments, uid, gid,
- calendarDataInDefaultLocation, addressbookDataInDefaultLocation,
- newServerRoot)
+ # Old Calendar DocumentRoot
+ if oldCalDocumentRootValueProcessed:
+ if diskAccessor.exists(oldCalDocumentRootValueProcessed):
+ # Must be on an external volume if we see it existing at the point
+ # so don't copy it
+ newDocumentRoot = newDocumentRootValue = \
oldCalDocumentRootValueProcessed + elif \
diskAccessor.exists(absolutePathWithRoot(sourceRoot, \
oldCalDocumentRootValueProcessed)): + diskAccessor.ditto(
+ absolutePathWithRoot(sourceRoot, oldCalDocumentRootValueProcessed),
+ newDocumentRoot
+ )
+ diskAccessor.chown(newDocumentRoot, uid, gid, recursive=True)
- return newServerRootValue
+ # Old Calendar DataRoot
+ if oldCalDataRootValueProcessed:
+ if diskAccessor.exists(oldCalDataRootValueProcessed):
+ # Must be on an external volume if we see it existing at the point
+ # so don't copy it
+ newDataRootValue = oldCalDataRootValueProcessed
+ elif diskAccessor.exists(
+ absolutePathWithRoot(sourceRoot, oldCalDataRootValueProcessed)
+ ):
+ diskAccessor.ditto(
+ absolutePathWithRoot(sourceRoot, oldCalDataRootValueProcessed),
+ newDataRoot
+ )
+ diskAccessor.chown(newDataRoot, uid, gid, recursive=True)
-def relocateData(oldCalDocuments, oldCalData, oldABDocuments, uid, gid,
- calendarDataInDefaultLocation, addressbookDataInDefaultLocation,
- newServerRoot):
- """
- Relocates existing calendar data to the new default location iff the data
- was previously in the old default location; otherwise the old calendar
- DocumentRoot becomes the new ServerRoot directory, the contents of the
- old DocumentRoot are moved into ServerRoot/Documents and the contents of
- old DataRoot are copied/moved into ServerRoot/Data. If there is addressbook
- data, a symlink is created as ServerRoot/Documents/addressbooks pointing
- to the old addressbook directory so that the import-to-PostgreSQL will
- find it.
- """
+ # Old AddressBook DocumentRoot
+ if oldABDocumentRootValue:
+ newAddressBooks = os.path.join(newDocumentRoot, "addressbooks")
+ if diskAccessor.exists(oldABDocumentRootValue):
+ # Must be on an external volume if we see it existing at the point
+ diskAccessor.ditto(
+ os.path.join(oldABDocumentRootValue, "addressbooks"),
+ newAddressBooks
+ )
+ elif diskAccessor.exists(
+ absolutePathWithRoot(sourceRoot, oldABDocumentRootValue)
+ ):
+ diskAccessor.ditto(
+ absolutePathWithRoot(
+ sourceRoot,
+ os.path.join(oldABDocumentRootValue, "addressbooks")
+ ),
+ os.path.join(newDocumentRoot, "addressbooks")
+ )
+ if diskAccessor.exists(newAddressBooks):
+ diskAccessor.chown(newAddressBooks, uid, gid, recursive=True)
- log("RelocateData: cal documents=%s, cal data=%s, ab documents=%s, new server \
root=%s"
- % (oldCalDocuments, oldCalData, oldABDocuments, newServerRoot))
- if oldCalDocuments and os.path.exists(oldCalDocuments):
+ newServerRootValue, newDocumentRootValue = relativize(newServerRootValue,
+ newDocumentRootValue)
+ newServerRootValue, newDataRootValue = relativize(newServerRootValue,
+ newDataRootValue)
- if calendarDataInDefaultLocation:
- # We're in the default location, relocate to new location
- newCalDocuments = os.path.join(newServerRoot, "Documents")
- if not os.path.exists(newCalDocuments):
- os.mkdir(newCalDocuments)
- newCalData = os.path.join(newServerRoot, "Data")
- if not os.path.exists(newCalData):
- os.mkdir(newCalData)
- if os.path.exists(oldCalDocuments):
- # Move evertying from oldCalDocuments
- for item in list(os.listdir(oldCalDocuments)):
- source = os.path.join(oldCalDocuments, item)
- dest = os.path.join(newCalDocuments, item)
- log("Relocating %s to %s" % (source, dest))
- os.rename(source, dest)
- else:
- log("Warning: %s does not exist; nothing to migrate" % \
(oldCalDocuments,))
- else:
- # The admin has moved calendar data to a non-standard location so
- # we're going to leave it there, but move things down a level so
- # that the old DocumentRoot becomes new ServerRoot
+ return (
+ newServerRoot,
+ newServerRootValue,
+ newDocumentRootValue,
+ newDataRootValue
+ )
- # Create "Documents" directory with same ownership as oldCalDocuments
- newCalDocuments = os.path.join(newServerRoot, "Documents")
- log("New documents directory: %s" % (newCalDocuments,))
- newCalData = os.path.join(newServerRoot, "Data")
- log("New data directory: %s" % (newCalData,))
- os.mkdir(newCalDocuments)
- os.mkdir(newCalData)
- for item in list(os.listdir(newServerRoot)):
- if item not in ("Documents", "Data"):
- source = os.path.join(newServerRoot, item)
- dest = os.path.join(newCalDocuments, item)
- log("Relocating %s to %s" % (source, dest))
- os.rename(source, dest)
- # Relocate calendar DataRoot, copying all files
- if os.path.exists(oldCalData):
- if not os.path.exists(newCalData):
- os.mkdir(newCalData)
- for item in list(os.listdir(oldCalData)):
- source = os.path.join(oldCalData, item)
- if not os.path.isfile(source):
- continue
- dest = os.path.join(newCalData, item)
- log("Relocating %s to %s" % (source, dest))
- try:
- os.rename(source, dest)
- except OSError:
- # Can't rename because it's cross-volume; must copy/delete
- shutil.copy2(source, dest)
- os.remove(source)
+def relativize(parent, child):
+ """
+ If child is really a child of parent, make child relative to parent.
+ """
+ if child.startswith(parent):
+ parent = parent.rstrip("/")
+ child = child[len(parent):].strip("/")
+ return parent.rstrip("/"), child.rstrip("/")
- # Symlink to AB document root so server will find it an import to
- # PostgreSQL
- if oldABDocuments and os.path.exists(oldABDocuments):
- oldAddressBooks = os.path.join(oldABDocuments, "addressbooks")
- newAddressBooks = os.path.join(newCalDocuments, "addressbooks")
- log("Symlinking AddressBook data: %s to %s" % (newAddressBooks, \
oldAddressBooks))
- os.symlink(oldAddressBooks, newAddressBooks)
-
- elif oldABDocuments and os.path.exists(oldABDocuments):
- # No calendar data, only addressbook data
-
- if addressbookDataInDefaultLocation:
- # We're in the default location, relocate to new location
- newABDocuments = os.path.join(newServerRoot, "Documents")
- if os.path.exists(newABDocuments):
- # Move evertying from oldABDocuments
- for item in list(os.listdir(oldABDocuments)):
- source = os.path.join(oldABDocuments, item)
- dest = os.path.join(newABDocuments, item)
- log("Relocating %s to %s" % (source, dest))
- os.rename(source, dest)
- else:
- log("Error: %s does not exist" % (newABDocuments,))
- else:
- # The admin has moved addressbook data to a non-standard location so
- # we're going to leave it there, but move things down a level so
- # that the old DocumentRoot becomes new ServerRoot
-
- # Create "Documents" directory with same ownership as oldABDocuments
- newABDocuments = os.path.join(newServerRoot, "Documents")
- newABData = os.path.join(newServerRoot, "Data")
- log("New documents directory: %s" % (newABDocuments,))
- os.mkdir(newABDocuments)
- os.mkdir(newABData)
- for item in list(os.listdir(newServerRoot)):
- if item not in ("Documents", "Data"):
- source = os.path.join(newServerRoot, item)
- dest = os.path.join(newABDocuments, item)
- log("Relocating %s to %s" % (source, dest))
- os.rename(source, dest)
-
- if newServerRoot and os.path.exists(newServerRoot):
- """
- Change onwnership of entire ServerRoot
- """
- os.chown(newServerRoot, uid, gid)
- for root, dirs, files in os.walk(newServerRoot, followlinks=True):
- for name in dirs:
- os.chown(os.path.join(root, name), uid, gid)
- for name in files:
- os.chown(os.path.join(root, name), uid, gid)
-
-
-
def getServerIDs(plist):
"""
Given a caldavd.plist, return the userid and groupid for the UserName and
@@ -773,6 +768,7 @@
gid = grp.getgrnam(plist["GroupName"]).gr_gid
return uid, gid
+
def absolutePathWithRoot(root, path):
"""
Combine root and path as long as path does not start with /Volumes/
@@ -783,5 +779,59 @@
path = path.strip("/")
return os.path.join(root, path)
+
+class DiskAccessor(object):
+ """
+ A wrapper around various disk access methods so that unit tests can easily
+ replace these with a stub that doesn't actually require disk access.
+ """
+
+ def exists(self, path):
+ return os.path.exists(path)
+
+ def readFile(self, path):
+ input = file(path)
+ contents = input.read()
+ input.close()
+ return contents
+
+ def mkdir(self, path):
+ return os.mkdir(path)
+
+ def rename(self, before, after):
+ try:
+ return os.rename(before, after)
+ except OSError:
+ # Can't rename because it's cross-volume; must copy/delete
+ shutil.copy2(before, after)
+ return os.remove(before)
+
+ def isfile(self, path):
+ return os.path.isfile(path)
+
+ def symlink(self, orig, link):
+ return os.symlink(orig, link)
+
+ def chown(self, path, uid, gid, recursive=False):
+ os.chown(path, uid, gid)
+ if recursive:
+ for root, dirs, files in os.walk(path, followlinks=True):
+ for name in dirs:
+ os.chown(os.path.join(root, name), uid, gid)
+ for name in files:
+ os.chown(os.path.join(root, name), uid, gid)
+
+
+ def walk(self, path, followlinks=True):
+ return os.walk(path, followlinks=followlinks)
+
+ def listdir(self, path):
+ return list(os.listdir(path))
+
+ def ditto(self, src, dest):
+ log("Copying with ditto: %s to %s" % (src, dest))
+ return subprocess.call([DITTO, src, dest])
+
+
if __name__ == '__main__':
main()
Modified: CalendarServer/branches/users/cdaboo/pods/contrib/migration/test/test_migrator.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/migration/test/test_migrator.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/migration/test/test_migrator.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -15,13 +15,21 @@
##
import twistedcaldav.test.util
-from contrib.migration.calendarmigrator import mergePlist
+from contrib.migration.calendarmigrator import (
+ mergePlist, examinePreviousSystem, relocateData, relativize
+)
+import contrib.migration.calendarmigrator
class MigrationTests(twistedcaldav.test.util.TestCase):
"""
Calendar Server Migration Tests
"""
+ def setUp(self):
+ # Disable logging during tests
+ self.patch(contrib.migration.calendarmigrator, "log", lambda _: None)
+
+
def test_mergeSSL(self):
# SSL on for both services
@@ -237,6 +245,33 @@
mergePlist(oldCalDAV, oldCardDAV, newCombined)
self.assertEquals(newCombined, expected)
+ # Only CalDAV (Lion -> Lion)
+ oldCalDAV = {
+ "BindHTTPPorts": [],
+ "BindSSLPorts": [],
+ "HTTPPort": 8008,
+ "RedirectHTTPToHTTPS": False,
+ "SSLAuthorityChain": "/etc/certificates/test.chain.pem",
+ "SSLCertificate": "/etc/certificates/test.cert.pem",
+ "SSLPort": 8443,
+ "SSLPrivateKey": "/etc/certificates/test.key.pem",
+ }
+ oldCardDAV = {
+ }
+ expected = {
+ "BindHTTPPorts": [8008, 8800],
+ "BindSSLPorts": [8443, 8843],
+ "EnableSSL" : True,
+ "HTTPPort": 8008,
+ "RedirectHTTPToHTTPS": True,
+ "SSLAuthorityChain": "/etc/certificates/test.chain.pem",
+ "SSLCertificate": "/etc/certificates/test.cert.pem",
+ "SSLPort": 8443,
+ "SSLPrivateKey": "/etc/certificates/test.key.pem",
+ }
+ newCombined = { }
+ mergePlist(oldCalDAV, oldCardDAV, newCombined)
+ self.assertEquals(newCombined, expected)
# All settings missing!
@@ -256,3 +291,1055 @@
newCombined = { }
mergePlist(oldCalDAV, oldCardDAV, newCombined)
self.assertEquals(newCombined, expected)
+
+
+ def test_examinePreviousSystem(self):
+ """
+ Set up a virtual system in various configurations, then ensure the
+ examinePreviousSystem( ) method detects/returns the expected values.
+
+ 'info' is an array of tuples, each tuple containing:
+ - Description of configuration
+ - Layout of disk as a dictionary of paths plus file contents
+ - Expected return values
+ """
+
+ info = [
+
+ (
+ "Snow -> Lion Migration, all in default locations",
+ ("/Volumes/old", "/"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/Library/CalendarServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/Library/CalendarServer/Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/Library/AddressBookServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/Library/AddressBookServer/Data</string>
+ </dict>
+ </plist>
+ """,
+ "/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/Library/CalendarServer/Documents/calendars/" : True,
+ "/Volumes/old/Library/CalendarServer/Data/" : True,
+ "/Volumes/old/Library/AddressBookServer/Documents/addressbooks/" : \
True, + "/Volumes/old/Library/AddressBookServer/Data/" : True,
+ },
+ (
+ None, # Old ServerRoot value
+ "/Library/CalendarServer/Documents", # Old Cal DocRoot value
+ "/Library/CalendarServer/Data", # Old Cal DataRoot value
+ "/Library/AddressBookServer/Documents", # Old AB DocRoot value
+ 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Snow -> Lion Migration, all in default locations, non-/ target",
+ ("/Volumes/old", "/Volumes/new"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/Library/CalendarServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/Library/CalendarServer/Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/Library/AddressBookServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/Library/AddressBookServer/Data</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/Library/CalendarServer/Documents/calendars/" : True,
+ "/Volumes/old/Library/CalendarServer/Data/" : True,
+ "/Volumes/old/Library/AddressBookServer/Documents/addressbooks/" : \
True, + "/Volumes/old/Library/AddressBookServer/Data/" : True,
+ },
+ (
+ None, # Old ServerRoot value
+ "/Library/CalendarServer/Documents", # Old Cal DocRoot value
+ "/Library/CalendarServer/Data", # Old Cal DataRoot value
+ "/Library/AddressBookServer/Documents", # Old AB DocRoot value
+ 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Snow -> Lion Migration, not in default locations",
+ ("/Volumes/old", "/"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/NonStandard/CalendarServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/NonStandard/CalendarServer/Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/NonStandard/AddressBookServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/NonStandard/AddressBookServer/Data</string>
+ </dict>
+ </plist>
+ """,
+ "/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/NonStandard/CalendarServer/Documents/calendars/" : \
True, + "/Volumes/old/NonStandard/CalendarServer/Data/" : True,
+ "/Volumes/old/NonStandard/AddressBookServer/Documents/addressbooks/" \
: True, + "/Volumes/old/NonStandard/AddressBookServer/Data/" : True,
+ },
+ (
+ None, # Old ServerRoot value
+ "/NonStandard/CalendarServer/Documents", # Old Cal DocRoot Value
+ "/NonStandard/CalendarServer/Data", # Old Cal DataRoot Value
+ "/NonStandard/AddressBookServer/Documents", # Old AB DocRoot Value
+ 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Snow -> Lion Migration, in internal/external locations",
+ ("/Volumes/old", "/"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/Volumes/External/CalendarServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/Volumes/External/CalendarServer/Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/Library/AddressBookServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/Library/AddressBookServer/Data</string>
+ </dict>
+ </plist>
+ """,
+ "/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/External/CalendarServer/Documents/calendars/" : True,
+ "/Volumes/External/CalendarServer/Data/" : True,
+ "/Volumes/old/Library/AddressBookServer/Documents/addressbooks/" : \
True, + "/Volumes/old/Library/AddressBookServer/Data/" : True,
+ },
+ (
+ None, # Old ServerRoot value
+ "/Volumes/External/CalendarServer/Documents", # Old Cal DocRoot \
Value + "/Volumes/External/CalendarServer/Data", # Old Cal DataRoot \
Value + "/Library/AddressBookServer/Documents", # Old AB DocRoot Value
+ 93, 93, # user id, group id
+ )
+ ),
+
+
+ (
+ "Snow -> Lion Migration, only AddressBook data",
+ ("/Volumes/old", "/"),
+ {
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/Library/AddressBookServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/Library/AddressBookServer/Data</string>
+ </dict>
+ </plist>
+ """,
+ "/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/Library/AddressBookServer/Documents/addressbooks/" : \
True, + "/Volumes/old/Library/AddressBookServer/Data/" : True,
+ },
+ (
+ None, # Old ServerRoot value
+ None, # Old Cal DocRoot value
+ None, # Old Cal DataRoot value
+ "/Library/AddressBookServer/Documents", # Old AB DocRoot value
+ 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Lion -> Lion Migration, all in default locations",
+ ("/Volumes/old", "/"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/Library/Server/Calendar and Contacts/Documents/" : \
True, + "/Volumes/old/Library/Server/Calendar and Contacts/Data/" : \
True, + },
+ (
+ "/Library/Server/Calendar and Contacts", # Old ServerRoot value
+ "Documents", # Old Cal DocRoot value
+ "Data", # Old Cal DataRoot value
+ None, # Old AB Docs
+ 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Lion -> Lion Migration, not in default locations",
+ ("/Volumes/old", "/"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/NonStandard/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>/Volumes/External/Calendar/Documents</string>
+ <key>DataRoot</key>
+ <string>/Volumes/External/Calendar/Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/NonStandard/Calendar and Contacts/Documents/" : True,
+ "/Volumes/old/NonStandard/Calendar and Contacts/Data/" : True,
+ },
+ (
+ "/NonStandard/Calendar and Contacts", # Old ServerRoot value
+ "/Volumes/External/Calendar/Documents", # Old Cal DocRoot value
+ "/Volumes/External/Calendar/Data", # Old Cal DataRoot value
+ None, # Old AB Docs
+ 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Lion -> Lion Migration, non-/ targetRoot",
+ ("/Volumes/old", "/Volumes/new"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/Library/Server/Calendar and Contacts/Documents/" : \
True, + "/Volumes/old/Library/Server/Calendar and Contacts/Data/" : \
True, + },
+ (
+ "/Library/Server/Calendar and Contacts", # Old ServerRoot value
+ "Documents", # Old Cal DocRoot value
+ "Data", # Old Cal DocRoot value
+ None, # Old AB Docs
+ 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Lion -> Lion Migration, external ServerRoot with absolute external \
DocumentRoot and internal DataRoot", + ("/Volumes/old", "/Volumes/new"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Volumes/External/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>/Volumes/External/CalendarDocuments/</string>
+ <key>DataRoot</key>
+ <string>/CalendarData</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/External/Library/Server/Calendar and Contacts/" : True,
+ "/Volumes/External/CalendarDocuments/" : True,
+ "/Volumes/old/CalendarData" : True,
+ "/Volumes/new/Library/Server/Calendar and Contacts/" : True,
+ },
+ (
+ "/Volumes/External/Server/Calendar and Contacts", # Old ServerRoot \
value + "/Volumes/External/CalendarDocuments/", # Old Cal DocRoot \
value + "/CalendarData", # Old Cal DocRoot value
+ None, # Old AB Docs
+ 93, 93, # user id, group id
+ )
+ ),
+
+
+
+ (
+ "Empty migration, nothing exists",
+ ("/Volumes/old", "/Volumes/new"),
+ {
+ },
+ (
+ None, # Old ServerRoot value
+ None, # Old Cal DocRoot value
+ None, # Old Cal DocRoot value
+ None, # Old AB Docs
+ -1, -1, # user id, group id
+ )
+ ),
+
+
+ ]
+
+ for description, (source, target), paths, expected in info:
+ # print "-=-=-=- %s -=-=-=-" % (description,)
+ accessor = StubDiskAccessor(paths)
+ actual = examinePreviousSystem(source, target, diskAccessor=accessor)
+ self.assertEquals(expected, actual)
+
+
+ def test_relocateData(self):
+
+ info = [
+
+ (
+ "Snow -> Lion Migration, all in default locations",
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/Library/CalendarServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/Library/CalendarServer/Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/Library/AddressBookServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/Library/AddressBookServer/Data</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/Library/CalendarServer/Documents/calendars/" : True,
+ "/Volumes/old/Library/CalendarServer/Data/" : True,
+ "/Volumes/old/Library/AddressBookServer/Documents/addressbooks/" : \
True, + "/Volumes/old/Library/AddressBookServer/Data/" : True,
+ "/Volumes/new/Library/Server/Calendar and Contacts" : True,
+ },
+ ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ None, # oldServerRootValue
+ "/Library/CalendarServer/Documents", # oldCalDocumentRootValue
+ "/Library/CalendarServer/Data", # oldCalDataRootValue
+ "/Library/AddressBookServer/Documents", # oldABDocumentRootValue
+ 93, 93, # user id, group id
+ ),
+ ( # expected return values
+ "/Volumes/new/Library/Server/Calendar and Contacts",
+ "/Library/Server/Calendar and Contacts",
+ "Documents",
+ "Data"
+ ),
+ [ # expected DiskAccessor history
+ ('ditto', '/Volumes/old/Library/CalendarServer/Documents', \
'/Volumes/new/Library/Server/Calendar and Contacts/Documents'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and Contacts/Documents', \
93, 93), + ('ditto', '/Volumes/old/Library/CalendarServer/Data', \
'/Volumes/new/Library/Server/Calendar and Contacts/Data'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and Contacts/Data', 93, \
93), + ('ditto', \
'/Volumes/old/Library/AddressBookServer/Documents/addressbooks', \
'/Volumes/new/Library/Server/Calendar and Contacts/Documents/addressbooks'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and \
Contacts/Documents/addressbooks', 93, 93), + ]
+ ),
+
+ (
+ "Snow -> Lion Migration, in non-standard locations",
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/NonStandard/CalendarServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/NonStandard/CalendarServer/Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/NonStandard/AddressBookServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/NonStandard/AddressBookServer/Data</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/NonStandard/CalendarServer/Documents/calendars/" : \
True, + "/Volumes/old/NonStandard/CalendarServer/Data/" : True,
+ "/Volumes/old/NonStandard/AddressBookServer/Documents/addressbooks/" \
: True, + "/Volumes/old/NonStandard/AddressBookServer/Data/" : True,
+ "/Volumes/new/Library/Server/Calendar and Contacts" : True,
+ },
+ ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ None, # oldServerRootValue
+ "/NonStandard/CalendarServer/Documents", # oldCalDocumentRootValue
+ "/NonStandard/CalendarServer/Data", # oldCalDataRootValue
+ "/NonStandard/AddressBookServer/Documents", # oldABDocumentRootValue
+ 93, 93, # user id, group id
+ ),
+ ( # expected return values
+ "/Volumes/new/Library/Server/Calendar and Contacts",
+ "/Library/Server/Calendar and Contacts",
+ "Documents",
+ "Data"
+ ),
+ [
+ ('ditto', '/Volumes/old/NonStandard/CalendarServer/Documents', \
'/Volumes/new/Library/Server/Calendar and Contacts/Documents'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and Contacts/Documents', \
93, 93), + ('ditto', '/Volumes/old/NonStandard/CalendarServer/Data', \
'/Volumes/new/Library/Server/Calendar and Contacts/Data'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and Contacts/Data', 93, \
93), + ('ditto', \
'/Volumes/old/NonStandard/AddressBookServer/Documents/addressbooks', \
'/Volumes/new/Library/Server/Calendar and Contacts/Documents/addressbooks'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and \
Contacts/Documents/addressbooks', 93, 93), + ]
+ ),
+
+ (
+ "Snow -> Lion Migration, internal AB, external Cal",
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/Volumes/External/CalendarServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/Volumes/External/CalendarServer/Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ <string>/Library/AddressBookServer/Documents</string>
+ <key>DataRoot</key>
+ <string>/Library/AddressBookServer/Data</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/External/CalendarServer/Documents" : True,
+ "/Volumes/External/CalendarServer/Data" : True,
+ "/Volumes/old/Library/AddressBookServer/Documents/addressbooks/" : \
True, + "/Volumes/old/Library/AddressBookServer/Data/" : True,
+ "/Volumes/new/Library/Server/Calendar and Contacts" : True,
+ },
+ ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ None, # oldServerRootValue
+ "/Volumes/External/CalendarServer/Documents", # \
oldCalDocumentRootValue + "/Volumes/External/CalendarServer/Data", # \
oldCalDataRootValue + "/Library/AddressBookServer/Documents", # \
oldABDocumentRootValue + 93, 93, # user id, group id
+ ),
+ ( # expected return values
+ "/Volumes/new/Library/Server/Calendar and Contacts",
+ "/Library/Server/Calendar and Contacts",
+ "/Volumes/External/CalendarServer/Documents",
+ "/Volumes/External/CalendarServer/Data"
+ ),
+ [
+ ('ditto', \
'/Volumes/old/Library/AddressBookServer/Documents/addressbooks', \
'/Volumes/External/CalendarServer/Documents/addressbooks'), + \
('chown-recursive', '/Volumes/External/CalendarServer/Documents/addressbooks', 93, \
93), + ]
+ ),
+
+ (
+ "Lion -> Lion Migration, all in default locations",
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/Library/Server/Calendar and Contacts/Documents/" : \
True, + "/Volumes/old/Library/Server/Calendar and Contacts/Data/" : \
True, + "/Volumes/new/Library/Server/Calendar and Contacts/" : True,
+ },
+ ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ "/Library/Server/Calendar and Contacts", # oldServerRootValue
+ "Documents", # oldCalDocumentRootValue
+ "Data", # oldCalDataRootValue
+ None, # oldABDocumentRootValue
+ 93, 93, # user id, group id
+ ),
+ ( # expected return values
+ "/Volumes/new/Library/Server/Calendar and Contacts",
+ "/Library/Server/Calendar and Contacts",
+ "Documents",
+ "Data"
+ ),
+ [
+ ('ditto', '/Volumes/old/Library/Server/Calendar and \
Contacts/Documents', '/Volumes/new/Library/Server/Calendar and Contacts/Documents'), \
+ ('chown-recursive', '/Volumes/new/Library/Server/Calendar and \
Contacts/Documents', 93, 93), + ('ditto', \
'/Volumes/old/Library/Server/Calendar and Contacts/Data', \
'/Volumes/new/Library/Server/Calendar and Contacts/Data'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and Contacts/Data', 93, \
93), + ]
+ ),
+
+ (
+ "Lion -> Lion Migration, external ServerRoot with relative DocumentRoot \
and DataRoot", + {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Volumes/External/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/External/Library/Server/Calendar and Contacts/Documents/" \
: True, + "/Volumes/External/Library/Server/Calendar and \
Contacts/Data/" : True, + "/Volumes/new/Library/Server/Calendar and \
Contacts/" : True, + },
+ ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ "/Volumes/External/Library/Server/Calendar and Contacts", # \
oldServerRootValue + "Documents", # oldCalDocumentRootValue
+ "Data", # oldCalDataRootValue
+ None, # oldABDocumentRootValue
+ 93, 93, # user id, group id
+ ),
+ ( # expected return values
+ "/Volumes/External/Library/Server/Calendar and Contacts",
+ "/Volumes/External/Library/Server/Calendar and Contacts",
+ "Documents",
+ "Data"
+ ),
+ [
+ ]
+ ),
+
+
+ (
+ "Lion -> Lion Migration, external ServerRoot with absolute external \
DocumentRoot and internal DataRoot", + {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Volumes/External/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>/Volumes/External/CalendarDocuments/</string>
+ <key>DataRoot</key>
+ <string>/CalendarData</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and Contacts</string>
+ <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/External/Library/Server/Calendar and Contacts/" : True,
+ "/Volumes/External/CalendarDocuments/" : True,
+ "/Volumes/old/CalendarData" : True,
+ "/Volumes/new/Library/Server/Calendar and Contacts/" : True,
+ },
+ ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ "/Volumes/External/Library/Server/Calendar and Contacts", # \
oldServerRootValue + "/Volumes/External/CalendarDocuments/", # \
oldCalDocumentRootValue + "/CalendarData", # oldCalDataRootValue
+ None, # oldABDocumentRootValue
+ 93, 93, # user id, group id
+ ),
+ ( # expected return values
+ "/Volumes/External/Library/Server/Calendar and Contacts",
+ "/Volumes/External/Library/Server/Calendar and Contacts",
+ "/Volumes/External/CalendarDocuments",
+ "Data" # Note that DataRoot was copied over to external volume
+ ),
+ [
+ ('ditto', '/Volumes/old/CalendarData', \
'/Volumes/External/Library/Server/Calendar and Contacts/Data'), + \
('chown-recursive', '/Volumes/External/Library/Server/Calendar and Contacts/Data', \
93, 93), + ]
+ ),
+
+ (
+ "Empty migration",
+ { # no files
+ },
+ ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ None, # oldServerRootValue
+ None, # oldCalDocumentRootValue
+ None, # oldCalDataRootValue
+ None, # oldABDocumentRootValue
+ -1, -1, # user id, group id
+ ),
+ ( # expected return values
+ "/Volumes/new/Library/Server/Calendar and Contacts",
+ "/Library/Server/Calendar and Contacts",
+ "Documents",
+ "Data"
+ ),
+ [ # no history
+ ]
+ ),
+
+ ]
+
+ for description, paths, args, expected, history in info:
+ # print "-=-=-=- %s -=-=-=-" % (description,)
+ accessor = StubDiskAccessor(paths)
+ actual = relocateData(*args, diskAccessor=accessor)
+ self.assertEquals(expected, actual)
+ self.assertEquals(history, accessor.history)
+
+
+ def test_stubDiskAccessor(self):
+
+ paths = {
+ "/a/b/c/d" : "foo",
+ "/a/b/c/e" : "bar",
+ "/x/y/z/" : True,
+ }
+ accessor = StubDiskAccessor(paths)
+
+ shouldExist = ["/a", "/a/", "/a/b", "/a/b/", "/a/b/c/d", "/x/y/z"]
+ shouldNotExist = ["/b", "/x/y/z/Z"]
+
+ for path in shouldExist:
+ self.assertTrue(accessor.exists(path))
+ for path in shouldNotExist:
+ self.assertFalse(accessor.exists(path))
+
+ for key, value in paths.iteritems():
+ if value is not True:
+ self.assertEquals(accessor.readFile(key), value)
+
+
+ def test_relativize(self):
+ """
+ Make sure child paths are made relative to their parent
+ """
+ info = [
+ (("/abc/", "/abc/def"), ("/abc", "def")),
+ (("/abc", "/abc/def"), ("/abc", "def")),
+ (("/abc", "/def"), ("/abc", "/def")),
+ ]
+ for args, expected in info:
+ self.assertEquals(expected, relativize(*args))
+
+
+class StubDiskAccessor(object):
+ """
+ A stub which allows testing without actually having real files
+ """
+
+ def __init__(self, paths):
+ self.paths = paths
+ self._fillInDirectories()
+
+ self.reset()
+
+ def _fillInDirectories(self):
+ for key in self.paths.keys():
+ parts = key.split("/")
+ for i in xrange(len(parts)):
+ path = "/".join(parts[:i])
+ self.paths[path] = True
+
+ def addPath(self, path, value):
+ self.paths[path] = value
+ self._fillInDirectories()
+
+ def reset(self):
+ self.history = []
+
+ def exists(self, path):
+ return self.paths.has_key(path.rstrip("/"))
+
+ def readFile(self, path):
+ return self.paths[path]
+
+ def mkdir(self, path):
+ self.history.append(("mkdir", path))
+ self.addPath(path, True)
+
+ def rename(self, before, after):
+ self.history.append(("rename", before, after))
+
+ def isfile(self, path):
+ # FIXME: probably want a better way to denote a directory than "True"
+ return self.exists(path) and self.paths[path] is not True
+
+ def symlink(self, orig, link):
+ self.history.append(("symlink", orig, link))
+
+ def chown(self, path, uid, gid, recursive=False):
+ self.history.append(("chown-recursive" if recursive else "chown", path, uid, \
gid)) +
+ def walk(self, path, followlinks=True):
+ yield [], [], []
+
+ def listdir(self, path):
+ return []
+
+ def ditto(self, src, dest):
+ self.history.append(("ditto", src, dest))
+ self.addPath(dest, True)
+
Modified: CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/config.plist
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/config.plist 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/config.plist 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -66,6 +66,8 @@
<key>observers</key>
<array>
<string>loadtest.population.ReportStatistics</string>
+ <string>loadtest.ical.RequestLogger</string>
+ <string>loadtest.profiles.OperationLogger</string>
</array>
</dict>
Modified: CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/ical.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/ical.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/ical.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -19,6 +19,7 @@
from operator import getitem
from pprint import pformat
from datetime import datetime
+from urlparse import urlparse, urlunparse
from xml.etree import ElementTree
ElementTree.QName.__repr__ = lambda self: '<QName %r>' % (self.text,)
@@ -196,17 +197,14 @@
# not both.
after = self.reactor.seconds()
- # XXX If the response code is wrong, there's probably not
- # point passing the response down the callback chain.
- # errback?
success = response.code == expectedResponseCode
# if not success:
# import pdb; pdb.set_trace()
msg(
type="response", success=success, method=method,
- headers=headers, body=body,
- duration=(after - before), url=url)
+ headers=headers, body=body, code=response.code,
+ user=self.user, duration=(after - before), url=url)
if success:
return response
@@ -665,11 +663,24 @@
class RequestLogger(object):
+ format = u"%(user)s request %(code)s%(success)s[%(duration)5.2f s] %(method)8s \
%(url)s" + success = u"\N{CHECK MARK}"
+ failure = u"\N{BALLOT X}"
+
def observe(self, event):
- if event.get("type") == "request":
- print event["user"], event["method"], event["url"]
+ if event.get("type") == "response":
+ event['url'] = urlunparse(('', '') + urlparse(event['url'])[2:])
+ if event['success']:
+ event['success'] = self.success
+ else:
+ event['success'] = self.failure
+ print (self.format % event).encode('utf-8')
+ def report(self):
+ pass
+
+
def main():
from urllib2 import HTTPDigestAuthHandler
Copied: CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/logger.py \
(from rev 7364, CalendarServer/trunk/contrib/performance/loadtest/logger.py) \
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/logger.py \
(rev 0)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/logger.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -0,0 +1,52 @@
+from stats import mean, median, stddev, mad
+
+class SummarizingMixin(object):
+ def printHeader(self, fields):
+ """
+ Print a header for the summarization data which will be reported.
+
+ @param fields: A C{list} of two-tuples. Each tuple describes one
+ column in the summary. The first element gives a label to appear
+ at the top of the column. The second element gives the width of
+ the column.
+ """
+ format = []
+ labels = []
+ for (label, width) in fields:
+ format.append('%%%ds' % (width,))
+ labels.append(label)
+ print ' '.join(format) % tuple(labels)
+
+
+ def _summarizeData(self, operation, data):
+ failed = 0
+ threesec = 0
+ durations = []
+ for (success, duration) in data:
+ if not success:
+ failed += 1
+ if duration > 3:
+ threesec += 1
+ durations.append(duration)
+
+ return operation, len(data), failed, threesec, mean(durations), \
median(durations) +
+
+ def _printRow(self, formats, values):
+ format = ' '.join(formats)
+ print format % values
+
+
+ def printData(self, formats, perOperationTimes):
+ """
+ Print one or more rows of data with the given formatting.
+
+ @param formats: A C{list} of C{str} giving formats into which each
+ data field will be interpolated.
+
+ @param perOperationTimes: A C{list} of all of the data to summarize.
+ Each element is a two-tuple of whether the operation succeeded
+ (C{True} if so, C{False} if not) and how long the operation took.
+ """
+ for method, data in perOperationTimes:
+ self._printRow(formats, self._summarizeData(method, data))
Modified: CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/population.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/population.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/population.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -26,10 +26,10 @@
from twisted.python.log import msg, err
from stats import mean, median, stddev, mad
+from loadtest.logger import SummarizingMixin
from loadtest.ical import SnowLeopard, RequestLogger
from loadtest.profiles import Eventer, Inviter, Accepter
-
class ClientType(object, FancyEqMixin):
"""
@ivar clientType: An L{ICalendarClient} implementation
@@ -206,7 +206,7 @@
-class ReportStatistics(StatisticsBase):
+class ReportStatistics(StatisticsBase, SummarizingMixin):
_fields = [
('operation', 10, '%10s'),
('count', 8, '%8s'),
@@ -225,39 +225,15 @@
dataset.append((event['success'], event['duration']))
- def _printHeader(self):
- format = []
- labels = []
- for (label, width, fmt) in self._fields:
- format.append('%%%ds' % (width,))
- labels.append(label)
- print ''.join(format) % tuple(labels)
-
-
- def _summarizeData(self, method, data):
- failed = 0
- threesec = 0
- durations = []
- for (success, duration) in data:
- if not success:
- failed += 1
- if duration > 3:
- threesec += 1
- durations.append(duration)
-
- return method, len(data), failed, threesec, mean(durations), \
median(durations)
-
-
- def _printData(self, *values):
- format = ''.join(fmt for (label, width, fmt) in self._fields)
- print format % values
-
-
def report(self):
print
- self._printHeader()
- for method, data in self._perMethodTimes.iteritems():
- self._printData(*self._summarizeData(method, data))
+ self.printHeader([
+ (label, width)
+ for (label, width, fmt)
+ in self._fields])
+ self.printData(
+ [fmt for (label, width, fmt) in self._fields],
+ sorted(self._perMethodTimes.items()))
def main():
Modified: CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/profiles.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/profiles.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/profiles.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -30,11 +30,14 @@
from protocol.caldav.definitions import caldavxml
+from twisted.python import context
from twisted.python.log import msg
+from twisted.python.failure import Failure
from twisted.internet.defer import succeed, fail
from twisted.internet.task import LoopingCall
from twisted.web.http import PRECONDITION_FAILED
+from loadtest.logger import SummarizingMixin
from loadtest.ical import IncorrectResponseCode
@@ -59,6 +62,41 @@
if cal.resourceType == calendarType]
+ def _isSelfAttendee(self, attendee):
+ """
+ Try to match one of the attendee's identifiers against one of
+ C{self._client}'s identifiers. Return C{True} if something matches,
+ C{False} otherwise.
+ """
+ return attendee.params[u'EMAIL'][0] == self._client.email[len('mailto:'):]
+
+
+ def _newOperation(self, label, deferred):
+ """
+ Helper to emit a log event when a new operation is started and
+ another one when it completes.
+ """
+ # If this is a scheduled request, record the lag in the
+ # scheduling now so it can be reported when the response is
+ # received.
+ lag = context.get('lag', None)
+
+ before = self._reactor.seconds()
+ msg(type="operation", phase="start",
+ user=self._client.user, label=label, lag=lag)
+
+ def finished(passthrough):
+ success = not isinstance(passthrough, Failure)
+ after = self._reactor.seconds()
+ msg(type="operation", phase="end", duration=after - before,
+ user=self._client.user, label=label, success=success)
+ return passthrough
+ deferred.addBoth(finished)
+ return deferred
+
+
+
+
class CannotAddAttendee(Exception):
"""
Indicates no new attendees can be invited to a particular event.
@@ -143,10 +181,16 @@
if event is None:
continue
+ vevent = event.contents[u'vevent'][0]
+ organizer = vevent.contents.get('organizer', [None])[0]
+ if organizer is not None and not self._isSelfAttendee(organizer):
+ # This event was organized by someone else, don't try to invite \
someone to it. + continue
+
href = calendar.url + uuid
# Find out who might attend
- attendees = event.contents['vevent'][0].contents.get('attendee', [])
+ attendees = vevent.contents.get('attendee', [])
d = self._addAttendee(event, attendees)
d.addCallbacks(
@@ -154,7 +198,7 @@
self._client.addEventAttendee(
href, attendee),
lambda reason: reason.trap(CannotAddAttendee))
- return d
+ return self._newOperation("invite", d)
@@ -189,14 +233,13 @@
# NEEDS-ACTION PARTSTAT.
attendees = vevent.contents['vevent'][0].contents.get('attendee', [])
for attendee in attendees:
- if attendee.params[u'EMAIL'][0] == self._client.email[len('mailto:'):]:
+ if self._isSelfAttendee(attendee):
if attendee.params[u'PARTSTAT'][0] == 'NEEDS-ACTION':
# XXX Base this on something real
delay = self.random.gauss(10, 2)
self._accepting.add(href)
self._reactor.callLater(
delay, self._acceptInvitation, href, attendee)
- return
def _acceptInvitation(self, href, attendee):
@@ -234,7 +277,7 @@
self._accepting.remove(href)
return passthrough
d.addBoth(finished)
- return d
+ return self._newOperation("accept", d)
def _makeAcceptedAttendee(self, attendee):
@@ -315,4 +358,54 @@
href = '%s%s.ics' % (
calendar.url, vevent.contents[u'uid'][0].value)
- return self._client.addEvent(href, vcalendar)
+ d = self._client.addEvent(href, vcalendar)
+ return self._newOperation("create", d)
+
+
+class OperationLogger(SummarizingMixin):
+ """
+ Profiles will initiate operations which may span multiple requests. Start
+ and stop log messages are emitted for these operations and logged by this
+ logger.
+ """
+ formats = {
+ u"start": u"%(user)s - - - - - - - - - - - %(label)8s BEGIN %(lag)s",
+ u"end" : u"%(user)s - - - - - - - - - - - %(label)8s END [%(duration)5.2f \
s]", + }
+
+ lagFormat = u'{lag %5.2f ms}'
+
+ _fields = [
+ ('operation', 10, '%10s'),
+ ('count', 8, '%8s'),
+ ('failed', 8, '%8s'),
+ ('>3sec', 8, '%8s'),
+ ('mean', 8, '%8.4f'),
+ ('median', 8, '%8.4f'),
+ ]
+
+ def __init__(self):
+ self._perOperationTimes = {}
+
+
+ def observe(self, event):
+ if event.get("type") == "operation":
+ if event.get('lag') is None:
+ event['lag'] = ''
+ else:
+ event['lag'] = self.lagFormat % (event['lag'] * 1000.0,)
+ print (self.formats[event[u'phase']] % event).encode('utf-8')
+ if event[u'phase'] == u'end':
+ dataset = self._perOperationTimes.setdefault(event[u'label'], [])
+ dataset.append((event[u'success'], event[u'duration']))
+
+
+ def report(self):
+ print
+ self.printHeader([
+ (label, width)
+ for (label, width, fmt)
+ in self._fields])
+ self.printData(
+ [fmt for (label, width, fmt) in self._fields],
+ sorted(self._perOperationTimes.items()))
Modified: CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/sim.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/sim.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/sim.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -20,6 +20,7 @@
from plistlib import readPlist
from collections import namedtuple
+from twisted.python import context
from twisted.python.filepath import FilePath
from twisted.python.log import addObserver
from twisted.python.usage import UsageError, Options
@@ -32,6 +33,30 @@
CalendarClientSimulator)
+class LagTrackingReactor(object):
+ """
+ This reactor wraps another reactor and proxies all attribute
+ access (including method calls). It only changes the behavior of
+ L{IReactorTime.callLater} to insert a C{"lag"} key into the
+ context which delayed function calls are invoked with. This key
+ has a float value which gives the difference in time between when
+ the call was original scheduled and when the call actually took
+ place.
+ """
+ def __init__(self, reactor):
+ self._reactor = reactor
+
+ def __getattr__(self, name):
+ return getattr(self._reactor, name)
+
+ def callLater(self, delay, function, *args, **kwargs):
+ expected = self._reactor.seconds() + delay
+ def modifyContext():
+ now = self._reactor.seconds()
+ context.call({'lag': now - expected}, function, *args, **kwargs)
+ return self._reactor.callLater(delay, modifyContext)
+
+
class SimOptions(Options):
"""
Command line configuration options for the load simulator.
@@ -103,7 +128,7 @@
self.arrival = arrival
self.parameters = parameters
self.observers = observers
- self.reactor = reactor
+ self.reactor = LagTrackingReactor(reactor)
@classmethod
Modified: CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/test_profiles.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/test_profiles.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/test_profiles.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -334,7 +334,7 @@
If the inviter randomly selects a user which is already an
invitee on the event, a different user is added instead.
"""
- selfNumber = 13
+ selfNumber = 1
vevent, event, calendar, client = self._simpleAccount(
selfNumber, INVITED_EVENT)
@@ -376,7 +376,27 @@
self.assertEquals(len(attendees), 2)
+ def test_doNotInviteToSomeoneElsesEvent(self):
+ """
+ If there are events on our calendar which are being organized
+ by someone else, the inviter does not attempt to invite new
+ users to them.
+ """
+ selfNumber = 2
+ vevent, event, calendar, client = self._simpleAccount(
+ selfNumber, INVITED_EVENT)
+ inviter = Inviter(None, client, selfNumber)
+ # Try to send an invitation, but with only one event on the
+ # calendar, of which we are not the organizer. It should be
+ # unchanged afterwards.
+ inviter._invite()
+ attendees = event.vevent.contents[u'vevent'][0].contents[u'attendee']
+ self.assertEqual(len(attendees), 2)
+ self.assertEqual(attendees[0].params['CN'], [u'User 01'])
+ self.assertEqual(attendees[1].params['CN'], [u'User 02'])
+
+
class AccepterTests(TestCase):
"""
Tests for loadtest.profiles.Accepter.
Property changes on: \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/sim \
___________________________________________________________________
Added: svn:executable
+ *
Modified: CalendarServer/branches/users/cdaboo/pods/doc/calendarserver_export.8
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/doc/calendarserver_export.8 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/doc/calendarserver_export.8 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -31,7 +31,7 @@
.Sh DESCRIPTION
.Nm
is a tool that generates a single iCalendar file containing all of the
-iCalendar components found from all specifies input sources, providing
+iCalendar components found from all specified input sources, providing
server administrators a means by which to export data from the Darwin
Calandar Server into a format that can be viewed and/or manipulated by
other tools. Multiple input sources may be specified; the resulting
Modified: CalendarServer/branches/users/cdaboo/pods/python
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/python 2011-04-27 20:10:49 UTC (rev \
7376)
+++ CalendarServer/branches/users/cdaboo/pods/python 2011-04-27 21:09:24 UTC (rev \
7377) @@ -1,7 +1,8 @@
-#!/bin/sh
+#!/bin/bash
wd="$(cd "$(dirname "$0")" && pwd)";
-export PYTHONPATH="$("${wd}/run" -p)";
+. "${wd}/support/shell.sh"
-exec python "$@";
+exec "${python}" "$@";
+
Modified: CalendarServer/branches/users/cdaboo/pods/support/Makefile.Apple
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/support/Makefile.Apple 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/support/Makefile.Apple 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -108,6 +108,10 @@
$(_v) $(INSTALL_DIRECTORY) "$(DSTROOT)/usr/libexec/changeip"
$(_v) $(INSTALL_FILE) "$(Sources)/calendarserver/tools/changeip_calendar.py" \
"$(DSTROOT)/usr/libexec/changeip/changeip_calendar.py" $(_v) chmod ugo+x \
"$(DSTROOT)/usr/libexec/changeip/changeip_calendar.py" + @echo "Installing \
certificate update scripts..." + $(_v) $(INSTALL_DIRECTORY) \
"$(DSTROOT)/usr/libexec/certupdate" + $(_v) $(INSTALL_FILE) \
"$(Sources)/contrib/certupdate/calendarcertupdate.py" \
"$(DSTROOT)/usr/libexec/certupdate/calendarcertupdate.py" + $(_v) chmod ugo+x \
"$(DSTROOT)/usr/libexec/certupdate/calendarcertupdate.py"
install::
@echo "Installing CalDAVTester package..."
Modified: CalendarServer/branches/users/cdaboo/pods/support/build.sh
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/support/build.sh 2011-04-27 20:10:49 \
UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/support/build.sh 2011-04-27 21:09:24 \
UTC (rev 7377) @@ -644,7 +644,7 @@
"http://svn.osafoundation.org/vobject/trunk";
# XXX actually PyCalendar should be imported in-place.
- py_dependency -fe -i "src" -r 154 \
+ py_dependency -fe -i "src" -r 156 \
"pycalendar" "pycalendar" "pycalendar" \
"http://svn.mulberrymail.com/repos/PyCalendar/branches/server";
Property changes on: CalendarServer/branches/users/cdaboo/pods/support/build.sh
___________________________________________________________________
Modified: svn:mergeinfo
- /CalendarServer/branches/config-separation/support/build.sh:4379-4443
/CalendarServer/branches/egg-info-351/support/build.sh:4589-4615
/CalendarServer/branches/generic-sqlstore/support/build.sh:6167-6191
/CalendarServer/branches/new-store/support/build.sh:5594-5934
/CalendarServer/branches/new-store-no-caldavfile/support/build.sh:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2/support/build.sh:5936-5981
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/support/build.sh:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/support/build.sh:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/support/build.sh:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/support/build.sh:4465-4957
/CalendarServer/branches/users/cdaboo/pycalendar/support/build.sh:7085-7206
/CalendarServer/branches/users/cdaboo/pycard/support/build.sh:7227-7237
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/support/build.sh:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/build.sh:5188-5440
/CalendarServer/branches/users/glyph/conn-limit/support/build.sh:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge/support/build.sh:4971-5080
/CalendarServer/branches/users/glyph/dalify/support/build.sh:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect/support/build.sh:6824-6876
/CalendarServer/branches/users/glyph/dont-start-postgres/support/build.sh:6592-6614
/CalendarServer/branches/users/glyph/linux-tests/support/build.sh:6893-6900
/CalendarServer/branches/users/glyph/more-deferreds-6/support/build.sh:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7/support/build.sh:6369-6445
/CalendarServer/branches/users/glyph/sendfdport/support/build.sh:5388-5424
/CalendarServer/branches/users/glyph/sharedpool/support/build.sh:6490-6550
/CalendarServer/branches/users/glyph/sql-store/support/build.sh:5929-6073
/CalendarServer/branches/users/glyph/subtransactions/support/build.sh:7248-7258
/CalendarServer/branches/users/glyph/use-system-twisted/support/build.sh:5084-5149
/CalendarServer/branches/users/sagen/locations-resources/support/build.sh:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2/support/build.sh:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events/support/build.sh:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038/support/build.sh:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/support/build.sh:4068-4075
/CalendarServer/branches/users/sagen/resources-2/support/build.sh:5084-5093
/CalendarServer/branches/users/wsanchez/transations/support/build.sh:5515-5593
+ /CalendarServer/branches/config-separation/support/build.sh:4379-4443
/CalendarServer/branches/egg-info-351/support/build.sh:4589-4615
/CalendarServer/branches/generic-sqlstore/support/build.sh:6167-6191
/CalendarServer/branches/new-store-no-caldavfile-2/support/build.sh:5936-5981
/CalendarServer/branches/new-store-no-caldavfile/support/build.sh:5911-5935
/CalendarServer/branches/new-store/support/build.sh:5594-5934
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/support/build.sh:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/support/build.sh:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/support/build.sh:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/support/build.sh:4465-4957
/CalendarServer/branches/users/cdaboo/pycalendar/support/build.sh:7085-7206
/CalendarServer/branches/users/cdaboo/pycard/support/build.sh:7227-7237
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/support/build.sh:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/build.sh:5188-5440
/CalendarServer/branches/users/glyph/conn-limit/support/build.sh:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge/support/build.sh:4971-5080
/CalendarServer/branches/users/glyph/dalify/support/build.sh:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect/support/build.sh:6824-6876
/CalendarServer/branches/users/glyph/dont-start-postgres/support/build.sh:6592-6614
/CalendarServer/branches/users/glyph/linux-tests/support/build.sh:6893-6900
/CalendarServer/branches/users/glyph/more-deferreds-6/support/build.sh:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7/support/build.sh:6369-6445
/CalendarServer/branches/users/glyph/oracle-nulls/support/build.sh:7340-7351
/CalendarServer/branches/users/glyph/sendfdport/support/build.sh:5388-5424
/CalendarServer/branches/users/glyph/sharedpool/support/build.sh:6490-6550
/CalendarServer/branches/users/glyph/sql-store/support/build.sh:5929-6073
/CalendarServer/branches/users/glyph/subtransactions/support/build.sh:7248-7258
/CalendarServer/branches/users/glyph/use-system-twisted/support/build.sh:5084-5149
/CalendarServer/branches/users/sagen/locations-resources-2/support/build.sh:5052-5061
/CalendarServer/branches/users/sagen/locations-resources/support/build.sh:5032-5051
/CalendarServer/branches/users/sagen/purge_old_events/support/build.sh:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038/support/build.sh:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/support/build.sh:4068-4075
/CalendarServer/branches/users/sagen/resources-2/support/build.sh:5084-5093
/CalendarServer/branches/users/wsanchez/transations/support/build.sh:5515-5593
/CalendarServer/trunk/support/build.sh:7297-7364
Modified: CalendarServer/branches/users/cdaboo/pods/support/py.sh
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/support/py.sh 2011-04-27 20:10:49 UTC \
(rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/support/py.sh 2011-04-27 21:09:24 UTC \
(rev 7377) @@ -48,7 +48,7 @@
# Detect which version of Python to use, then print out which one was detected.
detect_python_version () {
- for v in "" "2.6" "2.5"
+ for v in "2.7" "2.6" "2.5" ""
do
for p in \
"${PYTHON:=}" \
@@ -120,12 +120,12 @@
# Compare version numbers
cmp_version () {
- local result=0;
-
local v="$1"; shift;
local mv="$1"; shift;
- while [ $result != 1 ]; do
+ local result;
+
+ while true; do
vh="${v%%.*}"; # Get highest-order segment
mvh="${mv%%.*}";
@@ -134,8 +134,14 @@
break;
fi;
+ if [ "${vh}" -lt "${mvh}" ]; then
+ result=0;
+ break;
+ fi;
+
if [ "${v}" == "${v#*.}" ]; then
# No dots left, so we're ok
+ result=0;
break;
fi;
Modified: CalendarServer/branches/users/cdaboo/pods/support/shell.sh
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/support/shell.sh 2011-04-27 20:10:49 \
UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/support/shell.sh 2011-04-27 21:09:24 \
UTC (rev 7377) @@ -21,7 +21,10 @@
# set up by the CalendarServer run script and are not otherwise installed on
# your system.
-wd="$(pwd)";
+if [ -z "${wd}" ]; then
+ wd="$(pwd)";
+fi;
+
. ${wd}/support/build.sh;
do_setup=false;
do_get=false;
Modified: CalendarServer/branches/users/cdaboo/pods/testserver
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/testserver 2011-04-27 20:10:49 UTC (rev \
7376)
+++ CalendarServer/branches/users/cdaboo/pods/testserver 2011-04-27 21:09:24 UTC (rev \
7377) @@ -61,12 +61,13 @@
# Do The Right Thing
##
-export PYTHONPATH=$("${wd}/run" -p);
+. "${wd}/support/shell.sh"
if [ ! -e "${documentroot}/calendars/__uids__/user09" ]; then
curl "http://localhost:8008/calendars/__uids__/user09/";
fi;
-python twistedcaldav/test/data/makelargecalendars.py -o 9 -d "${documentroot}";
+"${python}" twistedcaldav/test/data/makelargecalendars.py -o 9 -d "${documentroot}";
-cd "${cdt}" && python testcaldav.py -s "${serverinfo}" "$@";
+cd "${cdt}" && "${python}" testcaldav.py -s "${serverinfo}" "$@";
+
Modified: CalendarServer/branches/users/cdaboo/pods/twext/enterprise/adbapi2.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twext/enterprise/adbapi2.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twext/enterprise/adbapi2.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -703,8 +703,13 @@
def startService(self):
"""
- No startup necessary.
+ Increase the thread pool size of the reactor by the number of threads
+ that this service may consume. This is important because unlike most
+ L{IReactorThreads} users, the connection work units are very long-lived
+ and block until this service has been stopped.
"""
+ tp = self.reactor.getThreadPool()
+ self.reactor.suggestThreadPoolSize(tp.max + self.maxConnections)
@inlineCallbacks
@@ -742,7 +747,10 @@
# independently submitted from .abort() / .close().
yield self._free.pop()._releaseConnection()
+ tp = self.reactor.getThreadPool()
+ self.reactor.suggestThreadPoolSize(tp.max - self.maxConnections)
+
def _createHolder(self):
"""
Create a L{ThreadHolder}. (Test hook.)
Modified: CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/model.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/model.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/model.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -188,7 +188,8 @@
def needsValue(self):
"""
- Does this column require a value in INSERT statements which create rows?
+ Does this column require a value in C{INSERT} statements which create
+ rows?
@return: C{True} for L{Column}s with no default specified which also
cannot be NULL, C{False} otherwise.
Modified: CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/syntax.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/syntax.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/syntax.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -19,7 +19,7 @@
Syntax wrappers and generators for SQL.
"""
-import itertools
+from itertools import count, repeat
from zope.interface import implements
@@ -72,7 +72,7 @@
def __init__(self, dialect):
super(NumericPlaceholder, self).__init__(dialect)
- self._next = itertools.count(1).next
+ self._next = count(1).next
def placeholder(self):
@@ -121,10 +121,37 @@
def _extraVars(self, txn, metadata):
+ """
+ A hook for subclasses to provide additional keyword arguments to the
+ C{bind} call when L{_Statement.on} is executed. Currently this is used
+ only for 'out' parameters to capture results when executing statements
+ that do not normally have a result (L{Insert}, L{Delete}, L{Update}).
+ """
return {}
def _extraResult(self, result, outvars, metadata):
+ """
+ A hook for subclasses to manipulate the results of 'on', after they've
+ been retrieved by the database but before they've been given to
+ application code.
+
+ @param result: a L{Deferred} that will fire with the rows as returned by
+ the database.
+ @type result: C{list} of rows, which are C{list}s or C{tuple}s.
+
+ @param outvars: a dictionary of extra variables returned by
+ C{self._extraVars}.
+
+ @param metadata: information about the connection where the statement
+ was executed.
+
+ @type metadata: L{ConnectionMetadata} (a subclass thereof)
+
+ @return: the result to be returned from L{_Statement.on}.
+
+ @rtype: L{Deferred} firing result rows
+ """
return result
@@ -132,6 +159,19 @@
"""
Execute this statement on a given L{IAsyncTransaction} and return the
resulting L{Deferred}.
+
+ @param txn: the L{IAsyncTransaction} to execute this on.
+
+ @param raiseOnZeroRowCount: the exception to raise if no data was
+ affected or returned by this query.
+
+ @param kw: keyword arguments, mapping names of L{Parameter} objects
+ located somewhere in C{self}
+
+ @return: results from the database.
+
+ @rtype: a L{Deferred} firing a C{list} of records (C{tuple}s or
+ C{list}s)
"""
metadata = self._paramstyles[txn.paramstyle](txn.dialect)
outvars = self._extraVars(txn, metadata)
@@ -139,10 +179,60 @@
fragment = self.toSQL(metadata).bind(**kw)
result = txn.execSQL(fragment.text, fragment.parameters,
raiseOnZeroRowCount)
- return self._extraResult(result, outvars, metadata)
+ result = self._extraResult(result, outvars, metadata)
+ if metadata.dialect == ORACLE_DIALECT and result:
+ result.addCallback(self._fixOracleNulls)
+ return result
+ def _resultColumns(self):
+ """
+ Subclasses must implement this to return a description of the columns
+ expected to be returned. This is a list of L{ColumnSyntax} objects, and
+ possibly other expression syntaxes which will be converted to C{None}.
+ """
+ raise NotImplementedError(
+ "Each statement subclass must describe its result"
+ )
+
+ def _resultShape(self):
+ """
+ Process the result of the subclass's C{_resultColumns}, as described in
+ the docstring above.
+ """
+ for expectation in self._resultColumns():
+ if isinstance(expectation, ColumnSyntax):
+ yield expectation.model
+ else:
+ yield None
+
+
+ def _fixOracleNulls(self, rows):
+ """
+ Oracle treats empty strings as C{NULL}. Fix this by looking at the
+ columns we expect to have returned, and replacing any C{None}s with
+ empty strings in the appropriate position.
+ """
+ if rows is None:
+ return None
+ newRows = []
+ for row in rows:
+ newRow = []
+ for column, description in zip(row, self._resultShape()):
+ if ((description is not None and
+ # FIXME: "is the python type str" is what I mean; this list
+ # should be more centrally maintained
+ description.type.name in ('varchar', 'text', 'char') and
+ column is None
+ )):
+ column = ''
+ newRow.append(column)
+ newRows.append(newRow)
+ return newRows
+
+
+
class Syntax(object):
"""
Base class for syntactic convenience.
@@ -618,7 +708,6 @@
-
class Select(_Statement):
"""
'select' statement.
@@ -719,7 +808,25 @@
return result
+ def _resultColumns(self):
+ """
+ Determine the list of L{ColumnSyntax} objects that will represent the
+ result. Normally just the list of selected columns; if wildcard syntax
+ is used though, determine the ordering from the database.
+ """
+ if self.columns is ALL_COLUMNS:
+ # TODO: Possibly this rewriting should always be done, before even
+ # executing the query, so that if we develop a schema mismatch with
+ # the database (additional columns), the application will still see
+ # the right rows.
+ for table in self.From.tables():
+ for column in table:
+ yield column
+ else:
+ for column in self.columns.columns:
+ yield column
+
def _commaJoined(stmts):
first = True
cstatement = SQLFragment()
@@ -783,7 +890,9 @@
Add a dialect-appropriate 'returning' clause to the end of the given SQL
statement.
- @param metadata: describes the database we are generating the statement for.
+ @param metadata: describes the database we are generating the statement
+ for.
+
@type metadata: L{ConnectionMetadata}
@param stmt: the SQL fragment generated without the 'returning' clause
@@ -839,8 +948,16 @@
return result
+ def _resultColumns(self):
+ return self._returnAsList()
+
+
class _OracleOutParam(object):
+ """
+ A parameter that will be populated using the cx_Oracle API for host
+ variables.
+ """
implements(IDerivedParameter)
def __init__(self, columnSyntax):
@@ -848,9 +965,8 @@
def preQuery(self, cursor):
- self.columnSyntax
typeMap = {'integer': cx_Oracle.NUMBER,
- 'text': cx_Oracle.CLOB,
+ 'text': cx_Oracle.NCLOB,
'varchar': cx_Oracle.STRING,
'timestamp': cx_Oracle.TIMESTAMP}
typeID = self.columnSyntax.model.type.name.lower()
@@ -993,8 +1109,20 @@
-class Lock(_Statement):
+class _LockingStatement(_Statement):
"""
+ A statement related to lock management, which implicitly has no results.
+ """
+ def _resultColumns(self):
+ """
+ No columns should be expected, so return an infinite iterator of None.
+ """
+ return repeat(None)
+
+
+
+class Lock(_LockingStatement):
+ """
An SQL 'lock' statement.
"""
@@ -1015,7 +1143,7 @@
-class Savepoint(_Statement):
+class Savepoint(_LockingStatement):
"""
An SQL 'savepoint' statement.
"""
@@ -1028,7 +1156,7 @@
return SQLFragment('savepoint %s' % (self.name,))
-class RollbackToSavepoint(_Statement):
+class RollbackToSavepoint(_LockingStatement):
"""
An SQL 'rollback to savepoint' statement.
"""
@@ -1041,7 +1169,7 @@
return SQLFragment('rollback to savepoint %s' % (self.name,))
-class ReleaseSavepoint(_Statement):
+class ReleaseSavepoint(_LockingStatement):
"""
An SQL 'release savepoint' statement.
"""
Modified: CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/test/test_sqlsyntax.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/test/test_sqlsyntax.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/test/test_sqlsyntax.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -34,6 +34,7 @@
from twext.enterprise.adbapi2 import ConnectionPool
from twext.enterprise.test.test_adbapi2 import resultOf
from twext.enterprise.test.test_adbapi2 import FakeThreadHolder
+from twisted.internet.defer import succeed
from twisted.trial.unittest import TestCase
@@ -49,6 +50,25 @@
+class FakeCXOracleModule(object):
+ NUMBER = 'the NUMBER type'
+ STRING = 'a string type (for varchars)'
+ NCLOB = 'the NCLOB type. (for text)'
+ TIMESTAMP = 'for timestamps!'
+
+
+
+class NullTestingOracleTxn(object):
+ """
+ Fake transaction for testing oracle NULL behavior.
+ """
+
+ dialect = ORACLE_DIALECT
+ paramstyle = 'numeric'
+
+ def execSQL(self, text, params, exc):
+ return succeed([[None, None]])
+
class GenerationTests(TestCase):
"""
Tests for syntactic helpers to generate SQL queries.
@@ -63,7 +83,10 @@
create table OTHER (BAR integer,
FOO_BAR integer not null);
create table TEXTUAL (MYTEXT varchar(255));
- create table LEVELS (ACCESS integer, USERNAME varchar(255));
+ create table LEVELS (ACCESS integer,
+ USERNAME varchar(255));
+ create table NULLCHECK (ASTRING varchar(255) not null,
+ ANUMBER integer);
""")
self.schema = SchemaSyntax(s)
@@ -121,7 +144,7 @@
self.assertRaises(ValueError, sampleComparison)
- def test_nullComparison(self):
+ def test_compareWithNULL(self):
"""
Comparing a column with None results in the generation of an 'is null'
or 'is not null' SQL statement.
@@ -510,18 +533,14 @@
)
- def test_insertMultiReturnOnOracleTxn(self):
+ def simulateOracleConnection(self):
"""
- As described in L{test_insertMultiReturnOracle}, Oracle deals with
- 'returning' clauses by using out parameters. However, this is not quite
- enough, as the code needs to actually retrieve the values from the out
- parameters.
+ Create a fake oracle-ish connection pool without using real threads or a
+ real database.
+
+ @return: a 3-tuple of L{IAsyncTransaction}, L{ConnectionPool},
+ L{ConnectionFactory}.
"""
- class FakeCXOracleModule(object):
- NUMBER = 'the NUMBER type'
- STRING = 'a string type (for varchars)'
- CLOB = 'the clob type. (for text)'
- TIMESTAMP = 'for timestamps!'
self.patch(syntax, 'cx_Oracle', FakeCXOracleModule)
factory = ConnectionFactory()
pool = ConnectionPool(factory.connect, maxConnections=2,
@@ -531,6 +550,17 @@
pool._createHolder = lambda : FakeThreadHolder(self)
pool.startService()
conn = pool.connection()
+ return conn, pool, factory
+
+
+ def test_insertMultiReturnOnOracleTxn(self):
+ """
+ As described in L{test_insertMultiReturnOracle}, Oracle deals with
+ 'returning' clauses by using out parameters. However, this is not quite
+ enough, as the code needs to actually retrieve the values from the out
+ parameters.
+ """
+ conn, pool, factory = self.simulateOracleConnection()
i = Insert({self.schema.FOO.BAR: 40,
self.schema.FOO.BAZ: 50},
Return=(self.schema.FOO.BAR, self.schema.FOO.BAZ))
@@ -828,6 +858,57 @@
)
+ def test_rewriteOracleNULLs_Select(self):
+ """
+ Oracle databases cannot distinguish between the empty string and
+ C{NULL}. When you insert an empty string, C{cx_Oracle} therefore treats
+ it as a C{None} and will return that when you select it back again. We
+ address this in the schema by dropping 'not null' constraints.
+
+ Therefore, when executing a statement which includes a string column,
+ 'on' should rewrite None return values from C{cx_Oracle} to be empty
+ bytestrings, but only for string columns.
+ """
+
+ rows = resultOf(
+ Select([self.schema.NULLCHECK.ASTRING,
+ self.schema.NULLCHECK.ANUMBER],
+ From=self.schema.NULLCHECK).on(NullTestingOracleTxn()))[0]
+
+ self.assertEquals(rows, [['', None]])
+
+
+ def test_rewriteOracleNULLs_SelectAllColumns(self):
+ """
+ Same as L{test_rewriteOracleNULLs_Select}, but with the L{ALL_COLUMNS}
+ shortcut.
+ """
+ rows = resultOf(
+ Select(From=self.schema.NULLCHECK).on(NullTestingOracleTxn())
+ )[0]
+ self.assertEquals(rows, [['', None]])
+
+
+ def test_rewriteOracleNULLs_Insert(self):
+ """
+ The behavior described in L{test_rewriteOracleNULLs_Select} applies to
+ other statement types as well, specifically those with 'returning'
+ clauses.
+ """
+ conn, pool, factory = self.simulateOracleConnection()
+ # Add 2 cursor variable values so that these will be used by
+ # FakeVariable.getvalue.
+ factory.varvals.extend([None, None])
+ rows = resultOf(
+ Insert({self.schema.NULLCHECK.ASTRING: '',
+ self.schema.NULLCHECK.ANUMBER: None},
+ Return=[self.schema.NULLCHECK.ASTRING,
+ self.schema.NULLCHECK.ANUMBER]
+ ).on(conn))[0]
+
+ self.assertEquals(rows, [['', None]])
+
+
def test_nestedLogicalExpressions(self):
"""
Make sure that logical operator precedence inserts proper parenthesis
Modified: CalendarServer/branches/users/cdaboo/pods/twext/enterprise/test/test_adbapi2.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twext/enterprise/test/test_adbapi2.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twext/enterprise/test/test_adbapi2.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -20,6 +20,9 @@
from itertools import count
+from zope.interface.verify import verifyClass
+
+from twisted.python.threadpool import ThreadPool
from twisted.trial.unittest import TestCase
from twisted.internet.defer import execute
@@ -28,6 +31,8 @@
from twisted.internet.defer import Deferred
from twext.enterprise.ienterprise import ConnectionError
from twext.enterprise.ienterprise import AlreadyFinishedError
+from twisted.internet.interfaces import IReactorThreads
+from zope.interface.declarations import implements
from twext.enterprise.adbapi2 import ConnectionPool
@@ -192,6 +197,9 @@
def getvalue(self):
+ vv = self.cursor.connection.parent.varvals
+ if vv:
+ return vv.pop(0)
return self.cursor.variables.index(self) + 300
@@ -206,6 +214,7 @@
self.idcounter = count(1)
self._connectResultQueue = []
self.defaultConnect()
+ self.varvals = []
@property
@@ -324,6 +333,81 @@
+class ClockWithThreads(Clock):
+ """
+ A testing reactor that supplies L{IReactorTime} and L{IReactorThreads}.
+ """
+ implements(IReactorThreads)
+
+ def __init__(self):
+ super(ClockWithThreads, self).__init__()
+ self._pool = ThreadPool()
+
+
+ def getThreadPool(self):
+ """
+ Get the threadpool.
+ """
+ return self._pool
+
+
+ def suggestThreadPoolSize(self, size):
+ """
+ Approximate the behavior of a 'real' reactor.
+ """
+ self._pool.adjustPoolsize(maxthreads=size)
+
+
+ def callInThread(self, thunk, *a, **kw):
+ """
+ No implementation.
+ """
+
+
+ def callFromThread(self, thunk, *a, **kw):
+ """
+ No implementation.
+ """
+
+
+verifyClass(IReactorThreads, ClockWithThreads)
+
+
+
+class ConnectionPoolBootTests(TestCase):
+ """
+ Tests for the start-up phase of L{ConnectionPool}.
+ """
+
+ def test_threadCount(self):
+ """
+ The reactor associated with a L{ConnectionPool} will have its maximum
+ thread count adjusted when L{ConnectionPool.startService} is called, to
+ accomodate for L{ConnectionPool.maxConnections} additional threads.
+
+ Stopping the service should restore it to its original value, so that a
+ repeatedly re-started L{ConnectionPool} will not cause the thread
+ ceiling to grow without bound.
+ """
+ defaultMax = 27
+ connsMax = 45
+ combinedMax = defaultMax + connsMax
+ pool = ConnectionPool(None, maxConnections=connsMax)
+ pool.reactor = ClockWithThreads()
+ threadpool = pool.reactor.getThreadPool()
+ pool.reactor.suggestThreadPoolSize(defaultMax)
+ self.assertEquals(threadpool.max, defaultMax)
+ pool.startService()
+ self.assertEquals(threadpool.max, combinedMax)
+ justChecking = []
+ pool.stopService().addCallback(justChecking.append)
+ # No SQL run, so no threads started, so this deferred should fire
+ # immediately. If not, we're in big trouble, so sanity check.
+ self.assertEquals(justChecking, [None])
+ self.assertEquals(threadpool.max, defaultMax)
+
+
+
class ConnectionPoolTests(TestCase):
"""
Tests for L{ConnectionPool}.
@@ -340,7 +424,7 @@
self.pool = ConnectionPool(self.factory.connect,
maxConnections=2)
self.pool._createHolder = self.makeAHolder
- self.clock = self.pool.reactor = Clock()
+ self.clock = self.pool.reactor = ClockWithThreads()
self.pool.startService()
Modified: CalendarServer/branches/users/cdaboo/pods/twext/enterprise/util.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twext/enterprise/util.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twext/enterprise/util.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -30,8 +30,10 @@
@param column: a single value from a column.
@return: a converted value based on the type of the input; oracle CLOBs and
- datetime timestamps will be converted to strings, all other types will
- be left alone.
+ datetime timestamps will be converted to strings, unicode values will be
+ converted to UTF-8 encoded byte sequences (C{str}s), and floating point
+ numbers will be converted to integer types if they are integers. Any
+ other types will be left alone.
"""
if hasattr(column, 'read'):
# Try to detect large objects and format convert them to
@@ -40,7 +42,7 @@
# http://cx-oracle.sourceforge.net/html/lob.html - in
# particular, the part where it says "In particular, do not
# use the fetchall() method".
- return column.read()
+ column = column.read()
elif isinstance(column, datetime):
# cx_Oracle properly maps the type of timestamps to datetime
# objects. However, our code is mostly written against
@@ -50,12 +52,20 @@
# we'll do that.
return column.strftime(SQL_TIMESTAMP_FORMAT)
elif isinstance(column, float):
+ # cx_Oracle maps _all_ nubmers to float types, which is more consistent,
+ # but we expect the database to be able to store integers as integers
+ # (in fact almost all the values in our schema are integers), so we map
+ # those values which exactly match back into integers.
if int(column) == column:
return int(column)
else:
return column
- else:
- return column
+ if isinstance(column, unicode):
+ # Finally, we process all data as UTF-8 bytestrings in order to reduce
+ # memory consumption. Pass any unicode string values back to the
+ # application as unicode.
+ column = column.encode('utf-8')
+ return column
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/config.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/config.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/config.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -135,6 +135,8 @@
else:
self._provider = provider
self._updating = False
+ self._beforeResetHook = None
+ self._afterResetHook = None
self._preUpdateHooks = []
self._postUpdateHooks = []
self.reset()
@@ -179,15 +181,25 @@
lastDict[configItem] = defaultValue
return defaultValue
+ def addResetHooks(self, before, after):
+ """
+ Hooks for preserving config across reload( ) + reset( )
+
+ Each hook will be passed the config data; whatever the before hook
+ returns will be passed as the second arg to the after hook.
+ """
+ self._beforeResetHook = before
+ self._afterResetHook = after
+
def addPreUpdateHooks(self, hooks):
self._preUpdateHooks.extend(hooks)
-
+
def addPostUpdateHooks(self, hooks):
self._postUpdateHooks.extend(hooks)
def getProvider(self):
return self._provider
-
+
def setProvider(self, provider):
self._provider = provider
self.reset()
@@ -231,7 +243,16 @@
configDict = ConfigDict(self._provider.loadConfig())
configDict._reloading = True
if not self._provider.hasErrors():
+ if self._beforeResetHook:
+ # Give the beforeResetHook a chance to stash away values we want
+ # to preserve across the reload( )
+ preserved = self._beforeResetHook(self._data)
+ else:
+ preserved = None
self.reset()
+ if preserved and self._afterResetHook:
+ # Pass the preserved data back to the afterResetHook
+ self._afterResetHook(self._data, preserved)
self.update(configDict)
else:
raise ConfigurationError("Invalid configuration in %s"
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/appleopendirectory.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/appleopendirectory.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/appleopendirectory.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -422,12 +422,6 @@
recordGUID = value.get(dsattributes.kDS1AttrGeneratedUID)
- # Skip if group restriction is in place and guid is not
- # a member
- if self.restrictedGUIDs is not None:
- if str(recordGUID) not in self.restrictedGUIDs:
- continue
-
recordType = value.get(dsattributes.kDSNAttrRecordType)
if isinstance(recordType, list):
recordType = recordType[0]
@@ -435,6 +429,13 @@
continue
recordType = self._fromODRecordTypes[recordType]
+ # Skip if group restriction is in place and guid is not
+ # a member (but don't skip any groups)
+ if (recordType != self.recordType_groups and
+ self.restrictedGUIDs is not None):
+ if str(recordGUID) not in self.restrictedGUIDs:
+ continue
+
recordAuthIDs = self._setFromAttribute(
value.get(dsattributes.kDSNAttrAltSecurityIdentities))
recordFullName = value.get(
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/augment.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/augment.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/augment.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -420,7 +420,10 @@
"""
super(AugmentXMLDB, self).refresh()
try:
- self.db = self._parseXML()
+ results = self._parseXML()
+ # Only update the cache if _parseXML( ) returns anything
+ if results:
+ self.db = results
except RuntimeError:
log.error("Failed to parse XML augments file during cache refresh - \
ignoring") self.lastCached = time.time()
@@ -435,16 +438,18 @@
self.removeAugmentRecords(self.db.keys())
return succeed(None)
- def _shouldReparse(self, xmlFile):
+ def _shouldReparse(self, xmlFiles):
"""
- Check to see whether the given file has been modified since we last
- parsed it.
+ Check to see whether any of the given files have been modified since
+ we last parsed them.
"""
- oldModTime, oldSize = self.xmlFileStats.get(xmlFile, (0, 0))
- newModTime = os.path.getmtime(xmlFile)
- newSize = os.path.getsize(xmlFile)
- if (oldModTime != newModTime) or (oldSize != newSize):
- return True
+ for xmlFile in xmlFiles:
+ if os.path.exists(xmlFile):
+ oldModTime, oldSize = self.xmlFileStats.get(xmlFile, (0, 0))
+ newModTime = os.path.getmtime(xmlFile)
+ newSize = os.path.getsize(xmlFile)
+ if (oldModTime != newModTime) or (oldSize != newSize):
+ return True
return False
def _parseXML(self):
@@ -454,23 +459,13 @@
If none of the xmlFiles exist, create a default record.
"""
- # Do each file
results = {}
- allMissing = True
+ # If all augments files are missing, return a default record
for xmlFile in self.xmlFiles:
if os.path.exists(xmlFile):
- # Compare previously seen modification time and size of each
- # xml file. If unchanged, skip.
- if self._shouldReparse(xmlFile):
- # Creating a parser does the parse
- XMLAugmentsParser(xmlFile, results)
- newModTime = os.path.getmtime(xmlFile)
- newSize = os.path.getsize(xmlFile)
- self.xmlFileStats[xmlFile] = (newModTime, newSize)
- allMissing = False
-
- if allMissing:
+ break
+ else:
results["Default"] = AugmentRecord(
"Default",
enabled=True,
@@ -478,6 +473,17 @@
enabledForAddressBooks=True,
)
+ # Compare previously seen modification time and size of each
+ # xml file. If all are unchanged, skip.
+ if self._shouldReparse(self.xmlFiles):
+ for xmlFile in self.xmlFiles:
+ if os.path.exists(xmlFile):
+ # Creating a parser does the parse
+ XMLAugmentsParser(xmlFile, results)
+ newModTime = os.path.getmtime(xmlFile)
+ newSize = os.path.getsize(xmlFile)
+ self.xmlFileStats[xmlFile] = (newModTime, newSize)
+
return results
class AugmentADAPI(AugmentDB, AbstractADBAPIDatabase):
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/cachingdirectory.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/cachingdirectory.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/cachingdirectory.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -74,6 +74,7 @@
CachingDirectoryService.INDEX_TYPE_AUTHID : {},
}
self.directoryService = directoryService
+ self.lastPurgedTime = time.time()
def addRecord(self, record, indexType, indexKey, useMemcache=True,
neverExpire=False):
@@ -121,8 +122,22 @@
self.log_debug("Missing record index item; type: %s, item: \
%s" % (indexType, item))
def findRecord(self, indexType, indexKey):
+ self.purgeExpiredRecords()
return self.recordsIndexedBy[indexType].get(indexKey)
+
+ def purgeExpiredRecords(self):
+ """
+ Scan the cached records and remove any that have expired.
+ Does nothing if we've scanned within the past cacheTimeout seconds.
+ """
+ if time.time() - self.lastPurgedTime > self.directoryService.cacheTimeout:
+ for record in list(self.records):
+ if record.isExpired():
+ self.removeRecord(record)
+ self.lastPurgedTime = time.time()
+
+
class CachingDirectoryService(DirectoryService):
"""
Caching Directory implementation of L{IDirectoryService}.
@@ -272,10 +287,7 @@
record = self.recordCacheForType(recordType).findRecord(indexType, \
indexKey)
if record:
- if (
- record.cachedTime != 0 and
- time.time() - record.cachedTime > self.cacheTimeout
- ):
+ if record.isExpired():
self.recordCacheForType(recordType).removeRecord(record)
return None
else:
@@ -386,6 +398,20 @@
def neverExpire(self):
self.cachedTime = 0
+ def isExpired(self):
+ """
+ Returns True if this record was created more than cacheTimeout
+ seconds ago
+ """
+ if (
+ self.cachedTime != 0 and
+ time.time() - self.cachedTime > self.service.cacheTimeout
+ ):
+ return True
+ else:
+ return False
+
+
class DirectoryMemcacheError(DirectoryError):
"""
Error communicating with memcached.
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_augment.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_augment.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_augment.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -281,10 +281,20 @@
newxmlfile = FilePath(self.mktemp())
FilePath(xmlFile).copyTo(newxmlfile)
db = AugmentXMLDB((newxmlfile.path,))
- self.assertFalse(db._shouldReparse(newxmlfile.path)) # No need to parse
+ self.assertFalse(db._shouldReparse([newxmlfile.path])) # No need to parse
newxmlfile.setContent("") # Change the file
- self.assertTrue(db._shouldReparse(newxmlfile.path)) # Need to parse
+ self.assertTrue(db._shouldReparse([newxmlfile.path])) # Need to parse
+ def test_refresh(self):
+ """
+ Ensure that a refresh without any file changes doesn't zero out the
+ cache
+ """
+ dbxml = AugmentXMLDB((xmlFile,))
+ keys = dbxml.db.keys()
+ dbxml.refresh()
+ self.assertEquals(keys, dbxml.db.keys())
+
class AugmentSqliteTests(AugmentTests, AugmentTestsMixin):
def _db(self, dbpath=None):
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_opendirectory.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_opendirectory.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_opendirectory.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -322,6 +322,8 @@
dsattributes.kDSNAttrRecordType : \
dsattributes.kDSStdRecordTypeGroups, },
),
+ dsattributes.kDSStdRecordTypePlaces : (),
+ dsattributes.kDSStdRecordTypeResources : (),
}
def attributeMatches(fieldValue, value, caseless, matchType):
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/resource.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/resource.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/resource.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -1164,7 +1164,7 @@
if principal is None:
return (None, None, None)
else:
- return (principal.record.fullName.decode("utf-8"),
+ return (principal.record.fullName,
principal.record.guid,
principal.record.calendarUserAddresses)
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/scheduling/ischedule.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/scheduling/ischedule.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/scheduling/ischedule.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -303,7 +303,7 @@
if principal is None:
return (None, None, None)
else:
- return (principal.record.fullName.decode("utf-8"),
+ return (principal.record.fullName,
principal.record.guid,
principal.record.calendarUserAddresses)
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/servers.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/servers.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/servers.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -114,12 +114,20 @@
self.thisServer = parsed_uri.port in (config.SSLPort,) + \
tuple(config.BindSSLPorts)
# Need to cache IP addresses
- _ignore_host, _ignore_aliases, ips = \
socket.gethostbyname_ex(parsed_uri.hostname) + try:
+ _ignore_host, _ignore_aliases, ips = \
socket.gethostbyname_ex(parsed_uri.hostname) + except socket.gaierror, e:
+ log.error("Unable to lookup ip-addr for server '%s': %s" % \
(parsed_uri.hostname, str(e))) + ips = ()
self.ips = set(ips)
for uri in self.partitions.values():
parsed_uri = urlparse.urlparse(uri)
- _ignore_host, _ignore_aliases, ips = \
socket.gethostbyname_ex(parsed_uri.hostname) + try:
+ _ignore_host, _ignore_aliases, ips = \
socket.gethostbyname_ex(parsed_uri.hostname) + except socket.gaierror, e:
+ log.error("Unable to lookup ip-addr for partition '%s': %s" % \
(parsed_uri.hostname, str(e))) + ips = ()
self.partitions_ips.update(ips)
def checkThisIP(self, ip):
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/stdconfig.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/stdconfig.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/stdconfig.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -644,7 +644,7 @@
},
# Umask
- "umask": 0027,
+ "umask": 0022,
# A TCP port used for communication between the child and master
# processes (bound to 127.0.0.1). Specify 0 to let OS assign a port.
@@ -1263,3 +1263,34 @@
config.setProvider(PListConfigProvider(DEFAULT_CONFIG))
config.addPreUpdateHooks(PRE_UPDATE_HOOKS)
config.addPostUpdateHooks(POST_UPDATE_HOOKS)
+
+
+def _preserveConfig(configDict):
+ """
+ Preserve certain config keys across reset( ) because these can't be
+ re-fetched after the process has shed privileges
+ """
+ iMIP = configDict.Scheduling.iMIP
+ XMPP = configDict.Notifications.Services.XMPPNotifier
+ preserved = {
+ "iMIPPassword" : iMIP.Password,
+ "MailSendingPassword" : iMIP.Sending.Password,
+ "MailReceivingPassword" : iMIP.Receiving.Password,
+ "XMPPPassword" : XMPP.Password,
+ }
+ return preserved
+
+def _restoreConfig(configDict, preserved):
+ """
+ Restore certain config keys across reset( ) because these can't be
+ re-fetched after the process has shed privileges
+ """
+ iMIP = configDict.Scheduling.iMIP
+ XMPP = configDict.Notifications.Services.XMPPNotifier
+ iMIP.Password = preserved["iMIPPassword"]
+ iMIP.Sending.Password = preserved["MailSendingPassword"]
+ iMIP.Receiving.Password = preserved["MailReceivingPassword"]
+ XMPP.Password = preserved["XMPPPassword"]
+
+
+config.addResetHooks(_preserveConfig, _restoreConfig)
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_config.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_config.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_config.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -43,6 +43,37 @@
<string>debug</string>
</dict>
+ <key>Notifications</key>
+ <dict>
+ <key>Services</key>
+ <dict>
+ <key>XMPPNotifier</key>
+ <dict>
+ <key>Password</key>
+ <string>xmpp</string>
+ </dict>
+ </dict>
+ </dict>
+
+ <key>Scheduling</key>
+ <dict>
+ <key>iMIP</key>
+ <dict>
+ <key>Password</key>
+ <string>imip</string>
+ <key>Sending</key>
+ <dict>
+ <key>Password</key>
+ <string>sending</string>
+ </dict>
+ <key>Receiving</key>
+ <dict>
+ <key>Password</key>
+ <string>receiving</string>
+ </dict>
+ </dict>
+ </dict>
+
</dict>
</plist>
"""
@@ -127,6 +158,28 @@
self.assertEquals(config.HTTPPort, 8008)
+ def testPreserveAcrossReload(self):
+ self.assertEquals(config.Scheduling.iMIP.Password, "")
+ self.assertEquals(config.Scheduling.iMIP.Sending.Password, "")
+ self.assertEquals(config.Scheduling.iMIP.Receiving.Password, "")
+ self.assertEquals(config.Notifications.Services.XMPPNotifier.Password, "")
+
+ config.load(self.testConfig)
+
+ self.assertEquals(config.Scheduling.iMIP.Password, "imip")
+ self.assertEquals(config.Scheduling.iMIP.Sending.Password, "sending")
+ self.assertEquals(config.Scheduling.iMIP.Receiving.Password, "receiving")
+ self.assertEquals(config.Notifications.Services.XMPPNotifier.Password, \
"xmpp") +
+ writePlist({}, self.testConfig)
+
+ config.reload()
+
+ self.assertEquals(config.Scheduling.iMIP.Password, "imip")
+ self.assertEquals(config.Scheduling.iMIP.Sending.Password, "sending")
+ self.assertEquals(config.Scheduling.iMIP.Receiving.Password, "receiving")
+ self.assertEquals(config.Notifications.Services.XMPPNotifier.Password, \
"xmpp") +
def testSetAttr(self):
self.assertNotIn("BindAddresses", config.__dict__)
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_upgrade.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_upgrade.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_upgrade.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -30,6 +30,7 @@
import hashlib
import os, zlib, cPickle
+from txdav.caldav.datastore.index_file import db_basename
@@ -290,7 +291,11 @@
"calendars": {
"users": {
"wsanchez": {
- "calendar" : {},
+ "calendar" : {
+ db_basename : {
+ "@contents": "",
+ },
+ },
"notifications": {
"sample-notification.xml": {
"@contents": "<?xml \
version='1.0'>\n<should-be-ignored />" @@ -307,7 +312,11 @@
"64" : {
"23" : {
"6423F94A-6B76-4A3A-815B-D52CFD77935D" : {
- "calendar": {},
+ "calendar": {
+ db_basename : {
+ "@contents": "",
+ },
+ },
}
}
}
@@ -342,6 +351,9 @@
{
"calendar" :
{
+ db_basename : {
+ "@contents": "",
+ },
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" :
{
"@contents" : event01_before,
@@ -357,6 +369,9 @@
},
"inbox" :
{
+ db_basename : {
+ "@contents": "",
+ },
"@xattrs" :
{
# Pickled XML Doc
@@ -371,6 +386,9 @@
{
"calendar" :
{
+ db_basename : {
+ "@contents": "",
+ },
},
},
},
@@ -407,6 +425,9 @@
{
"calendar" :
{
+ db_basename : {
+ "@contents": "",
+ },
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" :
{
"@contents" : event01_after,
@@ -422,6 +443,9 @@
},
"inbox" :
{
+ db_basename : {
+ "@contents": "",
+ },
"@xattrs" :
{
freeBusyAttr : zlib.compress("<?xml \
version='1.0' encoding='UTF-8'?>\n<calendar-free-busy-set \
xmlns='urn:ietf:params:xml:ns:caldav'>\r\n <href \
xmlns='DAV:'>/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/calendar/</href>\r\n</calendar-free-busy-set>"),
@@ -438,6 +462,9 @@
{
"calendar" :
{
+ db_basename : {
+ "@contents": "",
+ },
},
},
},
@@ -645,13 +672,16 @@
"23" : {
"6423F94A-6B76-4A3A-815B-D52CFD77935D" : {
"calendar" : {
+ db_basename : {
+ "@contents": "",
+ },
},
"garbage.ics" : {
"@contents": "Oops, not actually an ICS file.",
},
"other-file.txt": {
"@contents": "Also not a calendar collection."
- }
+ },
}
}
},
@@ -727,6 +757,122 @@
@inlineCallbacks
+ def test_calendarsUpgradeWithNestedCollections(self):
+ """
+ Unknown files, including .DS_Store files at any point in the hierarchy,
+ as well as non-directory in a user's calendar home, will be ignored and not
+ interrupt an upgrade.
+ """
+
+ self.setUpXMLDirectory()
+
+ beforeUIDContents = {
+ "64" : {
+ "23" : {
+ "6423F94A-6B76-4A3A-815B-D52CFD77935D" : {
+ "calendar" : {
+ db_basename : {
+ "@contents": "",
+ },
+ },
+ "nested1": {
+ "nested2": {},
+ },
+ }
+ }
+ },
+ ".DS_Store" : {
+ "@contents" : "",
+ }
+ }
+
+ afterUIDContents = {
+ "64" : {
+ "23" : {
+ "6423F94A-6B76-4A3A-815B-D52CFD77935D" : {
+ "calendar" : {
+ db_basename : {
+ "@contents": "",
+ },
+ },
+ ".collection.nested1": {
+ "nested2": {},
+ },
+ }
+ }
+ },
+ ".DS_Store" : {
+ "@contents" : "",
+ }
+ }
+
+ before = {
+ ".DS_Store" :
+ {
+ "@contents" : "",
+ },
+ "calendars" :
+ {
+ ".DS_Store" :
+ {
+ "@contents" : "",
+ },
+ "__uids__" :beforeUIDContents,
+ },
+ "principals" :
+ {
+ ".DS_Store" :
+ {
+ "@contents" : "",
+ },
+ OLDPROXYFILE :
+ {
+ "@contents" : "",
+ }
+ }
+ }
+
+ after = {
+ ".DS_Store" :
+ {
+ "@contents" : "",
+ },
+ "tasks" :
+ {
+ "incoming" :
+ {
+ },
+ },
+ ".calendarserver_version" :
+ {
+ "@contents" : "2",
+ },
+ "calendars" :
+ {
+ ".DS_Store" :
+ {
+ "@contents" : "",
+ },
+ "__uids__" : afterUIDContents,
+ },
+ NEWPROXYFILE :
+ {
+ "@contents" : None,
+ },
+ MailGatewayTokensDatabase.dbFilename :
+ {
+ "@contents" : None,
+ },
+ "%s-journal" % (MailGatewayTokensDatabase.dbFilename,) :
+ {
+ "@contents" : None
+ },
+ }
+
+ (yield self.verifyDirectoryComparison(before, after, reverify=True))
+
+
+ @inlineCallbacks
def test_calendarsUpgradeWithUIDs(self):
"""
Verify that calendar homes in the /calendars/__uids__/<guid>/ form
@@ -744,6 +890,9 @@
{
"calendar" :
{
+ db_basename : {
+ "@contents": "",
+ },
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" :
{
"@contents" : event01_before,
@@ -751,6 +900,9 @@
},
"inbox" :
{
+ db_basename : {
+ "@contents": "",
+ },
"@xattrs" :
{
# Plain XML
@@ -792,6 +944,9 @@
{
"calendar" :
{
+ db_basename : {
+ "@contents": "",
+ },
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" :
{
"@contents" : event01_after,
@@ -803,6 +958,9 @@
},
"inbox" :
{
+ db_basename : {
+ "@contents": "",
+ },
"@xattrs" :
{
freeBusyAttr : zlib.compress("<?xml \
version='1.0' encoding='UTF-8'?>\n<calendar-free-busy-set \
xmlns='urn:ietf:params:xml:ns:caldav'>\r\n <href \
xmlns='DAV:'>/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/calendar/</href>\r\n</calendar-free-busy-set>"),
@@ -852,6 +1010,9 @@
{
"calendar" :
{
+ db_basename : {
+ "@contents": "",
+ },
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" :
{
"@contents" : event01_before,
@@ -868,6 +1029,9 @@
},
"inbox" :
{
+ db_basename : {
+ "@contents": "",
+ },
"@xattrs" :
{
# Zlib compressed XML
@@ -908,6 +1072,9 @@
{
"calendar" :
{
+ db_basename : {
+ "@contents": "",
+ },
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" :
{
"@contents" : event01_after,
@@ -924,6 +1091,9 @@
},
"inbox" :
{
+ db_basename : {
+ "@contents": "",
+ },
"@xattrs" :
{
freeBusyAttr : zlib.compress("<?xml \
version='1.0' encoding='UTF-8'?>\n<calendar-free-busy-set \
xmlns='urn:ietf:params:xml:ns:caldav'>\r\n <href \
xmlns='DAV:'>/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/calendar/</href>\r\n</calendar-free-busy-set>"),
@@ -972,6 +1142,9 @@
{
"calendar" :
{
+ db_basename : {
+ "@contents": "",
+ },
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" :
{
"@contents" : event01_after,
@@ -988,6 +1161,9 @@
},
"inbox" :
{
+ db_basename : {
+ "@contents": "",
+ },
"@xattrs" :
{
# Zlib compressed XML
@@ -1028,6 +1204,9 @@
{
"calendar" :
{
+ db_basename : {
+ "@contents": "",
+ },
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" :
{
"@contents" : event01_after,
@@ -1044,6 +1223,9 @@
},
"inbox" :
{
+ db_basename : {
+ "@contents": "",
+ },
"@xattrs" :
{
freeBusyAttr : zlib.compress("<?xml \
version='1.0' encoding='UTF-8'?>\r\n<calendar-free-busy-set \
xmlns='urn:ietf:params:xml:ns:caldav'>\r\n <href \
xmlns='DAV:'>/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/calendar/</href>\r\n</calendar-free-busy-set>\r\n"),
@@ -1093,6 +1275,9 @@
{
"calendar" :
{
+ db_basename : {
+ "@contents": "",
+ },
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" :
{
"@contents" : event01_before,
@@ -1133,6 +1318,9 @@
{
"calendar" :
{
+ db_basename : {
+ "@contents": "",
+ },
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" :
{
"@contents" : event01_after,
Modified: CalendarServer/branches/users/cdaboo/pods/twistedcaldav/upgrade.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/twistedcaldav/upgrade.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/upgrade.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -116,7 +116,7 @@
if principal is None:
return (None, None, None)
else:
- return (principal.record.fullName.decode("utf-8"),
+ return (principal.record.fullName,
principal.record.guid,
principal.record.calendarUserAddresses)
@@ -588,16 +588,23 @@
docRoot = config.DocumentRoot
if os.path.exists(docRoot):
calRoot = os.path.join(docRoot, "calendars")
- if os.path.exists(calRoot):
+ if os.path.exists(calRoot) and os.path.isdir(calRoot):
uidHomes = os.path.join(calRoot, "__uids__")
- for path1 in os.listdir(uidHomes):
- uidLevel1 = os.path.join(uidHomes, path1)
- for path2 in os.listdir(uidLevel1):
- uidLevel2 = os.path.join(uidLevel1, path2)
- for home in os.listdir(uidLevel2):
- calHome = os.path.join(uidLevel2, home)
- if not flattenHome(calHome):
- errorOccurred = True
+ if os.path.isdir(uidHomes):
+ for path1 in os.listdir(uidHomes):
+ uidLevel1 = os.path.join(uidHomes, path1)
+ if not os.path.isdir(uidLevel1):
+ continue
+ for path2 in os.listdir(uidLevel1):
+ uidLevel2 = os.path.join(uidLevel1, path2)
+ if not os.path.isdir(uidLevel2):
+ continue
+ for home in os.listdir(uidLevel2):
+ calHome = os.path.join(uidLevel2, home)
+ if not os.path.isdir(calHome):
+ continue
+ if not flattenHome(calHome):
+ errorOccurred = True
return errorOccurred
Modified: CalendarServer/branches/users/cdaboo/pods/txdav/base/datastore/dbapiclient.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/txdav/base/datastore/dbapiclient.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/txdav/base/datastore/dbapiclient.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -99,7 +99,13 @@
def execute(self, sql, args=()):
realArgs = []
for arg in args:
- if isinstance(arg, (str, unicode)) and len(arg) > 1024:
+ if isinstance(arg, str):
+ # We use NCLOB everywhere, so cx_Oracle requires a unicode-type
+ # input. But we mostly pass around utf-8 encoded bytes at the
+ # application layer as they consume less memory, so do the
+ # conversion here.
+ arg = arg.decode('utf-8')
+ if isinstance(arg, unicode) and len(arg) > 1024:
# This *may* cause a type mismatch, but none of the non-CLOB
# strings that we're passing would allow a value this large
# anyway. Smaller strings will be automatically converted by
@@ -107,7 +113,7 @@
# sure why cx_Oracle itself doesn't just do the following hack
# automatically and internally for larger values too, but, here
# it is:
- v = self.var(cx_Oracle.CLOB, len(arg) + 1)
+ v = self.var(cx_Oracle.NCLOB, len(arg) + 1)
v.setvalue(0, arg)
else:
v = arg
Property changes on: \
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/index_file.py \
___________________________________________________________________
Modified: svn:mergeinfo
- /CalendarServer/branches/config-separation/txdav/caldav/datastore/index_file.py:4379-4443
/CalendarServer/branches/egg-info-351/txdav/caldav/datastore/index_file.py:4589-4625
/CalendarServer/branches/generic-sqlstore/txdav/caldav/datastore/index_file.py:6167-6191
/CalendarServer/branches/new-store/txdav/caldav/datastore/index_file.py:5594-5934
/CalendarServer/branches/new-store-no-caldavfile/txdav/caldav/datastore/index_file.py:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2/txdav/caldav/datastore/index_file.py:5936-5981
/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/caldav/datastore/index_file.py:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/caldav/datastore/index_file.py:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/caldav/datastore/index_file.py:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/caldav/datastore/index_file.py:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/txdav/caldav/datastore/index_file.py:4465-4957
/CalendarServer/branches/users/cdaboo/pycalendar/txdav/caldav/datastore/index_file.py:7085-7206
/CalendarServer/branches/users/cdaboo/pycard/txdav/caldav/datastore/index_file.py:7227-7237
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/caldav/datastore/index_file.py:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/caldav/datastore/index_file.py:5188-5440
/CalendarServer/branches/users/glyph/conn-limit/txdav/caldav/datastore/index_file.py:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/caldav/datastore/index_file.py:4971-5080
/CalendarServer/branches/users/glyph/dalify/txdav/caldav/datastore/index_file.py:6932-7023
/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/caldav/datastore/index_file.py:6592-6614
/CalendarServer/branches/users/glyph/linux-tests/txdav/caldav/datastore/index_file.py:6893-6900
/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/caldav/datastore/index_file.py:6322-6334
/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py:6369
/CalendarServer/branches/users/glyph/oracle/txdav/caldav/datastore/index_file.py:7106-7155
/CalendarServer/branches/users/glyph/sendfdport/txdav/caldav/datastore/index_file.py:5388-5424
/CalendarServer/branches/users/glyph/sharedpool/txdav/caldav/datastore/index_file.py:6490-6550
/CalendarServer/branches/users/glyph/sql-store/txdav/caldav/datastore/index_file.py:5929-6073
/CalendarServer/branches/users/glyph/subtransactions/txdav/caldav/datastore/index_file.py:7248-7258
/CalendarServer/branches/users/glyph/use-system-twisted/txdav/caldav/datastore/index_file.py:5084-5149
/CalendarServer/branches/users/sagen/locations-resources/txdav/caldav/datastore/index_file.py:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2/txdav/caldav/datastore/index_file.py:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events/txdav/caldav/datastore/index_file.py:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/caldav/datastore/index_file.py:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/caldav/datastore/index_file.py:4068-4075
/CalendarServer/branches/users/sagen/resources-2/txdav/caldav/datastore/index_file.py:5084-5093
/CalendarServer/branches/users/wsanchez/transations/txdav/caldav/datastore/index_file.py:5515-5593
/CalendarServer/trunk/twistedcaldav/index.py:6322-6394
+ /CalendarServer/branches/config-separation/txdav/caldav/datastore/index_file.py:4379-4443
/CalendarServer/branches/egg-info-351/txdav/caldav/datastore/index_file.py:4589-4625
/CalendarServer/branches/generic-sqlstore/txdav/caldav/datastore/index_file.py:6167-6191
/CalendarServer/branches/new-store-no-caldavfile-2/txdav/caldav/datastore/index_file.py:5936-5981
/CalendarServer/branches/new-store-no-caldavfile/txdav/caldav/datastore/index_file.py:5911-5935
/CalendarServer/branches/new-store/txdav/caldav/datastore/index_file.py:5594-5934
/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/caldav/datastore/index_file.py:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/caldav/datastore/index_file.py:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/caldav/datastore/index_file.py:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/caldav/datastore/index_file.py:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/txdav/caldav/datastore/index_file.py:4465-4957
/CalendarServer/branches/users/cdaboo/pycalendar/txdav/caldav/datastore/index_file.py:7085-7206
/CalendarServer/branches/users/cdaboo/pycard/txdav/caldav/datastore/index_file.py:7227-7237
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/caldav/datastore/index_file.py:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/caldav/datastore/index_file.py:5188-5440
/CalendarServer/branches/users/glyph/conn-limit/txdav/caldav/datastore/index_file.py:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/caldav/datastore/index_file.py:4971-5080
/CalendarServer/branches/users/glyph/dalify/txdav/caldav/datastore/index_file.py:6932-7023
/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/caldav/datastore/index_file.py:6592-6614
/CalendarServer/branches/users/glyph/linux-tests/txdav/caldav/datastore/index_file.py:6893-6900
/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/caldav/datastore/index_file.py:6322-6334
/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py:6369
/CalendarServer/branches/users/glyph/oracle-nulls/txdav/caldav/datastore/index_file.py:7340-7351
/CalendarServer/branches/users/glyph/oracle/txdav/caldav/datastore/index_file.py:7106-7155
/CalendarServer/branches/users/glyph/sendfdport/txdav/caldav/datastore/index_file.py:5388-5424
/CalendarServer/branches/users/glyph/sharedpool/txdav/caldav/datastore/index_file.py:6490-6550
/CalendarServer/branches/users/glyph/sql-store/txdav/caldav/datastore/index_file.py:5929-6073
/CalendarServer/branches/users/glyph/subtransactions/txdav/caldav/datastore/index_file.py:7248-7258
/CalendarServer/branches/users/glyph/use-system-twisted/txdav/caldav/datastore/index_file.py:5084-5149
/CalendarServer/branches/users/sagen/locations-resources-2/txdav/caldav/datastore/index_file.py:5052-5061
/CalendarServer/branches/users/sagen/locations-resources/txdav/caldav/datastore/index_file.py:5032-5051
/CalendarServer/branches/users/sagen/purge_old_events/txdav/caldav/datastore/index_file.py:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/caldav/datastore/index_file.py:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/caldav/datastore/index_file.py:4068-4075
/CalendarServer/branches/users/sagen/resources-2/txdav/caldav/datastore/index_file.py:5084-5093
/CalendarServer/branches/users/wsanchez/transations/txdav/caldav/datastore/index_file.py:5515-5593
/CalendarServer/trunk/twistedcaldav/index.py:6322-6394
/CalendarServer/trunk/txdav/caldav/datastore/index_file.py:7297-7364
Modified: CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/calendar_store/ho/me/home1/calendar_1/1.ics
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/calendar_store/ho/me/home1/calendar_1/1.ics 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/calendar_store/ho/me/home1/calendar_1/1.ics 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -21,12 +21,12 @@
END:VTIMEZONE
BEGIN:VEVENT
ATTENDEE;CN="Wilfredo Sanchez";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailt
- o:wsanchez@apple.com
+ o:wsanchez@example.com
ATTENDEE;CN="Cyrus Daboo";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cda
- boo@apple.com
+ boo@example.com
DTEND;TZID=US/Pacific:20090324T124500
TRANSP:OPAQUE
-ORGANIZER;CN="Wilfredo Sanchez":mailto:wsanchez@apple.com
+ORGANIZER;CN="Wilfredo Sanchez":mailto:wsanchez@example.com
UID:uid1
DTSTAMP:20090326T145447Z
LOCATION:Wilfredo's Office
Modified: CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/common.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/common.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/common.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -944,7 +944,7 @@
"""
self.assertEquals(
(yield self.calendarObjectUnderTest()).organizer(),
- "mailto:wsanchez@apple.com"
+ "mailto:wsanchez@example.com"
)
Property changes on: \
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/test_index_file.py
___________________________________________________________________
Modified: svn:mergeinfo
- /CalendarServer/branches/config-separation/txdav/caldav/datastore/test/test_index_file.py:4379-4443
/CalendarServer/branches/egg-info-351/txdav/caldav/datastore/test/test_index_file.py:4589-4625
/CalendarServer/branches/generic-sqlstore/txdav/caldav/datastore/test/test_index_file.py:6167-6191
/CalendarServer/branches/new-store/txdav/caldav/datastore/test/test_index_file.py:5594-5934
/CalendarServer/branches/new-store-no-caldavfile/txdav/caldav/datastore/test/test_index_file.py:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2/txdav/caldav/datastore/test/test_index_file.py:5936-5981
/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/caldav/datastore/test/test_index_file.py:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/caldav/datastore/test/test_index_file.py:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/caldav/datastore/test/test_index_file.py:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/caldav/datastore/test/test_index_file.py:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/txdav/caldav/datastore/test/test_index_file.py:4465-4957
/CalendarServer/branches/users/cdaboo/pycalendar/txdav/caldav/datastore/test/test_index_file.py:7085-7206
/CalendarServer/branches/users/cdaboo/pycard/txdav/caldav/datastore/test/test_index_file.py:7227-7237
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/caldav/datastore/test/test_index_file.py:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/caldav/datastore/test/test_index_file.py:5188-5440
/CalendarServer/branches/users/glyph/conn-limit/txdav/caldav/datastore/test/test_index_file.py:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/caldav/datastore/test/test_index_file.py:4971-5080
/CalendarServer/branches/users/glyph/dalify/txdav/caldav/datastore/test/test_index_file.py:6932-7023
/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/caldav/datastore/test/test_index_file.py:6592-6614
/CalendarServer/branches/users/glyph/linux-tests/txdav/caldav/datastore/test/test_index_file.py:6893-6900
/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/caldav/datastore/test/test_index_file.py:6322-6334
/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py:6369
/CalendarServer/branches/users/glyph/oracle/txdav/caldav/datastore/test/test_index_file.py:7106-7155
/CalendarServer/branches/users/glyph/sendfdport/txdav/caldav/datastore/test/test_index_file.py:5388-5424
/CalendarServer/branches/users/glyph/sharedpool/txdav/caldav/datastore/test/test_index_file.py:6490-6550
/CalendarServer/branches/users/glyph/sql-store/txdav/caldav/datastore/test/test_index_file.py:5929-6073
/CalendarServer/branches/users/glyph/subtransactions/txdav/caldav/datastore/test/test_index_file.py:7248-7258
/CalendarServer/branches/users/glyph/use-system-twisted/txdav/caldav/datastore/test/test_index_file.py:5084-5149
/CalendarServer/branches/users/sagen/locations-resources/txdav/caldav/datastore/test/test_index_file.py:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2/txdav/caldav/datastore/test/test_index_file.py:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events/txdav/caldav/datastore/test/test_index_file.py:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/caldav/datastore/test/test_index_file.py:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/caldav/datastore/test/test_index_file.py:4068-4075
/CalendarServer/branches/users/sagen/resources-2/txdav/caldav/datastore/test/test_index_file.py:5084-5093
/CalendarServer/branches/users/wsanchez/transations/txdav/caldav/datastore/test/test_index_file.py:5515-5593
/CalendarServer/trunk/twistedcaldav/test/test_index.py:6322-6394
+ /CalendarServer/branches/config-separation/txdav/caldav/datastore/test/test_index_file.py:4379-4443
/CalendarServer/branches/egg-info-351/txdav/caldav/datastore/test/test_index_file.py:4589-4625
/CalendarServer/branches/generic-sqlstore/txdav/caldav/datastore/test/test_index_file.py:6167-6191
/CalendarServer/branches/new-store-no-caldavfile-2/txdav/caldav/datastore/test/test_index_file.py:5936-5981
/CalendarServer/branches/new-store-no-caldavfile/txdav/caldav/datastore/test/test_index_file.py:5911-5935
/CalendarServer/branches/new-store/txdav/caldav/datastore/test/test_index_file.py:5594-5934
/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/caldav/datastore/test/test_index_file.py:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/caldav/datastore/test/test_index_file.py:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/caldav/datastore/test/test_index_file.py:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/caldav/datastore/test/test_index_file.py:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/txdav/caldav/datastore/test/test_index_file.py:4465-4957
/CalendarServer/branches/users/cdaboo/pycalendar/txdav/caldav/datastore/test/test_index_file.py:7085-7206
/CalendarServer/branches/users/cdaboo/pycard/txdav/caldav/datastore/test/test_index_file.py:7227-7237
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/caldav/datastore/test/test_index_file.py:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/caldav/datastore/test/test_index_file.py:5188-5440
/CalendarServer/branches/users/glyph/conn-limit/txdav/caldav/datastore/test/test_index_file.py:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/caldav/datastore/test/test_index_file.py:4971-5080
/CalendarServer/branches/users/glyph/dalify/txdav/caldav/datastore/test/test_index_file.py:6932-7023
/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/caldav/datastore/test/test_index_file.py:6592-6614
/CalendarServer/branches/users/glyph/linux-tests/txdav/caldav/datastore/test/test_index_file.py:6893-6900
/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/caldav/datastore/test/test_index_file.py:6322-6334
/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py:6369
/CalendarServer/branches/users/glyph/oracle-nulls/txdav/caldav/datastore/test/test_index_file.py:7340-7351
/CalendarServer/branches/users/glyph/oracle/txdav/caldav/datastore/test/test_index_file.py:7106-7155
/CalendarServer/branches/users/glyph/sendfdport/txdav/caldav/datastore/test/test_index_file.py:5388-5424
/CalendarServer/branches/users/glyph/sharedpool/txdav/caldav/datastore/test/test_index_file.py:6490-6550
/CalendarServer/branches/users/glyph/sql-store/txdav/caldav/datastore/test/test_index_file.py:5929-6073
/CalendarServer/branches/users/glyph/subtransactions/txdav/caldav/datastore/test/test_index_file.py:7248-7258
/CalendarServer/branches/users/glyph/use-system-twisted/txdav/caldav/datastore/test/test_index_file.py:5084-5149
/CalendarServer/branches/users/sagen/locations-resources-2/txdav/caldav/datastore/test/test_index_file.py:5052-5061
/CalendarServer/branches/users/sagen/locations-resources/txdav/caldav/datastore/test/test_index_file.py:5032-5051
/CalendarServer/branches/users/sagen/purge_old_events/txdav/caldav/datastore/test/test_index_file.py:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/caldav/datastore/test/test_index_file.py:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/caldav/datastore/test/test_index_file.py:4068-4075
/CalendarServer/branches/users/sagen/resources-2/txdav/caldav/datastore/test/test_index_file.py:5084-5093
/CalendarServer/branches/users/wsanchez/transations/txdav/caldav/datastore/test/test_index_file.py:5515-5593
/CalendarServer/trunk/twistedcaldav/test/test_index.py:6322-6394
/CalendarServer/trunk/txdav/caldav/datastore/test/test_index_file.py:7297-7364
Property changes on: \
CalendarServer/branches/users/cdaboo/pods/txdav/carddav/datastore/index_file.py \
___________________________________________________________________
Modified: svn:mergeinfo
- /CalendarServer/branches/config-separation/txdav/carddav/datastore/index_file.py:4379-4443
/CalendarServer/branches/egg-info-351/txdav/carddav/datastore/index_file.py:4589-4625
/CalendarServer/branches/generic-sqlstore/txdav/carddav/datastore/index_file.py:6167-6191
/CalendarServer/branches/new-store/txdav/carddav/datastore/index_file.py:5594-5934
/CalendarServer/branches/new-store-no-caldavfile/txdav/carddav/datastore/index_file.py:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2/txdav/carddav/datastore/index_file.py:5936-5981
/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/carddav/datastore/index_file.py:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/carddav/datastore/index_file.py:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/carddav/datastore/index_file.py:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/carddav/datastore/index_file.py:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/txdav/carddav/datastore/index_file.py:4465-4957
/CalendarServer/branches/users/cdaboo/pycalendar/txdav/carddav/datastore/index_file.py:7085-7206
/CalendarServer/branches/users/cdaboo/pycard/txdav/carddav/datastore/index_file.py:7227-7237
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/carddav/datastore/index_file.py:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/carddav/datastore/index_file.py:5188-5440
/CalendarServer/branches/users/glyph/conn-limit/txdav/carddav/datastore/index_file.py:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/carddav/datastore/index_file.py:4971-5080
/CalendarServer/branches/users/glyph/dalify/txdav/carddav/datastore/index_file.py:6932-7023
/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/carddav/datastore/index_file.py:6592-6614
/CalendarServer/branches/users/glyph/linux-tests/txdav/carddav/datastore/index_file.py:6893-6900
/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/carddav/datastore/index_file.py:6322-6334
/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py:6369
/CalendarServer/branches/users/glyph/oracle/txdav/carddav/datastore/index_file.py:7106-7155
/CalendarServer/branches/users/glyph/sendfdport/txdav/carddav/datastore/index_file.py:5388-5424
/CalendarServer/branches/users/glyph/sharedpool/txdav/carddav/datastore/index_file.py:6490-6550
/CalendarServer/branches/users/glyph/sql-store/txdav/carddav/datastore/index_file.py:5929-6073
/CalendarServer/branches/users/glyph/subtransactions/txdav/carddav/datastore/index_file.py:7248-7258
/CalendarServer/branches/users/glyph/use-system-twisted/txdav/carddav/datastore/index_file.py:5084-5149
/CalendarServer/branches/users/sagen/locations-resources/txdav/carddav/datastore/index_file.py:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2/txdav/carddav/datastore/index_file.py:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events/txdav/carddav/datastore/index_file.py:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/carddav/datastore/index_file.py:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/carddav/datastore/index_file.py:4068-4075
/CalendarServer/branches/users/sagen/resources-2/txdav/carddav/datastore/index_file.py:5084-5093
/CalendarServer/branches/users/wsanchez/transations/txdav/carddav/datastore/index_file.py:5515-5593
/CalendarServer/trunk/twistedcaldav/vcardindex.py:6322-6394
+ /CalendarServer/branches/config-separation/txdav/carddav/datastore/index_file.py:4379-4443
/CalendarServer/branches/egg-info-351/txdav/carddav/datastore/index_file.py:4589-4625
/CalendarServer/branches/generic-sqlstore/txdav/carddav/datastore/index_file.py:6167-6191
/CalendarServer/branches/new-store-no-caldavfile-2/txdav/carddav/datastore/index_file.py:5936-5981
/CalendarServer/branches/new-store-no-caldavfile/txdav/carddav/datastore/index_file.py:5911-5935
/CalendarServer/branches/new-store/txdav/carddav/datastore/index_file.py:5594-5934
/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/carddav/datastore/index_file.py:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/carddav/datastore/index_file.py:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/carddav/datastore/index_file.py:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/carddav/datastore/index_file.py:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/txdav/carddav/datastore/index_file.py:4465-4957
/CalendarServer/branches/users/cdaboo/pycalendar/txdav/carddav/datastore/index_file.py:7085-7206
/CalendarServer/branches/users/cdaboo/pycard/txdav/carddav/datastore/index_file.py:7227-7237
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/carddav/datastore/index_file.py:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/carddav/datastore/index_file.py:5188-5440
/CalendarServer/branches/users/glyph/conn-limit/txdav/carddav/datastore/index_file.py:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/carddav/datastore/index_file.py:4971-5080
/CalendarServer/branches/users/glyph/dalify/txdav/carddav/datastore/index_file.py:6932-7023
/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/carddav/datastore/index_file.py:6592-6614
/CalendarServer/branches/users/glyph/linux-tests/txdav/carddav/datastore/index_file.py:6893-6900
/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/carddav/datastore/index_file.py:6322-6334
/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py:6369
/CalendarServer/branches/users/glyph/oracle-nulls/txdav/carddav/datastore/index_file.py:7340-7351
/CalendarServer/branches/users/glyph/oracle/txdav/carddav/datastore/index_file.py:7106-7155
/CalendarServer/branches/users/glyph/sendfdport/txdav/carddav/datastore/index_file.py:5388-5424
/CalendarServer/branches/users/glyph/sharedpool/txdav/carddav/datastore/index_file.py:6490-6550
/CalendarServer/branches/users/glyph/sql-store/txdav/carddav/datastore/index_file.py:5929-6073
/CalendarServer/branches/users/glyph/subtransactions/txdav/carddav/datastore/index_file.py:7248-7258
/CalendarServer/branches/users/glyph/use-system-twisted/txdav/carddav/datastore/index_file.py:5084-5149
/CalendarServer/branches/users/sagen/locations-resources-2/txdav/carddav/datastore/index_file.py:5052-5061
/CalendarServer/branches/users/sagen/locations-resources/txdav/carddav/datastore/index_file.py:5032-5051
/CalendarServer/branches/users/sagen/purge_old_events/txdav/carddav/datastore/index_file.py:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/carddav/datastore/index_file.py:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/carddav/datastore/index_file.py:4068-4075
/CalendarServer/branches/users/sagen/resources-2/txdav/carddav/datastore/index_file.py:5084-5093
/CalendarServer/branches/users/wsanchez/transations/txdav/carddav/datastore/index_file.py:5515-5593
/CalendarServer/trunk/twistedcaldav/vcardindex.py:6322-6394
/CalendarServer/trunk/txdav/carddav/datastore/index_file.py:7297-7364
Property changes on: \
CalendarServer/branches/users/cdaboo/pods/txdav/carddav/datastore/test/test_index_file.py
___________________________________________________________________
Modified: svn:mergeinfo
- /CalendarServer/branches/config-separation/txdav/carddav/datastore/test/test_index_file.py:4379-4443
/CalendarServer/branches/egg-info-351/txdav/carddav/datastore/test/test_index_file.py:4589-4625
/CalendarServer/branches/generic-sqlstore/txdav/carddav/datastore/test/test_index_file.py:6167-6191
/CalendarServer/branches/new-store/txdav/carddav/datastore/test/test_index_file.py:5594-5934
/CalendarServer/branches/new-store-no-caldavfile/txdav/carddav/datastore/test/test_index_file.py:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2/txdav/carddav/datastore/test/test_index_file.py:5936-5981
/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/carddav/datastore/test/test_index_file.py:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/carddav/datastore/test/test_index_file.py:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/carddav/datastore/test/test_index_file.py:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/carddav/datastore/test/test_index_file.py:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/txdav/carddav/datastore/test/test_index_file.py:4465-4957
/CalendarServer/branches/users/cdaboo/pycalendar/txdav/carddav/datastore/test/test_index_file.py:7085-7206
/CalendarServer/branches/users/cdaboo/pycard/txdav/carddav/datastore/test/test_index_file.py:7227-7237
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/carddav/datastore/test/test_index_file.py:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/carddav/datastore/test/test_index_file.py:5188-5440
/CalendarServer/branches/users/glyph/conn-limit/txdav/carddav/datastore/test/test_index_file.py:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/carddav/datastore/test/test_index_file.py:4971-5080
/CalendarServer/branches/users/glyph/dalify/txdav/carddav/datastore/test/test_index_file.py:6932-7023
/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/carddav/datastore/test/test_index_file.py:6592-6614
/CalendarServer/branches/users/glyph/linux-tests/txdav/carddav/datastore/test/test_index_file.py:6893-6900
/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/carddav/datastore/test/test_index_file.py:6322-6334
/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py:6369
/CalendarServer/branches/users/glyph/oracle/txdav/carddav/datastore/test/test_index_file.py:7106-7155
/CalendarServer/branches/users/glyph/sendfdport/txdav/carddav/datastore/test/test_index_file.py:5388-5424
/CalendarServer/branches/users/glyph/sharedpool/txdav/carddav/datastore/test/test_index_file.py:6490-6550
/CalendarServer/branches/users/glyph/sql-store/txdav/carddav/datastore/test/test_index_file.py:5929-6073
/CalendarServer/branches/users/glyph/subtransactions/txdav/carddav/datastore/test/test_index_file.py:7248-7258
/CalendarServer/branches/users/glyph/use-system-twisted/txdav/carddav/datastore/test/test_index_file.py:5084-5149
/CalendarServer/branches/users/sagen/locations-resources/txdav/carddav/datastore/test/test_index_file.py:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2/txdav/carddav/datastore/test/test_index_file.py:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events/txdav/carddav/datastore/test/test_index_file.py:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/carddav/datastore/test/test_index_file.py:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/carddav/datastore/test/test_index_file.py:4068-4075
/CalendarServer/branches/users/sagen/resources-2/txdav/carddav/datastore/test/test_index_file.py:5084-5093
/CalendarServer/branches/users/wsanchez/transations/txdav/carddav/datastore/test/test_index_file.py:5515-5593
/CalendarServer/trunk/twistedcaldav/test/test_vcardindex.py:6322-6394
+ /CalendarServer/branches/config-separation/txdav/carddav/datastore/test/test_index_file.py:4379-4443
/CalendarServer/branches/egg-info-351/txdav/carddav/datastore/test/test_index_file.py:4589-4625
/CalendarServer/branches/generic-sqlstore/txdav/carddav/datastore/test/test_index_file.py:6167-6191
/CalendarServer/branches/new-store-no-caldavfile-2/txdav/carddav/datastore/test/test_index_file.py:5936-5981
/CalendarServer/branches/new-store-no-caldavfile/txdav/carddav/datastore/test/test_index_file.py:5911-5935
/CalendarServer/branches/new-store/txdav/carddav/datastore/test/test_index_file.py:5594-5934
/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/carddav/datastore/test/test_index_file.py:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/carddav/datastore/test/test_index_file.py:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/carddav/datastore/test/test_index_file.py:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/carddav/datastore/test/test_index_file.py:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/txdav/carddav/datastore/test/test_index_file.py:4465-4957
/CalendarServer/branches/users/cdaboo/pycalendar/txdav/carddav/datastore/test/test_index_file.py:7085-7206
/CalendarServer/branches/users/cdaboo/pycard/txdav/carddav/datastore/test/test_index_file.py:7227-7237
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/carddav/datastore/test/test_index_file.py:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/carddav/datastore/test/test_index_file.py:5188-5440
/CalendarServer/branches/users/glyph/conn-limit/txdav/carddav/datastore/test/test_index_file.py:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/carddav/datastore/test/test_index_file.py:4971-5080
/CalendarServer/branches/users/glyph/dalify/txdav/carddav/datastore/test/test_index_file.py:6932-7023
/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/carddav/datastore/test/test_index_file.py:6592-6614
/CalendarServer/branches/users/glyph/linux-tests/txdav/carddav/datastore/test/test_index_file.py:6893-6900
/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/carddav/datastore/test/test_index_file.py:6322-6334
/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py:6369
/CalendarServer/branches/users/glyph/oracle-nulls/txdav/carddav/datastore/test/test_index_file.py:7340-7351
/CalendarServer/branches/users/glyph/oracle/txdav/carddav/datastore/test/test_index_file.py:7106-7155
/CalendarServer/branches/users/glyph/sendfdport/txdav/carddav/datastore/test/test_index_file.py:5388-5424
/CalendarServer/branches/users/glyph/sharedpool/txdav/carddav/datastore/test/test_index_file.py:6490-6550
/CalendarServer/branches/users/glyph/sql-store/txdav/carddav/datastore/test/test_index_file.py:5929-6073
/CalendarServer/branches/users/glyph/subtransactions/txdav/carddav/datastore/test/test_index_file.py:7248-7258
/CalendarServer/branches/users/glyph/use-system-twisted/txdav/carddav/datastore/test/test_index_file.py:5084-5149
/CalendarServer/branches/users/sagen/locations-resources-2/txdav/carddav/datastore/test/test_index_file.py:5052-5061
/CalendarServer/branches/users/sagen/locations-resources/txdav/carddav/datastore/test/test_index_file.py:5032-5051
/CalendarServer/branches/users/sagen/purge_old_events/txdav/carddav/datastore/test/test_index_file.py:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/carddav/datastore/test/test_index_file.py:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/carddav/datastore/test/test_index_file.py:4068-4075
/CalendarServer/branches/users/sagen/resources-2/txdav/carddav/datastore/test/test_index_file.py:5084-5093
/CalendarServer/branches/users/wsanchez/transations/txdav/carddav/datastore/test/test_index_file.py:5515-5593
/CalendarServer/trunk/twistedcaldav/test/test_vcardindex.py:6322-6394
/CalendarServer/trunk/txdav/carddav/datastore/test/test_index_file.py:7297-7364
Modified: CalendarServer/branches/users/cdaboo/pods/txdav/common/datastore/sql_tables.py
===================================================================
--- CalendarServer/branches/users/cdaboo/pods/txdav/common/datastore/sql_tables.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/txdav/common/datastore/sql_tables.py 2011-04-27 \
21:09:24 UTC (rev 7377) @@ -206,14 +206,13 @@
emit in oracle format.
"""
for sequence in schema.model.sequences:
- out.write('drop sequence %s; create sequence %s;\n' % (
- sequence.name, sequence.name))
+ out.write('create sequence %s;\n' % (sequence.name,))
for table in schema:
# The only table name which actually exceeds the length limit right now
# is CALENDAR_OBJECT_ATTACHMENTS_MODE, which isn't actually _used_
# anywhere, so we can fake it for now.
- out.write('drop table %s; create table %s (\n' % (
- table.model.name[:30], table.model.name[:30],))
+ shortName = table.model.name[:30]
+ out.write('create table %s (\n' % (shortName,))
first = True
for column in table:
if first:
@@ -222,7 +221,7 @@
out.write(",\n")
typeName = column.model.type.name
if typeName == 'text':
- typeName = 'clob'
+ typeName = 'nclob'
if typeName == 'boolean':
typeName = 'integer'
out.write(' "%s" %s' % (column.model.name, typeName))
@@ -247,7 +246,11 @@
elif default is False:
default = 0
out.write(" " + repr(default))
- if not column.model.canBeNull():
+ if ( (not column.model.canBeNull())
+ # Oracle treats empty strings as NULLs, so we have to accept
+ # NULL values in columns of a string type. Other types should
+ # be okay though.
+ and typeName not in ('text', 'varchar') ):
out.write(' not null')
if set([column.model]) in list(table.model.uniques()):
out.write(' unique')
[Attachment #5 (text/html)]
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[7377] CalendarServer/branches/users/cdaboo/pods</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: \
verdana,arial,helvetica,sans-serif; font-size: 10pt; } #msg dl a { font-weight: \
bold} #msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: \
bold; } #msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: \
6px; } #logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em \
0; } #logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg \
h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; } \
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; \
} #logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: \
-1.5em; padding-left: 1.5em; } #logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em \
1em 0 1em; background: white;} #logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid \
#fa0; border-bottom: 1px solid #fa0; background: #fff; } #logmsg table th { \
text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted \
#fa0; } #logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: \
0.2em 0.5em; } #logmsg table thead th { text-align: center; border-bottom: 1px solid \
#fa0; } #logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: \
6px; } #patch { width: 100%; }
#patch h4 {font-family: \
verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, \
#patch .copfile {border:1px solid #ccc;margin:10px 0;} #patch ins \
{background:#dfd;text-decoration:none;display:block;padding:0 10px;} #patch del \
{background:#fdd;text-decoration:none;display:block;padding:0 10px;} #patch .lines, \
.info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a \
href="http://trac.macosforge.org/projects/calendarserver/changeset/7377">7377</a></dd>
<dt>Author</dt> <dd>cdaboo@apple.com</dd>
<dt>Date</dt> <dd>2011-04-27 14:09:24 -0700 (Wed, 27 Apr 2011)</dd>
</dl>
<h3>Log Message</h3>
<pre>Merged from trunk and fixed some tests.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#CalendarServerbranchesuserscdaboopodsbincaldavd">CalendarServer/branches/users/cdaboo/pods/bin/caldavd</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscalendarserverplatformdarwinodopen \
directorypy">CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/opendirectory.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscalendarserverplatformdarwinodtest \
test_opendirectorypy">CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/test/test_opendirectory.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscalendarservertapcaldavpy">CalendarServer/branches/users/cdaboo/pods/calendarserver/tap/caldav.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscalendarservertoolstesttest_gatewa \
ypy">CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_gateway.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscalendarservertoolstesttest_princi \
palspy">CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_principals.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodsconfauthaccountstestxml">CalendarServer/branches/users/cdaboo/pods/conf/auth/accounts-test.xml</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodsconfcaldavdappleplist">CalendarServer/branches/users/cdaboo/pods/conf/caldavd-apple.plist</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodsconfcaldavdtestplist">CalendarServer/branches/users/cdaboo/pods/conf/caldavd-test.plist</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodsconfcaldavdplist">CalendarServer/branches/users/cdaboo/pods/conf/caldavd.plist</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribmigrationcalendarmigratorpy \
">CalendarServer/branches/users/cdaboo/pods/contrib/migration/calendarmigrator.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribmigrationtesttest_migratorp \
y">CalendarServer/branches/users/cdaboo/pods/contrib/migration/test/test_migrator.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribperformanceloadtestconfigpl \
ist">CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/config.plist</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribperformanceloadtesticalpy"> \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/ical.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribperformanceloadtestpopulati \
onpy">CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/population.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribperformanceloadtestprofiles \
py">CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/profiles.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribperformanceloadtestsimpy">C \
alendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/sim.py</a></li> \
<li><a href="#CalendarServerbranchesuserscdaboopodscontribperformanceloadtesttest_prof \
ilespy">CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/test_profiles.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodsdoccalendarserver_export8">CalendarServer/branches/users/cdaboo/pods/doc/calendarserver_export.8</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodspython">CalendarServer/branches/users/cdaboo/pods/python</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodssupportMakefileApple">CalendarServer/branches/users/cdaboo/pods/support/Makefile.Apple</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodssupportbuildsh">CalendarServer/branches/users/cdaboo/pods/support/build.sh</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodssupportpysh">CalendarServer/branches/users/cdaboo/pods/support/py.sh</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodssupportshellsh">CalendarServer/branches/users/cdaboo/pods/support/shell.sh</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstestserver">CalendarServer/branches/users/cdaboo/pods/testserver</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwextenterpriseadbapi2py">CalendarServer/branches/users/cdaboo/pods/twext/enterprise/adbapi2.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwextenterprisedalmodelpy">CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/model.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwextenterprisedalsyntaxpy">CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/syntax.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwextenterprisedaltesttest_sqlsynt \
axpy">CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/test/test_sqlsyntax.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwextenterprisetesttest_adbapi2py" \
>CalendarServer/branches/users/cdaboo/pods/twext/enterprise/test/test_adbapi2.py</a></li>
>
<li><a href="#CalendarServerbranchesuserscdaboopodstwextenterpriseutilpy">CalendarServer/branches/users/cdaboo/pods/twext/enterprise/util.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavconfigpy">CalendarServer/branches/users/cdaboo/pods/twistedcaldav/config.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavdirectoryappleopendir \
ectorypy">CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/appleopendirectory.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavdirectoryaugmentpy">C \
alendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/augment.py</a></li> \
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavdirectorycachingdirect \
orypy">CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/cachingdirectory.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavdirectorytesttest_aug \
mentpy">CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_augment.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavdirectorytesttest_ope \
ndirectorypy">CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_opendirectory.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavresourcepy">CalendarServer/branches/users/cdaboo/pods/twistedcaldav/resource.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavschedulingischedulepy \
">CalendarServer/branches/users/cdaboo/pods/twistedcaldav/scheduling/ischedule.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavserverspy">CalendarServer/branches/users/cdaboo/pods/twistedcaldav/servers.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavstdconfigpy">CalendarServer/branches/users/cdaboo/pods/twistedcaldav/stdconfig.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavtesttest_configpy">Ca \
lendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_config.py</a></li> \
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavtesttest_upgradepy">Ca \
lendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_upgrade.py</a></li> \
<li><a href="#CalendarServerbranchesuserscdaboopodstwistedcaldavupgradepy">CalendarServer/branches/users/cdaboo/pods/twistedcaldav/upgrade.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstxdavbasedatastoredbapiclientpy">C \
alendarServer/branches/users/cdaboo/pods/txdav/base/datastore/dbapiclient.py</a></li> \
<li><a href="#CalendarServerbranchesuserscdaboopodstxdavcaldavdatastoretestcalendar_st \
orehomehome1calendar_11ics">CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/calendar_store/ho/me/home1/calendar_1/1.ics</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstxdavcaldavdatastoretestcommonpy"> \
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/common.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstxdavcommondatastoresql_tablespy"> \
CalendarServer/branches/users/cdaboo/pods/txdav/common/datastore/sql_tables.py</a></li>
</ul>
<h3>Added Paths</h3>
<ul>
<li>CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/</li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribcertupdate__init__py">CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribcertupdatecalendarcertupdat \
epy">CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py</a></li>
<li>CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/</li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribcertupdatetest__init__py">Ca \
lendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py</a></li> \
<li><a href="#CalendarServerbranchesuserscdaboopodscontribcertupdatetesttest_certupdat \
epy">CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribperformanceloadtestloggerpy \
">CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/logger.py</a></li>
</ul>
<h3>Removed Paths</h3>
<ul>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribcertupdate__init__py">CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribcertupdatecalendarcertupdat \
epy">CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py</a></li>
<li>CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/</li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribcertupdatetest__init__py">Ca \
lendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py</a></li> \
<li><a href="#CalendarServerbranchesuserscdaboopodscontribcertupdatetesttest_certupdat \
epy">CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py</a></li>
</ul>
<h3>Property Changed</h3>
<ul>
<li><a href="#CalendarServerbranchesuserscdaboopods">CalendarServer/branches/users/cdaboo/pods/</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodscontribperformancesim">CalendarServer/branches/users/cdaboo/pods/contrib/performance/sim</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodssupportbuildsh">CalendarServer/branches/users/cdaboo/pods/support/build.sh</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstxdavcaldavdatastoreindex_filepy"> \
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/index_file.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstxdavcaldavdatastoretesttest_index \
_filepy">CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/test_index_file.py</a></li>
<li><a href="#CalendarServerbranchesuserscdaboopodstxdavcarddavdatastoreindex_filepy" \
>CalendarServer/branches/users/cdaboo/pods/txdav/carddav/datastore/index_file.py</a></li>
>
<li><a href="#CalendarServerbranchesuserscdaboopodstxdavcarddavdatastoretesttest_index \
_filepy">CalendarServer/branches/users/cdaboo/pods/txdav/carddav/datastore/test/test_index_file.py</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="CalendarServerbranchesuserscdaboopods"></a>
<div class="propset"><h4>Property changes: \
CalendarServer/branches/users/cdaboo/pods</h4> <pre class="diff"><span>
</span></pre></div>
<a id="svnmergeinfo"></a>
<div class="modfile"><h4>Modified: svn:mergeinfo</h4></div>
<span class="cx">/CalendarServer/branches/egg-info-351:4589-4625
</span><span class="cx">/CalendarServer/branches/generic-sqlstore:6167-6191
</span><span class="cx">/CalendarServer/branches/new-store:5594-5934
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile:5911-5935
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
</span><span class="cx">/CalendarServer/branches/users/cdaboo/batchupload-6699:6700-7198
</span><span class="cx">/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
</span><span class="cx">/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
</span><span class="cx">/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
</span><span class="cx">/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycalendar:7085-7206
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycard:7227-7237
</span><span class="cx">/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
</span><span class="cx">/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
</span><span class="cx">/CalendarServer/branches/users/glyph/conn-limit:6574-6577
</span><span class="cx">/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
</span><span class="cx">/CalendarServer/branches/users/glyph/dalify:6932-7023
</span><span class="cx">/CalendarServer/branches/users/glyph/db-reconnect:6824-6876
</span><span class="cx">/CalendarServer/branches/users/glyph/dont-start-postgres:6592-6614
</span><span class="cx">/CalendarServer/branches/users/glyph/linux-tests:6893-6900
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-7:6369-6445
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle:7106-7155
</span><span class="cx">/CalendarServer/branches/users/glyph/sendfdport:5388-5424
</span><span class="cx">/CalendarServer/branches/users/glyph/sharedpool:6490-6550
</span><span class="cx">/CalendarServer/branches/users/glyph/sql-store:5929-6073
</span><span class="cx">/CalendarServer/branches/users/glyph/subtransactions:7248-7258
</span><span class="cx">/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources:5032-5051
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
</span><span class="cx">/CalendarServer/branches/users/sagen/purge_old_events:6735-6746
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
</span><span class="cx">/CalendarServer/branches/users/sagen/resources-2:5084-5093
</span><span class="cx">/CalendarServer/branches/users/wsanchez/transations:5515-5593
</span><span class="cx"> + /CalendarServer/branches/config-separation:4379-4443
</span><span class="cx">/CalendarServer/branches/egg-info-351:4589-4625
</span><span class="cx">/CalendarServer/branches/generic-sqlstore:6167-6191
</span><span class="cx">/CalendarServer/branches/new-store:5594-5934
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile:5911-5935
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
</span><span class="cx">/CalendarServer/branches/users/cdaboo/batchupload-6699:6700-7198
</span><span class="cx">/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
</span><span class="cx">/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
</span><span class="cx">/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
</span><span class="cx">/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycalendar:7085-7206
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycard:7227-7237
</span><span class="cx">/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
</span><span class="cx">/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
</span><span class="cx">/CalendarServer/branches/users/glyph/conn-limit:6574-6577
</span><span class="cx">/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
</span><span class="cx">/CalendarServer/branches/users/glyph/dalify:6932-7023
</span><span class="cx">/CalendarServer/branches/users/glyph/db-reconnect:6824-6876
</span><span class="cx">/CalendarServer/branches/users/glyph/dont-start-postgres:6592-6614
</span><span class="cx">/CalendarServer/branches/users/glyph/linux-tests:6893-6900
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-7:6369-6445
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle:7106-7155
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle-nulls:7340-7351
</span><span class="cx">/CalendarServer/branches/users/glyph/sendfdport:5388-5424
</span><span class="cx">/CalendarServer/branches/users/glyph/sharedpool:6490-6550
</span><span class="cx">/CalendarServer/branches/users/glyph/sql-store:5929-6073
</span><span class="cx">/CalendarServer/branches/users/glyph/subtransactions:7248-7258
</span><span class="cx">/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources:5032-5051
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
</span><span class="cx">/CalendarServer/branches/users/sagen/purge_old_events:6735-6746
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
</span><span class="cx">/CalendarServer/branches/users/sagen/resources-2:5084-5093
</span><span class="cx">/CalendarServer/branches/users/wsanchez/transations:5515-5593
</span><span class="cx">/CalendarServer/trunk:7297-7364
</span><a id="CalendarServerbranchesuserscdaboopodsbincaldavd"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/bin/caldavd (7376 => 7377)</h4> <pre \
class="diff"><span> <span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/bin/caldavd 2011-04-27 20:10:49 UTC (rev \
7376)
+++ CalendarServer/branches/users/cdaboo/pods/bin/caldavd 2011-04-27 21:09:24 UTC \
(rev 7377) </span><span class="lines">@@ -51,7 +51,7 @@
</span><span class="cx"> return 0;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-for v in "" "2.6" "2.5"; do
</del><ins>+for v in "2.7" "2.6" "2.5" ""; do
</ins><span class="cx"> for p in \
\ </span><span class="cx"> "${PYTHON:=}" \
\ </span><span class="cx"> "python${v}" \
\ </span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscalendarserverplatformdarwinodopendirectorypy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/opendirectory.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/opendirectory.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/opendirectory.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -23,7 +23,23 @@
</span><span class="cx"> import dsattributes
</span><span class="cx"> import base64
</span><span class="cx"> from twext.python.log import Logger
</span><ins>+import Foundation
</ins><span class="cx">
</span><ins>+
+def autoPooled(f):
+ """
+ A decorator which creates an autorelease pool and deletes it, causing it
+ to drain
+ """
+ def autoPooledFunction(*args, **kwds):
+ pool = Foundation.NSAutoreleasePool.alloc().init()
+ try:
+ return f(*args, **kwds)
+ finally:
+ del pool
+ return autoPooledFunction
+
+
</ins><span class="cx"> log = Logger()
</span><span class="cx">
</span><span class="cx"> NUM_TRIES = 3
</span><span class="lines">@@ -136,6 +152,7 @@
</span><span class="cx"> return names, encodings
</span><span class="cx">
</span><span class="cx">
</span><ins>+@autoPooled
</ins><span class="cx"> def odInit(nodeName):
</span><span class="cx"> """
</span><span class="cx"> Create an Open Directory object to operate on the \
specified directory service node name. </span><span class="lines">@@ -167,6 +184,7 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx">
</span><ins>+@autoPooled
</ins><span class="cx"> def getNodeAttributes(directory, nodeName, attributes):
</span><span class="cx"> """
</span><span class="cx"> Return key attributes for the specified directory node. \
The attributes </span><span class="lines">@@ -198,6 +216,7 @@
</span><span class="cx"> raise ODError(error)
</span><span class="cx">
</span><span class="cx">
</span><ins>+@autoPooled
</ins><span class="cx"> def listAllRecordsWithAttributes_list(directory, recordType, \
attributes, count=0): </span><span class="cx"> """
</span><span class="cx"> List records in Open Directory, and return key \
attributes for each one. </span><span class="lines">@@ -212,7 +231,6 @@
</span><span class="cx"> for each record found, or C{None} otherwise.
</span><span class="cx"> """
</span><span class="cx"> results = []
</span><del>-
</del><span class="cx"> attributeNames, encodings = \
attributeNamesFromList(attributes) </span><span class="cx">
</span><span class="cx"> tries = NUM_TRIES
</span><span class="lines">@@ -246,6 +264,8 @@
</span><span class="cx"> log.error(error)
</span><span class="cx"> raise ODError(error)
</span><span class="cx">
</span><ins>+
+@autoPooled
</ins><span class="cx"> def queryRecordsWithAttribute_list(directory, attr, value, \
matchType, casei, recordType, attributes, count=0): </span><span class="cx"> \
""" </span><span class="cx"> List records in Open Directory \
matching specified attribute/value, and return key attributes for each one. \
</span><span class="lines">@@ -263,9 +283,7 @@ </span><span class="cx"> @return: \
C{list} containing a C{list} of C{str} (record name) and C{dict} attributes \
</span><span class="cx"> for each record found, or C{None} otherwise. \
</span><span class="cx"> """ </span><del>-
</del><span class="cx"> results = []
</span><del>-
</del><span class="cx"> attributeNames, encodings = \
attributeNamesFromList(attributes) </span><span class="cx">
</span><span class="cx"> tries = NUM_TRIES
</span><span class="lines">@@ -301,6 +319,7 @@
</span><span class="cx"> raise ODError(error)
</span><span class="cx">
</span><span class="cx">
</span><ins>+@autoPooled
</ins><span class="cx"> def queryRecordsWithAttributes_list(directory, compound, \
casei, recordType, attributes, count=0): </span><span class="cx"> \
""" </span><span class="cx"> List records in Open Directory \
matching specified criteria, and return key attributes for each one. </span><span \
class="lines">@@ -385,6 +404,7 @@ </span><span class="cx"> raise ODError(error)
</span><span class="cx">
</span><span class="cx">
</span><ins>+@autoPooled
</ins><span class="cx"> def authenticateUserBasic(directory, nodeName, user, \
password): </span><span class="cx"> """
</span><span class="cx"> Authenticate a user with a password to Open Directory.
</span><span class="lines">@@ -428,6 +448,7 @@
</span><span class="cx"> raise ODError(error)
</span><span class="cx">
</span><span class="cx">
</span><ins>+@autoPooled
</ins><span class="cx"> def authenticateUserDigest(directory, nodeName, user, \
challenge, response, method): </span><span class="cx"> """
</span><span class="cx"> Authenticate using HTTP Digest credentials to Open \
Directory. </span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscalendarserverplatformdarwinodtesttest_opendirectorypy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/test/test_opendirectory.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/test/test_opendirectory.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/calendarserver/platform/darwin/od/test/test_opendirectory.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -898,3 +898,13 @@
</span><span class="cx"> (["a", "b"], \
{"b":"base64"}), </span><span class="cx"> \
opendirectory.attributeNamesFromList(["a", ("b", \
"base64")]) </span><span class="cx"> )
</span><ins>+
+ def test_autoPooled(self):
+ """
+ Make sure no exception is raised by an autoPooled method
+ """
+ @opendirectory.autoPooled
+ def method(x):
+ return x + 1
+
+ self.assertEquals(2, method(1))
</ins></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscalendarservertapcaldavpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/calendarserver/tap/caldav.py (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/calendarserver/tap/caldav.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/calendarserver/tap/caldav.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -120,16 +120,35 @@
</span><span class="cx"> return (uid, gid)
</span><span class="cx">
</span><span class="cx">
</span><del>-PARENT_ENVIRONMENT = {
- "PATH": os.environ.get("PATH", ""),
- "PYTHONPATH": os.environ.get("PYTHONPATH", ""),
- "LD_LIBRARY_PATH": os.environ.get("LD_LIBRARY_PATH", \
""),
- "DYLD_LIBRARY_PATH": os.environ.get("DYLD_LIBRARY_PATH", \
""),
-}
</del><span class="cx">
</span><del>-if "KRB5_KTNAME" in os.environ:
- PARENT_ENVIRONMENT["KRB5_KTNAME"] = \
os.environ["KRB5_KTNAME"] </del><ins>+def _computeEnvVars(parent):
+ """
+ Compute environment variables to be propagated to child processes.
+ """
+ result = {}
+ requiredVars = [
+ "PATH",
+ "PYTHONPATH",
+ "LD_LIBRARY_PATH",
+ "DYLD_LIBRARY_PATH",
+ ]
</ins><span class="cx">
</span><ins>+ optionalVars = [
+ "KRB5_KTNAME",
+ "ORACLE_HOME",
+ ]
+
+ for varname in requiredVars:
+ result[varname] = parent.get(varname, "")
+ for varname in optionalVars:
+ if varname in parent:
+ result[varname] = parent[varname]
+ return result
+
+PARENT_ENVIRONMENT = _computeEnvVars(os.environ)
+
+
+
</ins><span class="cx"> class CalDAVStatisticsProtocol (Protocol):
</span><span class="cx">
</span><span class="cx"> def connectionMade(self):
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscalendarservertoolstesttest_gatewaypy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_gateway.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_gateway.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_gateway.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -15,6 +15,7 @@
</span><span class="cx"> ##
</span><span class="cx">
</span><span class="cx"> import os
</span><ins>+import sys
</ins><span class="cx"> from twext.python.plistlib import readPlistFromString
</span><span class="cx"> import xml
</span><span class="cx">
</span><span class="lines">@@ -81,7 +82,7 @@
</span><span class="cx"> command = command.encode("utf-8")
</span><span class="cx">
</span><span class="cx"> sourceRoot = \
os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(__file__)))) \
</span><del>- python = os.path.join(sourceRoot, "python") \
</del><ins>+ python = sys.executable </ins><span class="cx"> gateway = \
os.path.join(sourceRoot, "bin", "calendarserver_command_gateway") \
</span><span class="cx"> </span><span class="cx"> args = [python, gateway, \
"-f", self.configFileName] </span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscalendarservertoolstesttest_principalspy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_principals.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_principals.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/calendarserver/tools/test/test_principals.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -15,6 +15,7 @@
</span><span class="cx"> ##
</span><span class="cx">
</span><span class="cx"> import os
</span><ins>+import sys
</ins><span class="cx">
</span><span class="cx"> from twext.python.filepath import CachingFilePath as \
FilePath </span><span class="cx"> from twisted.internet import reactor
</span><span class="lines">@@ -83,7 +84,7 @@
</span><span class="cx"> Run calendarserver_manage_principals, passing \
additional as args. </span><span class="cx"> """
</span><span class="cx"> sourceRoot = \
os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(__file__)))) \
</span><del>- python = os.path.join(sourceRoot, "python") \
</del><ins>+ python = sys.executable </ins><span class="cx"> script = \
os.path.join(sourceRoot, "bin", \
"calendarserver_manage_principals") </span><span class="cx">
</span><span class="cx"> args = [python, script, "-f", \
self.configFileName] </span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodsconfauthaccountstestxml"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/conf/auth/accounts-test.xml (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/conf/auth/accounts-test.xml 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/conf/auth/accounts-test.xml 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -38,7 +38,7 @@
</span><span class="cx"> <user>
</span><span class="cx"> <uid>wsanchez</uid>
</span><span class="cx"> <guid>wsanchez</guid>
</span><del>- <email-address>wsanchez@apple.com</email-address>
</del><ins>+ <email-address>wsanchez@example.com</email-address>
</ins><span class="cx"> <password>test</password>
</span><span class="cx"> <name>Wilfredo Sanchez Vega</name>
</span><span class="cx"> <first-name>Wilfredo</first-name>
</span><span class="lines">@@ -47,7 +47,7 @@
</span><span class="cx"> <user>
</span><span class="cx"> <uid>cdaboo</uid>
</span><span class="cx"> <guid>cdaboo</guid>
</span><del>- <email-address>cdaboo@apple.com</email-address>
</del><ins>+ <email-address>cdaboo@example.com</email-address>
</ins><span class="cx"> <password>test</password>
</span><span class="cx"> <name>Cyrus Daboo</name>
</span><span class="cx"> <first-name>Cyrus</first-name>
</span><span class="lines">@@ -56,7 +56,7 @@
</span><span class="cx"> <user>
</span><span class="cx"> <uid>sagen</uid>
</span><span class="cx"> <guid>sagen</guid>
</span><del>- <email-address>sagen@apple.com</email-address>
</del><ins>+ <email-address>sagen@example.com</email-address>
</ins><span class="cx"> <password>test</password>
</span><span class="cx"> <name>Morgen Sagen</name>
</span><span class="cx"> <first-name>Morgen</first-name>
</span><span class="lines">@@ -65,7 +65,7 @@
</span><span class="cx"> <user>
</span><span class="cx"> <uid>dre</uid>
</span><span class="cx"> <guid>andre</guid>
</span><del>- <email-address>dre@apple.com</email-address>
</del><ins>+ <email-address>dre@example.com</email-address>
</ins><span class="cx"> <password>test</password>
</span><span class="cx"> <name>Andre LaBranche</name>
</span><span class="cx"> <first-name>Andre</first-name>
</span><span class="lines">@@ -74,12 +74,21 @@
</span><span class="cx"> <user>
</span><span class="cx"> <uid>glyph</uid>
</span><span class="cx"> <guid>glyph</guid>
</span><del>- <email-address>glyph@apple.com</email-address>
</del><ins>+ <email-address>glyph@example.com</email-address>
</ins><span class="cx"> <password>test</password>
</span><span class="cx"> <name>Glyph Lefkowitz</name>
</span><span class="cx"> <first-name>Glyph</first-name>
</span><span class="cx"> <last-name>Lefkowitz</last-name>
</span><span class="cx"> </user>
</span><ins>+ <user>
+ <uid>i18nuser</uid>
+ <guid>i18nuser</guid>
+ <email-address>i18nuser@example.com</email-address>
+ <password>i18nuser</password>
+ <name>ま </name>
+ <first-name>ま</first-name>
+ <last-name> </last-name>
+ </user>
</ins><span class="cx"> <user repeat="99">
</span><span class="cx"> <uid>user%02d</uid>
</span><span class="cx"> <uid>User %02d</uid>
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodsconfcaldavdappleplist"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/conf/caldavd-apple.plist (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/conf/caldavd-apple.plist 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/conf/caldavd-apple.plist 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -171,21 +171,6 @@
</span><span class="cx"> A variety of directory services are available for \
use. </span><span class="cx"> -->
</span><span class="cx">
</span><del>- <!-- XML File Directory Service -->
- <!--
- <key>DirectoryService</key>
- <dict>
- <key>type</key>
- <string>twistedcaldav.directory.xmlfile.XMLDirectoryService</string>
-
- <key>params</key>
- <dict>
- <key>xmlFile</key>
- <string>accounts.xml</string>
- </dict>
- </dict>
- -->
-
</del><span class="cx"> <!-- Open Directory Service (Mac OS X) -->
</span><span class="cx"> <key>DirectoryService</key>
</span><span class="cx"> <dict>
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodsconfcaldavdtestplist"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/conf/caldavd-test.plist (7376 => 7377)</h4> \
<pre class="diff"><span> <span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/conf/caldavd-test.plist 2011-04-27 20:10:49 \
UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/conf/caldavd-test.plist 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -94,6 +94,14 @@
</span><span class="cx"> <key>ServerRoot</key>
</span><span class="cx"> <string>./data</string>
</span><span class="cx">
</span><ins>+ <!-- Database connection -->
+ <!--
+ <key>DBType</key>
+ <string>postgres</string>
+ <key>DSN</key>
+ <string>:caldav:caldav:::</string>
+ -->
+
</ins><span class="cx"> <!-- Data root -->
</span><span class="cx"> <key>DataRoot</key>
</span><span class="cx"> <string>Data</string>
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodsconfcaldavdplist"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/conf/caldavd.plist (7376 => 7377)</h4> <pre \
class="diff"><span> <span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/conf/caldavd.plist 2011-04-27 20:10:49 UTC \
(rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/conf/caldavd.plist 2011-04-27 21:09:24 \
UTC (rev 7377) </span><span class="lines">@@ -80,6 +80,14 @@
</span><span class="cx"> <key>ServerRoot</key>
</span><span class="cx"> <string>/var/db/caldavd</string>
</span><span class="cx">
</span><ins>+ <!-- Database connection -->
+ <!--
+ <key>DBType</key>
+ <string>postgres</string>
+ <key>DSN</key>
+ <string>:caldav:caldav:::</string>
+ -->
+
</ins><span class="cx"> <!-- Data root -->
</span><span class="cx"> <key>DataRoot</key>
</span><span class="cx"> <string>Data</string>
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribcertupdate__init__py"></a>
<div class="delfile"><h4>Deleted: \
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py (7364 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- CalendarServer/trunk/contrib/certupdate/__init__.py 2011-04-27 \
18:04:16 UTC (rev 7364)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -1,15 +0,0 @@
</span><del>-##
-# Copyright (c) 2011 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
</del></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribcertupdate__init__pyfromrev7364CalendarServertrunkcontribcertupdate__init__py"></a>
<div class="copfile"><h4>Copied: \
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py (from rev \
7364, CalendarServer/trunk/contrib/certupdate/__init__.py) (0 => 7377)</h4> <pre \
class="diff"><span> <span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py \
(rev 0)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/__init__.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -0,0 +1,15 @@
</span><ins>+##
+# Copyright (c) 2011 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
</ins></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribcertupdatecalendarcertupdatepy"></a>
<div class="delfile"><h4>Deleted: \
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py \
(7364 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/trunk/contrib/certupdate/calendarcertupdate.py 2011-04-27 18:04:16 UTC \
(rev 7364)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -1,154 +0,0 @@
</span><del>-#!/usr/bin/env python
-#
-# CertUpdate script for calendar / addresbook service.
-#
-# This script will be called with the path to the cert file in
-# /etc/certificates and also the keychain persistent reference if
-# we have one available. For the remove command, the handler
-# returns 0 = don't care, 1 = please keep, 2 = an error occurred.
-# For the replace command the handler returns
-# 0 = don't care/ cert replaced, 2 = an error occurred.
-#
-# Copyright (c) 2011 Apple Inc. All Rights Reserved.
-#
-# IMPORTANT NOTE: This file is licensed only for use on Apple-labeled
-# computers and is subject to the terms and conditions of the Apple
-# Software License Agreement accompanying the package this file is a
-# part of. You may not port this file to another platform without
-# Apple's written consent.
-
-import datetime
-import os
-import subprocess
-import sys
-
-from plistlib import readPlist, readPlistFromString, writePlist
-
-LOG = "/var/log/caldavd/certupdate.log"
-SERVICE_NAME = "calendar"
-CALDAVD_PLIST = "/etc/caldavd/caldavd.plist"
-SERVER_ADMIN = "/usr/sbin/serveradmin"
-
-def main():
-
- log(sys.argv)
- numArgs = len(sys.argv) - 1
- if numArgs == 3:
- if sys.argv[1] != "remove":
- die("Bad command line; 'remove' expected", 2)
- if isThisMyCert(CALDAVD_PLIST, sys.argv[2]):
- die("%s is in use by calendar" % (sys.argv[2],), 1)
- else:
- die("%s is not in use by calendar" % (sys.argv[2],), 0)
-
- elif numArgs == 5:
- if sys.argv[1] != "replace":
- die("Bad command line; 'replace' expected", 2)
- if isThisMyCert(CALDAVD_PLIST, sys.argv[2]):
- try:
- replaceCert(CALDAVD_PLIST, sys.argv[4])
- restartService(CALDAVD_PLIST)
- die("Replaced calendar cert with %s" % (sys.argv[4],), 0)
- except Exception, e:
- die("Error replacing calendar cert with %s: %s" % \
(sys.argv[4], e), 2)
-
- else:
- die("%s is not in use by calendar" % (sys.argv[2],), 0)
-
- else:
- # Wrong number of args
- die("Bad command line; incorrect number of arguments", 2)
-
-
-def getMyCert(plistPath):
- """
- Return SSLCertificate from the plist at plistPath
- """
- plist = readPlist(plistPath)
- return plist.get("SSLCertificate", None)
-
-
-def isThisMyCert(plistPath, otherCert):
- """
- Compare otherCert against SSLCertificate from the plist at plistPath
- """
- myCert = getMyCert(plistPath)
- return otherCert == myCert
-
-
-def replaceCert(plistPath, otherCert):
- """
- Replace SSL settings in plist at plistPath based on otherCert path
- """
- log("Reading plist %s" % (plistPath,))
- plist = readPlist(plistPath)
- log("Read in plist %s" % (plistPath,))
-
- basePath = otherCert[:-len("cert.pem")]
- log("Base path is %s" % (basePath,))
-
- log("Setting SSLCertificate to %s" % (otherCert,))
- plist["SSLCertificate"] = otherCert
-
- otherChain = basePath + "chain.pem"
- log("Setting SSLAuthorityChain to %s" % (otherChain,))
- plist["SSLAuthorityChain"] = otherChain
-
- otherKey = basePath + "key.pem"
- log("Setting SSLPrivateKey to %s" % (otherKey,))
- plist["SSLPrivateKey"] = otherKey
-
- log("Writing plist %s" % (plistPath,))
- writePlist(plist, plistPath)
-
-
-def restartService(plistPath):
- """
- Use serveradmin to restart the service.
- """
-
- plist = readPlist(plistPath)
-
- if not plist.get("EnableSSL", False):
- log("SSL is not enabled, so no need to restart")
- return
-
- if plist.get("EnableCardDAV", False):
- log("Stopping addressbook service via serveradmin")
- ret = subprocess.call([SERVER_ADMIN, "stop", \
"addressbook"])
- log("serveradmin exited with %d" % (ret,))
- log("Starting addressbook service via serveradmin")
- ret = subprocess.call([SERVER_ADMIN, "start", \
"addressbook"])
- log("serveradmin exited with %d" % (ret,))
- elif plist.get("EnableCalDAV", False):
- log("Stopping calendar service via serveradmin")
- ret = subprocess.call([SERVER_ADMIN, "stop", \
"calendar"])
- log("serveradmin exited with %d" % (ret,))
- log("Starting calendar service via serveradmin")
- ret = subprocess.call([SERVER_ADMIN, "start", \
"calendar"])
- log("serveradmin exited with %d" % (ret,))
- else:
- log("Neither calendar nor addressbook services were running")
-
-
-def log(msg):
- try:
- timestamp = datetime.datetime.now().strftime("%b %d %H:%M:%S")
- msg = "calendarcertupdate: %s %s" % (timestamp, msg)
- with open(LOG, 'a') as output:
- output.write("%s\n" % (msg,)) # so it appears in our log
- except IOError:
- # Could not write to log
- pass
-
-
-def die(msg, exitCode):
- """
- Log msg and exit with exitCode
- """
- log(msg)
- sys.exit(exitCode)
-
-
-if __name__ == '__main__':
- main()
</del></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribcertupdatecalendarcertupdatepyfromrev7364CalendarServertrunkcontribcertupdatecalendarcertupdatepy"></a>
<div class="copfile"><h4>Copied: \
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py \
(from rev 7364, CalendarServer/trunk/contrib/certupdate/calendarcertupdate.py) (0 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py \
(rev 0)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/calendarcertupdate.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -0,0 +1,154 @@
</span><ins>+#!/usr/bin/env python
+#
+# CertUpdate script for calendar / addresbook service.
+#
+# This script will be called with the path to the cert file in
+# /etc/certificates and also the keychain persistent reference if
+# we have one available. For the remove command, the handler
+# returns 0 = don't care, 1 = please keep, 2 = an error occurred.
+# For the replace command the handler returns
+# 0 = don't care/ cert replaced, 2 = an error occurred.
+#
+# Copyright (c) 2011 Apple Inc. All Rights Reserved.
+#
+# IMPORTANT NOTE: This file is licensed only for use on Apple-labeled
+# computers and is subject to the terms and conditions of the Apple
+# Software License Agreement accompanying the package this file is a
+# part of. You may not port this file to another platform without
+# Apple's written consent.
+
+import datetime
+import os
+import subprocess
+import sys
+
+from plistlib import readPlist, readPlistFromString, writePlist
+
+LOG = "/var/log/caldavd/certupdate.log"
+SERVICE_NAME = "calendar"
+CALDAVD_PLIST = "/etc/caldavd/caldavd.plist"
+SERVER_ADMIN = "/usr/sbin/serveradmin"
+
+def main():
+
+ log(sys.argv)
+ numArgs = len(sys.argv) - 1
+ if numArgs == 3:
+ if sys.argv[1] != "remove":
+ die("Bad command line; 'remove' expected", 2)
+ if isThisMyCert(CALDAVD_PLIST, sys.argv[2]):
+ die("%s is in use by calendar" % (sys.argv[2],), 1)
+ else:
+ die("%s is not in use by calendar" % (sys.argv[2],), 0)
+
+ elif numArgs == 5:
+ if sys.argv[1] != "replace":
+ die("Bad command line; 'replace' expected", 2)
+ if isThisMyCert(CALDAVD_PLIST, sys.argv[2]):
+ try:
+ replaceCert(CALDAVD_PLIST, sys.argv[4])
+ restartService(CALDAVD_PLIST)
+ die("Replaced calendar cert with %s" % (sys.argv[4],), 0)
+ except Exception, e:
+ die("Error replacing calendar cert with %s: %s" % \
(sys.argv[4], e), 2) +
+ else:
+ die("%s is not in use by calendar" % (sys.argv[2],), 0)
+
+ else:
+ # Wrong number of args
+ die("Bad command line; incorrect number of arguments", 2)
+
+
+def getMyCert(plistPath):
+ """
+ Return SSLCertificate from the plist at plistPath
+ """
+ plist = readPlist(plistPath)
+ return plist.get("SSLCertificate", None)
+
+
+def isThisMyCert(plistPath, otherCert):
+ """
+ Compare otherCert against SSLCertificate from the plist at plistPath
+ """
+ myCert = getMyCert(plistPath)
+ return otherCert == myCert
+
+
+def replaceCert(plistPath, otherCert):
+ """
+ Replace SSL settings in plist at plistPath based on otherCert path
+ """
+ log("Reading plist %s" % (plistPath,))
+ plist = readPlist(plistPath)
+ log("Read in plist %s" % (plistPath,))
+
+ basePath = otherCert[:-len("cert.pem")]
+ log("Base path is %s" % (basePath,))
+
+ log("Setting SSLCertificate to %s" % (otherCert,))
+ plist["SSLCertificate"] = otherCert
+
+ otherChain = basePath + "chain.pem"
+ log("Setting SSLAuthorityChain to %s" % (otherChain,))
+ plist["SSLAuthorityChain"] = otherChain
+
+ otherKey = basePath + "key.pem"
+ log("Setting SSLPrivateKey to %s" % (otherKey,))
+ plist["SSLPrivateKey"] = otherKey
+
+ log("Writing plist %s" % (plistPath,))
+ writePlist(plist, plistPath)
+
+
+def restartService(plistPath):
+ """
+ Use serveradmin to restart the service.
+ """
+
+ plist = readPlist(plistPath)
+
+ if not plist.get("EnableSSL", False):
+ log("SSL is not enabled, so no need to restart")
+ return
+
+ if plist.get("EnableCardDAV", False):
+ log("Stopping addressbook service via serveradmin")
+ ret = subprocess.call([SERVER_ADMIN, "stop", \
"addressbook"]) + log("serveradmin exited with %d" % \
(ret,)) + log("Starting addressbook service via serveradmin")
+ ret = subprocess.call([SERVER_ADMIN, "start", \
"addressbook"]) + log("serveradmin exited with %d" % \
(ret,)) + elif plist.get("EnableCalDAV", False):
+ log("Stopping calendar service via serveradmin")
+ ret = subprocess.call([SERVER_ADMIN, "stop", \
"calendar"]) + log("serveradmin exited with %d" % (ret,))
+ log("Starting calendar service via serveradmin")
+ ret = subprocess.call([SERVER_ADMIN, "start", \
"calendar"]) + log("serveradmin exited with %d" % (ret,))
+ else:
+ log("Neither calendar nor addressbook services were running")
+
+
+def log(msg):
+ try:
+ timestamp = datetime.datetime.now().strftime("%b %d %H:%M:%S")
+ msg = "calendarcertupdate: %s %s" % (timestamp, msg)
+ with open(LOG, 'a') as output:
+ output.write("%s\n" % (msg,)) # so it appears in our log
+ except IOError:
+ # Could not write to log
+ pass
+
+
+def die(msg, exitCode):
+ """
+ Log msg and exit with exitCode
+ """
+ log(msg)
+ sys.exit(exitCode)
+
+
+if __name__ == '__main__':
+ main()
</ins></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribcertupdatetest__init__py"></a>
<div class="delfile"><h4>Deleted: \
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py (7364 \
=> 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/trunk/contrib/certupdate/test/__init__.py 2011-04-27 18:04:16 UTC (rev \
7364)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -1,15 +0,0 @@
</span><del>-##
-# Copyright (c) 2011 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
</del></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribcertupdatetest__init__pyfromrev7364CalendarServertrunkcontribcertupdatetest__init__py"></a>
<div class="copfile"><h4>Copied: \
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py (from \
rev 7364, CalendarServer/trunk/contrib/certupdate/test/__init__.py) (0 => 7377)</h4> \
<pre class="diff"><span> <span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py \
(rev 0)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/__init__.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -0,0 +1,15 @@
</span><ins>+##
+# Copyright (c) 2011 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
</ins></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribcertupdatetesttest_certupdatepy"></a>
<div class="delfile"><h4>Deleted: \
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py \
(7364 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/trunk/contrib/certupdate/test/test_certupdate.py 2011-04-27 18:04:16 \
UTC (rev 7364)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -1,65 +0,0 @@
</span><del>-##
-# Copyright (c) 2011 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
-
-from tempfile import mkstemp
-import os
-import twistedcaldav.test.util
-from plistlib import readPlist
-from contrib.certupdate.calendarcertupdate import (
- getMyCert, isThisMyCert, replaceCert
-)
-
-samplePlist = """<?xml version="1.0" \
encoding="UTF-8"?>
-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" \
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0">
-<dict>
- <key>SSLAuthorityChain</key>
- <string>/etc/certificates/original.chain.pem</string>
- <key>SSLCertificate</key>
- <string>/etc/certificates/original.cert.pem</string>
- <key>SSLPrivateKey</key>
- <string>/etc/certificates/original.key.pem</string>
-</dict>
-</plist>
-"""
-
-class CertUpdateTests(twistedcaldav.test.util.TestCase):
- """
- Calendar Server Certificate Update Tests
- """
-
- def setUp(self):
- self.fd, self.path = mkstemp(suffix=".plist")
- out = os.fdopen(self.fd, "w")
- out.write(samplePlist)
- out.close()
-
- def tearDown(self):
- os.remove(self.path)
-
- def test_getMyCert(self):
- self.assertEquals("/etc/certificates/original.cert.pem", \
getMyCert(self.path))
-
- def test_isThisMyCert(self):
- self.assertTrue(isThisMyCert(self.path, \
"/etc/certificates/original.cert.pem"))
- self.assertFalse(isThisMyCert(self.path, \
"/etc/certificates/not.cert.pem"))
-
- def test_replaceCert(self):
- replaceCert(self.path, "/etc/certificates/new.cert.pem")
- plist = readPlist(self.path)
- self.assertEquals(plist["SSLAuthorityChain"], \
"/etc/certificates/new.chain.pem")
- self.assertEquals(plist["SSLCertificate"], \
"/etc/certificates/new.cert.pem")
- self.assertEquals(plist["SSLPrivateKey"], \
"/etc/certificates/new.key.pem") </del></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribcertupdatetesttest_certupdatepyfromrev7364CalendarServertrunkcontribcertupdatetesttest_certupdatepy"></a>
<div class="copfile"><h4>Copied: \
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py \
(from rev 7364, CalendarServer/trunk/contrib/certupdate/test/test_certupdate.py) (0 \
=> 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py \
(rev 0)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/certupdate/test/test_certupdate.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -0,0 +1,65 @@
</span><ins>+##
+# Copyright (c) 2011 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from tempfile import mkstemp
+import os
+import twistedcaldav.test.util
+from plistlib import readPlist
+from contrib.certupdate.calendarcertupdate import (
+ getMyCert, isThisMyCert, replaceCert
+)
+
+samplePlist = """<?xml version="1.0" \
encoding="UTF-8"?> +<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST \
1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> \
+<plist version="1.0"> +<dict>
+ <key>SSLAuthorityChain</key>
+ <string>/etc/certificates/original.chain.pem</string>
+ <key>SSLCertificate</key>
+ <string>/etc/certificates/original.cert.pem</string>
+ <key>SSLPrivateKey</key>
+ <string>/etc/certificates/original.key.pem</string>
+</dict>
+</plist>
+"""
+
+class CertUpdateTests(twistedcaldav.test.util.TestCase):
+ """
+ Calendar Server Certificate Update Tests
+ """
+
+ def setUp(self):
+ self.fd, self.path = mkstemp(suffix=".plist")
+ out = os.fdopen(self.fd, "w")
+ out.write(samplePlist)
+ out.close()
+
+ def tearDown(self):
+ os.remove(self.path)
+
+ def test_getMyCert(self):
+ self.assertEquals("/etc/certificates/original.cert.pem", \
getMyCert(self.path)) +
+ def test_isThisMyCert(self):
+ self.assertTrue(isThisMyCert(self.path, \
"/etc/certificates/original.cert.pem")) + \
self.assertFalse(isThisMyCert(self.path, "/etc/certificates/not.cert.pem")) \
+ + def test_replaceCert(self):
+ replaceCert(self.path, "/etc/certificates/new.cert.pem")
+ plist = readPlist(self.path)
+ self.assertEquals(plist["SSLAuthorityChain"], \
"/etc/certificates/new.chain.pem") + \
self.assertEquals(plist["SSLCertificate"], \
"/etc/certificates/new.cert.pem") + \
self.assertEquals(plist["SSLPrivateKey"], \
"/etc/certificates/new.key.pem") </ins></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribmigrationcalendarmigratorpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/contrib/migration/calendarmigrator.py (7376 \
=> 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/migration/calendarmigrator.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/migration/calendarmigrator.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -29,7 +29,7 @@
</span><span class="cx"> import subprocess
</span><span class="cx"> import sys
</span><span class="cx">
</span><del>-from plistlib import readPlist, writePlist
</del><ins>+from plistlib import readPlist, readPlistFromString, writePlist
</ins><span class="cx">
</span><span class="cx"> CALDAV_LAUNCHD_KEY = \
"org.calendarserver.calendarserver" </span><span class="cx"> \
CARDDAV_LAUNCHD_KEY = "org.addressbookserver.addressbookserver" \
</span><span class="lines">@@ -45,6 +45,7 @@ </span><span class="cx"> \
RESOURCE_MIGRATION_TRIGGER = "trigger_resource_migration" </span><span \
class="cx"> SERVER_ADMIN = "/usr/sbin/serveradmin" </span><span class="cx"> \
LAUNCHCTL = "/bin/launchctl" </span><ins>+DITTO = \
"/usr/bin/ditto" </ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> verbatimKeys = """
</span><span class="lines">@@ -214,13 +215,50 @@
</span><span class="cx"> if enableCalDAV:
</span><span class="cx"> unloadService(options, CALDAV_LAUNCHD_KEY)
</span><span class="cx">
</span><del>- newServerRootValue = migrateData(options)
- migrateConfiguration(options, newServerRootValue, enableCalDAV,
- enableCardDAV)
</del><ins>+ # Pull values out of previous plists
+ (
+ oldServerRootValue,
+ oldCalDocumentRootValue,
+ oldCalDataRootValue,
+ oldABDocumentRootValue,
+ uid,
+ gid
+ ) = examinePreviousSystem(
+ options.sourceRoot,
+ options.targetRoot
+ )
</ins><span class="cx">
</span><ins>+ # Copy data as needed
+ (
+ newServerRoot,
+ newServerRootValue,
+ newDocumentRootValue,
+ newDataRootValue
+ ) = relocateData(
+ options.sourceRoot,
+ options.targetRoot,
+ oldServerRootValue,
+ oldCalDocumentRootValue,
+ oldCalDataRootValue,
+ oldABDocumentRootValue,
+ uid,
+ gid
+ )
+
+ # Combine old and new plists
+ migrateConfiguration(
+ options,
+ newServerRootValue,
+ newDocumentRootValue,
+ newDataRootValue,
+ enableCalDAV,
+ enableCardDAV
+ )
+
</ins><span class="cx"> configureNotifications()
</span><span class="cx">
</span><del>- triggerResourceMigration(newServerRootValue)
</del><ins>+ triggerResourceMigration(newServerRoot)
+
</ins><span class="cx"> setRunState(options, enableCalDAV, enableCardDAV)
</span><span class="cx">
</span><span class="cx"> else:
</span><span class="lines">@@ -310,7 +348,8 @@
</span><span class="cx"> open(triggerPath, "w").close()
</span><span class="cx">
</span><span class="cx">
</span><del>-def migrateConfiguration(options, newServerRootValue, enableCalDAV, \
enableCardDAV): </del><ins>+def migrateConfiguration(options, newServerRootValue,
+ newDocumentRootValue, newDataRootValue, enableCalDAV, enableCardDAV):
</ins><span class="cx"> """
</span><span class="cx"> Copy files/directories/symlinks from previous system's \
/etc/caldavd </span><span class="cx"> and /etc/carddavd
</span><span class="lines">@@ -325,7 +364,6 @@
</span><span class="cx"> log("New configuration directory does not \
exist: %s" % (newConfigDir,)) </span><span class="cx"> return
</span><span class="cx">
</span><del>-
</del><span class="cx"> for configDir in (CALDAVD_CONFIG_DIR, \
CARDDAVD_CONFIG_DIR): </span><span class="cx">
</span><span class="cx"> oldConfigDir = os.path.join(options.sourceRoot, \
configDir) </span><span class="lines">@@ -387,8 +425,8 @@
</span><span class="cx"> mergePlist(oldCalDAVDPlist, oldCardDAVDPlist, \
newCalDAVDPlist) </span><span class="cx">
</span><span class="cx"> newCalDAVDPlist["ServerRoot"] = \
newServerRootValue </span><del>- newCalDAVDPlist["DocumentRoot"] = \
"Documents"
- newCalDAVDPlist["DataRoot"] = "Data"
</del><ins>+ newCalDAVDPlist["DocumentRoot"] = newDocumentRootValue
+ newCalDAVDPlist["DataRoot"] = newDataRootValue
</ins><span class="cx">
</span><span class="cx"> newCalDAVDPlist["EnableCalDAV"] = enableCalDAV
</span><span class="cx"> newCalDAVDPlist["EnableCardDAV"] = \
enableCardDAV </span><span class="lines">@@ -516,250 +554,207 @@
</span><span class="cx">
</span><span class="cx"> def log(msg):
</span><span class="cx"> try:
</span><ins>+ timestamp = datetime.datetime.now().strftime("%b %d \
%H:%M:%S") + msg = "calendarmigrator: %s %s" % (timestamp, msg)
+ print msg # so it appears in Setup.log
</ins><span class="cx"> with open(LOG, 'a') as output:
</span><del>- timestamp = datetime.datetime.now().strftime("%b %d \
%H:%M:%S")
- msg = "calendarmigrator: %s %s" % (timestamp, msg)
</del><span class="cx"> output.write("%s\n" % (msg,)) # so it \
appears in our log </span><del>- print msg # so it appears in Setup.log
</del><span class="cx"> except IOError:
</span><span class="cx"> # Could not write to log
</span><span class="cx"> pass
</span><span class="cx">
</span><del>-def migrateData(options):
</del><ins>+def examinePreviousSystem(sourceRoot, targetRoot, diskAccessor=None):
</ins><span class="cx"> """
</span><span class="cx"> Examines the old caldavd.plist and carddavd.plist to see \
where data </span><del>- lives in the previous system. If there is old data, \
calls relocateData( ) </del><ins>+ lives in the previous system.
</ins><span class="cx"> """
</span><span class="cx">
</span><del>- oldCalDocuments = None
- oldCalData = None
- oldABDocuments = None
- calendarDataInDefaultLocation = True
- addressbookDataInDefaultLocation = True
- uid = -1
- gid = -1
- newServerRoot = None # actual path
- newServerRootValue = NEW_SERVER_ROOT # value to put in plist
</del><ins>+ if diskAccessor is None:
+ diskAccessor = DiskAccessor()
</ins><span class="cx">
</span><del>- oldCalConfigDir = os.path.join(options.sourceRoot, \
CALDAVD_CONFIG_DIR) </del><ins>+ oldServerRootValue = None
+ oldCalDocumentRootValue = None
+ oldCalDataRootValue = None
+ oldABDocumentRootValue = None
+
+ # Get uid and gid from new caldavd.plist
+ newCalConfigDir = os.path.join(targetRoot, CALDAVD_CONFIG_DIR)
+ newCalPlistPath = os.path.join(newCalConfigDir, CALDAVD_PLIST)
+ if diskAccessor.exists(newCalPlistPath):
+ contents = diskAccessor.readFile(newCalPlistPath)
+ newCalPlist = readPlistFromString(contents)
+ uid, gid = getServerIDs(newCalPlist)
+ log("ServerIDs from %s: %d, %d" % (newCalPlistPath, uid, gid))
+ else:
+ uid = gid = -1
+ log("Can't find new calendar plist at %s" % (newCalPlistPath,))
+
+ # Try and read old caldavd.plist
+ oldCalConfigDir = os.path.join(sourceRoot, CALDAVD_CONFIG_DIR)
</ins><span class="cx"> oldCalPlistPath = os.path.join(oldCalConfigDir, \
CALDAVD_PLIST) </span><del>- if os.path.exists(oldCalPlistPath):
- oldCalPlist = readPlist(oldCalPlistPath)
- uid, gid = getServerIDs(oldCalPlist)
- log("ServerIDs: %d, %d" % (uid, gid))
</del><ins>+ if diskAccessor.exists(oldCalPlistPath):
+ contents = diskAccessor.readFile(oldCalPlistPath)
+ oldCalPlist = readPlistFromString(contents)
+ log("Found previous caldavd plist at %s" % (oldCalPlistPath,))
+
+ oldServerRootValue = oldCalPlist.get("ServerRoot", None)
+ oldCalDocumentRootValue = oldCalPlist.get("DocumentRoot", None)
+ oldCalDataRootValue = oldCalPlist.get("DataRoot", None)
+
</ins><span class="cx"> else:
</span><span class="cx"> log("Can't find previous calendar plist at \
%s" % (oldCalPlistPath,)) </span><span class="cx"> oldCalPlist = None
</span><del>- newCalConfigDir = os.path.join(options.targetRoot, \
CALDAVD_CONFIG_DIR)
- newCalPlistPath = os.path.join(newCalConfigDir, CALDAVD_PLIST)
- if os.path.exists(newCalPlistPath):
- newCalPlist = readPlist(newCalPlistPath)
- uid, gid = getServerIDs(newCalPlist)
- log("ServerIDs: %d, %d" % (uid, gid))
</del><span class="cx">
</span><ins>+ # Try and read old carddavd.plist
+ oldABConfigDir = os.path.join(sourceRoot, CARDDAVD_CONFIG_DIR)
+ oldABPlistPath = os.path.join(oldABConfigDir, CARDDAVD_PLIST)
+ if diskAccessor.exists(oldABPlistPath):
+ contents = diskAccessor.readFile(oldABPlistPath)
+ oldABPlist = readPlistFromString(contents)
+ log("Found previous carddavd plist at %s" % (oldABPlistPath,))
</ins><span class="cx">
</span><del>- oldABConfigDir = os.path.join(options.sourceRoot, \
CARDDAVD_CONFIG_DIR)
- oldABPlistPath = os.path.join(oldABConfigDir, CARDDAVD_PLIST)
- if os.path.exists(oldABPlistPath):
- oldABPlist = readPlist(oldABPlistPath)
</del><ins>+ oldABDocumentRootValue = oldABPlist.get("DocumentRoot", \
None) </ins><span class="cx"> else:
</span><del>- log("Can't find previous addressbook plist at %s" % \
(oldABPlistPath,)) </del><ins>+ log("Can't find previous carddavd plist \
at %s" % (oldABPlistPath,)) </ins><span class="cx"> oldABPlist = None
</span><span class="cx">
</span><del>- if oldCalPlist is not None:
- # See if there is actually any calendar data
</del><ins>+ return (
+ oldServerRootValue,
+ oldCalDocumentRootValue,
+ oldCalDataRootValue,
+ oldABDocumentRootValue,
+ uid,
+ gid
+ )
</ins><span class="cx">
</span><del>- oldDocumentRoot = oldCalPlist["DocumentRoot"]
- if oldDocumentRoot.rstrip("/") != \
"/Library/CalendarServer/Documents":
- log("Calendar data in non-standard location: %s" % \
(oldDocumentRoot,))
- calendarDataInDefaultLocation = False
- else:
- log("Calendar data in standard location: %s" % \
(oldDocumentRoot,)) </del><span class="cx">
</span><del>- oldDataRoot = oldCalPlist["DataRoot"]
</del><ins>+def relocateData(sourceRoot, targetRoot, oldServerRootValue,
+ oldCalDocumentRootValue, oldCalDataRootValue, oldABDocumentRootValue,
+ uid, gid, diskAccessor=None):
+ """
+ Copy data from sourceRoot to targetRoot, except when data is on another
+ volume in which case we just refer to it there.
+ """
</ins><span class="cx">
</span><del>- oldCalendarsPath = os.path.join(oldDocumentRoot, \
"calendars")
- if os.path.exists(oldCalendarsPath):
- # There is calendar data
- oldCalDocuments = oldDocumentRoot
- oldCalData = oldDataRoot
- log("Calendar data to migrate from %s and %s" %
- (oldCalDocuments, oldCalData))
</del><ins>+ if diskAccessor is None:
+ diskAccessor = DiskAccessor()
</ins><span class="cx">
</span><del>- if calendarDataInDefaultLocation:
- newServerRoot = absolutePathWithRoot(options.targetRoot,
- NEW_SERVER_ROOT)
- newServerRootValue = NEW_SERVER_ROOT
- else:
- newServerRoot = absolutePathWithRoot(options.targetRoot,
- oldDocumentRoot)
- newServerRootValue = oldDocumentRoot
- else:
- log("No calendar data to migrate")
</del><ins>+ log("RelocateData: sourceRoot=%s, targetRoot=%s, \
oldServerRootValue=%s, oldCalDocumentRootValue=%s, oldCalDataRootValue=%s, \
oldABDocumentRootValue=%s, uid=%d, gid=%d" % (sourceRoot, targetRoot, \
oldServerRootValue, oldCalDocumentRootValue, oldCalDataRootValue, \
oldABDocumentRootValue, uid, gid)) </ins><span class="cx">
</span><del>- if oldABPlist is not None:
- # See if there is actually any addressbook data
</del><span class="cx">
</span><del>- oldDocumentRoot = oldABPlist["DocumentRoot"]
- if oldDocumentRoot.rstrip("/") != \
"/Library/AddressBookServer/Documents":
- log("AddressBook data in non-standard location: %s" % \
(oldDocumentRoot,))
- addressbookDataInDefaultLocation = False
</del><ins>+ if oldServerRootValue:
+ newServerRootValue = oldServerRootValue
+ # Source is Lion; see if ServerRoot refers to an external volume
+ # or a directory in sourceRoot
+ if diskAccessor.exists(oldServerRootValue):
+ # refers to an external volume
+ newServerRoot = newServerRootValue
+ elif diskAccessor.exists(os.path.join(sourceRoot, oldServerRootValue)):
+ # refers to a directory on sourceRoot
+ newServerRoot = absolutePathWithRoot(targetRoot, newServerRootValue)
</ins><span class="cx"> else:
</span><del>- log("AddressBook data in standard location: %s" % \
(oldDocumentRoot,)) </del><ins>+ # It doesn't exist, so use default
+ newServerRootValue = NEW_SERVER_ROOT
+ newServerRoot = absolutePathWithRoot(targetRoot, newServerRootValue)
</ins><span class="cx">
</span><del>- oldAddressbooksPath = os.path.join(oldDocumentRoot, \
"addressbooks")
- if os.path.exists(oldAddressbooksPath):
- # There is addressbook data
- oldABDocuments = oldDocumentRoot
- log("AddressBook data to migrate from %s" % (oldABDocuments,))
</del><ins>+ # If there was an old ServerRoot value, process DocumentRoot and
+ # DataRoot because those could be relative to ServerRoot
+ oldCalDocumentRootValueProcessed = os.path.join(oldServerRootValue,
+ oldCalDocumentRootValue)
+ oldCalDataRootValueProcessed = os.path.join(oldServerRootValue,
+ oldCalDataRootValue)
+ else:
+ newServerRootValue = NEW_SERVER_ROOT
+ newServerRoot = absolutePathWithRoot(targetRoot, newServerRootValue)
+ oldCalDocumentRootValueProcessed = oldCalDocumentRootValue
+ oldCalDataRootValueProcessed = oldCalDataRootValue
</ins><span class="cx">
</span><del>- if newServerRoot is None:
- # don't override server root computed from calendar
- if addressbookDataInDefaultLocation:
- newServerRoot = absolutePathWithRoot(options.targetRoot,
- NEW_SERVER_ROOT)
- newServerRootValue = NEW_SERVER_ROOT
- else:
- newServerRoot = absolutePathWithRoot(options.targetRoot,
- oldDocumentRoot)
- newServerRootValue = oldDocumentRoot
- else:
- log("No addressbook data to migrate")
</del><ins>+ # Set default values for these, possibly overridden below:
+ newDocumentRootValue = "Documents"
+ newDocumentRoot = absolutePathWithRoot(
+ targetRoot,
+ os.path.join(newServerRootValue, newDocumentRootValue)
+ )
+ newDataRootValue = "Data"
+ newDataRoot = absolutePathWithRoot(
+ targetRoot,
+ os.path.join(newServerRootValue, newDataRootValue)
+ )
</ins><span class="cx">
</span><del>- if (oldCalDocuments or oldABDocuments) and newServerRoot:
- relocateData(oldCalDocuments, oldCalData, oldABDocuments, uid, gid,
- calendarDataInDefaultLocation, addressbookDataInDefaultLocation,
- newServerRoot)
</del><ins>+ # Old Calendar DocumentRoot
+ if oldCalDocumentRootValueProcessed:
+ if diskAccessor.exists(oldCalDocumentRootValueProcessed):
+ # Must be on an external volume if we see it existing at the point
+ # so don't copy it
+ newDocumentRoot = newDocumentRootValue = \
oldCalDocumentRootValueProcessed + elif \
diskAccessor.exists(absolutePathWithRoot(sourceRoot, \
oldCalDocumentRootValueProcessed)): + diskAccessor.ditto(
+ absolutePathWithRoot(sourceRoot, oldCalDocumentRootValueProcessed),
+ newDocumentRoot
+ )
+ diskAccessor.chown(newDocumentRoot, uid, gid, recursive=True)
</ins><span class="cx">
</span><del>- return newServerRootValue
</del><ins>+ # Old Calendar DataRoot
+ if oldCalDataRootValueProcessed:
+ if diskAccessor.exists(oldCalDataRootValueProcessed):
+ # Must be on an external volume if we see it existing at the point
+ # so don't copy it
+ newDataRootValue = oldCalDataRootValueProcessed
+ elif diskAccessor.exists(
+ absolutePathWithRoot(sourceRoot, oldCalDataRootValueProcessed)
+ ):
+ diskAccessor.ditto(
+ absolutePathWithRoot(sourceRoot, oldCalDataRootValueProcessed),
+ newDataRoot
+ )
+ diskAccessor.chown(newDataRoot, uid, gid, recursive=True)
</ins><span class="cx">
</span><del>-def relocateData(oldCalDocuments, oldCalData, oldABDocuments, uid, gid,
- calendarDataInDefaultLocation, addressbookDataInDefaultLocation,
- newServerRoot):
- """
- Relocates existing calendar data to the new default location iff the data
- was previously in the old default location; otherwise the old calendar
- DocumentRoot becomes the new ServerRoot directory, the contents of the
- old DocumentRoot are moved into ServerRoot/Documents and the contents of
- old DataRoot are copied/moved into ServerRoot/Data. If there is addressbook
- data, a symlink is created as ServerRoot/Documents/addressbooks pointing
- to the old addressbook directory so that the import-to-PostgreSQL will
- find it.
- """
</del><ins>+ # Old AddressBook DocumentRoot
+ if oldABDocumentRootValue:
+ newAddressBooks = os.path.join(newDocumentRoot, "addressbooks")
+ if diskAccessor.exists(oldABDocumentRootValue):
+ # Must be on an external volume if we see it existing at the point
+ diskAccessor.ditto(
+ os.path.join(oldABDocumentRootValue, "addressbooks"),
+ newAddressBooks
+ )
+ elif diskAccessor.exists(
+ absolutePathWithRoot(sourceRoot, oldABDocumentRootValue)
+ ):
+ diskAccessor.ditto(
+ absolutePathWithRoot(
+ sourceRoot,
+ os.path.join(oldABDocumentRootValue, "addressbooks")
+ ),
+ os.path.join(newDocumentRoot, "addressbooks")
+ )
+ if diskAccessor.exists(newAddressBooks):
+ diskAccessor.chown(newAddressBooks, uid, gid, recursive=True)
</ins><span class="cx">
</span><del>- log("RelocateData: cal documents=%s, cal data=%s, ab \
documents=%s, new server root=%s"
- % (oldCalDocuments, oldCalData, oldABDocuments, newServerRoot))
</del><span class="cx">
</span><del>- if oldCalDocuments and os.path.exists(oldCalDocuments):
</del><ins>+ newServerRootValue, newDocumentRootValue = \
relativize(newServerRootValue, + newDocumentRootValue)
+ newServerRootValue, newDataRootValue = relativize(newServerRootValue,
+ newDataRootValue)
</ins><span class="cx">
</span><del>- if calendarDataInDefaultLocation:
- # We're in the default location, relocate to new location
- newCalDocuments = os.path.join(newServerRoot, "Documents")
- if not os.path.exists(newCalDocuments):
- os.mkdir(newCalDocuments)
- newCalData = os.path.join(newServerRoot, "Data")
- if not os.path.exists(newCalData):
- os.mkdir(newCalData)
- if os.path.exists(oldCalDocuments):
- # Move evertying from oldCalDocuments
- for item in list(os.listdir(oldCalDocuments)):
- source = os.path.join(oldCalDocuments, item)
- dest = os.path.join(newCalDocuments, item)
- log("Relocating %s to %s" % (source, dest))
- os.rename(source, dest)
- else:
- log("Warning: %s does not exist; nothing to migrate" % \
(oldCalDocuments,))
- else:
- # The admin has moved calendar data to a non-standard location so
- # we're going to leave it there, but move things down a level so
- # that the old DocumentRoot becomes new ServerRoot
</del><ins>+ return (
+ newServerRoot,
+ newServerRootValue,
+ newDocumentRootValue,
+ newDataRootValue
+ )
</ins><span class="cx">
</span><del>- # Create "Documents" directory with same ownership \
as oldCalDocuments
- newCalDocuments = os.path.join(newServerRoot, "Documents")
- log("New documents directory: %s" % (newCalDocuments,))
- newCalData = os.path.join(newServerRoot, "Data")
- log("New data directory: %s" % (newCalData,))
- os.mkdir(newCalDocuments)
- os.mkdir(newCalData)
- for item in list(os.listdir(newServerRoot)):
- if item not in ("Documents", "Data"):
- source = os.path.join(newServerRoot, item)
- dest = os.path.join(newCalDocuments, item)
- log("Relocating %s to %s" % (source, dest))
- os.rename(source, dest)
</del><span class="cx">
</span><del>- # Relocate calendar DataRoot, copying all files
- if os.path.exists(oldCalData):
- if not os.path.exists(newCalData):
- os.mkdir(newCalData)
- for item in list(os.listdir(oldCalData)):
- source = os.path.join(oldCalData, item)
- if not os.path.isfile(source):
- continue
- dest = os.path.join(newCalData, item)
- log("Relocating %s to %s" % (source, dest))
- try:
- os.rename(source, dest)
- except OSError:
- # Can't rename because it's cross-volume; must copy/delete
- shutil.copy2(source, dest)
- os.remove(source)
</del><ins>+def relativize(parent, child):
+ """
+ If child is really a child of parent, make child relative to parent.
+ """
+ if child.startswith(parent):
+ parent = parent.rstrip("/")
+ child = child[len(parent):].strip("/")
+ return parent.rstrip("/"), child.rstrip("/")
</ins><span class="cx">
</span><del>- # Symlink to AB document root so server will find it an import \
to
- # PostgreSQL
- if oldABDocuments and os.path.exists(oldABDocuments):
- oldAddressBooks = os.path.join(oldABDocuments, "addressbooks")
- newAddressBooks = os.path.join(newCalDocuments, \
"addressbooks")
- log("Symlinking AddressBook data: %s to %s" % \
(newAddressBooks, oldAddressBooks))
- os.symlink(oldAddressBooks, newAddressBooks)
</del><span class="cx">
</span><del>-
- elif oldABDocuments and os.path.exists(oldABDocuments):
- # No calendar data, only addressbook data
-
- if addressbookDataInDefaultLocation:
- # We're in the default location, relocate to new location
- newABDocuments = os.path.join(newServerRoot, "Documents")
- if os.path.exists(newABDocuments):
- # Move evertying from oldABDocuments
- for item in list(os.listdir(oldABDocuments)):
- source = os.path.join(oldABDocuments, item)
- dest = os.path.join(newABDocuments, item)
- log("Relocating %s to %s" % (source, dest))
- os.rename(source, dest)
- else:
- log("Error: %s does not exist" % (newABDocuments,))
- else:
- # The admin has moved addressbook data to a non-standard location so
- # we're going to leave it there, but move things down a level so
- # that the old DocumentRoot becomes new ServerRoot
-
- # Create "Documents" directory with same ownership as \
oldABDocuments
- newABDocuments = os.path.join(newServerRoot, "Documents")
- newABData = os.path.join(newServerRoot, "Data")
- log("New documents directory: %s" % (newABDocuments,))
- os.mkdir(newABDocuments)
- os.mkdir(newABData)
- for item in list(os.listdir(newServerRoot)):
- if item not in ("Documents", "Data"):
- source = os.path.join(newServerRoot, item)
- dest = os.path.join(newABDocuments, item)
- log("Relocating %s to %s" % (source, dest))
- os.rename(source, dest)
-
- if newServerRoot and os.path.exists(newServerRoot):
- """
- Change onwnership of entire ServerRoot
- """
- os.chown(newServerRoot, uid, gid)
- for root, dirs, files in os.walk(newServerRoot, followlinks=True):
- for name in dirs:
- os.chown(os.path.join(root, name), uid, gid)
- for name in files:
- os.chown(os.path.join(root, name), uid, gid)
-
-
-
</del><span class="cx"> def getServerIDs(plist):
</span><span class="cx"> """
</span><span class="cx"> Given a caldavd.plist, return the userid and groupid for \
the UserName and </span><span class="lines">@@ -773,6 +768,7 @@
</span><span class="cx"> gid = \
grp.getgrnam(plist["GroupName"]).gr_gid </span><span class="cx"> return \
uid, gid </span><span class="cx">
</span><ins>+
</ins><span class="cx"> def absolutePathWithRoot(root, path):
</span><span class="cx"> """
</span><span class="cx"> Combine root and path as long as path does not start \
with /Volumes/ </span><span class="lines">@@ -783,5 +779,59 @@
</span><span class="cx"> path = path.strip("/")
</span><span class="cx"> return os.path.join(root, path)
</span><span class="cx">
</span><ins>+
+class DiskAccessor(object):
+ """
+ A wrapper around various disk access methods so that unit tests can easily
+ replace these with a stub that doesn't actually require disk access.
+ """
+
+ def exists(self, path):
+ return os.path.exists(path)
+
+ def readFile(self, path):
+ input = file(path)
+ contents = input.read()
+ input.close()
+ return contents
+
+ def mkdir(self, path):
+ return os.mkdir(path)
+
+ def rename(self, before, after):
+ try:
+ return os.rename(before, after)
+ except OSError:
+ # Can't rename because it's cross-volume; must copy/delete
+ shutil.copy2(before, after)
+ return os.remove(before)
+
+ def isfile(self, path):
+ return os.path.isfile(path)
+
+ def symlink(self, orig, link):
+ return os.symlink(orig, link)
+
+ def chown(self, path, uid, gid, recursive=False):
+ os.chown(path, uid, gid)
+ if recursive:
+ for root, dirs, files in os.walk(path, followlinks=True):
+ for name in dirs:
+ os.chown(os.path.join(root, name), uid, gid)
+ for name in files:
+ os.chown(os.path.join(root, name), uid, gid)
+
+
+ def walk(self, path, followlinks=True):
+ return os.walk(path, followlinks=followlinks)
+
+ def listdir(self, path):
+ return list(os.listdir(path))
+
+ def ditto(self, src, dest):
+ log("Copying with ditto: %s to %s" % (src, dest))
+ return subprocess.call([DITTO, src, dest])
+
+
</ins><span class="cx"> if __name__ == '__main__':
</span><span class="cx"> main()
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribmigrationtesttest_migratorpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/contrib/migration/test/test_migrator.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/migration/test/test_migrator.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/migration/test/test_migrator.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -15,13 +15,21 @@
</span><span class="cx"> ##
</span><span class="cx">
</span><span class="cx"> import twistedcaldav.test.util
</span><del>-from contrib.migration.calendarmigrator import mergePlist
</del><ins>+from contrib.migration.calendarmigrator import (
+ mergePlist, examinePreviousSystem, relocateData, relativize
+)
+import contrib.migration.calendarmigrator
</ins><span class="cx">
</span><span class="cx"> class MigrationTests(twistedcaldav.test.util.TestCase):
</span><span class="cx"> """
</span><span class="cx"> Calendar Server Migration Tests
</span><span class="cx"> """
</span><span class="cx">
</span><ins>+ def setUp(self):
+ # Disable logging during tests
+ self.patch(contrib.migration.calendarmigrator, "log", lambda _: \
None) +
+
</ins><span class="cx"> def test_mergeSSL(self):
</span><span class="cx">
</span><span class="cx"> # SSL on for both services
</span><span class="lines">@@ -237,6 +245,33 @@
</span><span class="cx"> mergePlist(oldCalDAV, oldCardDAV, newCombined)
</span><span class="cx"> self.assertEquals(newCombined, expected)
</span><span class="cx">
</span><ins>+ # Only CalDAV (Lion -> Lion)
+ oldCalDAV = {
+ "BindHTTPPorts": [],
+ "BindSSLPorts": [],
+ "HTTPPort": 8008,
+ "RedirectHTTPToHTTPS": False,
+ "SSLAuthorityChain": \
"/etc/certificates/test.chain.pem", + \
"SSLCertificate": "/etc/certificates/test.cert.pem", + \
"SSLPort": 8443, + "SSLPrivateKey": \
"/etc/certificates/test.key.pem", + }
+ oldCardDAV = {
+ }
+ expected = {
+ "BindHTTPPorts": [8008, 8800],
+ "BindSSLPorts": [8443, 8843],
+ "EnableSSL" : True,
+ "HTTPPort": 8008,
+ "RedirectHTTPToHTTPS": True,
+ "SSLAuthorityChain": \
"/etc/certificates/test.chain.pem", + \
"SSLCertificate": "/etc/certificates/test.cert.pem", + \
"SSLPort": 8443, + "SSLPrivateKey": \
"/etc/certificates/test.key.pem", + }
+ newCombined = { }
+ mergePlist(oldCalDAV, oldCardDAV, newCombined)
+ self.assertEquals(newCombined, expected)
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> # All settings missing!
</span><span class="lines">@@ -256,3 +291,1055 @@
</span><span class="cx"> newCombined = { }
</span><span class="cx"> mergePlist(oldCalDAV, oldCardDAV, newCombined)
</span><span class="cx"> self.assertEquals(newCombined, expected)
</span><ins>+
+
+ def test_examinePreviousSystem(self):
+ """
+ Set up a virtual system in various configurations, then ensure the
+ examinePreviousSystem( ) method detects/returns the expected values.
+
+ 'info' is an array of tuples, each tuple containing:
+ - Description of configuration
+ - Layout of disk as a dictionary of paths plus file contents
+ - Expected return values
+ """
+
+ info = [
+
+ (
+ "Snow -> Lion Migration, all in default locations",
+ ("/Volumes/old", "/"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/Library/CalendarServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/Library/CalendarServer/Data</string> + \
<key>UserName</key> + \
<string>calendar</string> + \
<key>GroupName</key> + \
<string>calendar</string> + </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/Library/AddressBookServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/Library/AddressBookServer/Data</string> + \
</dict> + </plist>
+ """,
+ "/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/Library/CalendarServer/Documents/calendars/" \
: True, + "/Volumes/old/Library/CalendarServer/Data/" : \
True, + \
"/Volumes/old/Library/AddressBookServer/Documents/addressbooks/" : True, + \
"/Volumes/old/Library/AddressBookServer/Data/" : True, + },
+ (
+ None, # Old ServerRoot value
+ "/Library/CalendarServer/Documents", # Old Cal DocRoot \
value + "/Library/CalendarServer/Data", # Old Cal DataRoot \
value + "/Library/AddressBookServer/Documents", # Old AB \
DocRoot value + 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Snow -> Lion Migration, all in default locations, non-/ \
target", + ("/Volumes/old", "/Volumes/new"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/Library/CalendarServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/Library/CalendarServer/Data</string> + \
<key>UserName</key> + \
<string>calendar</string> + \
<key>GroupName</key> + \
<string>calendar</string> + </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/Library/AddressBookServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/Library/AddressBookServer/Data</string> + \
</dict> + </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/Library/CalendarServer/Documents/calendars/" \
: True, + "/Volumes/old/Library/CalendarServer/Data/" : \
True, + \
"/Volumes/old/Library/AddressBookServer/Documents/addressbooks/" : True, + \
"/Volumes/old/Library/AddressBookServer/Data/" : True, + },
+ (
+ None, # Old ServerRoot value
+ "/Library/CalendarServer/Documents", # Old Cal DocRoot \
value + "/Library/CalendarServer/Data", # Old Cal DataRoot \
value + "/Library/AddressBookServer/Documents", # Old AB \
DocRoot value + 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Snow -> Lion Migration, not in default locations",
+ ("/Volumes/old", "/"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/NonStandard/CalendarServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/NonStandard/CalendarServer/Data</string> + \
<key>UserName</key> + \
<string>calendar</string> + \
<key>GroupName</key> + \
<string>calendar</string> + </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/NonStandard/AddressBookServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/NonStandard/AddressBookServer/Data</string> + \
</dict> + </plist>
+ """,
+ "/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/NonStandard/CalendarServer/Documents/calendars/" \
: True, + "/Volumes/old/NonStandard/CalendarServer/Data/" : \
True, + \
"/Volumes/old/NonStandard/AddressBookServer/Documents/addressbooks/" : \
True, + "/Volumes/old/NonStandard/AddressBookServer/Data/" : \
True, + },
+ (
+ None, # Old ServerRoot value
+ "/NonStandard/CalendarServer/Documents", # Old Cal DocRoot \
Value + "/NonStandard/CalendarServer/Data", # Old Cal \
DataRoot Value + "/NonStandard/AddressBookServer/Documents", \
# Old AB DocRoot Value + 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Snow -> Lion Migration, in internal/external locations",
+ ("/Volumes/old", "/"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/Volumes/External/CalendarServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/Volumes/External/CalendarServer/Data</string> + \
<key>UserName</key> + \
<string>calendar</string> + \
<key>GroupName</key> + \
<string>calendar</string> + </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/Library/AddressBookServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/Library/AddressBookServer/Data</string> + \
</dict> + </plist>
+ """,
+ "/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/External/CalendarServer/Documents/calendars/" : \
True, + "/Volumes/External/CalendarServer/Data/" : True,
+ "/Volumes/old/Library/AddressBookServer/Documents/addressbooks/" \
: True, + "/Volumes/old/Library/AddressBookServer/Data/" : \
True, + },
+ (
+ None, # Old ServerRoot value
+ "/Volumes/External/CalendarServer/Documents", # Old Cal \
DocRoot Value + "/Volumes/External/CalendarServer/Data", # \
Old Cal DataRoot Value + \
"/Library/AddressBookServer/Documents", # Old AB DocRoot Value + \
93, 93, # user id, group id + )
+ ),
+
+
+ (
+ "Snow -> Lion Migration, only AddressBook data",
+ ("/Volumes/old", "/"),
+ {
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/Library/AddressBookServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/Library/AddressBookServer/Data</string> + \
</dict> + </plist>
+ """,
+ "/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/Library/AddressBookServer/Documents/addressbooks/" \
: True, + "/Volumes/old/Library/AddressBookServer/Data/" : \
True, + },
+ (
+ None, # Old ServerRoot value
+ None, # Old Cal DocRoot value
+ None, # Old Cal DataRoot value
+ "/Library/AddressBookServer/Documents", # Old AB DocRoot \
value + 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Lion -> Lion Migration, all in default locations",
+ ("/Volumes/old", "/"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/Library/Server/Calendar and \
Contacts/Documents/" : True, + \
"/Volumes/old/Library/Server/Calendar and Contacts/Data/" : True, + \
}, + (
+ "/Library/Server/Calendar and Contacts", # Old ServerRoot \
value + "Documents", # Old Cal DocRoot value
+ "Data", # Old Cal DataRoot value
+ None, # Old AB Docs
+ 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Lion -> Lion Migration, not in default locations",
+ ("/Volumes/old", "/"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/NonStandard/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ \
<string>/Volumes/External/Calendar/Documents</string> + \
<key>DataRoot</key> + \
<string>/Volumes/External/Calendar/Data</string> + \
<key>UserName</key> + \
<string>calendar</string> + \
<key>GroupName</key> + \
<string>calendar</string> + </dict>
+ </plist>
+ """,
+ "/private/etc/caldavd/caldavd.plist" : """
+ <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/NonStandard/Calendar and \
Contacts/Documents/" : True, + \
"/Volumes/old/NonStandard/Calendar and Contacts/Data/" : True, + \
}, + (
+ "/NonStandard/Calendar and Contacts", # Old ServerRoot \
value + "/Volumes/External/Calendar/Documents", # Old Cal \
DocRoot value + "/Volumes/External/Calendar/Data", # Old Cal \
DataRoot value + None, # Old AB Docs
+ 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Lion -> Lion Migration, non-/ targetRoot",
+ ("/Volumes/old", "/Volumes/new"),
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/old/Library/Server/Calendar and \
Contacts/Documents/" : True, + \
"/Volumes/old/Library/Server/Calendar and Contacts/Data/" : True, + \
}, + (
+ "/Library/Server/Calendar and Contacts", # Old ServerRoot \
value + "Documents", # Old Cal DocRoot value
+ "Data", # Old Cal DocRoot value
+ None, # Old AB Docs
+ 93, 93, # user id, group id
+ )
+ ),
+
+ (
+ "Lion -> Lion Migration, external ServerRoot with absolute \
external DocumentRoot and internal DataRoot", + \
("/Volumes/old", "/Volumes/new"), + {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Volumes/External/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ \
<string>/Volumes/External/CalendarDocuments/</string> + \
<key>DataRoot</key> + \
<string>/CalendarData</string> + \
<key>UserName</key> + \
<string>calendar</string> + \
<key>GroupName</key> + \
<string>calendar</string> + </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/External/Library/Server/Calendar and Contacts/" \
: True, + "/Volumes/External/CalendarDocuments/" : True,
+ "/Volumes/old/CalendarData" : True,
+ "/Volumes/new/Library/Server/Calendar and Contacts/" : \
True, + },
+ (
+ "/Volumes/External/Server/Calendar and Contacts", # Old \
ServerRoot value + "/Volumes/External/CalendarDocuments/", # \
Old Cal DocRoot value + "/CalendarData", # Old Cal DocRoot \
value + None, # Old AB Docs
+ 93, 93, # user id, group id
+ )
+ ),
+
+
+
+ (
+ "Empty migration, nothing exists",
+ ("/Volumes/old", "/Volumes/new"),
+ {
+ },
+ (
+ None, # Old ServerRoot value
+ None, # Old Cal DocRoot value
+ None, # Old Cal DocRoot value
+ None, # Old AB Docs
+ -1, -1, # user id, group id
+ )
+ ),
+
+
+ ]
+
+ for description, (source, target), paths, expected in info:
+ # print "-=-=-=- %s -=-=-=-" % (description,)
+ accessor = StubDiskAccessor(paths)
+ actual = examinePreviousSystem(source, target, diskAccessor=accessor)
+ self.assertEquals(expected, actual)
+
+
+ def test_relocateData(self):
+
+ info = [
+
+ (
+ "Snow -> Lion Migration, all in default locations",
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/Library/CalendarServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/Library/CalendarServer/Data</string> + \
<key>UserName</key> + \
<string>calendar</string> + \
<key>GroupName</key> + \
<string>calendar</string> + </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/Library/AddressBookServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/Library/AddressBookServer/Data</string> + \
</dict> + </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/Library/CalendarServer/Documents/calendars/" \
: True, + "/Volumes/old/Library/CalendarServer/Data/" : \
True, + \
"/Volumes/old/Library/AddressBookServer/Documents/addressbooks/" : True, + \
"/Volumes/old/Library/AddressBookServer/Data/" : True, + \
"/Volumes/new/Library/Server/Calendar and Contacts" : True, + },
+ ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ None, # oldServerRootValue
+ "/Library/CalendarServer/Documents", # \
oldCalDocumentRootValue + "/Library/CalendarServer/Data", # \
oldCalDataRootValue + \
"/Library/AddressBookServer/Documents", # oldABDocumentRootValue + \
93, 93, # user id, group id + ),
+ ( # expected return values
+ "/Volumes/new/Library/Server/Calendar and Contacts",
+ "/Library/Server/Calendar and Contacts",
+ "Documents",
+ "Data"
+ ),
+ [ # expected DiskAccessor history
+ ('ditto', '/Volumes/old/Library/CalendarServer/Documents', \
'/Volumes/new/Library/Server/Calendar and Contacts/Documents'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and Contacts/Documents', \
93, 93), + ('ditto', '/Volumes/old/Library/CalendarServer/Data', \
'/Volumes/new/Library/Server/Calendar and Contacts/Data'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and Contacts/Data', 93, \
93), + ('ditto', \
'/Volumes/old/Library/AddressBookServer/Documents/addressbooks', \
'/Volumes/new/Library/Server/Calendar and Contacts/Documents/addressbooks'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and \
Contacts/Documents/addressbooks', 93, 93), + ]
+ ),
+
+ (
+ "Snow -> Lion Migration, in non-standard locations",
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/NonStandard/CalendarServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/NonStandard/CalendarServer/Data</string> + \
<key>UserName</key> + \
<string>calendar</string> + \
<key>GroupName</key> + \
<string>calendar</string> + </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/NonStandard/AddressBookServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/NonStandard/AddressBookServer/Data</string> + \
</dict> + </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/NonStandard/CalendarServer/Documents/calendars/" \
: True, + "/Volumes/old/NonStandard/CalendarServer/Data/" : \
True, + \
"/Volumes/old/NonStandard/AddressBookServer/Documents/addressbooks/" : \
True, + "/Volumes/old/NonStandard/AddressBookServer/Data/" : \
True, + "/Volumes/new/Library/Server/Calendar and Contacts" \
: True, + },
+ ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ None, # oldServerRootValue
+ "/NonStandard/CalendarServer/Documents", # \
oldCalDocumentRootValue + \
"/NonStandard/CalendarServer/Data", # oldCalDataRootValue + \
"/NonStandard/AddressBookServer/Documents", # oldABDocumentRootValue + \
93, 93, # user id, group id + ),
+ ( # expected return values
+ "/Volumes/new/Library/Server/Calendar and Contacts",
+ "/Library/Server/Calendar and Contacts",
+ "Documents",
+ "Data"
+ ),
+ [
+ ('ditto', '/Volumes/old/NonStandard/CalendarServer/Documents', \
'/Volumes/new/Library/Server/Calendar and Contacts/Documents'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and Contacts/Documents', \
93, 93), + ('ditto', '/Volumes/old/NonStandard/CalendarServer/Data', \
'/Volumes/new/Library/Server/Calendar and Contacts/Data'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and Contacts/Data', 93, \
93), + ('ditto', \
'/Volumes/old/NonStandard/AddressBookServer/Documents/addressbooks', \
'/Volumes/new/Library/Server/Calendar and Contacts/Documents/addressbooks'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and \
Contacts/Documents/addressbooks', 93, 93), + ]
+ ),
+
+ (
+ "Snow -> Lion Migration, internal AB, external Cal",
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/Volumes/External/CalendarServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/Volumes/External/CalendarServer/Data</string> + \
<key>UserName</key> + \
<string>calendar</string> + \
<key>GroupName</key> + \
<string>calendar</string> + </dict>
+ </plist>
+ """,
+ "/Volumes/old/private/etc/carddavd/carddavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>DocumentRoot</key>
+ \
<string>/Library/AddressBookServer/Documents</string> + \
<key>DataRoot</key> + \
<string>/Library/AddressBookServer/Data</string> + \
</dict> + </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/External/CalendarServer/Documents" : True,
+ "/Volumes/External/CalendarServer/Data" : True,
+ "/Volumes/old/Library/AddressBookServer/Documents/addressbooks/" \
: True, + "/Volumes/old/Library/AddressBookServer/Data/" : \
True, + "/Volumes/new/Library/Server/Calendar and Contacts" \
: True, + },
+ ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ None, # oldServerRootValue
+ "/Volumes/External/CalendarServer/Documents", # \
oldCalDocumentRootValue + \
"/Volumes/External/CalendarServer/Data", # oldCalDataRootValue + \
"/Library/AddressBookServer/Documents", # oldABDocumentRootValue + \
93, 93, # user id, group id + ),
+ ( # expected return values
+ "/Volumes/new/Library/Server/Calendar and Contacts",
+ "/Library/Server/Calendar and Contacts",
+ "/Volumes/External/CalendarServer/Documents",
+ "/Volumes/External/CalendarServer/Data"
+ ),
+ [
+ ('ditto', \
'/Volumes/old/Library/AddressBookServer/Documents/addressbooks', \
'/Volumes/External/CalendarServer/Documents/addressbooks'), + \
('chown-recursive', '/Volumes/External/CalendarServer/Documents/addressbooks', 93, \
93), + ]
+ ),
+
+ (
+ "Lion -> Lion Migration, all in default locations",
+ {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/old/Library/Server/Calendar and \
Contacts/Documents/" : True, + \
"/Volumes/old/Library/Server/Calendar and Contacts/Data/" : True, + \
"/Volumes/new/Library/Server/Calendar and Contacts/" : True, + \
}, + ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ "/Library/Server/Calendar and Contacts", # \
oldServerRootValue + "Documents", # oldCalDocumentRootValue
+ "Data", # oldCalDataRootValue
+ None, # oldABDocumentRootValue
+ 93, 93, # user id, group id
+ ),
+ ( # expected return values
+ "/Volumes/new/Library/Server/Calendar and Contacts",
+ "/Library/Server/Calendar and Contacts",
+ "Documents",
+ "Data"
+ ),
+ [
+ ('ditto', '/Volumes/old/Library/Server/Calendar and \
Contacts/Documents', '/Volumes/new/Library/Server/Calendar and Contacts/Documents'), \
+ ('chown-recursive', '/Volumes/new/Library/Server/Calendar and \
Contacts/Documents', 93, 93), + ('ditto', \
'/Volumes/old/Library/Server/Calendar and Contacts/Data', \
'/Volumes/new/Library/Server/Calendar and Contacts/Data'), + \
('chown-recursive', '/Volumes/new/Library/Server/Calendar and Contacts/Data', 93, \
93), + ]
+ ),
+
+ (
+ "Lion -> Lion Migration, external ServerRoot with relative \
DocumentRoot and DataRoot", + {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Volumes/External/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/External/Library/Server/Calendar and \
Contacts/Documents/" : True, + \
"/Volumes/External/Library/Server/Calendar and Contacts/Data/" : True, + \
"/Volumes/new/Library/Server/Calendar and Contacts/" : True, + \
}, + ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ "/Volumes/External/Library/Server/Calendar and Contacts", \
# oldServerRootValue + "Documents", # \
oldCalDocumentRootValue + "Data", # oldCalDataRootValue
+ None, # oldABDocumentRootValue
+ 93, 93, # user id, group id
+ ),
+ ( # expected return values
+ "/Volumes/External/Library/Server/Calendar and Contacts",
+ "/Volumes/External/Library/Server/Calendar and Contacts",
+ "Documents",
+ "Data"
+ ),
+ [
+ ]
+ ),
+
+
+ (
+ "Lion -> Lion Migration, external ServerRoot with absolute \
external DocumentRoot and internal DataRoot", + {
+ "/Volumes/old/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Volumes/External/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ \
<string>/Volumes/External/CalendarDocuments/</string> + \
<key>DataRoot</key> + \
<string>/CalendarData</string> + \
<key>UserName</key> + \
<string>calendar</string> + \
<key>GroupName</key> + \
<string>calendar</string> + </dict>
+ </plist>
+ """,
+ "/Volumes/new/private/etc/caldavd/caldavd.plist" : \
""" + <plist version="1.0">
+ <dict>
+ <key>ServerRoot</key>
+ <string>/Library/Server/Calendar and \
Contacts</string> + <key>DocumentRoot</key>
+ <string>Documents</string>
+ <key>DataRoot</key>
+ <string>Data</string>
+ <key>UserName</key>
+ <string>calendar</string>
+ <key>GroupName</key>
+ <string>calendar</string>
+ </dict>
+ </plist>
+ """,
+
+ "/Volumes/External/Library/Server/Calendar and Contacts/" \
: True, + "/Volumes/External/CalendarDocuments/" : True,
+ "/Volumes/old/CalendarData" : True,
+ "/Volumes/new/Library/Server/Calendar and Contacts/" : \
True, + },
+ ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ "/Volumes/External/Library/Server/Calendar and Contacts", \
# oldServerRootValue + \
"/Volumes/External/CalendarDocuments/", # oldCalDocumentRootValue + \
"/CalendarData", # oldCalDataRootValue + None, # \
oldABDocumentRootValue + 93, 93, # user id, group id
+ ),
+ ( # expected return values
+ "/Volumes/External/Library/Server/Calendar and Contacts",
+ "/Volumes/External/Library/Server/Calendar and Contacts",
+ "/Volumes/External/CalendarDocuments",
+ "Data" # Note that DataRoot was copied over to external \
volume + ),
+ [
+ ('ditto', '/Volumes/old/CalendarData', \
'/Volumes/External/Library/Server/Calendar and Contacts/Data'), + \
('chown-recursive', '/Volumes/External/Library/Server/Calendar and Contacts/Data', \
93, 93), + ]
+ ),
+
+ (
+ "Empty migration",
+ { # no files
+ },
+ ( # args
+ "/Volumes/old", # sourceRoot
+ "/Volumes/new", # targetRoot
+ None, # oldServerRootValue
+ None, # oldCalDocumentRootValue
+ None, # oldCalDataRootValue
+ None, # oldABDocumentRootValue
+ -1, -1, # user id, group id
+ ),
+ ( # expected return values
+ "/Volumes/new/Library/Server/Calendar and Contacts",
+ "/Library/Server/Calendar and Contacts",
+ "Documents",
+ "Data"
+ ),
+ [ # no history
+ ]
+ ),
+
+ ]
+
+ for description, paths, args, expected, history in info:
+ # print "-=-=-=- %s -=-=-=-" % (description,)
+ accessor = StubDiskAccessor(paths)
+ actual = relocateData(*args, diskAccessor=accessor)
+ self.assertEquals(expected, actual)
+ self.assertEquals(history, accessor.history)
+
+
+ def test_stubDiskAccessor(self):
+
+ paths = {
+ "/a/b/c/d" : "foo",
+ "/a/b/c/e" : "bar",
+ "/x/y/z/" : True,
+ }
+ accessor = StubDiskAccessor(paths)
+
+ shouldExist = ["/a", "/a/", "/a/b", \
"/a/b/", "/a/b/c/d", "/x/y/z"] + shouldNotExist \
= ["/b", "/x/y/z/Z"] +
+ for path in shouldExist:
+ self.assertTrue(accessor.exists(path))
+ for path in shouldNotExist:
+ self.assertFalse(accessor.exists(path))
+
+ for key, value in paths.iteritems():
+ if value is not True:
+ self.assertEquals(accessor.readFile(key), value)
+
+
+ def test_relativize(self):
+ """
+ Make sure child paths are made relative to their parent
+ """
+ info = [
+ (("/abc/", "/abc/def"), ("/abc", \
"def")), + (("/abc", "/abc/def"), \
("/abc", "def")), + (("/abc", \
"/def"), ("/abc", "/def")), + ]
+ for args, expected in info:
+ self.assertEquals(expected, relativize(*args))
+
+
+class StubDiskAccessor(object):
+ """
+ A stub which allows testing without actually having real files
+ """
+
+ def __init__(self, paths):
+ self.paths = paths
+ self._fillInDirectories()
+
+ self.reset()
+
+ def _fillInDirectories(self):
+ for key in self.paths.keys():
+ parts = key.split("/")
+ for i in xrange(len(parts)):
+ path = "/".join(parts[:i])
+ self.paths[path] = True
+
+ def addPath(self, path, value):
+ self.paths[path] = value
+ self._fillInDirectories()
+
+ def reset(self):
+ self.history = []
+
+ def exists(self, path):
+ return self.paths.has_key(path.rstrip("/"))
+
+ def readFile(self, path):
+ return self.paths[path]
+
+ def mkdir(self, path):
+ self.history.append(("mkdir", path))
+ self.addPath(path, True)
+
+ def rename(self, before, after):
+ self.history.append(("rename", before, after))
+
+ def isfile(self, path):
+ # FIXME: probably want a better way to denote a directory than \
"True" + return self.exists(path) and self.paths[path] is not True
+
+ def symlink(self, orig, link):
+ self.history.append(("symlink", orig, link))
+
+ def chown(self, path, uid, gid, recursive=False):
+ self.history.append(("chown-recursive" if recursive else \
"chown", path, uid, gid)) +
+ def walk(self, path, followlinks=True):
+ yield [], [], []
+
+ def listdir(self, path):
+ return []
+
+ def ditto(self, src, dest):
+ self.history.append(("ditto", src, dest))
+ self.addPath(dest, True)
+
</ins></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribperformanceloadtestconfigplist"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/config.plist \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/config.plist 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/config.plist 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -66,6 +66,8 @@
</span><span class="cx"> <key>observers</key>
</span><span class="cx"> <array>
</span><span class="cx"> \
<string>loadtest.population.ReportStatistics</string> </span><ins>+ \
<string>loadtest.ical.RequestLogger</string> + \
<string>loadtest.profiles.OperationLogger</string> </ins><span \
class="cx"> </array> </span><span class="cx">
</span><span class="cx"> </dict>
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribperformanceloadtesticalpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/ical.py (7376 \
=> 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/ical.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/ical.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -19,6 +19,7 @@
</span><span class="cx"> from operator import getitem
</span><span class="cx"> from pprint import pformat
</span><span class="cx"> from datetime import datetime
</span><ins>+from urlparse import urlparse, urlunparse
</ins><span class="cx">
</span><span class="cx"> from xml.etree import ElementTree
</span><span class="cx"> ElementTree.QName.__repr__ = lambda self: '<QName %r>' \
% (self.text,) </span><span class="lines">@@ -196,17 +197,14 @@
</span><span class="cx"> # not both.
</span><span class="cx"> after = self.reactor.seconds()
</span><span class="cx">
</span><del>- # XXX If the response code is wrong, there's probably not
- # point passing the response down the callback chain.
- # errback?
</del><span class="cx"> success = response.code == expectedResponseCode
</span><span class="cx">
</span><span class="cx"> # if not success:
</span><span class="cx"> # import pdb; pdb.set_trace()
</span><span class="cx"> msg(
</span><span class="cx"> type="response", success=success, \
method=method, </span><del>- headers=headers, body=body,
- duration=(after - before), url=url)
</del><ins>+ headers=headers, body=body, code=response.code,
+ user=self.user, duration=(after - before), url=url)
</ins><span class="cx">
</span><span class="cx"> if success:
</span><span class="cx"> return response
</span><span class="lines">@@ -665,11 +663,24 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx"> class RequestLogger(object):
</span><ins>+ format = u"%(user)s request %(code)s%(success)s[%(duration)5.2f \
s] %(method)8s %(url)s" + success = u"\N{CHECK MARK}"
+ failure = u"\N{BALLOT X}"
+
</ins><span class="cx"> def observe(self, event):
</span><del>- if event.get("type") == "request":
- print event["user"], event["method"], \
event["url"] </del><ins>+ if event.get("type") == \
"response": + event['url'] = urlunparse(('', '') + \
urlparse(event['url'])[2:]) + if event['success']:
+ event['success'] = self.success
+ else:
+ event['success'] = self.failure
+ print (self.format % event).encode('utf-8')
</ins><span class="cx">
</span><span class="cx">
</span><ins>+ def report(self):
+ pass
+
+
</ins><span class="cx">
</span><span class="cx"> def main():
</span><span class="cx"> from urllib2 import HTTPDigestAuthHandler
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribperformanceloadtestloggerpyfromrev7364CalendarServertrunkcontribperformanceloadtestloggerpy"></a>
<div class="copfile"><h4>Copied: \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/logger.py \
(from rev 7364, CalendarServer/trunk/contrib/performance/loadtest/logger.py) (0 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/logger.py \
(rev 0)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/logger.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -0,0 +1,52 @@
</span><ins>+from stats import mean, median, stddev, mad
+
+class SummarizingMixin(object):
+ def printHeader(self, fields):
+ """
+ Print a header for the summarization data which will be reported.
+
+ @param fields: A C{list} of two-tuples. Each tuple describes one
+ column in the summary. The first element gives a label to appear
+ at the top of the column. The second element gives the width of
+ the column.
+ """
+ format = []
+ labels = []
+ for (label, width) in fields:
+ format.append('%%%ds' % (width,))
+ labels.append(label)
+ print ' '.join(format) % tuple(labels)
+
+
+ def _summarizeData(self, operation, data):
+ failed = 0
+ threesec = 0
+ durations = []
+ for (success, duration) in data:
+ if not success:
+ failed += 1
+ if duration > 3:
+ threesec += 1
+ durations.append(duration)
+
+ return operation, len(data), failed, threesec, mean(durations), \
median(durations) +
+
+ def _printRow(self, formats, values):
+ format = ' '.join(formats)
+ print format % values
+
+
+ def printData(self, formats, perOperationTimes):
+ """
+ Print one or more rows of data with the given formatting.
+
+ @param formats: A C{list} of C{str} giving formats into which each
+ data field will be interpolated.
+
+ @param perOperationTimes: A C{list} of all of the data to summarize.
+ Each element is a two-tuple of whether the operation succeeded
+ (C{True} if so, C{False} if not) and how long the operation took.
+ """
+ for method, data in perOperationTimes:
+ self._printRow(formats, self._summarizeData(method, data))
</ins></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribperformanceloadtestpopulationpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/population.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/population.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/population.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -26,10 +26,10 @@
</span><span class="cx"> from twisted.python.log import msg, err
</span><span class="cx">
</span><span class="cx"> from stats import mean, median, stddev, mad
</span><ins>+from loadtest.logger import SummarizingMixin
</ins><span class="cx"> from loadtest.ical import SnowLeopard, RequestLogger
</span><span class="cx"> from loadtest.profiles import Eventer, Inviter, Accepter
</span><span class="cx">
</span><del>-
</del><span class="cx"> class ClientType(object, FancyEqMixin):
</span><span class="cx"> """
</span><span class="cx"> @ivar clientType: An L{ICalendarClient} implementation
</span><span class="lines">@@ -206,7 +206,7 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx">
</span><del>-class ReportStatistics(StatisticsBase):
</del><ins>+class ReportStatistics(StatisticsBase, SummarizingMixin):
</ins><span class="cx"> _fields = [
</span><span class="cx"> ('operation', 10, '%10s'),
</span><span class="cx"> ('count', 8, '%8s'),
</span><span class="lines">@@ -225,39 +225,15 @@
</span><span class="cx"> dataset.append((event['success'], \
event['duration'])) </span><span class="cx">
</span><span class="cx">
</span><del>- def _printHeader(self):
- format = []
- labels = []
- for (label, width, fmt) in self._fields:
- format.append('%%%ds' % (width,))
- labels.append(label)
- print ''.join(format) % tuple(labels)
-
-
- def _summarizeData(self, method, data):
- failed = 0
- threesec = 0
- durations = []
- for (success, duration) in data:
- if not success:
- failed += 1
- if duration > 3:
- threesec += 1
- durations.append(duration)
-
- return method, len(data), failed, threesec, mean(durations), \
median(durations)
-
-
- def _printData(self, *values):
- format = ''.join(fmt for (label, width, fmt) in self._fields)
- print format % values
-
-
</del><span class="cx"> def report(self):
</span><span class="cx"> print
</span><del>- self._printHeader()
- for method, data in self._perMethodTimes.iteritems():
- self._printData(*self._summarizeData(method, data))
</del><ins>+ self.printHeader([
+ (label, width)
+ for (label, width, fmt)
+ in self._fields])
+ self.printData(
+ [fmt for (label, width, fmt) in self._fields],
+ sorted(self._perMethodTimes.items()))
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> def main():
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribperformanceloadtestprofilespy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/profiles.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/profiles.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/profiles.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -30,11 +30,14 @@
</span><span class="cx">
</span><span class="cx"> from protocol.caldav.definitions import caldavxml
</span><span class="cx">
</span><ins>+from twisted.python import context
</ins><span class="cx"> from twisted.python.log import msg
</span><ins>+from twisted.python.failure import Failure
</ins><span class="cx"> from twisted.internet.defer import succeed, fail
</span><span class="cx"> from twisted.internet.task import LoopingCall
</span><span class="cx"> from twisted.web.http import PRECONDITION_FAILED
</span><span class="cx">
</span><ins>+from loadtest.logger import SummarizingMixin
</ins><span class="cx"> from loadtest.ical import IncorrectResponseCode
</span><span class="cx">
</span><span class="cx">
</span><span class="lines">@@ -59,6 +62,41 @@
</span><span class="cx"> if cal.resourceType == calendarType]
</span><span class="cx">
</span><span class="cx">
</span><ins>+ def _isSelfAttendee(self, attendee):
+ """
+ Try to match one of the attendee's identifiers against one of
+ C{self._client}'s identifiers. Return C{True} if something matches,
+ C{False} otherwise.
+ """
+ return attendee.params[u'EMAIL'][0] == self._client.email[len('mailto:'):]
+
+
+ def _newOperation(self, label, deferred):
+ """
+ Helper to emit a log event when a new operation is started and
+ another one when it completes.
+ """
+ # If this is a scheduled request, record the lag in the
+ # scheduling now so it can be reported when the response is
+ # received.
+ lag = context.get('lag', None)
+
+ before = self._reactor.seconds()
+ msg(type="operation", phase="start",
+ user=self._client.user, label=label, lag=lag)
+
+ def finished(passthrough):
+ success = not isinstance(passthrough, Failure)
+ after = self._reactor.seconds()
+ msg(type="operation", phase="end", duration=after - \
before, + user=self._client.user, label=label, success=success)
+ return passthrough
+ deferred.addBoth(finished)
+ return deferred
+
+
+
+
</ins><span class="cx"> class CannotAddAttendee(Exception):
</span><span class="cx"> """
</span><span class="cx"> Indicates no new attendees can be invited to a \
particular event. </span><span class="lines">@@ -143,10 +181,16 @@
</span><span class="cx"> if event is None:
</span><span class="cx"> continue
</span><span class="cx">
</span><ins>+ vevent = event.contents[u'vevent'][0]
+ organizer = vevent.contents.get('organizer', [None])[0]
+ if organizer is not None and not self._isSelfAttendee(organizer):
+ # This event was organized by someone else, don't try to invite \
someone to it. + continue
+
</ins><span class="cx"> href = calendar.url + uuid
</span><span class="cx">
</span><span class="cx"> # Find out who might attend
</span><del>- attendees = \
event.contents['vevent'][0].contents.get('attendee', []) </del><ins>+ \
attendees = vevent.contents.get('attendee', []) </ins><span class="cx">
</span><span class="cx"> d = self._addAttendee(event, attendees)
</span><span class="cx"> d.addCallbacks(
</span><span class="lines">@@ -154,7 +198,7 @@
</span><span class="cx"> self._client.addEventAttendee(
</span><span class="cx"> href, attendee),
</span><span class="cx"> lambda reason: \
reason.trap(CannotAddAttendee)) </span><del>- return d
</del><ins>+ return self._newOperation("invite", d)
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx">
</span><span class="lines">@@ -189,14 +233,13 @@
</span><span class="cx"> # NEEDS-ACTION PARTSTAT.
</span><span class="cx"> attendees = \
vevent.contents['vevent'][0].contents.get('attendee', []) </span><span class="cx"> \
for attendee in attendees: </span><del>- if attendee.params[u'EMAIL'][0] \
== self._client.email[len('mailto:'):]: </del><ins>+ if \
self._isSelfAttendee(attendee): </ins><span class="cx"> if \
attendee.params[u'PARTSTAT'][0] == 'NEEDS-ACTION': </span><span class="cx"> \
# XXX Base this on something real </span><span class="cx"> delay \
= self.random.gauss(10, 2) </span><span class="cx"> \
self._accepting.add(href) </span><span class="cx"> \
self._reactor.callLater( </span><span class="cx"> delay, \
self._acceptInvitation, href, attendee) </span><del>- return
</del><span class="cx">
</span><span class="cx">
</span><span class="cx"> def _acceptInvitation(self, href, attendee):
</span><span class="lines">@@ -234,7 +277,7 @@
</span><span class="cx"> self._accepting.remove(href)
</span><span class="cx"> return passthrough
</span><span class="cx"> d.addBoth(finished)
</span><del>- return d
</del><ins>+ return self._newOperation("accept", d)
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> def _makeAcceptedAttendee(self, attendee):
</span><span class="lines">@@ -315,4 +358,54 @@
</span><span class="cx">
</span><span class="cx"> href = '%s%s.ics' % (
</span><span class="cx"> calendar.url, \
vevent.contents[u'uid'][0].value) </span><del>- return \
self._client.addEvent(href, vcalendar) </del><ins>+ d = \
self._client.addEvent(href, vcalendar) + return \
self._newOperation("create", d) +
+
+class OperationLogger(SummarizingMixin):
+ """
+ Profiles will initiate operations which may span multiple requests. Start
+ and stop log messages are emitted for these operations and logged by this
+ logger.
+ """
+ formats = {
+ u"start": u"%(user)s - - - - - - - - - - - %(label)8s BEGIN \
%(lag)s", + u"end" : u"%(user)s - - - - - - - - - - - \
%(label)8s END [%(duration)5.2f s]", + }
+
+ lagFormat = u'{lag %5.2f ms}'
+
+ _fields = [
+ ('operation', 10, '%10s'),
+ ('count', 8, '%8s'),
+ ('failed', 8, '%8s'),
+ ('>3sec', 8, '%8s'),
+ ('mean', 8, '%8.4f'),
+ ('median', 8, '%8.4f'),
+ ]
+
+ def __init__(self):
+ self._perOperationTimes = {}
+
+
+ def observe(self, event):
+ if event.get("type") == "operation":
+ if event.get('lag') is None:
+ event['lag'] = ''
+ else:
+ event['lag'] = self.lagFormat % (event['lag'] * 1000.0,)
+ print (self.formats[event[u'phase']] % event).encode('utf-8')
+ if event[u'phase'] == u'end':
+ dataset = self._perOperationTimes.setdefault(event[u'label'], [])
+ dataset.append((event[u'success'], event[u'duration']))
+
+
+ def report(self):
+ print
+ self.printHeader([
+ (label, width)
+ for (label, width, fmt)
+ in self._fields])
+ self.printData(
+ [fmt for (label, width, fmt) in self._fields],
+ sorted(self._perOperationTimes.items()))
</ins></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribperformanceloadtestsimpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/sim.py (7376 \
=> 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/sim.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/sim.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -20,6 +20,7 @@
</span><span class="cx"> from plistlib import readPlist
</span><span class="cx"> from collections import namedtuple
</span><span class="cx">
</span><ins>+from twisted.python import context
</ins><span class="cx"> from twisted.python.filepath import FilePath
</span><span class="cx"> from twisted.python.log import addObserver
</span><span class="cx"> from twisted.python.usage import UsageError, Options
</span><span class="lines">@@ -32,6 +33,30 @@
</span><span class="cx"> CalendarClientSimulator)
</span><span class="cx">
</span><span class="cx">
</span><ins>+class LagTrackingReactor(object):
+ """
+ This reactor wraps another reactor and proxies all attribute
+ access (including method calls). It only changes the behavior of
+ L{IReactorTime.callLater} to insert a C{"lag"} key into the
+ context which delayed function calls are invoked with. This key
+ has a float value which gives the difference in time between when
+ the call was original scheduled and when the call actually took
+ place.
+ """
+ def __init__(self, reactor):
+ self._reactor = reactor
+
+ def __getattr__(self, name):
+ return getattr(self._reactor, name)
+
+ def callLater(self, delay, function, *args, **kwargs):
+ expected = self._reactor.seconds() + delay
+ def modifyContext():
+ now = self._reactor.seconds()
+ context.call({'lag': now - expected}, function, *args, **kwargs)
+ return self._reactor.callLater(delay, modifyContext)
+
+
</ins><span class="cx"> class SimOptions(Options):
</span><span class="cx"> """
</span><span class="cx"> Command line configuration options for the load \
simulator. </span><span class="lines">@@ -103,7 +128,7 @@
</span><span class="cx"> self.arrival = arrival
</span><span class="cx"> self.parameters = parameters
</span><span class="cx"> self.observers = observers
</span><del>- self.reactor = reactor
</del><ins>+ self.reactor = LagTrackingReactor(reactor)
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> @classmethod
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribperformanceloadtesttest_profilespy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/test_profiles.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/test_profiles.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/contrib/performance/loadtest/test_profiles.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -334,7 +334,7 @@
</span><span class="cx"> If the inviter randomly selects a user which is \
already an </span><span class="cx"> invitee on the event, a different user is \
added instead. </span><span class="cx"> """
</span><del>- selfNumber = 13
</del><ins>+ selfNumber = 1
</ins><span class="cx"> vevent, event, calendar, client = \
self._simpleAccount( </span><span class="cx"> selfNumber, INVITED_EVENT)
</span><span class="cx">
</span><span class="lines">@@ -376,7 +376,27 @@
</span><span class="cx"> self.assertEquals(len(attendees), 2)
</span><span class="cx">
</span><span class="cx">
</span><ins>+ def test_doNotInviteToSomeoneElsesEvent(self):
+ """
+ If there are events on our calendar which are being organized
+ by someone else, the inviter does not attempt to invite new
+ users to them.
+ """
+ selfNumber = 2
+ vevent, event, calendar, client = self._simpleAccount(
+ selfNumber, INVITED_EVENT)
+ inviter = Inviter(None, client, selfNumber)
+ # Try to send an invitation, but with only one event on the
+ # calendar, of which we are not the organizer. It should be
+ # unchanged afterwards.
+ inviter._invite()
+ attendees = event.vevent.contents[u'vevent'][0].contents[u'attendee']
+ self.assertEqual(len(attendees), 2)
+ self.assertEqual(attendees[0].params['CN'], [u'User 01'])
+ self.assertEqual(attendees[1].params['CN'], [u'User 02'])
</ins><span class="cx">
</span><ins>+
+
</ins><span class="cx"> class AccepterTests(TestCase):
</span><span class="cx"> """
</span><span class="cx"> Tests for loadtest.profiles.Accepter.
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodscontribperformancesim"></a>
<div class="propset"><h4>Property changes: \
CalendarServer/branches/users/cdaboo/pods/contrib/performance/sim</h4> <pre \
class="diff"><span> </span></pre></div>
<a id="svnexecutable"></a>
<div class="addfile"><h4>Added: svn:executable</h4></div>
<a id="CalendarServerbranchesuserscdaboopodsdoccalendarserver_export8"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/doc/calendarserver_export.8 (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/doc/calendarserver_export.8 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/doc/calendarserver_export.8 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -31,7 +31,7 @@
</span><span class="cx"> .Sh DESCRIPTION
</span><span class="cx"> .Nm
</span><span class="cx"> is a tool that generates a single iCalendar file containing \
all of the </span><del>-iCalendar components found from all specifies input sources, \
providing </del><ins>+iCalendar components found from all specified input sources, \
providing </ins><span class="cx"> server administrators a means by which to export \
data from the Darwin </span><span class="cx"> Calandar Server into a format that can \
be viewed and/or manipulated by </span><span class="cx"> other tools. Multiple input \
sources may be specified; the resulting </span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodspython"></a>
<div class="modfile"><h4>Modified: CalendarServer/branches/users/cdaboo/pods/python \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- CalendarServer/branches/users/cdaboo/pods/python 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/python 2011-04-27 21:09:24 UTC (rev \
7377) </span><span class="lines">@@ -1,7 +1,8 @@
</span><del>-#!/bin/sh
</del><ins>+#!/bin/bash
</ins><span class="cx">
</span><span class="cx"> wd="$(cd "$(dirname "$0")" \
&& pwd)"; </span><span class="cx">
</span><del>-export PYTHONPATH="$("${wd}/run" -p)";
</del><ins>+. "${wd}/support/shell.sh"
</ins><span class="cx">
</span><del>-exec python "$@";
</del><ins>+exec "${python}" "$@";
+
</ins></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodssupportMakefileApple"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/support/Makefile.Apple (7376 => 7377)</h4> \
<pre class="diff"><span> <span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/support/Makefile.Apple 2011-04-27 20:10:49 \
UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/support/Makefile.Apple 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -108,6 +108,10 @@
</span><span class="cx"> $(_v) $(INSTALL_DIRECTORY) \
"$(DSTROOT)/usr/libexec/changeip" </span><span class="cx"> $(_v) \
$(INSTALL_FILE) "$(Sources)/calendarserver/tools/changeip_calendar.py" \
"$(DSTROOT)/usr/libexec/changeip/changeip_calendar.py" </span><span \
class="cx"> $(_v) chmod ugo+x \
"$(DSTROOT)/usr/libexec/changeip/changeip_calendar.py" </span><ins>+ @echo \
"Installing certificate update scripts..." + $(_v) $(INSTALL_DIRECTORY) \
"$(DSTROOT)/usr/libexec/certupdate" + $(_v) $(INSTALL_FILE) \
"$(Sources)/contrib/certupdate/calendarcertupdate.py" \
"$(DSTROOT)/usr/libexec/certupdate/calendarcertupdate.py" + $(_v) chmod \
ugo+x "$(DSTROOT)/usr/libexec/certupdate/calendarcertupdate.py" </ins><span \
class="cx"> </span><span class="cx"> install::
</span><span class="cx"> @echo "Installing CalDAVTester package..."
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodssupportbuildsh"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/support/build.sh (7376 => 7377)</h4> <pre \
class="diff"><span> <span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/support/build.sh 2011-04-27 20:10:49 UTC \
(rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/support/build.sh 2011-04-27 21:09:24 \
UTC (rev 7377) </span><span class="lines">@@ -644,7 +644,7 @@
</span><span class="cx"> "http://svn.osafoundation.org/vobject/trunk";
</span><span class="cx">
</span><span class="cx"> # XXX actually PyCalendar should be imported in-place.
</span><del>- py_dependency -fe -i "src" -r 154 \
</del><ins>+ py_dependency -fe -i "src" -r 156 \
</ins><span class="cx"> "pycalendar" "pycalendar" \
"pycalendar" \ </span><span class="cx"> \
"http://svn.mulberrymail.com/repos/PyCalendar/branches/server"; \
</span><span class="cx"> </span><span class="cx">Property changes on: \
CalendarServer/branches/users/cdaboo/pods/support/build.sh </span><span \
class="cx">___________________________________________________________________ \
</span></span></pre></div> <a id="svnmergeinfo"></a>
<div class="modfile"><h4>Modified: svn:mergeinfo</h4></div>
<span class="cx">/CalendarServer/branches/egg-info-351/support/build.sh:4589-4615
</span><span class="cx">/CalendarServer/branches/generic-sqlstore/support/build.sh:6167-6191
</span><span class="cx">/CalendarServer/branches/new-store/support/build.sh:5594-5934
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile/support/build.sh:5911-5935
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile-2/support/build.sh:5936-5981
</span><span class="cx">/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/support/build.sh:5693-5702
</span><span class="cx">/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/support/build.sh:3628-3644
</span><span class="cx">/CalendarServer/branches/users/cdaboo/more-sharing-5591/support/build.sh:5592-5601
</span><span class="cx">/CalendarServer/branches/users/cdaboo/partition-4464/support/build.sh:4465-4957
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycalendar/support/build.sh:7085-7206
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycard/support/build.sh:7227-7237
</span><span class="cx">/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/support/build.sh:5071-5105
</span><span class="cx">/CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/build.sh:5188-5440
</span><span class="cx">/CalendarServer/branches/users/glyph/conn-limit/support/build.sh:6574-6577
</span><span class="cx">/CalendarServer/branches/users/glyph/contacts-server-merge/support/build.sh:4971-5080
</span><span class="cx">/CalendarServer/branches/users/glyph/dalify/support/build.sh:6932-7023
</span><span class="cx">/CalendarServer/branches/users/glyph/db-reconnect/support/build.sh:6824-6876
</span><span class="cx">/CalendarServer/branches/users/glyph/dont-start-postgres/support/build.sh:6592-6614
</span><span class="cx">/CalendarServer/branches/users/glyph/linux-tests/support/build.sh:6893-6900
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-6/support/build.sh:6322-6368
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-7/support/build.sh:6369-6445
</span><span class="cx">/CalendarServer/branches/users/glyph/sendfdport/support/build.sh:5388-5424
</span><span class="cx">/CalendarServer/branches/users/glyph/sharedpool/support/build.sh:6490-6550
</span><span class="cx">/CalendarServer/branches/users/glyph/sql-store/support/build.sh:5929-6073
</span><span class="cx">/CalendarServer/branches/users/glyph/subtransactions/support/build.sh:7248-7258
</span><span class="cx">/CalendarServer/branches/users/glyph/use-system-twisted/support/build.sh:5084-5149
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources/support/build.sh:5032-5051
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources-2/support/build.sh:5052-5061
</span><span class="cx">/CalendarServer/branches/users/sagen/purge_old_events/support/build.sh:6735-6746
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4038/support/build.sh:4040-4067
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4066/support/build.sh:4068-4075
</span><span class="cx">/CalendarServer/branches/users/sagen/resources-2/support/build.sh:5084-5093
</span><span class="cx">/CalendarServer/branches/users/wsanchez/transations/support/build.sh:5515-5593
</span><span class="cx"> + \
/CalendarServer/branches/config-separation/support/build.sh:4379-4443 </span><span \
class="cx">/CalendarServer/branches/egg-info-351/support/build.sh:4589-4615 \
</span><span class="cx">/CalendarServer/branches/generic-sqlstore/support/build.sh:6167-6191
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile-2/support/build.sh:5936-5981
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile/support/build.sh:5911-5935
</span><span class="cx">/CalendarServer/branches/new-store/support/build.sh:5594-5934
</span><span class="cx">/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/support/build.sh:5693-5702
</span><span class="cx">/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/support/build.sh:3628-3644
</span><span class="cx">/CalendarServer/branches/users/cdaboo/more-sharing-5591/support/build.sh:5592-5601
</span><span class="cx">/CalendarServer/branches/users/cdaboo/partition-4464/support/build.sh:4465-4957
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycalendar/support/build.sh:7085-7206
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycard/support/build.sh:7227-7237
</span><span class="cx">/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/support/build.sh:5071-5105
</span><span class="cx">/CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/build.sh:5188-5440
</span><span class="cx">/CalendarServer/branches/users/glyph/conn-limit/support/build.sh:6574-6577
</span><span class="cx">/CalendarServer/branches/users/glyph/contacts-server-merge/support/build.sh:4971-5080
</span><span class="cx">/CalendarServer/branches/users/glyph/dalify/support/build.sh:6932-7023
</span><span class="cx">/CalendarServer/branches/users/glyph/db-reconnect/support/build.sh:6824-6876
</span><span class="cx">/CalendarServer/branches/users/glyph/dont-start-postgres/support/build.sh:6592-6614
</span><span class="cx">/CalendarServer/branches/users/glyph/linux-tests/support/build.sh:6893-6900
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-6/support/build.sh:6322-6368
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-7/support/build.sh:6369-6445
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle-nulls/support/build.sh:7340-7351
</span><span class="cx">/CalendarServer/branches/users/glyph/sendfdport/support/build.sh:5388-5424
</span><span class="cx">/CalendarServer/branches/users/glyph/sharedpool/support/build.sh:6490-6550
</span><span class="cx">/CalendarServer/branches/users/glyph/sql-store/support/build.sh:5929-6073
</span><span class="cx">/CalendarServer/branches/users/glyph/subtransactions/support/build.sh:7248-7258
</span><span class="cx">/CalendarServer/branches/users/glyph/use-system-twisted/support/build.sh:5084-5149
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources-2/support/build.sh:5052-5061
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources/support/build.sh:5032-5051
</span><span class="cx">/CalendarServer/branches/users/sagen/purge_old_events/support/build.sh:6735-6746
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4038/support/build.sh:4040-4067
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4066/support/build.sh:4068-4075
</span><span class="cx">/CalendarServer/branches/users/sagen/resources-2/support/build.sh:5084-5093
</span><span class="cx">/CalendarServer/branches/users/wsanchez/transations/support/build.sh:5515-5593
</span><span class="cx">/CalendarServer/trunk/support/build.sh:7297-7364
</span><a id="CalendarServerbranchesuserscdaboopodssupportpysh"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/support/py.sh (7376 => 7377)</h4> <pre \
class="diff"><span> <span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/support/py.sh 2011-04-27 20:10:49 UTC (rev \
7376)
+++ CalendarServer/branches/users/cdaboo/pods/support/py.sh 2011-04-27 21:09:24 UTC \
(rev 7377) </span><span class="lines">@@ -48,7 +48,7 @@
</span><span class="cx"> # Detect which version of Python to use, then print out \
which one was detected. </span><span class="cx">
</span><span class="cx"> detect_python_version () {
</span><del>- for v in "" "2.6" "2.5"
</del><ins>+ for v in "2.7" "2.6" "2.5" ""
</ins><span class="cx"> do
</span><span class="cx"> for p in \
</span><span class="cx"> "${PYTHON:=}" \
</span><span class="lines">@@ -120,12 +120,12 @@
</span><span class="cx"> # Compare version numbers
</span><span class="cx">
</span><span class="cx"> cmp_version () {
</span><del>- local result=0;
-
</del><span class="cx"> local v="$1"; shift;
</span><span class="cx"> local mv="$1"; shift;
</span><span class="cx">
</span><del>- while [ $result != 1 ]; do
</del><ins>+ local result;
+
+ while true; do
</ins><span class="cx"> vh="${v%%.*}"; # Get highest-order segment
</span><span class="cx"> mvh="${mv%%.*}";
</span><span class="cx">
</span><span class="lines">@@ -134,8 +134,14 @@
</span><span class="cx"> break;
</span><span class="cx"> fi;
</span><span class="cx">
</span><ins>+ if [ "${vh}" -lt "${mvh}" ]; then
+ result=0;
+ break;
+ fi;
+
</ins><span class="cx"> if [ "${v}" == "${v#*.}" ]; then
</span><span class="cx"> # No dots left, so we're ok
</span><ins>+ result=0;
</ins><span class="cx"> break;
</span><span class="cx"> fi;
</span><span class="cx">
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodssupportshellsh"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/support/shell.sh (7376 => 7377)</h4> <pre \
class="diff"><span> <span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/support/shell.sh 2011-04-27 20:10:49 UTC \
(rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/support/shell.sh 2011-04-27 21:09:24 \
UTC (rev 7377) </span><span class="lines">@@ -21,7 +21,10 @@
</span><span class="cx"> # set up by the CalendarServer run script and are not \
otherwise installed on </span><span class="cx"> # your system.
</span><span class="cx">
</span><del>-wd="$(pwd)";
</del><ins>+if [ -z "${wd}" ]; then
+ wd="$(pwd)";
+fi;
+
</ins><span class="cx"> . ${wd}/support/build.sh;
</span><span class="cx"> do_setup=false;
</span><span class="cx"> do_get=false;
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstestserver"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/testserver (7376 => 7377)</h4> <pre \
class="diff"><span> <span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/testserver 2011-04-27 20:10:49 UTC (rev \
7376)
+++ CalendarServer/branches/users/cdaboo/pods/testserver 2011-04-27 21:09:24 UTC (rev \
7377) </span><span class="lines">@@ -61,12 +61,13 @@
</span><span class="cx"> # Do The Right Thing
</span><span class="cx"> ##
</span><span class="cx">
</span><del>-export PYTHONPATH=$("${wd}/run" -p);
</del><ins>+. "${wd}/support/shell.sh"
</ins><span class="cx">
</span><span class="cx"> if [ ! -e \
"${documentroot}/calendars/__uids__/user09" ]; then </span><span \
class="cx"> curl "http://localhost:8008/calendars/__uids__/user09/"; \
</span><span class="cx"> fi; </span><span class="cx">
</span><del>-python twistedcaldav/test/data/makelargecalendars.py -o 9 -d \
"${documentroot}"; </del><ins>+"${python}" \
twistedcaldav/test/data/makelargecalendars.py -o 9 -d "${documentroot}"; \
</ins><span class="cx"> </span><del>-cd "${cdt}" && python \
testcaldav.py -s "${serverinfo}" "$@"; </del><ins>+cd \
"${cdt}" && "${python}" testcaldav.py -s \
"${serverinfo}" "$@"; +
</ins></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwextenterpriseadbapi2py"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/adbapi2.py (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/adbapi2.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twext/enterprise/adbapi2.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -703,8 +703,13 @@
</span><span class="cx">
</span><span class="cx"> def startService(self):
</span><span class="cx"> """
</span><del>- No startup necessary.
</del><ins>+ Increase the thread pool size of the reactor by the number of \
threads + that this service may consume. This is important because unlike \
most + L{IReactorThreads} users, the connection work units are very long-lived
+ and block until this service has been stopped.
</ins><span class="cx"> """
</span><ins>+ tp = self.reactor.getThreadPool()
+ self.reactor.suggestThreadPoolSize(tp.max + self.maxConnections)
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> @inlineCallbacks
</span><span class="lines">@@ -742,7 +747,10 @@
</span><span class="cx"> # independently submitted from .abort() / \
.close(). </span><span class="cx"> yield \
self._free.pop()._releaseConnection() </span><span class="cx">
</span><ins>+ tp = self.reactor.getThreadPool()
+ self.reactor.suggestThreadPoolSize(tp.max - self.maxConnections)
</ins><span class="cx">
</span><ins>+
</ins><span class="cx"> def _createHolder(self):
</span><span class="cx"> """
</span><span class="cx"> Create a L{ThreadHolder}. (Test hook.)
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwextenterprisedalmodelpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/model.py (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/model.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/model.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -188,7 +188,8 @@
</span><span class="cx">
</span><span class="cx"> def needsValue(self):
</span><span class="cx"> """
</span><del>- Does this column require a value in INSERT statements which \
create rows? </del><ins>+ Does this column require a value in C{INSERT} \
statements which create + rows?
</ins><span class="cx">
</span><span class="cx"> @return: C{True} for L{Column}s with no default \
specified which also </span><span class="cx"> cannot be NULL, C{False} \
otherwise. </span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwextenterprisedalsyntaxpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/syntax.py (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/syntax.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/syntax.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -19,7 +19,7 @@
</span><span class="cx"> Syntax wrappers and generators for SQL.
</span><span class="cx"> """
</span><span class="cx">
</span><del>-import itertools
</del><ins>+from itertools import count, repeat
</ins><span class="cx">
</span><span class="cx"> from zope.interface import implements
</span><span class="cx">
</span><span class="lines">@@ -72,7 +72,7 @@
</span><span class="cx">
</span><span class="cx"> def __init__(self, dialect):
</span><span class="cx"> super(NumericPlaceholder, self).__init__(dialect)
</span><del>- self._next = itertools.count(1).next
</del><ins>+ self._next = count(1).next
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> def placeholder(self):
</span><span class="lines">@@ -121,10 +121,37 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx"> def _extraVars(self, txn, metadata):
</span><ins>+ """
+ A hook for subclasses to provide additional keyword arguments to the
+ C{bind} call when L{_Statement.on} is executed. Currently this is used
+ only for 'out' parameters to capture results when executing statements
+ that do not normally have a result (L{Insert}, L{Delete}, L{Update}).
+ """
</ins><span class="cx"> return {}
</span><span class="cx">
</span><span class="cx">
</span><span class="cx"> def _extraResult(self, result, outvars, metadata):
</span><ins>+ """
+ A hook for subclasses to manipulate the results of 'on', after they've
+ been retrieved by the database but before they've been given to
+ application code.
+
+ @param result: a L{Deferred} that will fire with the rows as returned by
+ the database.
+ @type result: C{list} of rows, which are C{list}s or C{tuple}s.
+
+ @param outvars: a dictionary of extra variables returned by
+ C{self._extraVars}.
+
+ @param metadata: information about the connection where the statement
+ was executed.
+
+ @type metadata: L{ConnectionMetadata} (a subclass thereof)
+
+ @return: the result to be returned from L{_Statement.on}.
+
+ @rtype: L{Deferred} firing result rows
+ """
</ins><span class="cx"> return result
</span><span class="cx">
</span><span class="cx">
</span><span class="lines">@@ -132,6 +159,19 @@
</span><span class="cx"> """
</span><span class="cx"> Execute this statement on a given \
L{IAsyncTransaction} and return the </span><span class="cx"> resulting \
L{Deferred}. </span><ins>+
+ @param txn: the L{IAsyncTransaction} to execute this on.
+
+ @param raiseOnZeroRowCount: the exception to raise if no data was
+ affected or returned by this query.
+
+ @param kw: keyword arguments, mapping names of L{Parameter} objects
+ located somewhere in C{self}
+
+ @return: results from the database.
+
+ @rtype: a L{Deferred} firing a C{list} of records (C{tuple}s or
+ C{list}s)
</ins><span class="cx"> """
</span><span class="cx"> metadata = \
self._paramstyles[txn.paramstyle](txn.dialect) </span><span class="cx"> \
outvars = self._extraVars(txn, metadata) </span><span class="lines">@@ -139,10 \
+179,60 @@ </span><span class="cx"> fragment = \
self.toSQL(metadata).bind(**kw) </span><span class="cx"> result = \
txn.execSQL(fragment.text, fragment.parameters, </span><span class="cx"> \
raiseOnZeroRowCount) </span><del>- return self._extraResult(result, outvars, \
metadata) </del><ins>+ result = self._extraResult(result, outvars, metadata)
+ if metadata.dialect == ORACLE_DIALECT and result:
+ result.addCallback(self._fixOracleNulls)
+ return result
</ins><span class="cx">
</span><span class="cx">
</span><ins>+ def _resultColumns(self):
+ """
+ Subclasses must implement this to return a description of the columns
+ expected to be returned. This is a list of L{ColumnSyntax} objects, and
+ possibly other expression syntaxes which will be converted to C{None}.
+ """
+ raise NotImplementedError(
+ "Each statement subclass must describe its result"
+ )
</ins><span class="cx">
</span><ins>+
+ def _resultShape(self):
+ """
+ Process the result of the subclass's C{_resultColumns}, as described in
+ the docstring above.
+ """
+ for expectation in self._resultColumns():
+ if isinstance(expectation, ColumnSyntax):
+ yield expectation.model
+ else:
+ yield None
+
+
+ def _fixOracleNulls(self, rows):
+ """
+ Oracle treats empty strings as C{NULL}. Fix this by looking at the
+ columns we expect to have returned, and replacing any C{None}s with
+ empty strings in the appropriate position.
+ """
+ if rows is None:
+ return None
+ newRows = []
+ for row in rows:
+ newRow = []
+ for column, description in zip(row, self._resultShape()):
+ if ((description is not None and
+ # FIXME: "is the python type str" is what I mean; \
this list + # should be more centrally maintained
+ description.type.name in ('varchar', 'text', 'char') and
+ column is None
+ )):
+ column = ''
+ newRow.append(column)
+ newRows.append(newRow)
+ return newRows
+
+
+
</ins><span class="cx"> class Syntax(object):
</span><span class="cx"> """
</span><span class="cx"> Base class for syntactic convenience.
</span><span class="lines">@@ -618,7 +708,6 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx">
</span><del>-
</del><span class="cx"> class Select(_Statement):
</span><span class="cx"> """
</span><span class="cx"> 'select' statement.
</span><span class="lines">@@ -719,7 +808,25 @@
</span><span class="cx"> return result
</span><span class="cx">
</span><span class="cx">
</span><ins>+ def _resultColumns(self):
+ """
+ Determine the list of L{ColumnSyntax} objects that will represent the
+ result. Normally just the list of selected columns; if wildcard syntax
+ is used though, determine the ordering from the database.
+ """
+ if self.columns is ALL_COLUMNS:
+ # TODO: Possibly this rewriting should always be done, before even
+ # executing the query, so that if we develop a schema mismatch with
+ # the database (additional columns), the application will still see
+ # the right rows.
+ for table in self.From.tables():
+ for column in table:
+ yield column
+ else:
+ for column in self.columns.columns:
+ yield column
</ins><span class="cx">
</span><ins>+
</ins><span class="cx"> def _commaJoined(stmts):
</span><span class="cx"> first = True
</span><span class="cx"> cstatement = SQLFragment()
</span><span class="lines">@@ -783,7 +890,9 @@
</span><span class="cx"> Add a dialect-appropriate 'returning' clause to the \
end of the given SQL </span><span class="cx"> statement.
</span><span class="cx">
</span><del>- @param metadata: describes the database we are generating the \
statement for. </del><ins>+ @param metadata: describes the database we are \
generating the statement + for.
+
</ins><span class="cx"> @type metadata: L{ConnectionMetadata}
</span><span class="cx">
</span><span class="cx"> @param stmt: the SQL fragment generated without the \
'returning' clause </span><span class="lines">@@ -839,8 +948,16 @@
</span><span class="cx"> return result
</span><span class="cx">
</span><span class="cx">
</span><ins>+ def _resultColumns(self):
+ return self._returnAsList()
</ins><span class="cx">
</span><ins>+
+
</ins><span class="cx"> class _OracleOutParam(object):
</span><ins>+ """
+ A parameter that will be populated using the cx_Oracle API for host
+ variables.
+ """
</ins><span class="cx"> implements(IDerivedParameter)
</span><span class="cx">
</span><span class="cx"> def __init__(self, columnSyntax):
</span><span class="lines">@@ -848,9 +965,8 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx"> def preQuery(self, cursor):
</span><del>- self.columnSyntax
</del><span class="cx"> typeMap = {'integer': cx_Oracle.NUMBER,
</span><del>- 'text': cx_Oracle.CLOB,
</del><ins>+ 'text': cx_Oracle.NCLOB,
</ins><span class="cx"> 'varchar': cx_Oracle.STRING,
</span><span class="cx"> 'timestamp': cx_Oracle.TIMESTAMP}
</span><span class="cx"> typeID = self.columnSyntax.model.type.name.lower()
</span><span class="lines">@@ -993,8 +1109,20 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx">
</span><del>-class Lock(_Statement):
</del><ins>+class _LockingStatement(_Statement):
</ins><span class="cx"> """
</span><ins>+ A statement related to lock management, which implicitly has no \
results. + """
+ def _resultColumns(self):
+ """
+ No columns should be expected, so return an infinite iterator of None.
+ """
+ return repeat(None)
+
+
+
+class Lock(_LockingStatement):
+ """
</ins><span class="cx"> An SQL 'lock' statement.
</span><span class="cx"> """
</span><span class="cx">
</span><span class="lines">@@ -1015,7 +1143,7 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx">
</span><del>-class Savepoint(_Statement):
</del><ins>+class Savepoint(_LockingStatement):
</ins><span class="cx"> """
</span><span class="cx"> An SQL 'savepoint' statement.
</span><span class="cx"> """
</span><span class="lines">@@ -1028,7 +1156,7 @@
</span><span class="cx"> return SQLFragment('savepoint %s' % (self.name,))
</span><span class="cx">
</span><span class="cx">
</span><del>-class RollbackToSavepoint(_Statement):
</del><ins>+class RollbackToSavepoint(_LockingStatement):
</ins><span class="cx"> """
</span><span class="cx"> An SQL 'rollback to savepoint' statement.
</span><span class="cx"> """
</span><span class="lines">@@ -1041,7 +1169,7 @@
</span><span class="cx"> return SQLFragment('rollback to savepoint %s' % \
(self.name,)) </span><span class="cx">
</span><span class="cx">
</span><del>-class ReleaseSavepoint(_Statement):
</del><ins>+class ReleaseSavepoint(_LockingStatement):
</ins><span class="cx"> """
</span><span class="cx"> An SQL 'release savepoint' statement.
</span><span class="cx"> """
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwextenterprisedaltesttest_sqlsyntaxpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/test/test_sqlsyntax.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/test/test_sqlsyntax.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twext/enterprise/dal/test/test_sqlsyntax.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -34,6 +34,7 @@
</span><span class="cx"> from twext.enterprise.adbapi2 import ConnectionPool
</span><span class="cx"> from twext.enterprise.test.test_adbapi2 import resultOf
</span><span class="cx"> from twext.enterprise.test.test_adbapi2 import \
FakeThreadHolder </span><ins>+from twisted.internet.defer import succeed
</ins><span class="cx"> from twisted.trial.unittest import TestCase
</span><span class="cx">
</span><span class="cx">
</span><span class="lines">@@ -49,6 +50,25 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx">
</span><ins>+class FakeCXOracleModule(object):
+ NUMBER = 'the NUMBER type'
+ STRING = 'a string type (for varchars)'
+ NCLOB = 'the NCLOB type. (for text)'
+ TIMESTAMP = 'for timestamps!'
+
+
+
+class NullTestingOracleTxn(object):
+ """
+ Fake transaction for testing oracle NULL behavior.
+ """
+
+ dialect = ORACLE_DIALECT
+ paramstyle = 'numeric'
+
+ def execSQL(self, text, params, exc):
+ return succeed([[None, None]])
+
</ins><span class="cx"> class GenerationTests(TestCase):
</span><span class="cx"> """
</span><span class="cx"> Tests for syntactic helpers to generate SQL queries.
</span><span class="lines">@@ -63,7 +83,10 @@
</span><span class="cx"> create table OTHER (BAR integer,
</span><span class="cx"> FOO_BAR integer \
not null); </span><span class="cx"> create table TEXTUAL \
(MYTEXT varchar(255)); </span><del>- create table LEVELS \
(ACCESS integer, USERNAME varchar(255)); </del><ins>+ create \
table LEVELS (ACCESS integer, + USERNAME \
varchar(255)); + create table NULLCHECK (ASTRING varchar(255) \
not null, + ANUMBER integer);
</ins><span class="cx"> """)
</span><span class="cx"> self.schema = SchemaSyntax(s)
</span><span class="cx">
</span><span class="lines">@@ -121,7 +144,7 @@
</span><span class="cx"> self.assertRaises(ValueError, sampleComparison)
</span><span class="cx">
</span><span class="cx">
</span><del>- def test_nullComparison(self):
</del><ins>+ def test_compareWithNULL(self):
</ins><span class="cx"> """
</span><span class="cx"> Comparing a column with None results in the \
generation of an 'is null' </span><span class="cx"> or 'is not null' SQL \
statement. </span><span class="lines">@@ -510,18 +533,14 @@
</span><span class="cx"> )
</span><span class="cx">
</span><span class="cx">
</span><del>- def test_insertMultiReturnOnOracleTxn(self):
</del><ins>+ def simulateOracleConnection(self):
</ins><span class="cx"> """
</span><del>- As described in L{test_insertMultiReturnOracle}, Oracle deals \
with
- 'returning' clauses by using out parameters. However, this is not quite
- enough, as the code needs to actually retrieve the values from the out
- parameters.
</del><ins>+ Create a fake oracle-ish connection pool without using real \
threads or a + real database.
+
+ @return: a 3-tuple of L{IAsyncTransaction}, L{ConnectionPool},
+ L{ConnectionFactory}.
</ins><span class="cx"> """
</span><del>- class FakeCXOracleModule(object):
- NUMBER = 'the NUMBER type'
- STRING = 'a string type (for varchars)'
- CLOB = 'the clob type. (for text)'
- TIMESTAMP = 'for timestamps!'
</del><span class="cx"> self.patch(syntax, 'cx_Oracle', FakeCXOracleModule)
</span><span class="cx"> factory = ConnectionFactory()
</span><span class="cx"> pool = ConnectionPool(factory.connect, \
maxConnections=2, </span><span class="lines">@@ -531,6 +550,17 @@
</span><span class="cx"> pool._createHolder = lambda : FakeThreadHolder(self)
</span><span class="cx"> pool.startService()
</span><span class="cx"> conn = pool.connection()
</span><ins>+ return conn, pool, factory
+
+
+ def test_insertMultiReturnOnOracleTxn(self):
+ """
+ As described in L{test_insertMultiReturnOracle}, Oracle deals with
+ 'returning' clauses by using out parameters. However, this is not quite
+ enough, as the code needs to actually retrieve the values from the out
+ parameters.
+ """
+ conn, pool, factory = self.simulateOracleConnection()
</ins><span class="cx"> i = Insert({self.schema.FOO.BAR: 40,
</span><span class="cx"> self.schema.FOO.BAZ: 50},
</span><span class="cx"> Return=(self.schema.FOO.BAR, \
self.schema.FOO.BAZ)) </span><span class="lines">@@ -828,6 +858,57 @@
</span><span class="cx"> )
</span><span class="cx">
</span><span class="cx">
</span><ins>+ def test_rewriteOracleNULLs_Select(self):
+ """
+ Oracle databases cannot distinguish between the empty string and
+ C{NULL}. When you insert an empty string, C{cx_Oracle} therefore treats
+ it as a C{None} and will return that when you select it back again. We
+ address this in the schema by dropping 'not null' constraints.
+
+ Therefore, when executing a statement which includes a string column,
+ 'on' should rewrite None return values from C{cx_Oracle} to be empty
+ bytestrings, but only for string columns.
+ """
+
+ rows = resultOf(
+ Select([self.schema.NULLCHECK.ASTRING,
+ self.schema.NULLCHECK.ANUMBER],
+ From=self.schema.NULLCHECK).on(NullTestingOracleTxn()))[0]
+
+ self.assertEquals(rows, [['', None]])
+
+
+ def test_rewriteOracleNULLs_SelectAllColumns(self):
+ """
+ Same as L{test_rewriteOracleNULLs_Select}, but with the L{ALL_COLUMNS}
+ shortcut.
+ """
+ rows = resultOf(
+ Select(From=self.schema.NULLCHECK).on(NullTestingOracleTxn())
+ )[0]
+ self.assertEquals(rows, [['', None]])
+
+
+ def test_rewriteOracleNULLs_Insert(self):
+ """
+ The behavior described in L{test_rewriteOracleNULLs_Select} applies to
+ other statement types as well, specifically those with 'returning'
+ clauses.
+ """
+ conn, pool, factory = self.simulateOracleConnection()
+ # Add 2 cursor variable values so that these will be used by
+ # FakeVariable.getvalue.
+ factory.varvals.extend([None, None])
+ rows = resultOf(
+ Insert({self.schema.NULLCHECK.ASTRING: '',
+ self.schema.NULLCHECK.ANUMBER: None},
+ Return=[self.schema.NULLCHECK.ASTRING,
+ self.schema.NULLCHECK.ANUMBER]
+ ).on(conn))[0]
+
+ self.assertEquals(rows, [['', None]])
+
+
</ins><span class="cx"> def test_nestedLogicalExpressions(self):
</span><span class="cx"> """
</span><span class="cx"> Make sure that logical operator precedence inserts \
proper parenthesis </span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwextenterprisetesttest_adbapi2py"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/test/test_adbapi2.py (7376 \
=> 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/test/test_adbapi2.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twext/enterprise/test/test_adbapi2.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -20,6 +20,9 @@
</span><span class="cx">
</span><span class="cx"> from itertools import count
</span><span class="cx">
</span><ins>+from zope.interface.verify import verifyClass
+
+from twisted.python.threadpool import ThreadPool
</ins><span class="cx"> from twisted.trial.unittest import TestCase
</span><span class="cx">
</span><span class="cx"> from twisted.internet.defer import execute
</span><span class="lines">@@ -28,6 +31,8 @@
</span><span class="cx"> from twisted.internet.defer import Deferred
</span><span class="cx"> from twext.enterprise.ienterprise import ConnectionError
</span><span class="cx"> from twext.enterprise.ienterprise import \
AlreadyFinishedError </span><ins>+from twisted.internet.interfaces import \
IReactorThreads +from zope.interface.declarations import implements
</ins><span class="cx"> from twext.enterprise.adbapi2 import ConnectionPool
</span><span class="cx">
</span><span class="cx">
</span><span class="lines">@@ -192,6 +197,9 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx"> def getvalue(self):
</span><ins>+ vv = self.cursor.connection.parent.varvals
+ if vv:
+ return vv.pop(0)
</ins><span class="cx"> return self.cursor.variables.index(self) + 300
</span><span class="cx">
</span><span class="cx">
</span><span class="lines">@@ -206,6 +214,7 @@
</span><span class="cx"> self.idcounter = count(1)
</span><span class="cx"> self._connectResultQueue = []
</span><span class="cx"> self.defaultConnect()
</span><ins>+ self.varvals = []
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> @property
</span><span class="lines">@@ -324,6 +333,81 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx">
</span><ins>+class ClockWithThreads(Clock):
+ """
+ A testing reactor that supplies L{IReactorTime} and L{IReactorThreads}.
+ """
+ implements(IReactorThreads)
+
+ def __init__(self):
+ super(ClockWithThreads, self).__init__()
+ self._pool = ThreadPool()
+
+
+ def getThreadPool(self):
+ """
+ Get the threadpool.
+ """
+ return self._pool
+
+
+ def suggestThreadPoolSize(self, size):
+ """
+ Approximate the behavior of a 'real' reactor.
+ """
+ self._pool.adjustPoolsize(maxthreads=size)
+
+
+ def callInThread(self, thunk, *a, **kw):
+ """
+ No implementation.
+ """
+
+
+ def callFromThread(self, thunk, *a, **kw):
+ """
+ No implementation.
+ """
+
+
+verifyClass(IReactorThreads, ClockWithThreads)
+
+
+
+class ConnectionPoolBootTests(TestCase):
+ """
+ Tests for the start-up phase of L{ConnectionPool}.
+ """
+
+ def test_threadCount(self):
+ """
+ The reactor associated with a L{ConnectionPool} will have its maximum
+ thread count adjusted when L{ConnectionPool.startService} is called, to
+ accomodate for L{ConnectionPool.maxConnections} additional threads.
+
+ Stopping the service should restore it to its original value, so that a
+ repeatedly re-started L{ConnectionPool} will not cause the thread
+ ceiling to grow without bound.
+ """
+ defaultMax = 27
+ connsMax = 45
+ combinedMax = defaultMax + connsMax
+ pool = ConnectionPool(None, maxConnections=connsMax)
+ pool.reactor = ClockWithThreads()
+ threadpool = pool.reactor.getThreadPool()
+ pool.reactor.suggestThreadPoolSize(defaultMax)
+ self.assertEquals(threadpool.max, defaultMax)
+ pool.startService()
+ self.assertEquals(threadpool.max, combinedMax)
+ justChecking = []
+ pool.stopService().addCallback(justChecking.append)
+ # No SQL run, so no threads started, so this deferred should fire
+ # immediately. If not, we're in big trouble, so sanity check.
+ self.assertEquals(justChecking, [None])
+ self.assertEquals(threadpool.max, defaultMax)
+
+
+
</ins><span class="cx"> class ConnectionPoolTests(TestCase):
</span><span class="cx"> """
</span><span class="cx"> Tests for L{ConnectionPool}.
</span><span class="lines">@@ -340,7 +424,7 @@
</span><span class="cx"> self.pool = \
ConnectionPool(self.factory.connect, </span><span class="cx"> \
maxConnections=2) </span><span class="cx"> self.pool._createHolder = \
self.makeAHolder </span><del>- self.clock = self.pool.reactor = \
Clock() </del><ins>+ self.clock = self.pool.reactor = \
ClockWithThreads() </ins><span class="cx"> self.pool.startService()
</span><span class="cx">
</span><span class="cx">
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwextenterpriseutilpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/util.py (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twext/enterprise/util.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twext/enterprise/util.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -30,8 +30,10 @@
</span><span class="cx"> @param column: a single value from a column.
</span><span class="cx">
</span><span class="cx"> @return: a converted value based on the type of the \
input; oracle CLOBs and </span><del>- datetime timestamps will be converted to \
strings, all other types will
- be left alone.
</del><ins>+ datetime timestamps will be converted to strings, unicode values \
will be + converted to UTF-8 encoded byte sequences (C{str}s), and floating \
point + numbers will be converted to integer types if they are integers. Any
+ other types will be left alone.
</ins><span class="cx"> """
</span><span class="cx"> if hasattr(column, 'read'):
</span><span class="cx"> # Try to detect large objects and format convert \
them to </span><span class="lines">@@ -40,7 +42,7 @@
</span><span class="cx"> # http://cx-oracle.sourceforge.net/html/lob.html - \
in </span><span class="cx"> # particular, the part where it says "In \
particular, do not </span><span class="cx"> # use the fetchall() \
method". </span><del>- return column.read()
</del><ins>+ column = column.read()
</ins><span class="cx"> elif isinstance(column, datetime):
</span><span class="cx"> # cx_Oracle properly maps the type of timestamps to \
datetime </span><span class="cx"> # objects. However, our code is mostly \
written against </span><span class="lines">@@ -50,12 +52,20 @@
</span><span class="cx"> # we'll do that.
</span><span class="cx"> return column.strftime(SQL_TIMESTAMP_FORMAT)
</span><span class="cx"> elif isinstance(column, float):
</span><ins>+ # cx_Oracle maps _all_ nubmers to float types, which is more \
consistent, + # but we expect the database to be able to store integers as \
integers + # (in fact almost all the values in our schema are integers), so we \
map + # those values which exactly match back into integers.
</ins><span class="cx"> if int(column) == column:
</span><span class="cx"> return int(column)
</span><span class="cx"> else:
</span><span class="cx"> return column
</span><del>- else:
- return column
</del><ins>+ if isinstance(column, unicode):
+ # Finally, we process all data as UTF-8 bytestrings in order to reduce
+ # memory consumption. Pass any unicode string values back to the
+ # application as unicode.
+ column = column.encode('utf-8')
+ return column
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx">
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavconfigpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/config.py (7376 => 7377)</h4> \
<pre class="diff"><span> <span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/config.py 2011-04-27 20:10:49 \
UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/config.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -135,6 +135,8 @@
</span><span class="cx"> else:
</span><span class="cx"> self._provider = provider
</span><span class="cx"> self._updating = False
</span><ins>+ self._beforeResetHook = None
+ self._afterResetHook = None
</ins><span class="cx"> self._preUpdateHooks = []
</span><span class="cx"> self._postUpdateHooks = []
</span><span class="cx"> self.reset()
</span><span class="lines">@@ -179,15 +181,25 @@
</span><span class="cx"> lastDict[configItem] = defaultValue
</span><span class="cx"> return defaultValue
</span><span class="cx">
</span><ins>+ def addResetHooks(self, before, after):
+ """
+ Hooks for preserving config across reload( ) + reset( )
+
+ Each hook will be passed the config data; whatever the before hook
+ returns will be passed as the second arg to the after hook.
+ """
+ self._beforeResetHook = before
+ self._afterResetHook = after
+
</ins><span class="cx"> def addPreUpdateHooks(self, hooks):
</span><span class="cx"> self._preUpdateHooks.extend(hooks)
</span><del>-
</del><ins>+
</ins><span class="cx"> def addPostUpdateHooks(self, hooks):
</span><span class="cx"> self._postUpdateHooks.extend(hooks)
</span><span class="cx">
</span><span class="cx"> def getProvider(self):
</span><span class="cx"> return self._provider
</span><del>-
</del><ins>+
</ins><span class="cx"> def setProvider(self, provider):
</span><span class="cx"> self._provider = provider
</span><span class="cx"> self.reset()
</span><span class="lines">@@ -231,7 +243,16 @@
</span><span class="cx"> configDict = ConfigDict(self._provider.loadConfig())
</span><span class="cx"> configDict._reloading = True
</span><span class="cx"> if not self._provider.hasErrors():
</span><ins>+ if self._beforeResetHook:
+ # Give the beforeResetHook a chance to stash away values we want
+ # to preserve across the reload( )
+ preserved = self._beforeResetHook(self._data)
+ else:
+ preserved = None
</ins><span class="cx"> self.reset()
</span><ins>+ if preserved and self._afterResetHook:
+ # Pass the preserved data back to the afterResetHook
+ self._afterResetHook(self._data, preserved)
</ins><span class="cx"> self.update(configDict)
</span><span class="cx"> else:
</span><span class="cx"> raise ConfigurationError("Invalid \
configuration in %s" </span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavdirectoryappleopendirectorypy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/appleopendirectory.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/appleopendirectory.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/appleopendirectory.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -422,12 +422,6 @@
</span><span class="cx">
</span><span class="cx"> recordGUID = \
value.get(dsattributes.kDS1AttrGeneratedUID) </span><span class="cx">
</span><del>- # Skip if group restriction is in place and guid is \
not
- # a member
- if self.restrictedGUIDs is not None:
- if str(recordGUID) not in self.restrictedGUIDs:
- continue
-
</del><span class="cx"> recordType = \
value.get(dsattributes.kDSNAttrRecordType) </span><span class="cx"> \
if isinstance(recordType, list): </span><span class="cx"> \
recordType = recordType[0] </span><span class="lines">@@ -435,6 +429,13 @@
</span><span class="cx"> continue
</span><span class="cx"> recordType = \
self._fromODRecordTypes[recordType] </span><span class="cx">
</span><ins>+ # Skip if group restriction is in place and guid is \
not + # a member (but don't skip any groups)
+ if (recordType != self.recordType_groups and
+ self.restrictedGUIDs is not None):
+ if str(recordGUID) not in self.restrictedGUIDs:
+ continue
+
</ins><span class="cx"> recordAuthIDs = self._setFromAttribute(
</span><span class="cx"> \
value.get(dsattributes.kDSNAttrAltSecurityIdentities)) </span><span class="cx"> \
recordFullName = value.get( </span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavdirectoryaugmentpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/augment.py (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/augment.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/augment.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -420,7 +420,10 @@
</span><span class="cx"> """
</span><span class="cx"> super(AugmentXMLDB, self).refresh()
</span><span class="cx"> try:
</span><del>- self.db = self._parseXML()
</del><ins>+ results = self._parseXML()
+ # Only update the cache if _parseXML( ) returns anything
+ if results:
+ self.db = results
</ins><span class="cx"> except RuntimeError:
</span><span class="cx"> log.error("Failed to parse XML augments \
file during cache refresh - ignoring") </span><span class="cx"> \
self.lastCached = time.time() </span><span class="lines">@@ -435,16 +438,18 @@
</span><span class="cx"> self.removeAugmentRecords(self.db.keys())
</span><span class="cx"> return succeed(None)
</span><span class="cx">
</span><del>- def _shouldReparse(self, xmlFile):
</del><ins>+ def _shouldReparse(self, xmlFiles):
</ins><span class="cx"> """
</span><del>- Check to see whether the given file has been modified since we \
last
- parsed it.
</del><ins>+ Check to see whether any of the given files have been modified \
since + we last parsed them.
</ins><span class="cx"> """
</span><del>- oldModTime, oldSize = self.xmlFileStats.get(xmlFile, (0, 0))
- newModTime = os.path.getmtime(xmlFile)
- newSize = os.path.getsize(xmlFile)
- if (oldModTime != newModTime) or (oldSize != newSize):
- return True
</del><ins>+ for xmlFile in xmlFiles:
+ if os.path.exists(xmlFile):
+ oldModTime, oldSize = self.xmlFileStats.get(xmlFile, (0, 0))
+ newModTime = os.path.getmtime(xmlFile)
+ newSize = os.path.getsize(xmlFile)
+ if (oldModTime != newModTime) or (oldSize != newSize):
+ return True
</ins><span class="cx"> return False
</span><span class="cx">
</span><span class="cx"> def _parseXML(self):
</span><span class="lines">@@ -454,23 +459,13 @@
</span><span class="cx"> If none of the xmlFiles exist, create a default \
record. </span><span class="cx"> """
</span><span class="cx">
</span><del>- # Do each file
</del><span class="cx"> results = {}
</span><span class="cx">
</span><del>- allMissing = True
</del><ins>+ # If all augments files are missing, return a default record
</ins><span class="cx"> for xmlFile in self.xmlFiles:
</span><span class="cx"> if os.path.exists(xmlFile):
</span><del>- # Compare previously seen modification time and size of \
each
- # xml file. If unchanged, skip.
- if self._shouldReparse(xmlFile):
- # Creating a parser does the parse
- XMLAugmentsParser(xmlFile, results)
- newModTime = os.path.getmtime(xmlFile)
- newSize = os.path.getsize(xmlFile)
- self.xmlFileStats[xmlFile] = (newModTime, newSize)
- allMissing = False
-
- if allMissing:
</del><ins>+ break
+ else:
</ins><span class="cx"> results["Default"] = AugmentRecord(
</span><span class="cx"> "Default",
</span><span class="cx"> enabled=True,
</span><span class="lines">@@ -478,6 +473,17 @@
</span><span class="cx"> enabledForAddressBooks=True,
</span><span class="cx"> )
</span><span class="cx">
</span><ins>+ # Compare previously seen modification time and size of each
+ # xml file. If all are unchanged, skip.
+ if self._shouldReparse(self.xmlFiles):
+ for xmlFile in self.xmlFiles:
+ if os.path.exists(xmlFile):
+ # Creating a parser does the parse
+ XMLAugmentsParser(xmlFile, results)
+ newModTime = os.path.getmtime(xmlFile)
+ newSize = os.path.getsize(xmlFile)
+ self.xmlFileStats[xmlFile] = (newModTime, newSize)
+
</ins><span class="cx"> return results
</span><span class="cx">
</span><span class="cx"> class AugmentADAPI(AugmentDB, AbstractADBAPIDatabase):
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavdirectorycachingdirectorypy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/cachingdirectory.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/cachingdirectory.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/cachingdirectory.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -74,6 +74,7 @@
</span><span class="cx"> CachingDirectoryService.INDEX_TYPE_AUTHID : \
{}, </span><span class="cx"> }
</span><span class="cx"> self.directoryService = directoryService
</span><ins>+ self.lastPurgedTime = time.time()
</ins><span class="cx">
</span><span class="cx"> def addRecord(self, record, indexType, indexKey, \
useMemcache=True, </span><span class="cx"> neverExpire=False):
</span><span class="lines">@@ -121,8 +122,22 @@
</span><span class="cx"> self.log_debug("Missing record \
index item; type: %s, item: %s" % (indexType, item)) </span><span class="cx"> \
</span><span class="cx"> def findRecord(self, indexType, indexKey):
</span><ins>+ self.purgeExpiredRecords()
</ins><span class="cx"> return self.recordsIndexedBy[indexType].get(indexKey)
</span><span class="cx">
</span><ins>+
+ def purgeExpiredRecords(self):
+ """
+ Scan the cached records and remove any that have expired.
+ Does nothing if we've scanned within the past cacheTimeout seconds.
+ """
+ if time.time() - self.lastPurgedTime > \
self.directoryService.cacheTimeout: + for record in list(self.records):
+ if record.isExpired():
+ self.removeRecord(record)
+ self.lastPurgedTime = time.time()
+
+
</ins><span class="cx"> class CachingDirectoryService(DirectoryService):
</span><span class="cx"> """
</span><span class="cx"> Caching Directory implementation of \
L{IDirectoryService}. </span><span class="lines">@@ -272,10 +287,7 @@
</span><span class="cx"> record = \
self.recordCacheForType(recordType).findRecord(indexType, indexKey) </span><span \
class="cx"> </span><span class="cx"> if record:
</span><del>- if (
- record.cachedTime != 0 and
- time.time() - record.cachedTime > self.cacheTimeout
- ):
</del><ins>+ if record.isExpired():
</ins><span class="cx"> \
self.recordCacheForType(recordType).removeRecord(record) </span><span class="cx"> \
return None </span><span class="cx"> else:
</span><span class="lines">@@ -386,6 +398,20 @@
</span><span class="cx"> def neverExpire(self):
</span><span class="cx"> self.cachedTime = 0
</span><span class="cx">
</span><ins>+ def isExpired(self):
+ """
+ Returns True if this record was created more than cacheTimeout
+ seconds ago
+ """
+ if (
+ self.cachedTime != 0 and
+ time.time() - self.cachedTime > self.service.cacheTimeout
+ ):
+ return True
+ else:
+ return False
+
+
</ins><span class="cx"> class DirectoryMemcacheError(DirectoryError):
</span><span class="cx"> """
</span><span class="cx"> Error communicating with memcached.
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavdirectorytesttest_augmentpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_augment.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_augment.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_augment.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -281,10 +281,20 @@
</span><span class="cx"> newxmlfile = FilePath(self.mktemp())
</span><span class="cx"> FilePath(xmlFile).copyTo(newxmlfile)
</span><span class="cx"> db = AugmentXMLDB((newxmlfile.path,))
</span><del>- self.assertFalse(db._shouldReparse(newxmlfile.path)) # No need \
to parse </del><ins>+ self.assertFalse(db._shouldReparse([newxmlfile.path])) # \
No need to parse </ins><span class="cx"> newxmlfile.setContent("") \
# Change the file </span><del>- \
self.assertTrue(db._shouldReparse(newxmlfile.path)) # Need to parse </del><ins>+ \
self.assertTrue(db._shouldReparse([newxmlfile.path])) # Need to parse </ins><span \
class="cx"> </span><ins>+ def test_refresh(self):
+ """
+ Ensure that a refresh without any file changes doesn't zero out the
+ cache
+ """
+ dbxml = AugmentXMLDB((xmlFile,))
+ keys = dbxml.db.keys()
+ dbxml.refresh()
+ self.assertEquals(keys, dbxml.db.keys())
+
</ins><span class="cx"> class AugmentSqliteTests(AugmentTests, AugmentTestsMixin):
</span><span class="cx">
</span><span class="cx"> def _db(self, dbpath=None):
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavdirectorytesttest_opendirectorypy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_opendirectory.py \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_opendirectory.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/directory/test/test_opendirectory.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -322,6 +322,8 @@
</span><span class="cx"> dsattributes.kDSNAttrRecordType \
: dsattributes.kDSStdRecordTypeGroups, </span><span class="cx"> \
}, </span><span class="cx"> ),
</span><ins>+ dsattributes.kDSStdRecordTypePlaces : (),
+ dsattributes.kDSStdRecordTypeResources : (),
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> def attributeMatches(fieldValue, value, \
caseless, matchType): </span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavresourcepy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/resource.py (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/resource.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/resource.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -1164,7 +1164,7 @@
</span><span class="cx"> if principal is None:
</span><span class="cx"> return (None, None, None)
</span><span class="cx"> else:
</span><del>- return \
(principal.record.fullName.decode("utf-8"), </del><ins>+ \
return (principal.record.fullName, </ins><span class="cx"> \
principal.record.guid, </span><span class="cx"> \
principal.record.calendarUserAddresses) </span><span class="cx">
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavschedulingischedulepy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/scheduling/ischedule.py (7376 \
=> 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/scheduling/ischedule.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/scheduling/ischedule.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -303,7 +303,7 @@
</span><span class="cx"> if principal is None:
</span><span class="cx"> return (None, None, None)
</span><span class="cx"> else:
</span><del>- return \
(principal.record.fullName.decode("utf-8"), </del><ins>+ \
return (principal.record.fullName, </ins><span class="cx"> \
principal.record.guid, </span><span class="cx"> \
principal.record.calendarUserAddresses) </span><span class="cx">
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavserverspy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/servers.py (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/servers.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/servers.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -114,12 +114,20 @@
</span><span class="cx"> self.thisServer = parsed_uri.port in \
(config.SSLPort,) + tuple(config.BindSSLPorts) </span><span class="cx">
</span><span class="cx"> # Need to cache IP addresses
</span><del>- _ignore_host, _ignore_aliases, ips = \
socket.gethostbyname_ex(parsed_uri.hostname) </del><ins>+ try:
+ _ignore_host, _ignore_aliases, ips = \
socket.gethostbyname_ex(parsed_uri.hostname) + except socket.gaierror, e:
+ log.error("Unable to lookup ip-addr for server '%s': %s" % \
(parsed_uri.hostname, str(e))) + ips = ()
</ins><span class="cx"> self.ips = set(ips)
</span><span class="cx">
</span><span class="cx"> for uri in self.partitions.values():
</span><span class="cx"> parsed_uri = urlparse.urlparse(uri)
</span><del>- _ignore_host, _ignore_aliases, ips = \
socket.gethostbyname_ex(parsed_uri.hostname) </del><ins>+ try:
+ _ignore_host, _ignore_aliases, ips = \
socket.gethostbyname_ex(parsed_uri.hostname) + except socket.gaierror, e:
+ log.error("Unable to lookup ip-addr for partition '%s': \
%s" % (parsed_uri.hostname, str(e))) + ips = ()
</ins><span class="cx"> self.partitions_ips.update(ips)
</span><span class="cx">
</span><span class="cx"> def checkThisIP(self, ip):
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavstdconfigpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/stdconfig.py (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/stdconfig.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/stdconfig.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -644,7 +644,7 @@
</span><span class="cx"> },
</span><span class="cx">
</span><span class="cx"> # Umask
</span><del>- "umask": 0027,
</del><ins>+ "umask": 0022,
</ins><span class="cx">
</span><span class="cx"> # A TCP port used for communication between the child \
and master </span><span class="cx"> # processes (bound to 127.0.0.1). Specify 0 \
to let OS assign a port. </span><span class="lines">@@ -1263,3 +1263,34 @@
</span><span class="cx"> config.setProvider(PListConfigProvider(DEFAULT_CONFIG))
</span><span class="cx"> config.addPreUpdateHooks(PRE_UPDATE_HOOKS)
</span><span class="cx"> config.addPostUpdateHooks(POST_UPDATE_HOOKS)
</span><ins>+
+
+def _preserveConfig(configDict):
+ """
+ Preserve certain config keys across reset( ) because these can't be
+ re-fetched after the process has shed privileges
+ """
+ iMIP = configDict.Scheduling.iMIP
+ XMPP = configDict.Notifications.Services.XMPPNotifier
+ preserved = {
+ "iMIPPassword" : iMIP.Password,
+ "MailSendingPassword" : iMIP.Sending.Password,
+ "MailReceivingPassword" : iMIP.Receiving.Password,
+ "XMPPPassword" : XMPP.Password,
+ }
+ return preserved
+
+def _restoreConfig(configDict, preserved):
+ """
+ Restore certain config keys across reset( ) because these can't be
+ re-fetched after the process has shed privileges
+ """
+ iMIP = configDict.Scheduling.iMIP
+ XMPP = configDict.Notifications.Services.XMPPNotifier
+ iMIP.Password = preserved["iMIPPassword"]
+ iMIP.Sending.Password = preserved["MailSendingPassword"]
+ iMIP.Receiving.Password = preserved["MailReceivingPassword"]
+ XMPP.Password = preserved["XMPPPassword"]
+
+
+config.addResetHooks(_preserveConfig, _restoreConfig)
</ins></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavtesttest_configpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_config.py (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_config.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_config.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -43,6 +43,37 @@
</span><span class="cx"> <string>debug</string>
</span><span class="cx"> </dict>
</span><span class="cx">
</span><ins>+ <key>Notifications</key>
+ <dict>
+ <key>Services</key>
+ <dict>
+ <key>XMPPNotifier</key>
+ <dict>
+ <key>Password</key>
+ <string>xmpp</string>
+ </dict>
+ </dict>
+ </dict>
+
+ <key>Scheduling</key>
+ <dict>
+ <key>iMIP</key>
+ <dict>
+ <key>Password</key>
+ <string>imip</string>
+ <key>Sending</key>
+ <dict>
+ <key>Password</key>
+ <string>sending</string>
+ </dict>
+ <key>Receiving</key>
+ <dict>
+ <key>Password</key>
+ <string>receiving</string>
+ </dict>
+ </dict>
+ </dict>
+
</ins><span class="cx"> </dict>
</span><span class="cx"> </plist>
</span><span class="cx"> """
</span><span class="lines">@@ -127,6 +158,28 @@
</span><span class="cx">
</span><span class="cx"> self.assertEquals(config.HTTPPort, 8008)
</span><span class="cx">
</span><ins>+ def testPreserveAcrossReload(self):
+ self.assertEquals(config.Scheduling.iMIP.Password, "")
+ self.assertEquals(config.Scheduling.iMIP.Sending.Password, "")
+ self.assertEquals(config.Scheduling.iMIP.Receiving.Password, "")
+ self.assertEquals(config.Notifications.Services.XMPPNotifier.Password, \
"") +
+ config.load(self.testConfig)
+
+ self.assertEquals(config.Scheduling.iMIP.Password, "imip")
+ self.assertEquals(config.Scheduling.iMIP.Sending.Password, \
"sending") + \
self.assertEquals(config.Scheduling.iMIP.Receiving.Password, "receiving") + \
self.assertEquals(config.Notifications.Services.XMPPNotifier.Password, \
"xmpp") +
+ writePlist({}, self.testConfig)
+
+ config.reload()
+
+ self.assertEquals(config.Scheduling.iMIP.Password, "imip")
+ self.assertEquals(config.Scheduling.iMIP.Sending.Password, \
"sending") + \
self.assertEquals(config.Scheduling.iMIP.Receiving.Password, "receiving") + \
self.assertEquals(config.Notifications.Services.XMPPNotifier.Password, \
"xmpp") +
</ins><span class="cx"> def testSetAttr(self):
</span><span class="cx"> self.assertNotIn("BindAddresses", \
config.__dict__) </span><span class="cx">
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavtesttest_upgradepy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_upgrade.py (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_upgrade.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/test/test_upgrade.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -30,6 +30,7 @@
</span><span class="cx">
</span><span class="cx"> import hashlib
</span><span class="cx"> import os, zlib, cPickle
</span><ins>+from txdav.caldav.datastore.index_file import db_basename
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx">
</span><span class="lines">@@ -290,7 +291,11 @@
</span><span class="cx"> "calendars": {
</span><span class="cx"> "users": {
</span><span class="cx"> "wsanchez": {
</span><del>- "calendar" : {},
</del><ins>+ "calendar" : {
+ db_basename : {
+ "@contents": "",
+ },
+ },
</ins><span class="cx"> "notifications": {
</span><span class="cx"> \
"sample-notification.xml": { </span><span class="cx"> \
"@contents": "<?xml version='1.0'>\n<should-be-ignored \
/>" </span><span class="lines">@@ -307,7 +312,11 @@
</span><span class="cx"> "64" : {
</span><span class="cx"> "23" : {
</span><span class="cx"> \
"6423F94A-6B76-4A3A-815B-D52CFD77935D" : { </span><del>- \
"calendar": {}, </del><ins>+ \
"calendar": { + db_basename : {
+ "@contents": "",
+ },
+ },
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="lines">@@ -342,6 +351,9 @@
</span><span class="cx"> {
</span><span class="cx"> "calendar" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> \
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" : </span><span class="cx"> \
{ </span><span class="cx"> "@contents" : \
event01_before, </span><span class="lines">@@ -357,6 +369,9 @@
</span><span class="cx"> },
</span><span class="cx"> "inbox" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> "@xattrs" :
</span><span class="cx"> {
</span><span class="cx"> # Pickled XML Doc
</span><span class="lines">@@ -371,6 +386,9 @@
</span><span class="cx"> {
</span><span class="cx"> "calendar" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> },
</span><span class="cx"> },
</span><span class="cx"> },
</span><span class="lines">@@ -407,6 +425,9 @@
</span><span class="cx"> {
</span><span class="cx"> "calendar" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> \
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" : </span><span class="cx"> \
{ </span><span class="cx"> \
"@contents" : event01_after, </span><span class="lines">@@ -422,6 +443,9 @@
</span><span class="cx"> },
</span><span class="cx"> "inbox" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> "@xattrs" :
</span><span class="cx"> {
</span><span class="cx"> freeBusyAttr : \
zlib.compress("<?xml version='1.0' \
encoding='UTF-8'?>\n<calendar-free-busy-set \
xmlns='urn:ietf:params:xml:ns:caldav'>\r\n <href \
xmlns='DAV:'>/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/calendar/</href>\r\n</calendar-free-busy-set>"),
</span><span class="lines">@@ -438,6 +462,9 @@
</span><span class="cx"> {
</span><span class="cx"> "calendar" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> },
</span><span class="cx"> },
</span><span class="cx"> },
</span><span class="lines">@@ -645,13 +672,16 @@
</span><span class="cx"> "23" : {
</span><span class="cx"> \
"6423F94A-6B76-4A3A-815B-D52CFD77935D" : { </span><span class="cx"> \
"calendar" : { </span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> },
</span><span class="cx"> "garbage.ics" : {
</span><span class="cx"> "@contents": \
"Oops, not actually an ICS file.", </span><span class="cx"> \
}, </span><span class="cx"> "other-file.txt": {
</span><span class="cx"> "@contents": \
"Also not a calendar collection." </span><del>- }
</del><ins>+ },
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx"> },
</span><span class="lines">@@ -727,6 +757,122 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx"> @inlineCallbacks
</span><ins>+ def test_calendarsUpgradeWithNestedCollections(self):
+ """
+ Unknown files, including .DS_Store files at any point in the hierarchy,
+ as well as non-directory in a user's calendar home, will be ignored and not
+ interrupt an upgrade.
+ """
+
+ self.setUpXMLDirectory()
+
+ beforeUIDContents = {
+ "64" : {
+ "23" : {
+ "6423F94A-6B76-4A3A-815B-D52CFD77935D" : {
+ "calendar" : {
+ db_basename : {
+ "@contents": "",
+ },
+ },
+ "nested1": {
+ "nested2": {},
+ },
+ }
+ }
+ },
+ ".DS_Store" : {
+ "@contents" : "",
+ }
+ }
+
+ afterUIDContents = {
+ "64" : {
+ "23" : {
+ "6423F94A-6B76-4A3A-815B-D52CFD77935D" : {
+ "calendar" : {
+ db_basename : {
+ "@contents": "",
+ },
+ },
+ ".collection.nested1": {
+ "nested2": {},
+ },
+ }
+ }
+ },
+ ".DS_Store" : {
+ "@contents" : "",
+ }
+ }
+
+ before = {
+ ".DS_Store" :
+ {
+ "@contents" : "",
+ },
+ "calendars" :
+ {
+ ".DS_Store" :
+ {
+ "@contents" : "",
+ },
+ "__uids__" :beforeUIDContents,
+ },
+ "principals" :
+ {
+ ".DS_Store" :
+ {
+ "@contents" : "",
+ },
+ OLDPROXYFILE :
+ {
+ "@contents" : "",
+ }
+ }
+ }
+
+ after = {
+ ".DS_Store" :
+ {
+ "@contents" : "",
+ },
+ "tasks" :
+ {
+ "incoming" :
+ {
+ },
+ },
+ ".calendarserver_version" :
+ {
+ "@contents" : "2",
+ },
+ "calendars" :
+ {
+ ".DS_Store" :
+ {
+ "@contents" : "",
+ },
+ "__uids__" : afterUIDContents,
+ },
+ NEWPROXYFILE :
+ {
+ "@contents" : None,
+ },
+ MailGatewayTokensDatabase.dbFilename :
+ {
+ "@contents" : None,
+ },
+ "%s-journal" % (MailGatewayTokensDatabase.dbFilename,) :
+ {
+ "@contents" : None
+ },
+ }
+
+ (yield self.verifyDirectoryComparison(before, after, reverify=True))
+
+
+ @inlineCallbacks
</ins><span class="cx"> def test_calendarsUpgradeWithUIDs(self):
</span><span class="cx"> """
</span><span class="cx"> Verify that calendar homes in the \
/calendars/__uids__/<guid>/ form </span><span class="lines">@@ -744,6 +890,9 @@
</span><span class="cx"> {
</span><span class="cx"> "calendar" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> \
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" : </span><span class="cx"> \
{ </span><span class="cx"> "@contents" : \
event01_before, </span><span class="lines">@@ -751,6 +900,9 @@
</span><span class="cx"> },
</span><span class="cx"> "inbox" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> "@xattrs" :
</span><span class="cx"> {
</span><span class="cx"> # Plain XML
</span><span class="lines">@@ -792,6 +944,9 @@
</span><span class="cx"> {
</span><span class="cx"> "calendar" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> \
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" : </span><span class="cx"> \
{ </span><span class="cx"> \
"@contents" : event01_after, </span><span class="lines">@@ -803,6 +958,9 @@
</span><span class="cx"> },
</span><span class="cx"> "inbox" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> "@xattrs" :
</span><span class="cx"> {
</span><span class="cx"> freeBusyAttr : \
zlib.compress("<?xml version='1.0' \
encoding='UTF-8'?>\n<calendar-free-busy-set \
xmlns='urn:ietf:params:xml:ns:caldav'>\r\n <href \
xmlns='DAV:'>/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/calendar/</href>\r\n</calendar-free-busy-set>"),
</span><span class="lines">@@ -852,6 +1010,9 @@
</span><span class="cx"> {
</span><span class="cx"> "calendar" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> \
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" : </span><span class="cx"> \
{ </span><span class="cx"> \
"@contents" : event01_before, </span><span class="lines">@@ -868,6 +1029,9 \
@@ </span><span class="cx"> },
</span><span class="cx"> "inbox" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> "@xattrs" :
</span><span class="cx"> {
</span><span class="cx"> # Zlib compressed \
XML </span><span class="lines">@@ -908,6 +1072,9 @@
</span><span class="cx"> {
</span><span class="cx"> "calendar" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> \
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" : </span><span class="cx"> \
{ </span><span class="cx"> \
"@contents" : event01_after, </span><span class="lines">@@ -924,6 +1091,9 \
@@ </span><span class="cx"> },
</span><span class="cx"> "inbox" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> "@xattrs" :
</span><span class="cx"> {
</span><span class="cx"> freeBusyAttr : \
zlib.compress("<?xml version='1.0' \
encoding='UTF-8'?>\n<calendar-free-busy-set \
xmlns='urn:ietf:params:xml:ns:caldav'>\r\n <href \
xmlns='DAV:'>/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/calendar/</href>\r\n</calendar-free-busy-set>"),
</span><span class="lines">@@ -972,6 +1142,9 @@
</span><span class="cx"> {
</span><span class="cx"> "calendar" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> \
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" : </span><span class="cx"> \
{ </span><span class="cx"> \
"@contents" : event01_after, </span><span class="lines">@@ -988,6 +1161,9 \
@@ </span><span class="cx"> },
</span><span class="cx"> "inbox" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> "@xattrs" :
</span><span class="cx"> {
</span><span class="cx"> # Zlib compressed \
XML </span><span class="lines">@@ -1028,6 +1204,9 @@
</span><span class="cx"> {
</span><span class="cx"> "calendar" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> \
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" : </span><span class="cx"> \
{ </span><span class="cx"> \
"@contents" : event01_after, </span><span class="lines">@@ -1044,6 +1223,9 \
@@ </span><span class="cx"> },
</span><span class="cx"> "inbox" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> "@xattrs" :
</span><span class="cx"> {
</span><span class="cx"> freeBusyAttr : \
zlib.compress("<?xml version='1.0' \
encoding='UTF-8'?>\r\n<calendar-free-busy-set \
xmlns='urn:ietf:params:xml:ns:caldav'>\r\n <href \
xmlns='DAV:'>/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/calendar/</href>\r\n</calendar-free-busy-set>\r\n"),
</span><span class="lines">@@ -1093,6 +1275,9 @@
</span><span class="cx"> {
</span><span class="cx"> "calendar" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> \
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" : </span><span class="cx"> \
{ </span><span class="cx"> \
"@contents" : event01_before, </span><span class="lines">@@ -1133,6 +1318,9 \
@@ </span><span class="cx"> {
</span><span class="cx"> "calendar" :
</span><span class="cx"> {
</span><ins>+ db_basename : {
+ "@contents": "",
+ },
</ins><span class="cx"> \
"1E238CA1-3C95-4468-B8CD-C8A399F78C72.ics" : </span><span class="cx"> \
{ </span><span class="cx"> \
"@contents" : event01_after, </span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstwistedcaldavupgradepy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/upgrade.py (7376 => \
7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/twistedcaldav/upgrade.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/twistedcaldav/upgrade.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -116,7 +116,7 @@
</span><span class="cx"> if principal is None:
</span><span class="cx"> return (None, None, None)
</span><span class="cx"> else:
</span><del>- return \
(principal.record.fullName.decode("utf-8"), </del><ins>+ \
return (principal.record.fullName, </ins><span class="cx"> \
principal.record.guid, </span><span class="cx"> \
principal.record.calendarUserAddresses) </span><span class="cx">
</span><span class="lines">@@ -588,16 +588,23 @@
</span><span class="cx"> docRoot = config.DocumentRoot
</span><span class="cx"> if os.path.exists(docRoot):
</span><span class="cx"> calRoot = os.path.join(docRoot, \
"calendars") </span><del>- if os.path.exists(calRoot):
</del><ins>+ if os.path.exists(calRoot) and os.path.isdir(calRoot):
</ins><span class="cx"> uidHomes = os.path.join(calRoot, \
"__uids__") </span><del>- for path1 in os.listdir(uidHomes):
- uidLevel1 = os.path.join(uidHomes, path1)
- for path2 in os.listdir(uidLevel1):
- uidLevel2 = os.path.join(uidLevel1, path2)
- for home in os.listdir(uidLevel2):
- calHome = os.path.join(uidLevel2, home)
- if not flattenHome(calHome):
- errorOccurred = True
</del><ins>+ if os.path.isdir(uidHomes):
+ for path1 in os.listdir(uidHomes):
+ uidLevel1 = os.path.join(uidHomes, path1)
+ if not os.path.isdir(uidLevel1):
+ continue
+ for path2 in os.listdir(uidLevel1):
+ uidLevel2 = os.path.join(uidLevel1, path2)
+ if not os.path.isdir(uidLevel2):
+ continue
+ for home in os.listdir(uidLevel2):
+ calHome = os.path.join(uidLevel2, home)
+ if not os.path.isdir(calHome):
+ continue
+ if not flattenHome(calHome):
+ errorOccurred = True
</ins><span class="cx">
</span><span class="cx"> return errorOccurred
</span><span class="cx">
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstxdavbasedatastoredbapiclientpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/txdav/base/datastore/dbapiclient.py (7376 \
=> 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/txdav/base/datastore/dbapiclient.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/txdav/base/datastore/dbapiclient.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -99,7 +99,13 @@
</span><span class="cx"> def execute(self, sql, args=()):
</span><span class="cx"> realArgs = []
</span><span class="cx"> for arg in args:
</span><del>- if isinstance(arg, (str, unicode)) and len(arg) > 1024:
</del><ins>+ if isinstance(arg, str):
+ # We use NCLOB everywhere, so cx_Oracle requires a unicode-type
+ # input. But we mostly pass around utf-8 encoded bytes at the
+ # application layer as they consume less memory, so do the
+ # conversion here.
+ arg = arg.decode('utf-8')
+ if isinstance(arg, unicode) and len(arg) > 1024:
</ins><span class="cx"> # This *may* cause a type mismatch, but none \
of the non-CLOB </span><span class="cx"> # strings that we're passing \
would allow a value this large </span><span class="cx"> # anyway. \
Smaller strings will be automatically converted by </span><span class="lines">@@ \
-107,7 +113,7 @@ </span><span class="cx"> # sure why cx_Oracle itself \
doesn't just do the following hack </span><span class="cx"> # \
automatically and internally for larger values too, but, here </span><span \
class="cx"> # it is: </span><del>- v = \
self.var(cx_Oracle.CLOB, len(arg) + 1) </del><ins>+ v = \
self.var(cx_Oracle.NCLOB, len(arg) + 1) </ins><span class="cx"> \
v.setvalue(0, arg) </span><span class="cx"> else:
</span><span class="cx"> v = arg
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstxdavcaldavdatastoreindex_filepy"></a>
<div class="propset"><h4>Property changes: \
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/index_file.py</h4> \
<pre class="diff"><span> </span></pre></div>
<a id="svnmergeinfo"></a>
<div class="modfile"><h4>Modified: svn:mergeinfo</h4></div>
<span class="cx">/CalendarServer/branches/egg-info-351/txdav/caldav/datastore/index_file.py:4589-4625
</span><span class="cx">/CalendarServer/branches/generic-sqlstore/txdav/caldav/datastore/index_file.py:6167-6191
</span><span class="cx">/CalendarServer/branches/new-store/txdav/caldav/datastore/index_file.py:5594-5934
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile/txdav/caldav/datastore/index_file.py:5911-5935
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile-2/txdav/caldav/datastore/index_file.py:5936-5981
</span><span class="cx">/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/caldav/datastore/index_file.py:6700-7198
</span><span class="cx">/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/caldav/datastore/index_file.py:5693-5702
</span><span class="cx">/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/caldav/datastore/index_file.py:3628-3644
</span><span class="cx">/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/caldav/datastore/index_file.py:5592-5601
</span><span class="cx">/CalendarServer/branches/users/cdaboo/partition-4464/txdav/caldav/datastore/index_file.py:4465-4957
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycalendar/txdav/caldav/datastore/index_file.py:7085-7206
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycard/txdav/caldav/datastore/index_file.py:7227-7237
</span><span class="cx">/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/caldav/datastore/index_file.py:5071-5105
</span><span class="cx">/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/caldav/datastore/index_file.py:5188-5440
</span><span class="cx">/CalendarServer/branches/users/glyph/conn-limit/txdav/caldav/datastore/index_file.py:6574-6577
</span><span class="cx">/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/caldav/datastore/index_file.py:4971-5080
</span><span class="cx">/CalendarServer/branches/users/glyph/dalify/txdav/caldav/datastore/index_file.py:6932-7023
</span><span class="cx">/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/caldav/datastore/index_file.py:6592-6614
</span><span class="cx">/CalendarServer/branches/users/glyph/linux-tests/txdav/caldav/datastore/index_file.py:6893-6900
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/caldav/datastore/index_file.py:6322-6334
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py:6369
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle/txdav/caldav/datastore/index_file.py:7106-7155
</span><span class="cx">/CalendarServer/branches/users/glyph/sendfdport/txdav/caldav/datastore/index_file.py:5388-5424
</span><span class="cx">/CalendarServer/branches/users/glyph/sharedpool/txdav/caldav/datastore/index_file.py:6490-6550
</span><span class="cx">/CalendarServer/branches/users/glyph/sql-store/txdav/caldav/datastore/index_file.py:5929-6073
</span><span class="cx">/CalendarServer/branches/users/glyph/subtransactions/txdav/caldav/datastore/index_file.py:7248-7258
</span><span class="cx">/CalendarServer/branches/users/glyph/use-system-twisted/txdav/caldav/datastore/index_file.py:5084-5149
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources/txdav/caldav/datastore/index_file.py:5032-5051
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources-2/txdav/caldav/datastore/index_file.py:5052-5061
</span><span class="cx">/CalendarServer/branches/users/sagen/purge_old_events/txdav/caldav/datastore/index_file.py:6735-6746
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/caldav/datastore/index_file.py:4040-4067
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/caldav/datastore/index_file.py:4068-4075
</span><span class="cx">/CalendarServer/branches/users/sagen/resources-2/txdav/caldav/datastore/index_file.py:5084-5093
</span><span class="cx">/CalendarServer/branches/users/wsanchez/transations/txdav/caldav/datastore/index_file.py:5515-5593
</span><span class="cx">/CalendarServer/trunk/twistedcaldav/index.py:6322-6394
</span><span class="cx"> + \
/CalendarServer/branches/config-separation/txdav/caldav/datastore/index_file.py:4379-4443
</span><span class="cx">/CalendarServer/branches/egg-info-351/txdav/caldav/datastore/index_file.py:4589-4625
</span><span class="cx">/CalendarServer/branches/generic-sqlstore/txdav/caldav/datastore/index_file.py:6167-6191
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile-2/txdav/caldav/datastore/index_file.py:5936-5981
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile/txdav/caldav/datastore/index_file.py:5911-5935
</span><span class="cx">/CalendarServer/branches/new-store/txdav/caldav/datastore/index_file.py:5594-5934
</span><span class="cx">/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/caldav/datastore/index_file.py:6700-7198
</span><span class="cx">/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/caldav/datastore/index_file.py:5693-5702
</span><span class="cx">/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/caldav/datastore/index_file.py:3628-3644
</span><span class="cx">/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/caldav/datastore/index_file.py:5592-5601
</span><span class="cx">/CalendarServer/branches/users/cdaboo/partition-4464/txdav/caldav/datastore/index_file.py:4465-4957
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycalendar/txdav/caldav/datastore/index_file.py:7085-7206
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycard/txdav/caldav/datastore/index_file.py:7227-7237
</span><span class="cx">/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/caldav/datastore/index_file.py:5071-5105
</span><span class="cx">/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/caldav/datastore/index_file.py:5188-5440
</span><span class="cx">/CalendarServer/branches/users/glyph/conn-limit/txdav/caldav/datastore/index_file.py:6574-6577
</span><span class="cx">/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/caldav/datastore/index_file.py:4971-5080
</span><span class="cx">/CalendarServer/branches/users/glyph/dalify/txdav/caldav/datastore/index_file.py:6932-7023
</span><span class="cx">/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/caldav/datastore/index_file.py:6592-6614
</span><span class="cx">/CalendarServer/branches/users/glyph/linux-tests/txdav/caldav/datastore/index_file.py:6893-6900
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/caldav/datastore/index_file.py:6322-6334
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py:6369
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle-nulls/txdav/caldav/datastore/index_file.py:7340-7351
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle/txdav/caldav/datastore/index_file.py:7106-7155
</span><span class="cx">/CalendarServer/branches/users/glyph/sendfdport/txdav/caldav/datastore/index_file.py:5388-5424
</span><span class="cx">/CalendarServer/branches/users/glyph/sharedpool/txdav/caldav/datastore/index_file.py:6490-6550
</span><span class="cx">/CalendarServer/branches/users/glyph/sql-store/txdav/caldav/datastore/index_file.py:5929-6073
</span><span class="cx">/CalendarServer/branches/users/glyph/subtransactions/txdav/caldav/datastore/index_file.py:7248-7258
</span><span class="cx">/CalendarServer/branches/users/glyph/use-system-twisted/txdav/caldav/datastore/index_file.py:5084-5149
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources-2/txdav/caldav/datastore/index_file.py:5052-5061
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources/txdav/caldav/datastore/index_file.py:5032-5051
</span><span class="cx">/CalendarServer/branches/users/sagen/purge_old_events/txdav/caldav/datastore/index_file.py:6735-6746
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/caldav/datastore/index_file.py:4040-4067
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/caldav/datastore/index_file.py:4068-4075
</span><span class="cx">/CalendarServer/branches/users/sagen/resources-2/txdav/caldav/datastore/index_file.py:5084-5093
</span><span class="cx">/CalendarServer/branches/users/wsanchez/transations/txdav/caldav/datastore/index_file.py:5515-5593
</span><span class="cx">/CalendarServer/trunk/twistedcaldav/index.py:6322-6394
</span><span class="cx">/CalendarServer/trunk/txdav/caldav/datastore/index_file.py:7297-7364
</span><a id="CalendarServerbranchesuserscdaboopodstxdavcaldavdatastoretestcalendar_storehomehome1calendar_11ics"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/calendar_store/ho/me/home1/calendar_1/1.ics \
(7376 => 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/calendar_store/ho/me/home1/calendar_1/1.ics 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/calendar_store/ho/me/home1/calendar_1/1.ics 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -21,12 +21,12 @@
</span><span class="cx"> END:VTIMEZONE
</span><span class="cx"> BEGIN:VEVENT
</span><span class="cx"> ATTENDEE;CN="Wilfredo \
Sanchez";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailt </span><del>- \
o:wsanchez@apple.com </del><ins>+ o:wsanchez@example.com
</ins><span class="cx"> ATTENDEE;CN="Cyrus \
Daboo";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cda </span><del>- \
boo@apple.com </del><ins>+ boo@example.com
</ins><span class="cx"> DTEND;TZID=US/Pacific:20090324T124500
</span><span class="cx"> TRANSP:OPAQUE
</span><del>-ORGANIZER;CN="Wilfredo Sanchez":mailto:wsanchez@apple.com
</del><ins>+ORGANIZER;CN="Wilfredo Sanchez":mailto:wsanchez@example.com
</ins><span class="cx"> UID:uid1
</span><span class="cx"> DTSTAMP:20090326T145447Z
</span><span class="cx"> LOCATION:Wilfredo's Office
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstxdavcaldavdatastoretestcommonpy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/common.py (7376 \
=> 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/common.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/common.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -944,7 +944,7 @@
</span><span class="cx"> """
</span><span class="cx"> self.assertEquals(
</span><span class="cx"> (yield \
self.calendarObjectUnderTest()).organizer(), </span><del>- \
"mailto:wsanchez@apple.com" </del><ins>+ \
"mailto:wsanchez@example.com" </ins><span class="cx"> )
</span><span class="cx">
</span><span class="cx">
</span></span></pre></div>
<a id="CalendarServerbranchesuserscdaboopodstxdavcaldavdatastoretesttest_index_filepy"></a>
<div class="propset"><h4>Property changes: \
CalendarServer/branches/users/cdaboo/pods/txdav/caldav/datastore/test/test_index_file.py</h4>
<pre class="diff"><span>
</span></pre></div>
<a id="svnmergeinfo"></a>
<div class="modfile"><h4>Modified: svn:mergeinfo</h4></div>
<span class="cx">/CalendarServer/branches/egg-info-351/txdav/caldav/datastore/test/test_index_file.py:4589-4625
</span><span class="cx">/CalendarServer/branches/generic-sqlstore/txdav/caldav/datastore/test/test_index_file.py:6167-6191
</span><span class="cx">/CalendarServer/branches/new-store/txdav/caldav/datastore/test/test_index_file.py:5594-5934
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile/txdav/caldav/datastore/test/test_index_file.py:5911-5935
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile-2/txdav/caldav/datastore/test/test_index_file.py:5936-5981
</span><span class="cx">/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/caldav/datastore/test/test_index_file.py:6700-7198
</span><span class="cx">/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/caldav/datastore/test/test_index_file.py:5693-5702
</span><span class="cx">/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/caldav/datastore/test/test_index_file.py:3628-3644
</span><span class="cx">/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/caldav/datastore/test/test_index_file.py:5592-5601
</span><span class="cx">/CalendarServer/branches/users/cdaboo/partition-4464/txdav/caldav/datastore/test/test_index_file.py:4465-4957
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycalendar/txdav/caldav/datastore/test/test_index_file.py:7085-7206
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycard/txdav/caldav/datastore/test/test_index_file.py:7227-7237
</span><span class="cx">/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/caldav/datastore/test/test_index_file.py:5071-5105
</span><span class="cx">/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/caldav/datastore/test/test_index_file.py:5188-5440
</span><span class="cx">/CalendarServer/branches/users/glyph/conn-limit/txdav/caldav/datastore/test/test_index_file.py:6574-6577
</span><span class="cx">/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/caldav/datastore/test/test_index_file.py:4971-5080
</span><span class="cx">/CalendarServer/branches/users/glyph/dalify/txdav/caldav/datastore/test/test_index_file.py:6932-7023
</span><span class="cx">/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/caldav/datastore/test/test_index_file.py:6592-6614
</span><span class="cx">/CalendarServer/branches/users/glyph/linux-tests/txdav/caldav/datastore/test/test_index_file.py:6893-6900
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/caldav/datastore/test/test_index_file.py:6322-6334
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py:6369
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle/txdav/caldav/datastore/test/test_index_file.py:7106-7155
</span><span class="cx">/CalendarServer/branches/users/glyph/sendfdport/txdav/caldav/datastore/test/test_index_file.py:5388-5424
</span><span class="cx">/CalendarServer/branches/users/glyph/sharedpool/txdav/caldav/datastore/test/test_index_file.py:6490-6550
</span><span class="cx">/CalendarServer/branches/users/glyph/sql-store/txdav/caldav/datastore/test/test_index_file.py:5929-6073
</span><span class="cx">/CalendarServer/branches/users/glyph/subtransactions/txdav/caldav/datastore/test/test_index_file.py:7248-7258
</span><span class="cx">/CalendarServer/branches/users/glyph/use-system-twisted/txdav/caldav/datastore/test/test_index_file.py:5084-5149
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources/txdav/caldav/datastore/test/test_index_file.py:5032-5051
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources-2/txdav/caldav/datastore/test/test_index_file.py:5052-5061
</span><span class="cx">/CalendarServer/branches/users/sagen/purge_old_events/txdav/caldav/datastore/test/test_index_file.py:6735-6746
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/caldav/datastore/test/test_index_file.py:4040-4067
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/caldav/datastore/test/test_index_file.py:4068-4075
</span><span class="cx">/CalendarServer/branches/users/sagen/resources-2/txdav/caldav/datastore/test/test_index_file.py:5084-5093
</span><span class="cx">/CalendarServer/branches/users/wsanchez/transations/txdav/caldav/datastore/test/test_index_file.py:5515-5593
</span><span class="cx">/CalendarServer/trunk/twistedcaldav/test/test_index.py:6322-6394
</span><span class="cx"> + \
/CalendarServer/branches/config-separation/txdav/caldav/datastore/test/test_index_file.py:4379-4443
</span><span class="cx">/CalendarServer/branches/egg-info-351/txdav/caldav/datastore/test/test_index_file.py:4589-4625
</span><span class="cx">/CalendarServer/branches/generic-sqlstore/txdav/caldav/datastore/test/test_index_file.py:6167-6191
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile-2/txdav/caldav/datastore/test/test_index_file.py:5936-5981
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile/txdav/caldav/datastore/test/test_index_file.py:5911-5935
</span><span class="cx">/CalendarServer/branches/new-store/txdav/caldav/datastore/test/test_index_file.py:5594-5934
</span><span class="cx">/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/caldav/datastore/test/test_index_file.py:6700-7198
</span><span class="cx">/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/caldav/datastore/test/test_index_file.py:5693-5702
</span><span class="cx">/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/caldav/datastore/test/test_index_file.py:3628-3644
</span><span class="cx">/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/caldav/datastore/test/test_index_file.py:5592-5601
</span><span class="cx">/CalendarServer/branches/users/cdaboo/partition-4464/txdav/caldav/datastore/test/test_index_file.py:4465-4957
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycalendar/txdav/caldav/datastore/test/test_index_file.py:7085-7206
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycard/txdav/caldav/datastore/test/test_index_file.py:7227-7237
</span><span class="cx">/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/caldav/datastore/test/test_index_file.py:5071-5105
</span><span class="cx">/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/caldav/datastore/test/test_index_file.py:5188-5440
</span><span class="cx">/CalendarServer/branches/users/glyph/conn-limit/txdav/caldav/datastore/test/test_index_file.py:6574-6577
</span><span class="cx">/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/caldav/datastore/test/test_index_file.py:4971-5080
</span><span class="cx">/CalendarServer/branches/users/glyph/dalify/txdav/caldav/datastore/test/test_index_file.py:6932-7023
</span><span class="cx">/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/caldav/datastore/test/test_index_file.py:6592-6614
</span><span class="cx">/CalendarServer/branches/users/glyph/linux-tests/txdav/caldav/datastore/test/test_index_file.py:6893-6900
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/caldav/datastore/test/test_index_file.py:6322-6334
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py:6369
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle-nulls/txdav/caldav/datastore/test/test_index_file.py:7340-7351
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle/txdav/caldav/datastore/test/test_index_file.py:7106-7155
</span><span class="cx">/CalendarServer/branches/users/glyph/sendfdport/txdav/caldav/datastore/test/test_index_file.py:5388-5424
</span><span class="cx">/CalendarServer/branches/users/glyph/sharedpool/txdav/caldav/datastore/test/test_index_file.py:6490-6550
</span><span class="cx">/CalendarServer/branches/users/glyph/sql-store/txdav/caldav/datastore/test/test_index_file.py:5929-6073
</span><span class="cx">/CalendarServer/branches/users/glyph/subtransactions/txdav/caldav/datastore/test/test_index_file.py:7248-7258
</span><span class="cx">/CalendarServer/branches/users/glyph/use-system-twisted/txdav/caldav/datastore/test/test_index_file.py:5084-5149
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources-2/txdav/caldav/datastore/test/test_index_file.py:5052-5061
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources/txdav/caldav/datastore/test/test_index_file.py:5032-5051
</span><span class="cx">/CalendarServer/branches/users/sagen/purge_old_events/txdav/caldav/datastore/test/test_index_file.py:6735-6746
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/caldav/datastore/test/test_index_file.py:4040-4067
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/caldav/datastore/test/test_index_file.py:4068-4075
</span><span class="cx">/CalendarServer/branches/users/sagen/resources-2/txdav/caldav/datastore/test/test_index_file.py:5084-5093
</span><span class="cx">/CalendarServer/branches/users/wsanchez/transations/txdav/caldav/datastore/test/test_index_file.py:5515-5593
</span><span class="cx">/CalendarServer/trunk/twistedcaldav/test/test_index.py:6322-6394
</span><span class="cx">/CalendarServer/trunk/txdav/caldav/datastore/test/test_index_file.py:7297-7364
</span><a id="CalendarServerbranchesuserscdaboopodstxdavcarddavdatastoreindex_filepy"></a>
<div class="propset"><h4>Property changes: \
CalendarServer/branches/users/cdaboo/pods/txdav/carddav/datastore/index_file.py</h4> \
<pre class="diff"><span> </span></pre></div>
<a id="svnmergeinfo"></a>
<div class="modfile"><h4>Modified: svn:mergeinfo</h4></div>
<span class="cx">/CalendarServer/branches/egg-info-351/txdav/carddav/datastore/index_file.py:4589-4625
</span><span class="cx">/CalendarServer/branches/generic-sqlstore/txdav/carddav/datastore/index_file.py:6167-6191
</span><span class="cx">/CalendarServer/branches/new-store/txdav/carddav/datastore/index_file.py:5594-5934
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile/txdav/carddav/datastore/index_file.py:5911-5935
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile-2/txdav/carddav/datastore/index_file.py:5936-5981
</span><span class="cx">/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/carddav/datastore/index_file.py:6700-7198
</span><span class="cx">/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/carddav/datastore/index_file.py:5693-5702
</span><span class="cx">/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/carddav/datastore/index_file.py:3628-3644
</span><span class="cx">/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/carddav/datastore/index_file.py:5592-5601
</span><span class="cx">/CalendarServer/branches/users/cdaboo/partition-4464/txdav/carddav/datastore/index_file.py:4465-4957
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycalendar/txdav/carddav/datastore/index_file.py:7085-7206
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycard/txdav/carddav/datastore/index_file.py:7227-7237
</span><span class="cx">/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/carddav/datastore/index_file.py:5071-5105
</span><span class="cx">/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/carddav/datastore/index_file.py:5188-5440
</span><span class="cx">/CalendarServer/branches/users/glyph/conn-limit/txdav/carddav/datastore/index_file.py:6574-6577
</span><span class="cx">/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/carddav/datastore/index_file.py:4971-5080
</span><span class="cx">/CalendarServer/branches/users/glyph/dalify/txdav/carddav/datastore/index_file.py:6932-7023
</span><span class="cx">/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/carddav/datastore/index_file.py:6592-6614
</span><span class="cx">/CalendarServer/branches/users/glyph/linux-tests/txdav/carddav/datastore/index_file.py:6893-6900
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/carddav/datastore/index_file.py:6322-6334
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py:6369
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle/txdav/carddav/datastore/index_file.py:7106-7155
</span><span class="cx">/CalendarServer/branches/users/glyph/sendfdport/txdav/carddav/datastore/index_file.py:5388-5424
</span><span class="cx">/CalendarServer/branches/users/glyph/sharedpool/txdav/carddav/datastore/index_file.py:6490-6550
</span><span class="cx">/CalendarServer/branches/users/glyph/sql-store/txdav/carddav/datastore/index_file.py:5929-6073
</span><span class="cx">/CalendarServer/branches/users/glyph/subtransactions/txdav/carddav/datastore/index_file.py:7248-7258
</span><span class="cx">/CalendarServer/branches/users/glyph/use-system-twisted/txdav/carddav/datastore/index_file.py:5084-5149
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources/txdav/carddav/datastore/index_file.py:5032-5051
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources-2/txdav/carddav/datastore/index_file.py:5052-5061
</span><span class="cx">/CalendarServer/branches/users/sagen/purge_old_events/txdav/carddav/datastore/index_file.py:6735-6746
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/carddav/datastore/index_file.py:4040-4067
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/carddav/datastore/index_file.py:4068-4075
</span><span class="cx">/CalendarServer/branches/users/sagen/resources-2/txdav/carddav/datastore/index_file.py:5084-5093
</span><span class="cx">/CalendarServer/branches/users/wsanchez/transations/txdav/carddav/datastore/index_file.py:5515-5593
</span><span class="cx">/CalendarServer/trunk/twistedcaldav/vcardindex.py:6322-6394
</span><span class="cx"> + \
/CalendarServer/branches/config-separation/txdav/carddav/datastore/index_file.py:4379-4443
</span><span class="cx">/CalendarServer/branches/egg-info-351/txdav/carddav/datastore/index_file.py:4589-4625
</span><span class="cx">/CalendarServer/branches/generic-sqlstore/txdav/carddav/datastore/index_file.py:6167-6191
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile-2/txdav/carddav/datastore/index_file.py:5936-5981
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile/txdav/carddav/datastore/index_file.py:5911-5935
</span><span class="cx">/CalendarServer/branches/new-store/txdav/carddav/datastore/index_file.py:5594-5934
</span><span class="cx">/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/carddav/datastore/index_file.py:6700-7198
</span><span class="cx">/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/carddav/datastore/index_file.py:5693-5702
</span><span class="cx">/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/carddav/datastore/index_file.py:3628-3644
</span><span class="cx">/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/carddav/datastore/index_file.py:5592-5601
</span><span class="cx">/CalendarServer/branches/users/cdaboo/partition-4464/txdav/carddav/datastore/index_file.py:4465-4957
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycalendar/txdav/carddav/datastore/index_file.py:7085-7206
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycard/txdav/carddav/datastore/index_file.py:7227-7237
</span><span class="cx">/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/carddav/datastore/index_file.py:5071-5105
</span><span class="cx">/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/carddav/datastore/index_file.py:5188-5440
</span><span class="cx">/CalendarServer/branches/users/glyph/conn-limit/txdav/carddav/datastore/index_file.py:6574-6577
</span><span class="cx">/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/carddav/datastore/index_file.py:4971-5080
</span><span class="cx">/CalendarServer/branches/users/glyph/dalify/txdav/carddav/datastore/index_file.py:6932-7023
</span><span class="cx">/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/carddav/datastore/index_file.py:6592-6614
</span><span class="cx">/CalendarServer/branches/users/glyph/linux-tests/txdav/carddav/datastore/index_file.py:6893-6900
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/carddav/datastore/index_file.py:6322-6334
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py:6369
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle-nulls/txdav/carddav/datastore/index_file.py:7340-7351
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle/txdav/carddav/datastore/index_file.py:7106-7155
</span><span class="cx">/CalendarServer/branches/users/glyph/sendfdport/txdav/carddav/datastore/index_file.py:5388-5424
</span><span class="cx">/CalendarServer/branches/users/glyph/sharedpool/txdav/carddav/datastore/index_file.py:6490-6550
</span><span class="cx">/CalendarServer/branches/users/glyph/sql-store/txdav/carddav/datastore/index_file.py:5929-6073
</span><span class="cx">/CalendarServer/branches/users/glyph/subtransactions/txdav/carddav/datastore/index_file.py:7248-7258
</span><span class="cx">/CalendarServer/branches/users/glyph/use-system-twisted/txdav/carddav/datastore/index_file.py:5084-5149
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources-2/txdav/carddav/datastore/index_file.py:5052-5061
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources/txdav/carddav/datastore/index_file.py:5032-5051
</span><span class="cx">/CalendarServer/branches/users/sagen/purge_old_events/txdav/carddav/datastore/index_file.py:6735-6746
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/carddav/datastore/index_file.py:4040-4067
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/carddav/datastore/index_file.py:4068-4075
</span><span class="cx">/CalendarServer/branches/users/sagen/resources-2/txdav/carddav/datastore/index_file.py:5084-5093
</span><span class="cx">/CalendarServer/branches/users/wsanchez/transations/txdav/carddav/datastore/index_file.py:5515-5593
</span><span class="cx">/CalendarServer/trunk/twistedcaldav/vcardindex.py:6322-6394
</span><span class="cx">/CalendarServer/trunk/txdav/carddav/datastore/index_file.py:7297-7364
</span><a id="CalendarServerbranchesuserscdaboopodstxdavcarddavdatastoretesttest_index_filepy"></a>
<div class="propset"><h4>Property changes: \
CalendarServer/branches/users/cdaboo/pods/txdav/carddav/datastore/test/test_index_file.py</h4>
<pre class="diff"><span>
</span></pre></div>
<a id="svnmergeinfo"></a>
<div class="modfile"><h4>Modified: svn:mergeinfo</h4></div>
<span class="cx">/CalendarServer/branches/egg-info-351/txdav/carddav/datastore/test/test_index_file.py:4589-4625
</span><span class="cx">/CalendarServer/branches/generic-sqlstore/txdav/carddav/datastore/test/test_index_file.py:6167-6191
</span><span class="cx">/CalendarServer/branches/new-store/txdav/carddav/datastore/test/test_index_file.py:5594-5934
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile/txdav/carddav/datastore/test/test_index_file.py:5911-5935
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile-2/txdav/carddav/datastore/test/test_index_file.py:5936-5981
</span><span class="cx">/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/carddav/datastore/test/test_index_file.py:6700-7198
</span><span class="cx">/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/carddav/datastore/test/test_index_file.py:5693-5702
</span><span class="cx">/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/carddav/datastore/test/test_index_file.py:3628-3644
</span><span class="cx">/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/carddav/datastore/test/test_index_file.py:5592-5601
</span><span class="cx">/CalendarServer/branches/users/cdaboo/partition-4464/txdav/carddav/datastore/test/test_index_file.py:4465-4957
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycalendar/txdav/carddav/datastore/test/test_index_file.py:7085-7206
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycard/txdav/carddav/datastore/test/test_index_file.py:7227-7237
</span><span class="cx">/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/carddav/datastore/test/test_index_file.py:5071-5105
</span><span class="cx">/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/carddav/datastore/test/test_index_file.py:5188-5440
</span><span class="cx">/CalendarServer/branches/users/glyph/conn-limit/txdav/carddav/datastore/test/test_index_file.py:6574-6577
</span><span class="cx">/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/carddav/datastore/test/test_index_file.py:4971-5080
</span><span class="cx">/CalendarServer/branches/users/glyph/dalify/txdav/carddav/datastore/test/test_index_file.py:6932-7023
</span><span class="cx">/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/carddav/datastore/test/test_index_file.py:6592-6614
</span><span class="cx">/CalendarServer/branches/users/glyph/linux-tests/txdav/carddav/datastore/test/test_index_file.py:6893-6900
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/carddav/datastore/test/test_index_file.py:6322-6334
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py:6369
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle/txdav/carddav/datastore/test/test_index_file.py:7106-7155
</span><span class="cx">/CalendarServer/branches/users/glyph/sendfdport/txdav/carddav/datastore/test/test_index_file.py:5388-5424
</span><span class="cx">/CalendarServer/branches/users/glyph/sharedpool/txdav/carddav/datastore/test/test_index_file.py:6490-6550
</span><span class="cx">/CalendarServer/branches/users/glyph/sql-store/txdav/carddav/datastore/test/test_index_file.py:5929-6073
</span><span class="cx">/CalendarServer/branches/users/glyph/subtransactions/txdav/carddav/datastore/test/test_index_file.py:7248-7258
</span><span class="cx">/CalendarServer/branches/users/glyph/use-system-twisted/txdav/carddav/datastore/test/test_index_file.py:5084-5149
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources/txdav/carddav/datastore/test/test_index_file.py:5032-5051
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources-2/txdav/carddav/datastore/test/test_index_file.py:5052-5061
</span><span class="cx">/CalendarServer/branches/users/sagen/purge_old_events/txdav/carddav/datastore/test/test_index_file.py:6735-6746
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/carddav/datastore/test/test_index_file.py:4040-4067
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/carddav/datastore/test/test_index_file.py:4068-4075
</span><span class="cx">/CalendarServer/branches/users/sagen/resources-2/txdav/carddav/datastore/test/test_index_file.py:5084-5093
</span><span class="cx">/CalendarServer/branches/users/wsanchez/transations/txdav/carddav/datastore/test/test_index_file.py:5515-5593
</span><span class="cx">/CalendarServer/trunk/twistedcaldav/test/test_vcardindex.py:6322-6394
</span><span class="cx"> + \
/CalendarServer/branches/config-separation/txdav/carddav/datastore/test/test_index_file.py:4379-4443
</span><span class="cx">/CalendarServer/branches/egg-info-351/txdav/carddav/datastore/test/test_index_file.py:4589-4625
</span><span class="cx">/CalendarServer/branches/generic-sqlstore/txdav/carddav/datastore/test/test_index_file.py:6167-6191
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile-2/txdav/carddav/datastore/test/test_index_file.py:5936-5981
</span><span class="cx">/CalendarServer/branches/new-store-no-caldavfile/txdav/carddav/datastore/test/test_index_file.py:5911-5935
</span><span class="cx">/CalendarServer/branches/new-store/txdav/carddav/datastore/test/test_index_file.py:5594-5934
</span><span class="cx">/CalendarServer/branches/users/cdaboo/batchupload-6699/txdav/carddav/datastore/test/test_index_file.py:6700-7198
</span><span class="cx">/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/carddav/datastore/test/test_index_file.py:5693-5702
</span><span class="cx">/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/carddav/datastore/test/test_index_file.py:3628-3644
</span><span class="cx">/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/carddav/datastore/test/test_index_file.py:5592-5601
</span><span class="cx">/CalendarServer/branches/users/cdaboo/partition-4464/txdav/carddav/datastore/test/test_index_file.py:4465-4957
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycalendar/txdav/carddav/datastore/test/test_index_file.py:7085-7206
</span><span class="cx">/CalendarServer/branches/users/cdaboo/pycard/txdav/carddav/datastore/test/test_index_file.py:7227-7237
</span><span class="cx">/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/carddav/datastore/test/test_index_file.py:5071-5105
</span><span class="cx">/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/carddav/datastore/test/test_index_file.py:5188-5440
</span><span class="cx">/CalendarServer/branches/users/glyph/conn-limit/txdav/carddav/datastore/test/test_index_file.py:6574-6577
</span><span class="cx">/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/carddav/datastore/test/test_index_file.py:4971-5080
</span><span class="cx">/CalendarServer/branches/users/glyph/dalify/txdav/carddav/datastore/test/test_index_file.py:6932-7023
</span><span class="cx">/CalendarServer/branches/users/glyph/dont-start-postgres/txdav/carddav/datastore/test/test_index_file.py:6592-6614
</span><span class="cx">/CalendarServer/branches/users/glyph/linux-tests/txdav/carddav/datastore/test/test_index_file.py:6893-6900
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/carddav/datastore/test/test_index_file.py:6322-6334
</span><span class="cx">/CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py:6369
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle-nulls/txdav/carddav/datastore/test/test_index_file.py:7340-7351
</span><span class="cx">/CalendarServer/branches/users/glyph/oracle/txdav/carddav/datastore/test/test_index_file.py:7106-7155
</span><span class="cx">/CalendarServer/branches/users/glyph/sendfdport/txdav/carddav/datastore/test/test_index_file.py:5388-5424
</span><span class="cx">/CalendarServer/branches/users/glyph/sharedpool/txdav/carddav/datastore/test/test_index_file.py:6490-6550
</span><span class="cx">/CalendarServer/branches/users/glyph/sql-store/txdav/carddav/datastore/test/test_index_file.py:5929-6073
</span><span class="cx">/CalendarServer/branches/users/glyph/subtransactions/txdav/carddav/datastore/test/test_index_file.py:7248-7258
</span><span class="cx">/CalendarServer/branches/users/glyph/use-system-twisted/txdav/carddav/datastore/test/test_index_file.py:5084-5149
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources-2/txdav/carddav/datastore/test/test_index_file.py:5052-5061
</span><span class="cx">/CalendarServer/branches/users/sagen/locations-resources/txdav/carddav/datastore/test/test_index_file.py:5032-5051
</span><span class="cx">/CalendarServer/branches/users/sagen/purge_old_events/txdav/carddav/datastore/test/test_index_file.py:6735-6746
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/carddav/datastore/test/test_index_file.py:4040-4067
</span><span class="cx">/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/carddav/datastore/test/test_index_file.py:4068-4075
</span><span class="cx">/CalendarServer/branches/users/sagen/resources-2/txdav/carddav/datastore/test/test_index_file.py:5084-5093
</span><span class="cx">/CalendarServer/branches/users/wsanchez/transations/txdav/carddav/datastore/test/test_index_file.py:5515-5593
</span><span class="cx">/CalendarServer/trunk/twistedcaldav/test/test_vcardindex.py:6322-6394
</span><span class="cx">/CalendarServer/trunk/txdav/carddav/datastore/test/test_index_file.py:7297-7364
</span><a id="CalendarServerbranchesuserscdaboopodstxdavcommondatastoresql_tablespy"></a>
<div class="modfile"><h4>Modified: \
CalendarServer/branches/users/cdaboo/pods/txdav/common/datastore/sql_tables.py (7376 \
=> 7377)</h4> <pre class="diff"><span>
<span class="info">--- \
CalendarServer/branches/users/cdaboo/pods/txdav/common/datastore/sql_tables.py 2011-04-27 \
20:10:49 UTC (rev 7376)
+++ CalendarServer/branches/users/cdaboo/pods/txdav/common/datastore/sql_tables.py 2011-04-27 \
21:09:24 UTC (rev 7377) </span><span class="lines">@@ -206,14 +206,13 @@
</span><span class="cx"> emit in oracle format.
</span><span class="cx"> """
</span><span class="cx"> for sequence in schema.model.sequences:
</span><del>- out.write('drop sequence %s; create sequence %s;\n' % (
- sequence.name, sequence.name))
</del><ins>+ out.write('create sequence %s;\n' % (sequence.name,))
</ins><span class="cx"> for table in schema:
</span><span class="cx"> # The only table name which actually exceeds the \
length limit right now </span><span class="cx"> # is \
CALENDAR_OBJECT_ATTACHMENTS_MODE, which isn't actually _used_ </span><span \
class="cx"> # anywhere, so we can fake it for now. </span><del>- \
out.write('drop table %s; create table %s (\n' % (
- table.model.name[:30], table.model.name[:30],))
</del><ins>+ shortName = table.model.name[:30]
+ out.write('create table %s (\n' % (shortName,))
</ins><span class="cx"> first = True
</span><span class="cx"> for column in table:
</span><span class="cx"> if first:
</span><span class="lines">@@ -222,7 +221,7 @@
</span><span class="cx"> out.write(",\n")
</span><span class="cx"> typeName = column.model.type.name
</span><span class="cx"> if typeName == 'text':
</span><del>- typeName = 'clob'
</del><ins>+ typeName = 'nclob'
</ins><span class="cx"> if typeName == 'boolean':
</span><span class="cx"> typeName = 'integer'
</span><span class="cx"> out.write(' "%s" %s' % \
(column.model.name, typeName)) </span><span class="lines">@@ -247,7 +246,11 @@
</span><span class="cx"> elif default is False:
</span><span class="cx"> default = 0
</span><span class="cx"> out.write(" " + \
repr(default)) </span><del>- if not column.model.canBeNull():
</del><ins>+ if ( (not column.model.canBeNull())
+ # Oracle treats empty strings as NULLs, so we have to accept
+ # NULL values in columns of a string type. Other types should
+ # be okay though.
+ and typeName not in ('text', 'varchar') ):
</ins><span class="cx"> out.write(' not null')
</span><span class="cx"> if set([column.model]) in \
list(table.model.uniques()): </span><span class="cx"> out.write(' \
unique') </span></span></pre>
</div>
</div>
</body>
</html>
_______________________________________________
calendarserver-changes mailing list
calendarserver-changes@lists.macosforge.org
http://lists.macosforge.org/mailman/listinfo.cgi/calendarserver-changes
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic