[prev in list] [next in list] [prev in thread] [next in thread] 

List:       mtos-dev
Subject:    Re: [MTOS-dev] New "Meta" Data System
From:       Mark Paschal <mark () sixapart ! com>
Date:       2008-04-24 17:35:11
Message-ID: 4810C4CF.7080001 () sixapart ! com
[Download RAW message or body]

Mark Carey wrote:
> Can someone provide a high level overview of the new approach to
> "meta" columns, which just landed in beta 4?

Prior releases use serialized blob storage for metadata, where 
everything is stored as an opaque blob representing the data structure 
containing all the object's metadata in the meta column of the 
affected object (such as entry_meta in the mt_entry table).

As of beta 4, we're using a narrow table technique where each 
additional metadata "column" is actually a *row* in a *separate* table 
(the meta table, such as mt_entry_meta for records in mt_entry).

This is the same technique used in Six Apart's other Data:: 
ObjectDriver based products. I reused as much of that code as possible 
to ensure the same correctness and reliability, but we did have to 
adapt it to integrate with the existing API and the upgrading system.


> What do plugin developers need to know, and need to change, for
> plugins that define "meta" columns?

We retained the existing API for defining meta columns (the 
MT::Object::install_meta method), so your plugins should continue to 
work with no modification necessary.

But Beta 4 is your opportunity to make sure this is true! Please 
please test it!

The new method of defining metadata is to define a column in the same 
"column_defs" property as your object's real columns, but with the 
additional "meta" keyword. This allows the use of the typed columns 
defined in the meta tables. As one doesn't define a type when using 
install_meta(), all metadata fields defined with install_meta() are 
stored as the "blob" type. That provides (a) at least as much space 
for string values and (b) the continued ability to automatically store 
Perl data structures, so anything your plugin stored before will still 
be stored.

The upgrade step converts all objects it sees in the object_types 
section of the registry that define metadata fields, so when your 
plugin is installed during the upgrade to beta 4 (or 4.15) and you've 
defined an object_type entry for your objects that have metadata, your 
objects' metadata will automatically be converted to the narrow table 
implementation. As the upgrader is implemented as a core upgrade step, 
you should also be able to reuse it in a plugin-defined upgrade step 
if your plugin has special needs.

Allow me to note that it was this metadata upgrade step that was 
broken in the first release of beta 4, and to say again: PLEASE test 
beta 4 and your metadata-using plugins!


> And does this mean 4.15 will support search of CF data, or does this
> simply lay the foundation for that in a future release?

This is mainly foundation-laying. IIRC we intend to support ordering 
entries by custom field values in the Entries template tag in Cal, but 
not much further functionality. The other turn-key advantages are no 
more single storage limit across *all* a record's metadata, and 
customers can browse metadata more easily with database inspection 
tools, even if we don't provide searching by it in the app search.

HTH, and let me know if you have any other questions.


Mark Paschal
Software developer, Movable Type
mark@sixapart.com

_______________________________________________
MTOS-dev mailing list
MTOS-dev@sixapart.com
http://www.sixapart.com/mailman/listinfo/mtos-dev
[prev in list] [next in list] [prev in thread] [next in thread]