[prev in list] [next in list] [prev in thread] [next in thread] 

List:       mapguide-internals
Subject:    RE: [mapguide-internals] RE: RFC110 Profiling API enhancement
From:       Traian Stanev <traian.stanev () autodesk ! com>
Date:       2011-04-28 18:50:17
Message-ID: 0D757A4B577F7E42B5166013D347B6DA33ED8C89 () 005-TK5MPN1-003 ! MGDADSK ! autodesk ! com
[Download RAW message or body]

[Attachment #2 (unknown)]


Hi Arthur,

> 1. As Traian mentioned, add a compile time flag to turn profile function on and \
> off?  No performance lost at runtime, but not all end users could use it unless he \
> get a profile enabled build.

The reason I ask for this is because the stylization and rendering code are used by \
Map3D for example, where the code is much, much more performance sensitive than \
MapGuide because it has to be called on every repaint.

Thanks,
Traian


-----Original Message-----
From: mapguide-internals-bounces@lists.osgeo.org \
                [mailto:mapguide-internals-bounces@lists.osgeo.org] On Behalf Of \
                Arthur Liu
Sent: Thursday, April 28, 2011 3:41 AM
To: MapGuide Internals Mail List
Subject: RE: [mapguide-internals] RE: RFC110 Profiling API enhancement

How about other two alternatives?

1. As Traian mentioned, add a compile time flag to turn profile function on and off? \
No performance lost at runtime, but not all end users could use it unless he get a \
profile enabled build.

2. Mirror a copy of current implementation in MgServerProfilingService. 

The implementation may like:

class MgProfiledRenderingService : public MgRenderingService{
   public:
 
       //override the implementation of RenderMap
       MgByteReader* RenderMap(...);
}

class MgProfilingService{
   public：
        MgByteReader* ProfileRenderMap(...);
        ......
   private:
        MgProfiledRenderingService* m_profiledRenderingSvc;
        ......
}

Some private members in MgRenderingService may also needed in \
MgProfiledRenderingService, more maintenances are also needed.  10 to 20 classes \
would be mirrored depends on detailed level of the information we are going to fetch.

Regards,
Arthur


-----Original Message-----
From: mapguide-internals-bounces@lists.osgeo.org \
                [mailto:mapguide-internals-bounces@lists.osgeo.org] On Behalf Of \
                Arthur Liu
Sent: Thursday, April 28, 2011 11:18 AM
To: MapGuide Internals Mail List
Subject: RE: [mapguide-internals] RE: RFC110 Profiling API enhancement

Hi Traian,

Following code shows what I thought on how to implement the profiling API in MG.

MgProflilable defines the interfaces to Set/Get profile result to services which \
support profiling.

class MgProfilable{
    public:
        void SetProfileResut(MgProfileResult*);
        MgProfileResult* GetProfileResult();
    
    protected:
        MgProfileResult* m_profileResult;

}

class MgRenderingService : MgProfilable{        
}

... Other services that would be profiled should also inherit from MgProfilable \
(MgResourceService,MgFeatureService,MgMappingService, Renderer...)

MgProfilingService wrappers the services which would be profiled (MgResourceService \
and MgRenderingService)

class MgServerProfilingService {
    public:
        MgByteReader* ProfileRenderMap(.....);

    private:
        MgRenderingService* m_renderingSvc;
        MgResourceService* m_ResourceSvc; }

MgByteReader*  MgServerProfilingService::ProfileRenderMap(.....) {
    Ptr<MgByteReader> result;

    // Create an empty profile result and set to rendering service
    MgProfileResult* pr = new MgProfileResult();
    renderingSvc ->SetProfileResult(pr);
    renderingSvc->RenderMap(....);

    result =  pr.serialize();
    return result.Detach();
}

MgByteReader*   MgServerRenderingService::RenderMap(...) {
    .... original render code....
    If(NULL != m_profileResult)  {
        m_profileResult.StartProfileRenderMap();
    }
    
    if(NULL != m_profileResult) {
       m_renderer.SetProfileResult(m_profileResult);
    }  
    m_renderer ->DoSth(...)
    .... code related to renderer in RenderMap
    
    .... original render code....
    If(NULL != m_profileResult) {
        m_profileResult.EndProfileRenderMap();
    }
}

I think the dlls which will be affected are MgPlatformBase,  MgMapGuideCommon, \
MgRenderers, MgStylization, MgServerFeatureService, MgServerMappingService, \
MgServerResourceService, MgRenderingService.

Regards,
Arthur


-----Original Message-----
From: mapguide-internals-bounces@lists.osgeo.org \
                [mailto:mapguide-internals-bounces@lists.osgeo.org] On Behalf Of \
                Traian Stanev
Sent: Monday, April 25, 2011 1:48 PM
To: MapGuide Internals Mail List
Subject: RE: [mapguide-internals] RE: RFC110 Profiling API enhancement


Hi Arthur,

I am worried that you are also concerned about the performance implications -- I was \
actually hoping that you will give me a definitive answer that there will not be any \
performance implications. 

Given this, can you give some specific examples of what you expect the profiling code \
snippets to be like, and also a list of DLLs which will be affected, and also tell us \
whether there will be a compile time flag to turn them off (in case core DLLs are \
affected such a flag would be a must).

Thanks,
Traian




-----Original Message-----
From: mapguide-internals-bounces@lists.osgeo.org \
                [mailto:mapguide-internals-bounces@lists.osgeo.org] On Behalf Of \
                Arthur Liu
Sent: Sunday, April 24, 2011 10:49 PM
To: MapGuide Internals Mail List
Subject: RE: [mapguide-internals] RE: RFC110 Profiling API enhancement

Hi Traian and Zac,

Thanks for your feedback, these information are very helpful to me.

As Zac said, besides developers of MG, end users could also get benefits from this \
profiling API. We have an idea about providing a dashboard to demonstrate the profile \
result, so that end users also could improve the map rendering performance by \
reorganizing the resources in map.

I also has some concern about injecting too many code into current MG code. A higher \
proportion of profiling code coverage means more detailed information we could get.  \
It's very important that we should balance the two factor.

Traian also raised a very good question about the CPU timing tool. I noticed that the \
timing tool used for debugging in MG was GetTickCount(), which works only on Windows \
platform and has accurate and overflow issues.  I would like to use \
QueryPerformanceFrequency() and QueryPerformanceCounter() API provided by Windows and \
gettimeofday() for Linux to implement an accurate timer.

Taking the advantage of existing logging mechanism is interesting,  which also \
answers my second question about the MdfModel and MdfParser.  However, the current \
log information is not detailed enough as profiling result, not sure if an enriched \
log is useful for other end users who don't care the details. When it comes to \
multiple users, the content in log getting chaos, and the implementation of the \
analytics script could be complex.  It's a great suggestion, I will consider to this \
solution more deep. 

Regards,
Arthur

-----Original Message-----
From: mapguide-internals-bounces@lists.osgeo.org \
                [mailto:mapguide-internals-bounces@lists.osgeo.org] On Behalf Of Zac \
                Spitzer
Sent: Saturday, April 23, 2011 10:26 AM
To: MapGuide Internals Mail List
Subject: Re: [mapguide-internals] RE: RFC110 Profiling API enhancement

absolutely we want to avoid any production overhead.

it would be nice to use this information to profile builds, to see how new \
features/fixes affect overall performance, similiar to what the Firefox folks do

I was thinking the detailed logging already caputures a lot of this information, in a \
verbose and unstructed manner, perhaps an analytics script for the log files could \
produce similiar info?

On Sat, Apr 23, 2011 at 12:20 PM, Traian Stanev <traian.stanev@autodesk.com> wrote:
> 
> In such case, my preference would be to write the code in such a way that this \
> profiling can be either compiled out of the code or at least to keep it outside of \
> the stylization and renderer core, in order to not force everyone to pay for the \
> reporting overhead. 
> Ideally, it would be something that wraps calls to the renderer/feature/resource \
> service/etc. from the rendering service. 
> Also, I would be interested to know what CPU timing tools will be used for this.
> 
> Traian
> 
> 
> -----Original Message-----
> From: mapguide-internals-bounces@lists.osgeo.org
> [mailto:mapguide-internals-bounces@lists.osgeo.org] On Behalf Of Zac 
> Spitzer
> Sent: Friday, April 22, 2011 9:56 PM
> To: MapGuide Internals Mail List
> Subject: Re: [mapguide-internals] RE: RFC110 Profiling API enhancement
> 
> I read this as being something that could be wrapped up so end users could leverage \
> this info for tuning maps. If the performance overhead is roughtly consistant, the \
> information would still be very useful. 
> Maestro already has some basic profiling in it, in MG Classic Author, being able to \
> see how much data was being returned for a layer was a great way to optimise \
> performance. 
> It's a tangent off this RFC, but I think some sort of support for 
> generalizing datasets would really help performance. Sort of like an 
> automated layer atop a featuresource which allows the generalized data 
> to be cached in sqlite? (fdo class naming would be a problem)
> 
> 
> z
> 
> 
> 
> 
> 
> 
> On Sat, Apr 23, 2011 at 11:34 AM, Traian Stanev <traian.stanev@autodesk.com> wrote:
> > 
> > Hi Arthur,
> > 
> > Who would be the target user of such an API? It seems to me it would be a \
> > MapGuide core developer interested in optimizing MapGuide performance. In such \
> > case, what would the API provide above and beyond the Visual Studio built-in \
> > sampling profiler? 
> > Wouldn't the time spent in injecting all those checkpoints in the code (which \
> > might themselves add a further performance overhead) be better spent optimizing \
> > the actual code using existing profiling tools, like the built-in Visual Studio \
> > profiler or VTune. 
> > Having spent a lot of time optimizing MapGuide rendering, I find that the \
> > above-mentioned tools do an excellent job already. 
> > Traian
> > 
> > 
> > -----Original Message-----
> > From: mapguide-internals-bounces@lists.osgeo.org
> > [mailto:mapguide-internals-bounces@lists.osgeo.org] On Behalf Of 
> > Arthur Liu
> > Sent: Friday, April 22, 2011 2:37 AM
> > To: MapGuide Internals Mail List
> > Subject: [mapguide-internals] RFC110 Profiling API enhancement
> 
> > Hi all,
> > 
> > There's an RFC110(http://trac.osgeo.org/mapguide/wiki/MapGuideRfc110)  about \
> > providing some profiling APIs to help identify MapGuide performance bottlenecks. 
> > Before making this RFC be reviewed formally, I would like to know you experts' \
> > ideas about following questions: 
> > 
> > 1.      Which of the following implementation is better?
> > 
> > a.      Solution 1:
> > 
> > Copy the implementation of profiling related APIs to MgServerProfilingService and \
> > add profiling code into the copied version. 
> > For example, to profile the map rendering process, we need to copy the \
> > implementation of RenderMap in MgServerRenderingService(and all other MG server \
> > APIs being invoked in RenderMap) to MgServerProfilingService. The advantage and \
> > disadvantage of this implementation is obvious: less influence on the original \
> > implementation but heavy maintenance to do in the future. 
> > 
> > 
> > b.      Solution 2:
> > 
> > Inject the profiling code into the original functions being profiled directly, \
> > which means MgServerServices should be extended to accept a ProfileResult and the \
> > implementation of some APIs will updated to support profiling. 
> > For example, to support profiling RenderMap, all the MgServerServices in the \
> > RenderMap workflow will be affected - MgServerRenderingService, \
> > MgServerResourceService, MgServerFeatureService, MgServerDescribeSchema, \
> > MgServerSelectFeatures, etc.  Some new internal interface like \
> > Get/SetProfileResult should be added to these MgServerServices. 
> > 
> > For both solutions, how many code should be copied or updated will be decided by \
> > how detailed the information would be gathered. 
> > 
> > 
> > 2.      Is it a good idea that we take the advantage of MdfModel and MdfParser to \
> > store the ProfileResult object model and do the serialization? 
> > 
> > Thanks,
> > Arthur
> > _______________________________________________
> > mapguide-internals mailing list
> > mapguide-internals@lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> > _______________________________________________
> > mapguide-internals mailing list
> > mapguide-internals@lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> > 
> 
> 
> 
> --
> Zac Spitzer
> Solution Architect / Director
> Ennoble Consultancy Australia
> http://www.ennoble.com.au
> http://zacster.blogspot.com
> +61 405 847 168
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> 
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> 
> 



--
Zac Spitzer
Solution Architect / Director
Ennoble Consultancy Australia
http://www.ennoble.com.au
http://zacster.blogspot.com
+61 405 847 168
_______________________________________________
mapguide-internals mailing list
mapguide-internals@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals



_______________________________________________
mapguide-internals mailing list
mapguide-internals@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals


[prev in list] [next in list] [prev in thread] [next in thread]