Products » Support Forums 

Support Forums

HomeHomeDotNetNuke Modu...DotNetNuke Modu...Friendly Url Pr...Friendly Url Pr...What causes Tab Dictionary Rebuild????What causes Tab Dictionary Rebuild????
Previous
 
Next
New Post
8/12/2010 5:38 AM
 

Hi Bruce, we've been using your Friendly URL provider for quite a while and everything's been going ok, until recently. We are on a pretty old version of DNN also, so we figured it might be a DNN problem.

In a nutshell, what's happening, is every so often the database server spikes right up to 100% for several minutes for no apparent reason - we haven't changed code in months. After discussing with my boss we've decided to move to the most current version of DNN. But we have to do it in steps. So I've got everything from production running on a test box on Version 4.9.5 and the most current version of your friendly URL provider. Everything was going swimmingly, then blam, the same problem happens on the test box this afternoon, again for no apparent reason. So the test box is on 4.9.5 with the most current version of Friendly URL provider.

After looking at the event logs for the test box, I realized I saw two Tab Dictionary removed messages in like 2 minutes, at about the same time as the db spiked. I looked in our production db, and it appears we are getting Tab Dictionary removed messages a couple of times a day, and they SEEM to corespond to when the Tab Dictionary's are getting removed - I think what's happening is that we are getting a couple rebuilds back to back, and since there are 208 portals with 54061 tabs and 749 alias, I think the tab rebuilds are taking too much time, or one is started, and the next one starts before the first finishes.

So my question is: What causes a Tab Dictionary rebuild? I haven't been able to get it to consistently do it on my machine - I would have thought adding or deleting tabs would cause it, but that doesn't seem to be the case. If I could cause it to happen every time, then I would be able to test my hypothesis.

This is baffling to me because 90% of the time the db server runs along at something like 1-10%, about 5% of the time it spikes up, then comes right back down, and then 5% of the time it just pegs at 100%

Your thoughts please

Thanks.

 
New Post
8/12/2010 6:12 AM
 

Rick, there are several things that cause a dictionary / page index rebuild. The dictionary or page index is a in-memory cache of the available portal aliases and page names, which allows the provider to do a fast lookup of Urls when requested.

The principal ways the page index is rebuilt are:

1) application pool recycle, which drops the cached index and forces it to be rebuilt

2) available/required memory resources in the server cause asp.net to drop the item from the cache, which forces it to be rebuilt

3) a change is made to the site where a postback Url matches the regex value in the 'triggerDictionaryRebuild' regex attribute.

The actual message you will see in the logs will point to which one of these is the culprit, although it's pretty much impossible to tell the difference between 1 and 2 unless you can tie corroborating evidence (such as an app pool recycle message) into the timeframe. If it's (3) then you can manage this by reducing the matches down. It could be administrators making changes (adding, removing tabs, etc) that causes the issue, but I assume you'll know the answer to this straight away. The reason for getting two changes back to back are harder to discern - it could be a timing issue as two threads both find the page index empty and so both begin the process of rebuilding at the same time. That type of thing is hard to test for, and even harder to code for.

Now, the size of the page index is a function of the sum of the portal aliases x the sum of the pages. A portal with 2 aliases and 20 pages will equal 40 entries + 5 or so 'common' entries = so about 45. With the sorts of numbers of tabs, aliases and portals you're running, this is going to be a sizeable sized page index - the arithmetic is complex but you can probably assume something like 100,000 entries at a minimum. The friendly url provider also uses standard DNN stored procedures to retrieve the information, so you're looking at running the 'GetTabs' stored procedure.

You might want to narrow down where the cpu usage is originating from ; if it's the database process, then there *may* be scope for optimising the query (some of the standard DNN queries are not at all optimised for performance). If it's the looping process of reading through the large size of the result set, then there isn't much you can do about it short of rewriting that part of the code. The module is built around the assumption that a Url either exists in the index, or it doesn't exist and should return a 404. That's why there is so much code devoted to keeping the index current.

If you have a test server you can look at using the Url Master 2.0 version to see if you see any improvement. This has a crucial difference in the way the page index is rebuilt. For a start, you can specify a 'chosen portal alias' (though I suspect you may wish to use multiple aliases) which immediately reduces the size of the index to roughly equal to the number of tabs (because each tab effectively only has one valid portal alias). The second improvement is that the page index is built incrementally, one portal at a time. So, if you have 200 portals, and only 10 of them get any serious use, then as each request comes in for each additional portal, the page index will be incrementally increased, rather than being built all in one big go. Obviously the big difference is the cost between the free module and not, so that's a choice that you can make. There isn't any chance of the Url Master 2.0 changes being backported into the Friendly Url Provider because the codebase has been irretrievably forked such is the nature of the fundamental changes in the new version.

 
New Post
8/12/2010 8:11 AM
 

Hi Bruce thanks for the follow up.

I figured it was those things, however for some odd reason, adding and deleting a page(at least for me) doesn't seem to cause a cache rebuild, which has me REALLY scratching my head, because to me, it seems logical that would happen. I need to investigate that further.

One thing I have done is break up the tab dictionary into dictionary's for each portal, instead of one huge dictionary. Basically, what I did was use the Portal ID when I'm storing the TabDictionary in the cache. Basically instead of calling GetAllTab, I call GetTabs(PortalID) in the BuildTabDictionary code, I also store the tabDictionary with the TabID in the key value so instead of just tabDict being passed to DataCache.SetCache it's TabDictKey+PortalID.ToString(). This means only the tabs for the requested portal are loaded up. One thing I saw with the original code was the memory foot print of the W3WP went up to 800 megs and kept climbing. This change keeps the memory foot print much lower.

I do have one other question too: At Line 122 of UrlRewriteModule.cs, you declare a PortalAliasInfo, and as trying to GetPortalAlias from the AbsoluteUri.. then there's an if (alias !=null) test there, which is fine, but I'm wondering under what circumstances would the alias be null? Wouldn't a throw be better here? Just trying to figure out what the else to that if should be?

Your thoughts? Thanks for the quick response.

 
New Post
8/12/2010 9:25 AM
 

Sigh, double posts...

Is it possible the page index could be rebuilt based on just a strict time limit?

Let's say two items are put in the cache at the same time (say app start), then some period of time passes - the system won't rebuild them unless they are requested and are found to be stale correct? Or does some timer come along and say, oh you've been here for the max allotted time, your getting rebuilt?

 
New Post
8/12/2010 12:30 PM
 

Well, there is a time limit on the cache - this is set by using the attributes of 'cacheTime', which is set in minutes. The default is 1440, which is 24 hours. There's also 'cachePersistRestart=true' which is supposed to persist the cached item through an app pool recycle. In theory this should give you the cache for a set time, then dump it and recreate. In practice I've never had a lot of luck in testing this accurately. The timespan and cache persist restart go directly into the asp.net datacache methods, so it should work OK.

All rebuilds work on the basis of a request coming in and finding a null dictionary - there's no timer or scheduled task to continually make sure it is current.

It's possible that you're getting a rebuild when you add a new page, but it's just not being logged. If you add a new page, and then you can navigate to the new page, then it's rebuilding the page index. You'd get a 404 error otherwise.

Your mods to build one portal at a time is pretty much the same sort of change made in the Url Master 2.0 code, so that should be helping to both incrementally build the dictionary and also to keep the memory footprint down. If you're into modifying the code, maybe a place to look at is in the CacheController class, in the 'RemovedCallBack' method. This method is called by the .net runtime when the dictionary object is dropped from the cache. This is where your messages come from, and with some creative coding you might be able to determine what conditions are present at time of the unload, which might help track down the issue further. You could even get creative and check things like the time the object was in cache, and system information like free memory / threads/ etc (not sure on the details, but should all be possible).

The check for the portal alias is because it's possible to get the site to respond to a domain when there is no matching portal alias entry. This is very common - such as not entering 'example.com' (only having www.example.com as an alias) but having IIS respond to example.com. Thus, you'll find there is no matching alias for the request. The reason there is no 'else' clause is that further down the processing logic, there's another attempt to identify the alias based on the tabid and domain name. Identifying the alias up front helps with the rewriting logic if it can be done, but it's not a big deal if it can't. IIRC that code was added more to identify the portal id than the alias. So you don't need an 'else' clause because the code will have another stab at identifying the alias further on.

 
Previous
 
Next
HomeHomeDotNetNuke Modu...DotNetNuke Modu...Friendly Url Pr...Friendly Url Pr...What causes Tab Dictionary Rebuild????What causes Tab Dictionary Rebuild????


Support Guidelines.. Please read before posting

To get support on iFinity products and services, please search the forums for the the answer to the problem you are seeking. If you cannot find a solution, post a question in the relevant forum.   Ensure that you specify the relevant versions of the problem, and the actual error message or a detailed description of the problem.    You will need to register with this site to post on the forum.  If you have a Microsoft Live (Hotmail/Passport) account you can use that.  If you have a Open Id account you can use that.  If you neither of these, you will need to register a user Id and password.