about
search
post article
Documentation
Mailing Lists
Bug Tracking
Development
Installation
Upgrading
Download
admin
rdf
main
|
Possibly non-scalable portion of Squishsite.py
|
Posted by on Monday September 18, 09:22PM, 2006
from the muppet dept.
My Squishdot instance at jonathanmark.com/Blog slows at 3000 postings. Is there a nonscalable section of the Squishsite.py code?
Lines 552 to 557 in Squishsite.py state:
def rev_id_list(self):
# """ returns reversed id list of reviewed articles """
rlist = map(None,self.ids)
rlist = filter(lambda x,p=self : p.data[x].reviewed, rlist)
rlist.reverse()
return rlist
Reading this code, it seems to me that the entire sequence of articles is returned.
This could result in Squishdot slowing down appreciably on older machines. I run Red Hat 7.3 on a 512 MB box dating from at least 2002.
It would be better to navigate the BTree using last(). Then pick only the most recent 30 articles, or whatever the maximum number of articles to be displayed is.
Then set rlist to only the reverse of those last 30 articles, instead of to every article.
Using pseudo-code
rlist = IOBtree.last()
for i in 1 to MaxArticlesDisplayed:
rlist += IOBtree.last().previous()
rlist.reverse()
This would produce a sequence of only the thirty articles that one wanted to display on the front page.
It would avoid the problem of bringing every article into memory, even those which by definition cannot be on the front page.
< Oops... | Squishdot within Plone? >
|
Related Links
|
|
|
The Fine Print: The following comments are owned by whoever posted them.
( Reply )
|
Not quite...
by on Monday September 18, 09:29PM, 2006
|
The code doesn't return a sequence of articles, it returns a sequence of article ids.
That on its own is fine, the problem is the "p.data[x].reviewed" which is used to filter out articles that haven't been reviewed. This ends up dragging all the postings into memory which may be slowing things down on your rather underspecced box ;-)
Instead of using the rev_id_list function, try treating the SquishSite as the ZCatalog that it is and iterating over the following instead:
cheers,
Chris |
[ Reply to this ]
|
Re: Possibly non-scalable portion of Squishsite.py
by on Monday April 02, 03:38PM, 2007
|
I know this is an older post, but I thought id let you know that if you think 3000 articles is bad, try 18,000 - (we're using squishdot in a way it was never intended) and performance really takes a hit.
|
[ Reply to this ]
|
-
Interesting!
by on Monday April 09, 02:34PM, 2007
|
Hey Robin,
Well, that's certainly interesting.
What are you using Squishdot for?
Where have you noticed the performance problems?
Feel free to drop me a mail if you don't want to reply here...
Chris
|
[ Reply to this ]
|
|
The Fine Print: The following comments are owned by whoever posted them.
( Reply )
|
|