Welcome to Squishdot Websites Squishdot How-To
 about
 search
 post article
 Documentation
 Mailing Lists
 Bug Tracking
 Development
 Installation
 Upgrading
 Download
 admin
 rdf

 main


Possibly non-scalable portion of Squishsite.py
Squishdot Posted by on Monday September 18, 09:22PM, 2006
from the muppet dept.
My Squishdot instance at jonathanmark.com/Blog slows at 3000 postings. Is there a nonscalable section of the Squishsite.py code?

Lines 552 to 557 in Squishsite.py state:

def rev_id_list(self):
# """ returns reversed id list of reviewed articles """
rlist = map(None,self.ids)
rlist = filter(lambda x,p=self : p.data[x].reviewed, rlist)
rlist.reverse()
return rlist



Reading this code, it seems to me that the entire sequence of articles is returned.

This could result in Squishdot slowing down appreciably on older machines. I run Red Hat 7.3 on a 512 MB box dating from at least 2002.

It would be better to navigate the BTree using last(). Then pick only the most recent 30 articles, or whatever the maximum number of articles to be displayed is.

Then set rlist to only the reverse of those last 30 articles, instead of to every article.

Using pseudo-code

rlist = IOBtree.last()
for i in 1 to MaxArticlesDisplayed:
rlist += IOBtree.last().previous()
rlist.reverse()

This would produce a sequence of only the thirty articles that one wanted to display on the front page.

It would avoid the problem of bringing every article into memory, even those which by definition cannot be on the front page.





Oops... | Squishdot within Plone?  >

 

Related Links

The Fine Print: The following comments are owned by whoever posted them.
( Reply )

Not quite...
by on Monday September 18, 09:29PM, 2006
The code doesn't return a sequence of articles, it returns a sequence of article ids.

That on its own is fine, the problem is the "p.data[x].reviewed" which is used to filter out articles that haven't been reviewed. This ends up dragging all the postings into memory which may be slowing things down on your rather underspecced box ;-)

Instead of using the rev_id_list function, try treating the SquishSite as the ZCatalog that it is and iterating over the following instead:



cheers,

Chris
[ Reply to this ]
Re: Possibly non-scalable portion of Squishsite.py
by on Monday April 02, 03:38PM, 2007
I know this is an older post, but I thought id let you know that if you think 3000 articles is bad, try 18,000 - (we're using squishdot in a way it was never intended) and performance really takes a hit.

[ Reply to this ]
  • Interesting!
    by on Monday April 09, 02:34PM, 2007
    Hey Robin,

    Well, that's certainly interesting.
    What are you using Squishdot for?
    Where have you noticed the performance problems?

    Feel free to drop me a mail if you don't want to reply here...

    Chris
    [ Reply to this ]

 
The Fine Print: The following comments are owned by whoever posted them.
( Reply )

Powered by Zope  Squishdot Powered
  "Any system that depends on reliability is unreliable." -- Nogg's Postulate
All trademarks and copyrights on this page are owned by their respective companies. Comments are owned by the Poster. The Rest ©1999 , ©2000-2002 .