Server speed

by **Bob Sihler** » Wed Mar 02, 2011 3:15 am

It has been a lot faster for me lately. Very few slowdowns, if any at all. Thanks for your efforts on this.

by **phlipdascrip** » Wed Mar 02, 2011 4:18 am

out of curiosity, what did you change?

by **Josh Lewis** » Wed Mar 02, 2011 5:34 am

It's good on my end. Matt your doing very well!

Thanks for your support! A Few months it got real bad, but slowly it got better, get it... slowly! :lol:

(Just had to say that for kicks and giggles)

by **rgg** » Wed Mar 02, 2011 2:08 pm

Yep, I noticed that it was a lot faster most of the time.

Now I can't get much real work done between posting pictures anymore ! :wink:

by **Sarah Simon** » Wed Mar 02, 2011 5:15 pm

Matt,

I posted several images for a route last evening, which is usually the "dreadfully slow" SP time, based on my experience.

Yesterday evening I experienced no delays - very fast.

Thanks!
Sarah

by **Mountainjeff** » Wed Mar 02, 2011 6:46 pm

Everything seems to be running faster for me than it was a few weeks ago.

by **phlipdascrip** » Wed Mar 02, 2011 11:03 pm

Montana Matt wrote:I eliminated a query that was happening on every page load. The data that was being saved wasn't absolutely necessary.

just as incentives:
* did you check if access logs are being kept? this is an unnecessary disk I/O evil as well.
* since you mention an outdated DB, making sure proper keys and indexes are set on tables can make a huge difference on queries. are you keeping a slow query log?
* tweaking the last ms out of frequently run queries themselves can also make quite the difference.
* is there static (config..) data that you could move from the db to static code (e.g. arrays)? memcache is prolly too much work to set up properly.

if you checked those already, never mind - don't mean to be presumptuous

by **phlipdascrip** » Thu Mar 03, 2011 1:29 am

Montana Matt wrote:Yes access logs are kept and getting rid of them probably isn't going to happen.

why not? aren't error logs sufficient?

Montana Matt wrote:It's not that the database is outdated. It's that it has grown so large that the storage engine we use for the main table isn't a good choice anymore.

I see. is the page content stored in the DB? did you consider writing it to files instead? plain text or xml, for example (e.g. [object ID].xml and then nodes for the different page sections). wouldn't that reduce the table size by what 90%? easily?

Montana Matt wrote:
phlipdascrip wrote:tweaking the last ms out of frequently run queries themselves can also make quite the difference.

I don't know what "ms" is in that sentence. But frequently run queries have been optimized quite well at this point.

milliseconds.

by **phlipdascrip** » Fri Mar 04, 2011 6:00 pm

File access may be a bit slower than mysql, but for larger amounts of unindexed data my guess is the difference is not much. Of course XML parsing time comes on top of that, but with only few nodes inside each XML file I think this is negligible as well. The overhead put on mysql by storing those large amounts of data inside mysql however may be the bigger evil. To know for sure though you'd probably have to run some kind of benchmark.

Also not to forget the number of times you need to retrieve full page content is much smaller than querying for page properties (title, owner, rank etc.). You'd only need to access an XML file when a page is actually viewed.

Did you look at mysql's cache statistics and config as well (opened_tables, open_tables, table_cache etc.)? I remember we were able to improve performance by tweaking those values too.

by **phlipdascrip** » Fri Mar 04, 2011 6:05 pm

Oh forgot, you could also set up memcache for caching just XML data. Frequently opened pages (e.g. featured on front page) would stay in the cache.

by **Josh Lewis** » Sat Mar 05, 2011 3:38 am

Fascinated information.

Glad we have you as our techy guy here.

by **Josh Lewis** » Sun Mar 06, 2011 5:34 pm

I'm sad to say that sp seems to be loosing some of it's speed, still manageable but am starting to wonder about later on.

Edit: Well it's still fine right now...

by **phlipdascrip** » Tue Mar 08, 2011 11:09 pm

I see, well seems like you know what the last remaining issue is, albeit it being the biggest one I believe - a simple conversion from myisam to innodb isn't quite possible is it? I was just thinking, if you'd source actual page data out into xml files and add a mechanism to only update the corresponding meta data in the database IF that meta data has actually changed during a page edit, you may get a useful fix without having to convert the table type - only the xml file would need to be updated, not the db record if page meta data hasn't changed.

by **lcarreau** » Wed Mar 09, 2011 12:18 am

phlipdascrip wrote:I see, well seems like you know what the last remaining issue is, albeit it being the biggest one I believe - a simple conversion from myisam to innodb isn't quite possible is it? I was just thinking, if you'd source actual page data out into xml files and add a mechanism to only update the corresponding meta data in the database IF that meta data has actually changed during a page edit, you may get a useful fix without having to convert the table type - only the xml file would need to be updated, not the db record if page meta data hasn't changed.

The following user would like to thank lcarreau for this post
Buz Groshong

by **mvs** » Wed Mar 09, 2011 8:28 am

Apparently Wikipedia uses InnoDB. Probably InnoDB is the right architecture for the kind of site Summitpost is, basically a wiki for mountains. It's amazing we got so far with a global lock on the table level, I am surprised!

Why would conversion to InnoDB be hard? Because current code doesn't have to be transaction-aware?

Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Re: Server speed

Who is online