PDA

View Full Version : Server Maintenance, please read.


Phobia
09-13-2004, 02:11 PM
I think pruning the DB down is a great idea.

However, I don't want to lose some of our more classic threads.

We now have a classic threads forum. Please take a few minutes (or hours) to browse our archives to find the threads you would like to see retained. Place a link to those threads in the classic threads designated thread or PM a moderator. They will then be moved.

I'll prune the remaining archives before gameday. I'll also lock down viewing to registered users only on gameday.

You have 5 days to get the classic threads identified. After that, they're gone.

ptlyon
09-13-2004, 02:13 PM
How far back you thinking on going?

Matt Helm
09-13-2004, 02:13 PM
Does the size of the strings have an effect on performance?

Not that it is a bad thing, but does the girlie thread cause any slowdowns?

Phobia
09-13-2004, 02:14 PM
How far back you thinking on going?

I dunno. Let's start with a year back all the way to inception.

ptlyon
09-13-2004, 02:16 PM
That'll clean alot up.

HC_Chief
09-13-2004, 02:17 PM
Why delete the old stuff? Just archive it to another separate DB.

Phobia
09-13-2004, 02:20 PM
Kyle said that's a lot more difficult than it sounds. I'm tired of the performance issues and the archives are used so infrequently, I think performance would be a great tradeoff.

htismaqe
09-13-2004, 02:22 PM
Was a decision ever made concering the e-mail issue AustinChief brought to light some time ago? If we're taking actions, might as well take care of that as well.

I agree.

I would prefer to fix that FIRST, before removing content.

htismaqe
09-13-2004, 02:22 PM
Kyle said that's a lot more difficult than it sounds. I'm tired of the performance issues and the archives are used so infrequently, I think performance would be a great tradeoff.

Is the data permanently lost?

Iowanian
09-13-2004, 02:27 PM
There are alot of "nothing" threads............few posts, just jibba jabba............I know this will be blasphemy....but what about "trimming down" half of the gjpgthread?

I could probably easily locate 5 threads I've started to nuke......hell....take them all if you want. You can nuke every post I have and I'll start over if it will help the board performance.

morphius
09-13-2004, 02:31 PM
Why do we need to keep the old threads, I thought thats why we kept Titus around here...

Iowanian
09-13-2004, 02:33 PM
Would it help to trim "accounts" that haven't been logged into in say....over a year?

Braincase
09-13-2004, 02:34 PM
First of all, delete all old threads with fewer than 10 posts... then we do a query on this with <=20. Delete anything that ENDelt was the only poster...

Raiderhater
09-13-2004, 02:35 PM
Why do we need to keep the old threads, I thought thats why we kept Titus around here...


Heh, Yeah but links are nice to have, for those who feel the need to argue ("I'll never post there").

htismaqe
09-13-2004, 02:36 PM
I wouldn't get rid of anything that Chiefs (or football) related. That kind of stuff gets "re-used" all the time...

Phobia
09-13-2004, 02:37 PM
First of all, delete all old threads with fewer than 10 posts...

This has been processing for the past half hour.

Raiderhater
09-13-2004, 02:38 PM
I'm not a big fan of deleting content, however there clearly is no sense in keeping the old stuff from the original board that has DB issues. Might as well go ahead and whack that stuff now.

Matt Helm
09-13-2004, 02:38 PM
It's not a big load, but after Nov 2. (3rd, 4th, or whenever the court case is finished) the WDC could be wiped.

Raiderhater
09-13-2004, 02:42 PM
It's not a big load, but after Nov 2. (3rd, 4th, or whenever the court case is finished) the WDC could be wiped.


Only threads that are in essence duplicates of the same story.

There have been a lot of things said that one would like to be able to revisit from time to time in the future.

Skip Towne
09-13-2004, 02:42 PM
I'll send you a list of people I want banned. That should help some.

|Zach|
09-13-2004, 03:42 PM
I'll send you a list of people I want banned. That should help some.
uh oh

KCTitus
09-13-2004, 03:47 PM
I really dont like the idea of deleting good content...trim everything else first. Especially the forums no one is using.

It would be a shame to delete the thread that has many folks talking about how bad the signing of Holmes was.

Imon Yourside
09-13-2004, 03:59 PM
can anyone answer will the old stuff be backed up to say dvdr or simply deleted. i would be in favor of it IF it could be backed up for arguments sake.

AustinChief
09-13-2004, 04:02 PM
I am looking at 2 or 3 options for "splitting" the posts table and getting it down to a reasonable size. I will keep everyone informed of the progress...

--Kyle

AustinChief
09-13-2004, 04:19 PM
As an alternative idea...

Why not grab the latest & greatest vBulletin... set it up with a brand new empty database.. and dub that ChiefsPlanet. Make a link to the current bloated database, and consider it an archive.

It would be kinda weird at the start, I suppose... there'd be no posts in the "main" bb. Folks would probably still be referencing the "archive" a lot...and having slow response.

But, after awhile I think it would work out. Much traffic on the "new" board... occasional access to the "archive" for old stuff.

I thought of that idea... but there are some serious drawbacks to it... I can actually do just that and "shut off" the old BBS so that no new posts are made... and we don't have to start from scratch.. I can copy the entire DB .. then prune everything over a year old...

...but there are still issues involved... I am working those out right now though...

AustinChief
09-13-2004, 04:25 PM
Please share.
mostly just dealing with "locking" it down completely so that no one gets confused and thinks that is the "real" site... and also getting it "integrated" enough with the "real" site that people would be able to find and use it... I would prefer to just split the posts table and rewrite the software.. but that is looking less likely... so the seperate archive BBS is looking more and more likely.

The other drawback is that no one would be able to post on those old threads.. so once a thread is "moved" there... it is set in stone...

--Kyle

Phobia
09-13-2004, 04:28 PM
mostly just dealing with "locking" it down completely so that no one gets confused and thinks that is the "real" site... and also getting it "integrated" enough with the "real" site that people would be able to find and use it... I would prefer to just split the posts table and rewrite the software.. but that is looking less likely... so the seperate archive BBS is looking more and more likely.

The other drawback is that no one would be able to post on those old threads.. so once a thread is "moved" there... it is set in stone...

--Kyle

As long as we don't have performance hiccups 20x per day, I think we can live with those drawbacks.

Frazod
09-13-2004, 05:24 PM
I'll send you a list of people I want banned. That should help some.

We could start with all the Kansas fans. They suck. :p



Seriously, I'm glad you guys are doing something. This shit is getting out of hand. Thanks.

:thumb:

Skip Towne
09-13-2004, 05:29 PM
We could start with all the Kansas fans. They suck. :p



Seriously, I'm glad you guys are doing something. This shit is getting out of hand. Thanks.

:thumb:
I certainly don't want rid of the MU fans. They're fun to kick around.

dirk digler
09-13-2004, 06:01 PM
I think pruning the DB down is a great idea.

However, I don't want to lose some of our more classic threads.

We now have a classic threads forum. Please take a few minutes (or hours) to browse our archives to find the threads you would like to see retained. Place a link to those threads in the classic threads designated thread or PM a moderator. They will then be moved.

I'll prune the remaining archives before gameday. I'll also lock down viewing to registered users only on gameday.

You have 5 days to get the classic threads identified. After that, they're gone.

Thanks Kyle and Phobia for trying to resolve this problem. You can delete all of my threads since I don't post anything important anyway.

Phobia
09-13-2004, 06:05 PM
Thanks Kyle and Phobia for trying to resolve this problem. You can delete all of my threads since I don't post anything important anyway.

We did that weeks ago.

royr17
09-13-2004, 06:12 PM
Here you go phobia :

http://www.chiefsplanet.com/BB/showthread.php?t=96048&highlight=royr17

digi2fish
09-13-2004, 06:22 PM
I have to say Kyle has done a great job already.
I was wondering how this site maintain such a huge loading and data storage when I first got here, IMO to remove some older data is necessary, somehow I think we can keep the backup in DVDs or something.

thanks, Kyle and Phob :thumb:

dirk digler
09-13-2004, 06:25 PM
We did that weeks ago.


ROFL Thanks!

AustinChief
09-13-2004, 06:46 PM
Oh yeah... not to repeat myself or anything... but was that e-mail thing determined to not actually be negatively impacting performance... or what?

That was mostly resolved... so it isn't an issue anymore.

--Kyle

Iowanian
09-13-2004, 06:51 PM
Here you go phobia :

http://www.chiefsplanet.com/BB/showthread.php?t=96048&highlight=royr17

no..no...no............that jewel is destine for the HoF.....

now get out there and lay that pipe, tigger.

Hydrae
09-13-2004, 07:09 PM
I sent a PM to Kyle a couple of weeks ago about another thought I had had related to server speed. It seems to me that every page I go to, I have to wait for all the little graphics to load. This includes all the "profile", "send PM", etc links that are at the bottom of each post. If those were to be cookied onto each users system, it will save on the time it takes each page to load and thus decrease the over all load on the server.

2bikemike
09-13-2004, 08:03 PM
I would support just about any move you make to correct the problems we have been having. I don't understand half of this stuff it takes to run this site. I just know I like hanging out here.

go bo
09-13-2004, 08:09 PM
I would support just about any move you make to correct the problems we have been having. I don't understand half of this stuff it takes to run this site. I just know I like hanging out here.me too... :thumb:

Straight, No Chaser
09-13-2004, 08:11 PM
I sent a PM to Kyle a couple of weeks ago about another thought I had had related to server speed. It seems to me that every page I go to, I have to wait for all the little graphics to load. This includes all the "profile", "send PM", etc links that are at the bottom of each post. If those were to be cookied onto each users system, it will save on the time it takes each page to load and thus decrease the over all load on the server.



I would like to suggest a "text only" option in the users profile prefs. I forget the software used here (is it VBulletin?); I'm guessing this feature is not yet available. If 10% of Planet users used a "text only" interface it would make a difference, especially during peak times.

Another more sophisticated solution would be to have an option available to the user, something like "Don't load images over 100k" for example. That way all the smaller stuff would come through and the monster images that some planeteers attach would not.

I bet 75% of the images you see on the planet could be slimmed down by anywhere from 10 - 100 times. Of course, you can't expect everyone to have the knowledge and tools it takes to balance compression with image quality. Plus, it takes extra time to perform those functions. Some software exists --seen on the many of the free photoblogs-- that actually makes the process transparent: published JPEGs are resized to nominal dimensions.

The past couple of days I have seen excruciating load times. The only real solution is to upgrade. The database could be archived, and stored for later when a viable solution arises.



---->

Coach
09-13-2004, 08:52 PM
Another more sophisticated solution would be to have an option available to the user, something like "Don't load images over 100k" for example. That way all the smaller stuff would come through and the monster images that some planeteers attach would not.

I bet 75% of the images you see on the planet could be slimmed down by anywhere from 10 - 100 times. Of course, you can't expect everyone to have the knowledge and tools it takes to balance compression with image quality. Plus, it takes extra time to perform those functions. Some software exists --seen on the many of the free photoblogs-- that actually makes the process transparent: published JPEGs are resized to nominal dimensions.


Looking at Taco's sig, that sig alone is 847K. Talk about taking up bandwith if you ask me.

KS Smitty
09-13-2004, 09:02 PM
First of all, delete all old threads with fewer than 10 posts... ...


This has been processing for the past half hour.


:deevee: There goes every thread I've ever started. :deevee:

David.
09-13-2004, 09:03 PM
I think we should start with the email notification. It was quick and easy for me to fix. Everyone else should do it too.

Mr. Laz
09-13-2004, 09:11 PM
As an alternative idea...

Why not grab the latest & greatest vBulletin... set it up with a brand new empty database.. and dub that ChiefsPlanet. Make a link to the current bloated database, and consider it an archive.

It would be kinda weird at the start, I suppose... there'd be no posts in the "main" bb. Folks would probably still be referencing the "archive" a lot...and having slow response.

But, after awhile I think it would work out. Much traffic on the "new" board... occasional access to the "archive" for old stuff.

could you divide up the board with several DB's

DB 1 - Main BB
DB 2 - Picture Forum
DB 3 - Royals, Gambling etc

but set the boards up to look the same ... people click on a forum link and go to another site without even really being aware of the change.

spreads the bandwidth around.... cuts down on the load


you dont lose any content, just manipulate the traffic a little

Straight, No Chaser
09-13-2004, 09:32 PM
Looking at Taco's sig, that sig alone is 847K. Talk about taking up bandwith if you ask me.

I'm no programmer or mathmatician but I'll try and play one:

[Bronco Week]
Server: Oooooohhhhhhhh No. Here comes another request for Taco's lame sig (874k). Here ya go... tic,tic,tic.
Server: Done.
Taco Hater: Ahh, lets see... another post from Taco. "What has that A-hole spewing now?". Click.
Server: Any second now Chief fans will want that damn sig aga...here it comes...#@!* 5 more requests.tic,tic,tic,tic,tic,tic,tic,tic,tic,tic,tic,tic.

<============>

Everytime a bloated image is spit out it wastes the server's capacity to handle requests. Multiply the fraction of a second it takes to perform that function by the times per hour it's requested x hours per day x hours per month etc. And that's a big number.

Add a solution for those who want the textual content but could careless for the visuals: "Back To the Future"



---->

PastorMikH
09-13-2004, 09:38 PM
I wonder how much space all those pics in the girly thread takes up???


:fire: :)

Hammock Parties
09-13-2004, 10:09 PM
I have a great idea.

Put a limit on the size (in KB and dimensions) of avatars and sigs. Especially sigs.

|Zach|
09-13-2004, 10:27 PM
I would understand more of a limit on sigs and avatars but I really enjoy them the way they are. I think they are a big reason that makes this place so special. They do alot to show personality and things like that.

Michael Michigan
09-13-2004, 10:51 PM
Don't worry about archives, CBS has s0me folks that can make up anything we need on a 197o's era typewriter that looks just like it for a small fee.

go bo
09-13-2004, 10:53 PM
I would understand more of a limit on sigs and avatars but I really enjoy them the way they are. I think they are a big reason that makes this place so special. They do alot to show personality and things like that.personality?

you have the personality of a monkey? :shrug:

i've met you and you have lots more personality than that... :D

more like this: http://instagiber.net/smiliesdotcom/otn/realhappy/jump.gif http://instagiber.net/smiliesdotcom/otn/realhappy/jump.gif http://instagiber.net/smiliesdotcom/otn/realhappy/jump.gif

|Zach|
09-13-2004, 10:54 PM
personality?

you have the personality of a beaver (or groundhog or whatever that thing is)? :shrug:

i've met you and you have lots more personality than that... :D

more like this: http://instagiber.net/smiliesdotcom/otn/realhappy/jump.gif
http://instagiber.net/smiliesdotcom/otn/realhappy/jump.gif

ROFL

My in person personality doesn't come off well in CP. They just don't transfer thats why I like meeting CPers in real life.

go bo
09-13-2004, 11:00 PM
ROFL

My in person personality doesn't come off well in CP. They just don't transfer thats why I like meeting CPers in real life.oh it comes off well enough...

are you coming to st. joe for the cp party?

|Zach|
09-13-2004, 11:03 PM
oh it comes off well enough...

are you coming to st. joe for the cp party? I believe I will be there. I am very excited for this one. Unless something strange comes about you can count me in.
:)

KC Jones
09-14-2004, 07:46 AM
There are a number of factors that could be related to performance. Bandwidth of the request/responses is only one. Frankly, once you've seen an image in somebodies sig or avatar your browser is likely caching it locally and it's only being transmitted on that first viewing (or after you've cleared your temp files).

Since it came from Kyle I think it's pretty safe to say he knows that it's DB congestion due to the size of the posts table. Just move all the threads that haven't been posted to in over 6 months to a new table. As you have time work on a new interface to this archive table. We can live without the old posts for a while, and it won't be such a horrible loss to have to view older threads in an archive.

Mr. Kotter
09-14-2004, 07:53 AM
I wonder how much space all those pics in the girly thread takes up???


:fire: :)

Those aren't really girly pics over there; it's actually a bible study group, they just want a little privacy....and didn't want to "flaunt" their religion....

htismaqe
09-14-2004, 07:55 AM
There are a number of factors that could be related to performance. Bandwidth of the request/responses is only one. Frankly, once you've seen an image in somebodies sig or avatar your browser is likely caching it locally and it's only being transmitted on that first viewing (or after you've cleared your temp files).

Since it came from Kyle I think it's pretty safe to say he knows that it's DB congestion due to the size of the posts table. Just move all the threads that haven't been posted to in over 6 months to a new table. As you have time work on a new interface to this archive table. We can live without the old posts for a while, and it won't be such a horrible loss to have to view older threads in an archive.

As long as those old threads are viewable, and thus QUOTABLE, I don't think anyone will care one bit.

Mr. Kotter
09-14-2004, 07:56 AM
As long as those old threads are viewable, and thus QUOTABLE, I don't think anyone will care one bit.

Exactly. :thumb:

KC Jones
09-14-2004, 08:06 AM
As long as those old threads are viewable, and thus QUOTABLE, I don't think anyone will care one bit.

Would it be ok if they weren't viewable for a month?

In other words make the split now and add the archive forum to access it later. You lose access to old material temporarily but gain performance immediately.

Not that I have any say in the matter.

htismaqe
09-14-2004, 08:16 AM
Would it be ok if they weren't viewable for a month?

In other words make the split now and add the archive forum to access it later. You lose access to old material temporarily but gain performance immediately.

Not that I have any say in the matter.

As long as no posts are PERMANENTLY deleted, I'm fine with just about anything.

Mr. Kotter
09-14-2004, 09:08 AM
As long as no posts are PERMANENTLY deleted, I'm fine with just about anything.

FWIW, I agree.

I suspect most here would be just fine with such an arrangement. :thumb:

Matt Helm
09-14-2004, 09:22 AM
We could start with all the Kansas fans. They suck. :p



Seriously, I'm glad you guys are doing something. This shit is getting out of hand. Thanks.

:thumb:

Funny, that's what I heard about misplaced Bears fans!!

BigMeatballDave
09-14-2004, 10:08 AM
Kyle said that's a lot more difficult than it sounds. I'm tired of the performance issues and the archives are used so infrequently, I think performance would be a great tradeoff.Agreed. I'd gladly trade old threads for improved performance...

PastorMikH
09-14-2004, 10:52 AM
Those aren't really girly pics over there; it's actually a bible study group, they just want a little privacy....and didn't want to "flaunt" their religion....


WOW! Really? Perhaps I'll quit ignoring it and join in then. You better be telling the truth though!!!:D

Coach
09-14-2004, 10:55 AM
WOW! Really? Perhaps I'll quit ignoring it and join in then. You better be telling the truth though!!!:D

Busted.....

Mr. Kotter
09-14-2004, 10:57 AM
WOW! Really? Perhaps I'll quit ignoring it and join in then. You better be telling the truth though!!!:D

Eh....Pastor, just a little joke there, eh....DON'T go there, if you're inclined to "lust" as Jimmy Carter once said. :)


However, it IS "clean" as far as "Girly Pics" go....I mean, BD does have STANDARDS and rules there, you know. Er.....eh.....so I have HEARD anyway. :thumb: :redface:

PastorMikH
09-14-2004, 11:04 AM
Eh....Pastor, just a little joke there, eh....DON'T go there, if you're inclined to "lust" as Jimmy Carter once said. :)



ROFL ROFL ROFL ROFL


Actually, I did click on the thread once when I first got here - by accident of course!!! I am well aware of what is there. Personally, I like the fact that people are respectful enough to post pics there instead of in random threads because there are times when I would get uncomfortable (well, I get uncomfortable at times now - especially when I'm reading post and someone comes in and thinks I'm looking at a sig or something) but I'd be REALLY uncomfortable if they were just pasted all over.

:)

BIG_DADDY
09-14-2004, 11:15 AM
ROFL ROFL ROFL ROFL


Actually, I did click on the thread once when I first got here - by accident of course!!! I am well aware of what is there. Personally, I like the fact that people are respectful enough to post pics there instead of in random threads because there are times when I would get uncomfortable (well, I get uncomfortable at times now - especially when I'm reading post and someone comes in and thinks I'm looking at a sig or something) but I'd be REALLY uncomfortable if they were just pasted all over.

:)

So those warm tingles your experiencing are uncomfortable to you?

Calcountry
09-14-2004, 11:57 AM
So those warm tingles your experiencing are uncomfortable to you?
Welcome back Big D.

:)

Mr. Kotter
09-14-2004, 12:34 PM
Welcome back Big D.

:)

Dittos. :thumb:

Valiant
09-14-2004, 03:20 PM
cant we just delete all the donxs and faiders fans accounts???

I would say bolts also, but have not seen a bolt fan in years...

KC Jones
09-14-2004, 03:38 PM
cant we just delete all the donxs and faiders fans accounts???

I would say bolts also, but have not seen a bolt fan in years...

I've heard rumors of some out on the West coast, but it's hard to know if they weren't just sightings of homeless guys wearing donated Natrone Means jersey's.

Skip Towne
09-14-2004, 04:43 PM
No, it has been established, the Chargers have 2 fans.

whoman69
09-14-2004, 04:46 PM
I think another move would be to delete unused accounts. The members list takes up space I'm sure. We have over 1000 accounts that have never had a post and another 900 that have less than 10. Some of those accounts have avatars that take up more space. No crying here if lost, especially when one considers that most were troll who came by here when their team was playing the Chiefs and they never came back.

Logical
09-14-2004, 06:20 PM
Not that my opinion matters but I really would miss the ability to look up old posts. If a locked archive is the answer that is vastly preferable IMO to deleting such a huge amount of history.

No matter what I support whatever it takes to return the BB to good performance.

Thanks again Kyle for all your hard work on this site.

Boozer
09-14-2004, 06:26 PM
Not that my opinion matters but I really would miss the ability to look up old posts. If a locked archive is the answer that is vastly preferable IMO to deleting such a huge amount of history.

No matter what I support whatever it takes to return the BB to good performance.

Thanks again Kyle for all your hard work on this site.

I'm sure my opinion means even less, but I heartily second Vlad's opinion.

penguinz
09-14-2004, 09:12 PM
Why the need to prune the DB down. I visit other sites that have many more posts and many more active users at all times and they have no issues. I would look more at hardware and connection than pruning the DB.

chiefsfan58
09-15-2004, 12:48 AM
Austin or Phobia,
Just out of curiousity, what type of box is this running off of (cpu, ram, hd) and how big is the entire DB?

DenverChief
09-15-2004, 12:52 AM
Austin or Phobia,
Just out of curiousity, what type of box is this running off of (cpu, ram, hd) and how big is the entire DB?

the Russians will answer no such questions!

Phobia
09-15-2004, 12:57 AM
the Russians will answer no such questions!

Nah - it's not hosted with them any longer. Kyle switched.

I don't know the exact specs on the server, but from what I understand, bandwidth, memory, storage, and processing are not an issue at all. My personal hunch is that the 3.03 version of vBulletin would help, but I don't really have any factual reason for believing that. Kyle says it is the size of the DB.

penguinz
09-15-2004, 07:32 AM
Upgrading to a release version of the forums would probably help considerably. But who cares about bugs and security issues in beta software. ;)

htismaqe
09-15-2004, 08:21 AM
Why the need to prune the DB down. I visit other sites that have many more posts and many more active users at all times and they have no issues. I would look more at hardware and connection than pruning the DB.

We've looked at hardware and connection.

Just a question, but how much do you know about database heirarchy and structure? Do you know about tables, table space, and indexes?

Many, many things contribute to the performance of a database. Sheer size many times has NOTHING to do with it.

htismaqe
09-15-2004, 08:23 AM
Upgrading to a release version of the forums would probably help considerably. But who cares about bugs and security issues in beta software. ;)

We care about bugs and security issues.

We also care about MANY customized features that this board has that no other vBulletin boards have.

Kyle would have to, at the least, port and, at the very most, complete re-write each and every one of those hacks.

He knows we need to upgrade. It takes time.

AustinChief
09-15-2004, 08:24 AM
Nah - it's not hosted with them any longer. Kyle switched.

I don't know the exact specs on the server, but from what I understand, bandwidth, memory, storage, and processing are not an issue at all. My personal hunch is that the 3.03 version of vBulletin would help, but I don't really have any factual reason for believing that. Kyle says it is the size of the DB.

The upgrade to VB 3.03 will help, but only marginally, most of the "fixes" in 3.03 are already hacked in here. The posts table is the only real issue... and the solution is to either throw hardware at it... or trim it down by splitting it into "active" and "archives"...

I will post more info on this in a sec...

Mr. Kotter
09-15-2004, 08:28 AM
Thanks for your efforts Kyle.

Will there be someone at the 37Forever Events to whom we can make a "donation" to the cause?

penguinz
09-15-2004, 08:28 AM
We've looked at hardware and connection.

Just a question, but how much do you know about database heirarchy and structure? Do you know about tables, table space, and indexes?

Many, many things contribute to the performance of a database. Sheer size many times has NOTHING to do with it.Very true. But their are utilites that can be run on a DB to optomize the tables which helps quite often.

HC_Chief
09-15-2004, 11:11 AM
I'm a data-tier architect. If you guys have any questions in regards to indexing, horizontal partitioning of tables, or any DB questions in general, shoot me a PM. I'll gladly assist.

Rain Man
09-15-2004, 11:13 AM
WOW! Really? Perhaps I'll quit ignoring it and join in then. You better be telling the truth though!!!:D

They're studying the Book of XXXodus now.

Mr. Laz
09-15-2004, 11:35 AM
I'm a data-tier architect. If you guys have any questions in regards to indexing, horizontal partitioning of tables, or any DB questions in general, shoot me a PM. I'll gladly assist.


well there ya go ....



ding,ding,ding .. we have a winner



turn HC loose on this bad boy :p

Phobia
09-15-2004, 11:41 AM
FWIW, I've received several offers of assistance via PM. I've forwarded them all to AustinChief. That's his department, so please forgive me if I didn't respond to each one of you.

KC Jones
09-15-2004, 11:42 AM
While everyone's volunteering things...

I know a guy that runs a pet store. I could get a new hamster to run on the wheel that powers this bad-boy on the cheap.

But what about yoru SLA needs?

I reccomend http://www.deadhost.net/package2.php for all your hamster powered IT needs:

Security Compromise Hamster Power*
All the features of our Security Compromise Basic package, plus:

* 35 grams hamster food
* Unlimited raisins
* Daily water bowl fill-ups
* 300% slower than the Security Compromise Basic package so your hamster can keep up
* Free unlimited website suspension/termination and legal action
* Free unlimited access to other members' credit card details
* Free publication of your personal details in ALL industry publications
* Free addition to over 10.5 million spam lists
* Server powered by revolutionary eco-friendly renewable hamster technology

And at and amazing £199.99/mo it's much more expensive than buying a real hamster!

htismaqe
09-15-2004, 12:28 PM
The upgrade to VB 3.03 will help, but only marginally, most of the "fixes" in 3.03 are already hacked in here. The posts table is the only real issue... and the solution is to either throw hardware at it... or trim it down by splitting it into "active" and "archives"...

I will post more info on this in a sec...

So you're going to split the table?

Doesn't that mean we'll just have to do it again?

Maybe HC or some of the other DB-saavy people here could suggest better ways to partition the tables...

penguinz
09-15-2004, 01:29 PM
Has gzip encoding been enabled on these boards?

AustinChief
09-15-2004, 02:11 PM
So you're going to split the table?

Doesn't that mean we'll just have to do it again?

Maybe HC or some of the other DB-saavy people here could suggest better ways to partition the tables...

Unfortunately, I know how to change the DB structure to help.. but that would require changing the code for nearly every query that VB uses. So splitting the table is probably our best bet. I am working on a "process" that we can use long term.

--Kyle

Iowanian
09-15-2004, 02:17 PM
Would Nuking the Rep Feature and history help at All?

htismaqe
09-15-2004, 03:06 PM
Unfortunately, I know how to change the DB structure to help.. but that would require changing the code for nearly every query that VB uses. So splitting the table is probably our best bet. I am working on a "process" that we can use long term.

--Kyle

Well that sucks.

stanleychief
09-15-2004, 10:49 PM
Would Nuking the Rep Feature and history help at All?

The more lookups and joins that have to be done, the slower it's going to be, so yes. Pretty much any feature that forces a recursive lookup into other tables is going to consume resources and time. In working with phpBB I think we ran across the same issue. In some areas it was doing some pretty inefficient lookups (getting a user record, getting all records from another table, then doing a foreach loop to map them to the users instead of a join)

It probably depends on how denormalized the rep feature is. A while back I looked into implementing a rep feature into phpBB2. I had planned on storing the user's current running total in the user table to prevent another lookup. I also worked on the MyForums feature over on http://www.kcsportschat.com but pretty much gave up due to the lack of interest.

For those familiar with vBulletin.. do you get the source when you license it, or do they obfuscate it? Sounds like we have a few DBAs and programmers here... we should have a powwow and discuss the architecture a bit. I'm pretty comfortable with MySQL and PHP, and although I can design to an extent, I don't claim to be an architect.

stanleychief
09-15-2004, 11:00 PM
Has gzip encoding been enabled on these boards?
Bad plan if you don't control the browser versions your user's have in their hot little hands. IE 4 and up and Netscape 4.5 and up are the versions when they started to handle compression correctly. I know this covers about 99% of the viewership, but there's always that 1% that are still running Windows 95a with AOL 4.0 :)

Funny you mention that though. I just implemented gzip and deflate compression on an Intranet site today for a large client. It's nice when you know all of your viewers will be using IE 5.5 and up. We saw total size drop ~200% on average, but page load times through 3 T1's wasn't really significantly reduced. Go figure? :p

ExtremeChief
09-16-2004, 06:05 AM
Bad plan if you don't control the browser versions your user's have in their hot little hands. IE 4 and up and Netscape 4.5 and up are the versions when they started to handle compression correctly. I know this covers about 99% of the viewership, but there's always that 1% that are still running Windows 95a with AOL 4.0 :)

Funny you mention that though. I just implemented gzip and deflate compression on an Intranet site today for a large client. It's nice when you know all of your viewers will be using IE 5.5 and up. We saw total size drop ~200% on average, but page load times through 3 T1's wasn't really significantly reduced. Go figure? :p


ummmm....huuuhhhhh....he said deflate :Elvis:

penguinz
09-16-2004, 08:35 AM
Bad plan if you don't control the browser versions your user's have in their hot little hands. IE 4 and up and Netscape 4.5 and up are the versions when they started to handle compression correctly. I know this covers about 99% of the viewership, but there's always that 1% that are still running Windows 95a with AOL 4.0 :)

Funny you mention that though. I just implemented gzip and deflate compression on an Intranet site today for a large client. It's nice when you know all of your viewers will be using IE 5.5 and up. We saw total size drop ~200% on average, but page load times through 3 T1's wasn't really significantly reduced. Go figure? :p
If they are not using an updated OS and browser then that is their problem. Do you really want people that ignorant posting here? Aren't there enough moronic posts to read? :P

Lzen
09-16-2004, 09:11 AM
Bad plan if you don't control the browser versions your user's have in their hot little hands. IE 4 and up and Netscape 4.5 and up are the versions when they started to handle compression correctly. I know this covers about 99% of the viewership, but there's always that 1% that are still running Windows 95a with AOL 4.0 :)

Funny you mention that though. I just implemented gzip and deflate compression on an Intranet site today for a large client. It's nice when you know all of your viewers will be using IE 5.5 and up. We saw total size drop ~200% on average, but page load times through 3 T1's wasn't really significantly reduced. Go figure? :p

If they are still running Win95 and any version of AOL, then they are probably too stupid to figure out how access this site anyway. ROFL

stanleychief
09-16-2004, 06:09 PM
If they are not using an updated OS and browser then that is their problem. Do you really want people that ignorant posting here? Aren't there enough moronic posts to read? :P

My feelings exactly, but alas.. 'tis a public site. Depends on the requirements I guess that the admin decides. Also.. anyone know what Mac browser compatibility is for gzip/deflate? Anyone care? :evil:

Calcountry
09-16-2004, 06:11 PM
While everyone's volunteering things...

I know a guy that runs a pet store. I could get a new hamster to run on the wheel that powers this bad-boy on the cheap.
Who might that be? :p

The hamsters are on strike, it will never work.

Calcountry
09-16-2004, 06:14 PM
Would Nuking the Rep Feature and history help at All?
I am sure that the server gets slowed down every time it has to load your avatar.