Jump to content

...hey, a new ANNOUNCEMENT: Forums Are Slow. We Are Searching for a Solution


puretube

Recommended Posts

  • Members

 

Quote Originally Posted by eti

View Post

Seems faster already!

 

kinda reminds me of the good old Aretha Franklin tune:




 










[edit]: This reply took another 20 minutes,


+10 minutes to edit the link...

BUT: the topic-reply-notifications are working quite well...

Link to comment
Share on other sites

  • Members

I've seen this happen before on forums with fewer visitors. The problem almost always boils down to the server being ill equipped to handle the size of the forum database. There are a variety of ways to manage this.


First, I assume HC isn't leasing space on a server along with other websites. This site, and especially this forum database, are just too big to be sharing a server. A good quality dedicated server in a good datacenter costs only a few hundred dollars a month.


Second, the thing to be avoided if at all possible is when the database manager has to hit the hard drive in order to complete a query. Every page view can generate dozens of database queries. If the database manager has to hit the hard drive to find the data then even a dozen simultaneous users can bring the server to it's knees. The database manager will try to cache as much in memory as possible, so the memory on the server needs to be maxed out, and non-essential services turned off to free up memory for the database manager.


If the database is large enough (this forum's database is probably at least 8GB) then even stuffing the maximum amount of memory in the server may not be enough to completely avoid hard drive thrashing. A solid state hard drive will help a lot. Multiple solid state drives, with one dedicated ONLY to the database manager, will help even more.


The ultimate solution is to distribute the work to multiple servers, all in the same datacenter with dedicated gigabit ethernet pipes between them. Very old threads that are rarely accessed should be moved to an archive database on it's own server. This will free up a lot of memory on the main database server for active threads. Keep that main database pared down so that it never exceeds the size of it's memory cache.

Link to comment
Share on other sites

  • Members

 

Quote Originally Posted by wilsoneffectsll

View Post

I'm totally amazed at the size of this forum and their lack of a solution to a problem they created facepalm.gif

 

Especially considering that being down/slow for any period of time permanently impacts traffic. Big facepalm.
Link to comment
Share on other sites

  • Members

 

Quote Originally Posted by amp_surgeon

View Post

I've seen this happen before on forums with fewer visitors. The problem almost always boils down to the server being ill equipped to handle the size of the forum database. There are a variety of ways to manage this.


First, I assume HC isn't leasing space on a server along with other websites. This site, and especially this forum database, are just too big to be sharing a server. A good quality dedicated server in a good datacenter costs only a few hundred dollars a month.


Second, the thing to be avoided if at all possible is when the database manager has to hit the hard drive in order to complete a query. Every page view can generate dozens of database queries. If the database manager has to hit the hard drive to find the data then even a dozen simultaneous users can bring the server to it's knees. The database manager will try to cache as much in memory as possible, so the memory on the server needs to be maxed out, and non-essential services turned off to free up memory for the database manager.


If the database is large enough (this forum's database is probably at least 8GB) then even stuffing the maximum amount of memory in the server may not be enough to completely avoid hard drive thrashing. A solid state hard drive will help a lot. Multiple solid state drives, with one dedicated ONLY to the database manager, will help even more.


The ultimate solution is to distribute the work to multiple servers, all in the same datacenter with dedicated gigabit ethernet pipes between them. Very old threads that are rarely accessed should be moved to an archive database on it's own server. This will free up a lot of memory on the main database server for active threads. Keep that main database pared down so that it never exceeds the size of it's memory cache.

 

I wouldn't mind paying for this forum to be honest if they would invest in infrastructure. We've heard multiple explanations from db crashes to archiving to looking for a solution. Seems like the problem is never found and it's time to get an expert involved.
Link to comment
Share on other sites

  • Members

 

I wouldn't mind paying for this forum to be honest if they would invest in infrastructure. We've heard multiple explanations from db crashes to archiving to looking for a solution. Seems like the problem is never found and it's time to get an expert involved.

 

 

That confirms that the database is the problem. The sheer size of the database is causing the database manager to hit the hard drive far too often. EVERYTHING is in the database, from the layout information, the sub-forum categories, the counters, the "latest post" info - everything. Even a visitor like a bot from a search engine needs to generate dozens of database queries just to view a single page. I imagine HC is also using one or more content management systems for the other parts of the site, which adds even more databases. If any of those databases is large enough, like this forum's database, then one server can't handle it alone.

 

If by "archiving" you mean breaking the forum database into multiple databases then that would help a lot, but the forum scripts would have to be able to accommodate multiple databases. I don't know if vBulletin is. If not, then the php scripts would have to be modified to permit it. Otherwise, links to threads archived in a different database would be broken. I've seen other forum admins simply cut off the forum for a day, and move the entire forum to another subdomain, and then start fresh with a new forum. This is easy to do, and dramatically cuts down on the database manager load because the archived forums aren't accessed very often. When they aren't accessed then they aren't cached in memory. The archived forum would also have to be locked so that nobody could add posts there.

 

Nobody should have to pay for this. That's why websites host advertisements. Experts are expensive, but if the server admins don't know how to deal with this problem then they need to bring in some professional help. There are other forums much bigger than HC that don't have this problem because they've learned to how to manage their databases. Look at Google. Their databases are massive, but their site is fast and efficient because they distribute the load to hundreds of servers. HC doesn't need to go anywhere near that far, but they've definitely outgrown the environment they're running in.

Link to comment
Share on other sites

  • Members

I think the HC Admins are in a no-win situation. The forum as a whole is very popular and very active. The sponsored ads certainly bring in some money for them. The fact that you Google search our posts and read them without logging helps to bring in additional traffic. But I wonder if the archiving is even necessary. If you have a thread that hasn't been updated in one year, how relevant is to current posters?

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...